ETL — People Still Do That?

While you won’t see “ETL” in headlines the way you see “big data” or “cloud,” ETL plays a key role in helping organizations take advantage of these technology trends. In fact, according to “The Forrester Wave:  Enterprise ETL, Q1 2012” report, the market for ETL is growing more than ever.

What is ETL?

ETL (extract, transform and load) often involves taking data from many systems and transferring it into your data warehouse for use with your analytics or enterprise resource planning (ERP) systems. In the process, the data is transformed to meet the business and technical needs of your database. It remains a critical part of many organizations’ successes. So, why don’t you hear about ETL now like you did fifteen years ago? It’s certainly not because ETL is no longer being deployed — if anything, there are more installations than ever before!  In some cases, the process has become such an integrated part of business intelligence (BI), ERP, customer relationship management (CRM) and other applications that people don’t even realize they are conducting ETL. In other organizations, ETL is still a separate business process.

The Forrester report shares how enterprise ETL is supporting many new and emerging data management initiatives. Oftentimes you hear the term ETL being used interchangeably with data integration. While that is not inaccurate, we know that ETL is only part of the data integration puzzle. Forrester shares (and SAP agrees) that ETL has evolved into new initiatives, including:

  • Data Virtualization — helps manage all the distributed data sources
  • Cloud Integration — ETL helps integration of on-premise data with the public and private cloud
  • Big Data — all those vast amounts of structured and unstructured data need to make it to your “big data” platform
  • Real-time data warehousing — needing access to data in seconds also impacts the need for “instant ETL”
  • Data migration and application retirement — moving data to new systems and retired old applications is becoming more prevalent
  • Master Data Management (MDM) — in order to achieve that holy grail of a master record, you need to be able to streamline all the various data sources and structures in your enterprise

The next time you think of ETL, don’t think of it as an antiquated or unused term, but remember that it is alive and kicking and still a major part of all data management initiatives. Take a look at the Forrester report for a nice snapshot of the current state of the market and to learn how your organization can best manage data from many sources: Forrester Wave report [PDF].

Trish Harman is a member of the Analytics Solution Marketing team at SAP, focused on enterprise information management solutions.  She focuses on helping  organizations understand how they can use EIM solutions from SAP to process any data type from any data source, enabling the delivery of integrated, accurate and timely information across the enterprise.