Igniting the Afterburners for Data Analysis

March 21, 2007 by admin

According to a survey by the market research firm of IDC, 30 percent of companies with annual revenues greater than US$500 million expect the amount of information stored in a central data warehouse to double in the next three years. But only 15 percent of the companies assume that the IT department delivers the right data to the right employees at the right time.

Easy connection to SAP NetWeaver

Business success depends on lightening-fast analysis of important data and information and extraction of market-relevant knowledge from piles of data. SAP NetWeaver Business Intelligence Accelerator (SAP NetWeaver BI Accelerator) was developed by SAP along with a technology partner, Intel. The software was shipped to selected SAP Ramp-Up customers at the end of 2005 and brought to market in the middle of 2006. Users now have the right tool for data analysis and extraction.
SAP NetWeaver BI Accelerator is based on proprietary SAP search and classification technology. It can be used as a preinstalled package solution only with the SAP NetWeaver platform, starting with SAP NetWeaver 2004s. The software makes analysis available on blade servers from Hewlett Packard and IBM, which have recently been equipped with Dual-Core Intel Xeon processors. The software runs on the 64-Bit Novell SuSe Linux operating system. SAP NetWeaver BI Accelerator is delivered as an appliance; the software is installed and configured on defined hardware. Activation simply requires a link to SAP NetWeaver Business Intelligence using a remote function call (RFC).

Dynamic summarization and flexible analysis

SAP NetWeaver BI Accelerator clearly enhances and improves the performance of reports based on SAP NetWeaver Business Intelligence. Execution of queries occurs an average of 10 to 100 times faster. That has advantages, especially when dealing with terabytes of data. The innovative key technologies of SAP NetWeaver BI Accelerator contribute a great deal to faster execution time for queries and more flexible analyses. Relational databases store data row by row, but SAP NetWeaver BI Accelerator stores data in columns and with a high degree of compression. That approach enables execution of query processes in main memory (also called in-memory-processing) and in parallel on all CPUs.
For parallel queries, the software sets up indexes with all the data from the InfoCubes of SAP NetWeaver Business Intelligence. The index contains all the data of the InfoCubes in a highly compressed, but not aggregated form. This approach differs from that of relational, OLAP-based query technology, which uses aggregated data. In SAP NetWeaver BI Accelerator, data is compressed dynamically during a query. That means that users can always search the data in real time and evaluate it according to specific criteria – like sales by week, region, or product.
When compared with technologies based on relational OLAP, this procedure offers advantages – especially with large quantities of data. OLAP reports must aggregate the business data according to certain characteristics even before a query is executed for good performance when accessing data. That requires the IT department to set up and maintain a new aggregate data store for that purpose, which requires a lot of time and can demand a great deal of maintenance when several different types of queries are involved.

Parallel queries, fast results

Architecture

Architecture

If SAP NetWeaver BI Accelerator is docked with the SAP NetWeaver platform, users can select InfoCubes in the business information solution for which they want to optimize response time. The InfoCubes are set up as multidimensional data models and contain characteristics and key figures from any source system for business management. The multidimensional data from the InfoCubes is transferred to SAP NetWeaver BI Accelerator and stored in an index. A delta mechanism ensures quick indexing of new data in the InfoCube.
When a query is executed for the first time, the indexes are loaded into the main memory of the SAP NetWeaver BI Accelerator software, where they are formatted and distributed over across the various physical partitions of the blade. Parallel queries across individual servers return results rapidly. According to the Winter Corporation, a consulting firm based in the United States, a SAP NetWeaver BI Accelerator software system with 10 blade servers and 20 partitions handles an InfoCube 20 times faster than a query without SAP NetWeaver BI Accelerator.
SAP NetWeaver BI Accelerator transfers the results of the query to the analytical engine of SAP NetWeaver Business Intelligence, formats the business information, and makes it available to users over front-end analytical software.

Minimizing data flows and saving memory

In addition, high rates of compression and column-based storage of InfoCube data enable fast query results. Technically, SAP NetWeaver BI Accelerator achieves high rates of compression with a dictionary-based procedure and numerical coding of the indexed data. Information and objects presented multiple times are stored as references to the item that occurs first. Each entry is assigned to exactly one number. This approach minimizes (by a factor of 10) the amount of data that must be stored temporarily during different processing steps – regardless of the database system. According to tests of the Winter Corporation, data sources of 670 GB from an InfoCube require only 55 GB of storage space in SAP NetWeaver BI Accelerator.
Column-based data storage enables SAP NetWeaver BI to access the data column relevant to a query at all times, directly, and specifically. Individual columns are sorted and the most important results placed at the beginning of the column. That approach is more efficient than the row-based storage used by relational database systems, which must read all fields, even those that are not relevant to a given query.

Less load on the database lowers costs

The use of SAP NetWeaver BI Accelerator avoids database storage when reading queries, reducing the load on the database. Because the license prices of databases often depend on the load on the CPU, overall operating costs and the space needed for the data base are reduced. And the IT department has less administrative effort because it no longer has to deal with the time-intensive design and setup of aggregates, like those for regions or sales organizations. For example, if customer or product master data changes, the indexes in SAP NetWeaver BI Accelerator simply need to be reloaded. When working with aggregates, however, changed master data or hierarchies normally require a completely new setup.
Large companies with large amounts of data can benefit from this approach – energy suppliers, telecommunications companies, companies with significant requirements for ad hoc reporting, and retail companies. For example, a retail chain can accumulate millions of point-of-sale (POS) data records every day. It can use SAP NetWeaver BI Accelerator to create analyses and queries directly with the POS data and receive a response in seconds. Important customer data is available directly, and changes in consumer behavior can be traced immediately. Processes like sales planning, procurement, and marketing activities can be quickly tailored to new customer requirements.

Queries up to 300 times faster

Response Times

Response Times

The in-memory method of analysis used by SAP NetWeaver BI Accelerator is verifiably faster than typical relational database queries. The Winter Corporation analyzed 1.3 billion data records with the software with an average response time of only 0.9 seconds. SAP NetWeaver BI Accelerator also convinced the analysts in terms of scalability and performance. SAP customers who have studied the analysis software in their companies have confirmed the results produced under laboratory conditions.
For example, in a pilot attempt, the oil company BP used SAP NetWeaver BI Accelerator for a control query that contained all the business data of 2004 in less than 2.6 seconds. The query used to take 13 minutes. The company wants to analyze all its business and market data – a total of 10 terabytes – with SAP NetWeaver BI Accelerator so that it can recognize market trends more quickly.
SAP NetWeaver BI Accelerator also convinced Brown-Forman, a beverage company and producer of Jack Daniel’s. Queries ran up to 300 times faster with 650 million data records. The indexes needed for an analysis were set up in 6.5 seconds; in the past, aggregates required 44 hours for creation. Indexes can be updated in less than one minute; rolling up delta data into the aggregates lasted 30 to 60 minutes.

Additional information:

http://www.intelalliance.com/sap

Dr. Andreas Schaffry

Dr. Andreas Schaffry

Tags: , ,

Leave a Reply