The implementation, enhancement, and updating of complex software solutions that connect several applications by means of interfaces (for example SAP R/3, SAP APO, external systems) requires system integration tests. For these tests, complex checking plans usually are drawn up that specify how to carry out and control manual procedures, and log the results. If a change is made to an existing system (new release, customizing change, enhanced interfaces, and so on), system operators often forego this extensive procedure. Instead, they expect users to notice any weak points during their day-to-day work. However, as not all processes are used to the same extent, problems and errors are often only discovered at a late stage – often too late, because by this stage it is usually very difficult to trace when and how the incorrect functions entered the system. As a result, it is not possible to make the connection between the error and the change that caused it. This problem results in a very high workload in searching for errors, and leads to frustration and accusations of blame on all sides. Users of complex software therefore want an automated process that checks the systems after each change, and possibly at regular intervals as well.
OR Soft uses automated tools within its SAP R/3 and SAP APO projects. These tools control whether the mapping of a model in both systems – SAP R/3 and SAP APO – is adequate, whether the most important processes function correctly, and whether both systems contain synchronous movement/transaction data.
System changes put to the test
The integration tests are not carried out within the SAP R/3 and SAP APO software that is to be tested, but instead outside of it, on a test suite set up by OR Soft. The test suite is an add-on to the main systems without separate data maintenance, but connected to both applications via standard transactions with read and write access. It sets up a local LiveCache, in order to merge the models from both systems. As an external add-on, the test suite allows the test object to be described independently of the concrete modeling. As a result, the tests can be reproduced without the influence of the system changes.
The test experts describe and define what exactly needs to be tested, and how, in test scripts. Test scripts and the test results that are obtained can be stored system-independently. The documentation enables the tests to be repeated in exactly the same way at any time. Once the changes have been made in the system, a comparison between the original test results and the new ones then produces clear indications.
For the tests, the necessary data is loaded from SAP R/3 and SAP APO, or generated from the test suite in SAP R/3 and SAP APO. The data objects loaded from the source systems arrive in the LiveCache of the PCs, where they can be compared with each other or with specified target values, and logged. The tests can be run automatically in the background. The log can be configured so that it only records peculiarities or deviations.
“Production model” test case
The way the automated system tests work can be explained using two examples. The first example looks at how different customizing settings and modeling in SAP APO and SAP R/3 planning leads to different results. If the scheduling of orders in the two systems is carried out under exactly the same conditions, then the modeling is correct. Specifically, this provides proof that the production process models (PPMs) in SAP APO are modeled correctly, and indicates that important customizing settings were made absolutely correctly from a modeling point of view and match the SAP R/3 settings.
A special checking routine is run for this purpose. It consists of several processing steps and simulates the weekends and working breaks by means of shift regimes. First, all PPMs are automatically scheduled one after the other as orders correctly from a system perspective, and then shifted by 6 sets of 24 hours (to test the weekend effects), increased (scaled by a factor of two) within the valid lot size, shifted back again and then shrunk by the same scaling factor. At the same time, the system tests observe the behavior of the related process order in SAP R/3. Here, even if the modeling is not the same, at least the start and end times of the corresponding process order must agree. If all operations, executed one after the other, reproduce the original situation, then the settings are correct (in accordance with what is defined in mathematics as the “necessary condition”). If deviations occur, then either the modelling is not correct from a formal point of view, or customizing settings relating to non-working times or order release strategies are incorrect.
“Stock development in the future” test case
The second example relates the agreement between material requirement and stock lists in SAP APO and SAP R/3 on the one hand, and the comparable MRP elements on the other. It is necessary to check whether the future situation is mapped in the same way in both systems, despite the fact that SAP APO has a higher degree of planning accuracy than SAP R/3. In this case, the checking routine compares the shared order categories (planned order, process order, customer requirements, forecast requirement) for all material requirement and stock lists for a planning scenario. If the basic data and corresponding elements agree, planning and confirmation in the two systems will not lead to contradictory information in the connected systems.
Simulation of extreme situations
The psychological value of the tests should not be underestimated. A change to the system often causes a certain amount of uncertainty among users. Intervention is expected to lead to errors in most cases, which could then come to light during productive operation. The system tests are a precaution against this. They are able to demonstrate to users that the solutions are stable. This increases user acceptance, relieves the burden on IT, and reduces the implementation and update times.
The same applies for customer-specific adjustments. The tests quickly and reliably check the functioning of the processes and the algorithms for software changes. If large volumes of data are involved, which cannot be handled manually, or in extreme situations (such as very long transactions or high interface load), they provide useful information in the early stages of the project. By simulating user behavior (simultaneous planning actions by different users, sequence optimization with intensive calculations, confirmation of a large number of orders), they enable reliable predictions to be made about the runtime behavior and memory requirements of the software.
Finally, system tests are also suitable for acceptance procedures in order to put the software and the performance parameters promised to the customer to the test. They also enable programming errors, conceptual errors, and poor data quality to be determined quickly in a software implementation project.
To date, OR Soft has applied these tests in a number of its own SAP APO and SAP R/3 projects, and reduced the time and costs for system integration as a result. The test suite is now sufficiently developed that it can be used as an independent software tool by IT departments for ongoing quality control and for reliable integration.