Organizations create a lot of underutilized data in their operations. During unexpected situations, analysts are looking to harvest their organization’s data and build very sophisticated analytics solutions around them with tools that build bridges between the siloed data within the organization. In addition, more and more external data is utilized.
To complicate the problem further, organizations need to collaborate with their ecosystem and ask their customers or suppliers to share their data, too. This data ecosystem allows them to enrich their own data and get better insights into processes and behaviors. Keep in mind that the majority of data is still created internally through automation by using robotic process automation (RPA) or the Internet of Things (IoT). Industry 4.0 increased the utilization of IoT for automated data gathering, creating a large amount of data that needed to be organized, analyzed, and used in the moment. When that last use criterion is not met, the available data is considered dark data.
As new challenges arise, organizations and analysts will utilize different sets of data when the questions change. Constant change drives continual process optimization and continual data expansion. Therefore, companies need to expand into external data based on collaboration with their ecosystem partners and external data sources.
Tools are needed to establish the collaboration, structure the data, and analyze in preparation for decision insights. Artificial intelligence (AI) and machine learning assist in finding ‘the needle in the haystack’ or improving the accuracy of decision models. For example, where earth observation data is used, there’s a high level of AI and machine learning adoption. Why? Satellites create super large data sets and individuals can create AI models to gain insight.
Satellites are IoT sensors for the world. They create all kinds of different data, not just the satellite pictures we typically imagine. For example, ESA Sentinel-1 satellite provides weather info and all-day and all-night radar info. Sentinel-2 provides multispectral, high-resolution images. Sentinel-6 carries a radar altimeter. They help to understand the current situation such as radar, color spectrums, and other data points that can be the foundation for predictions.
As enterprises discover the value of earth observation data, they will see that in the near future many of their processes can be enriched with information from a satellite. Many of the current use cases center around climate or agriculture, but we see an increasing trend within logistics and maintenance. For example, companies are optimizing logistics by determining best routes through incorporating real-time traffic and environmental restrictions. These are just a couple of examples. We will see many more use cases for sustainability.
Very often, new answers are determined based on past patterns. Here, space data can also help by answering difficult questions with historical data: How did the temperature change over the last five years? How did the population for specific cities change based on their footprint and buildings? How did the traffic change during COVID-19? Dark data can become valuable data. It all centers around the question to be answered. Having access to all the data will increase the ability to predict. We also need the data as we will change our questions or improve our AI models.
To get inspired on how you can use this dark data, check out this explainer video from Vox Media sponsored by SAP and Intel.
The success of future organizations will be based on their intelligent use of internal and external data to make optimal decisions. It is all in the question to be asked. As resources become more scarce, innovation cycles shorten, and the pressure to focus on customers and cut costs increases, companies need to utilize data differently and provide the insights with great visualizations so they can make intelligent, sustainable decisions.
How are you finding the dark data within your organization and broader ecosystem? Do you have use cases where earth observation data can be applied? SAP Business Technology Platform can help.
Torsten Welte is global head and vice president for Aerospace & Defense Industries at SAP.