Most companies face significant challenges with data integration. These challenges arise as data from multiple sources has to be integrated with already existing data, in order to add value to that data. This data could include business data that has been purchased externally, local user generated data, map-based data, data that has been sourced from the web (social media, e-mail) and data procured through government produced indicators (economic and other indicators). Many different kinds of analyses are supported by Data warehouses and data marts are created in order to make the activity size manageable. Yet, the integration problem is faced by organizations across the board.
The integration possible between various data types greatly varies, fundamentally because they vary in their origin and hence face compatibility issues. Let’s look at some of the issues with data integration:
Most data warehouses or data marts will only support one of two types of data. Although the data warehouses can be modified to support different data formats, the final outcome is full of compromises. Some formats that are integrated include internal RDBMS data, object data, geographic data and de-normalized purchased monthly data.
This is an opportunity for disintegration of the data mart front-end environment. The key software products available in the market offer groups for graphs, charts, data animation, data landscapes, cartographic or dashboard representations. There is also a chance that there is no one base analytical tool in the architecture, and in that case, multiple visualization and analytic data marts might be running off the base warehouse.
Data acquiring techniques are used by all corporations in order to complete the picture of the customer, market, competition and forecasting. This has ushered in a new industry that consists of data packagers and resellers. The data packagers and resellers cater to data subscription services and even offer a full service of analytics. However, this brings with it the usage of multiple proprietary databases that may not be able to give an organization a holistic view of all the data acquired.
The best solution to deal with the problem of data integration is by consolidating the data marts. In this, it is already acknowledged that the integration process will not be seamless. A plan needs to be developed to coordinate data marts. The coordination is done by using enterprise architecture mapping and a centralized planning is developed. The degree to which the data marts are integrated will vary, from a full segmentation of the business into targeted data marts, to an as-needed segmentation. The key challenges are recognized as limitations for data types, along with constraints related to vendor and data visualization. These key challenges are controlled and leveraged.
For data warehousing problems that are difficult to be controlled within the organization, consolidated data marts offer a practical solution.
It allows establishing points of controls to be established in the disintegrated data environment and a flexible architecture is put in place for all data marts to be supported and enhanced.
A metadata shell around the consolidated design offers credence to the idea of semantic packages that can be attached to the data.
This allows for the appropriate data mart to be accessed using the standard interface for metadata, which allows for the necessary information to be found and analyzed.
The user desktop can be customized by enabling shortcuts to preferred data marts and data.
These are the operational benefits of consolidating data. On the functional side, theses benefits are exponential.
Data integration is a requirement that most organizations need to have fulfilled. The more diverse the technical environment, the more the need for integration is necessary. If the integration fails to materialize or is not efficient for any reason, it will jeopardize the larger initiative and the implications to the business can be severe.