Data integrity is at the core of a successful data warehouse
Data integrity involves putting the necessary rules and data structures in place to ensure that the company’s data is consolidated, integrated and accurate. It’s a guarantee that the data is consistent, and gives the company a single version of ‘the truth’.
“If your data is not correct then the information you base your business decisions on may be false, and that can have a severe negative impact on the profitability of the organisation,” says Sean Longhurst, director of EnterpriseWorx.
“Data integrity means complete and accurate data that makes it possible to formulate predictions and make well-informed decisions.
“The reality for most organisations, however, is that their data resides in a number of unrelated systems and databases. For example, one system may handle enterprise resource management, another financial data and yet another human resources systems.
“Data warehousing unifies data lodged in diverse formats in different systems so as to enable the business to develop reliable, integrated reporting and analysis – a single version of ‘the truth’.”
Proper business and technical analysis processes leading up to the modelling of the data is critical to ensuring its accuracy and its successful application in the business. “This is the most important element in determining whether the data warehouse will be successful,” says Longhurst. “You need to ask the right people – both executives and end users – what they want out of the data warehouse. Frequently, different people have different views of the data and different perceptions of which data is correct, and these conflicts must be resolved.
“A key aspect to bear in mind is that it’s important to focus at the outset on what information you want to get out of the system. This is the only way to ensure that what you put into the data warehouse is appropriate. You can obtain a single view of the customer, for example, only by defining the key performance indicators you require – is it purchases, region, credit days outstanding?
“You have to define and design the system to generate the information you require, particularly if you are working towards implementing a business intelligence system at a later stage.”
The implementation of a data warehouse allows the organisation to apply universal business rules on a single source of data, rather than trying to apply the same standards across each disparate data source.
“The biggest risk is that the goalposts seem to shift all the time, particularly as the business’ focus changes,” says Longhurst. “The solution must be flexible and robust enough so it’s possible to make changes as you go along. If you follow sound design principles in modelling and building the database, this should not be difficult. You have to be patient and keep refining requirements. It’s an ongoing process, and there’s no golden rule. But you do need to make sure at the outset that your project planning time schedule is realistic.
“To ensure that the database is an accurate reflection of the universe it is modelling, it must have entity and referential integrity. This is done through the proper use of primary and foreign keys and automated auditing and balancing of data. This enables the consolidation of all data sources to ensure that data is complete, correct and stable, as well as fast and reusable for different purposes.
“Data integrity is the bedrock of the data warehouse and the foundation of business intelligence, leading to better and more accurate decision-making,” concludes Longhurst.