The enabler for an effective data warehousing and business intelligence strategy is automation. The process of loading a data warehouse and prepping the data for consumption, is analogous to building an assembly line or refinery that follows consistent processes to produce reliable data for decision making.
Like any “assembly line” whether producing cars or microwave ovens or clean reliable data , there needs to be continual inspection to make sure the quality of the output is maintained, because the quality of the input is suspect: business rules do change, source systems change, a new service representative may start using a free form text field for a purpose it was not designated, new codes get added to the system, agreements get brokered between departments for sharing data without consideration for the impact to the enterprise.
Having effective automated and human quality checks exposes problems early in the load process. An effective Data Governance strategy facilitates the sharing of the changes to processes and sources so that there is consistency of data at the source, whenever possible.
Just as there needs to be an agile project management process to build the data warehouse, there needs to be an agile data governance mechanism within the organization for identifying and affecting changes to the source systems, data warehouse and business intelligence environment so that the quality of the “refined” data is maintained.
More on Agile Data Governance later.
Loading ...
Loading ...
