Addressing the Elephant in the Room – TDAN.com

neub9
By neub9
4 Min Read

One of the most common observations made when visiting different clients’ data factories is the use of spreadsheets for data collection and purification. While these spreadsheets are essential for the data enrichment process and timely report generation, they also pose a significant control problem by exposing companies to the risks associated with inaccurate data.

It’s time to address the issue of excessive reliance on spreadsheets for data enrichment. The reluctance to acknowledge data gaps is understandable, especially after investing significant resources in new transactional systems or data warehouse projects. However, it’s essential to confront this reality and find a solution.

The solution lies in controlled data enrichment outside a company’s transactional systems. While it’s not feasible to provide an automated or transactional source for all data due to the complexity of business processes and the vast availability of data, addressing the missing data sources is critical. These missing sources often represent a high risk of financial reporting errors.

Master data management (MDM) systems have been used to attempt to solve this problem, but their integration throughout the organization is often not seamless, leading to an increase in the use of spreadsheets. As Lean Governance practitioners, we propose a solution that involves transforming raw data assets into finished goods (reports and models) with minimal time, effort, and resources through an integrated data governance framework.

This integrated governance framework encompasses the complete definition of enterprise governance, including organizational structure, information, data assets, business glossaries, business process, security, data loss prevention, and data quality controls. Adding a data enrichment module built around the core governance metadata that defines a company can significantly reduce the risks associated with bad data.

Data enrichment is a critical business tool that goes beyond traditional applications. It can be used to enrich data across the organization with domain codes, tags, or other taxonomy groupings, effectively eliminating the need for numerous spreadsheets designed for data enrichment, collection, or processing.

To ensure Lean Governance, a data enrichment facility should be designed in the context of required SOX or other controls and deployed in a production-controlled environment rather than locking spreadsheets away on secured drives, which does little to minimize overall data risk. Furthermore, auditability and access control are essential for retaining lineage across the entire data factory.

It’s time to confront the issue of excessive reliance on spreadsheets for data enrichment and bring these spreadsheets into full view with an eye on compliance and controls. By challenging the governance technology team to incorporate data enrichment into the organization’s governance framework, a new approach to data and information governance can be achieved.

Note: I happen to love elephants and have a deep respect for them. Their size, which makes them so amazing, is likely the reason they have been used to describe something so big in the room that no one can miss.

Share This Article
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *