Disruptive trends in master data management

Master data management (MDM) is a framework of processes and technologies for creating and maintaining an authoritative, reliable, sustainable, accurate, and secure data environment that represents a “single version of the truth,” which is an accepted system of record used both intra- and inter-enterprise across a diverse set of application systems, lines of business, and user communities. Master data, such as that about customer, prospect, patient and citizen, or sometimes supplier, product, securities, drug, etc., is foundational to business processes, usually widely distributed, and critical for an enterprise. When well managed, master data contributes directly to the success of information management initiatives and the organization overall. When not well managed, master data limits the agility of business systems and processes.

As more companies have deployed MDM technologies, the number and type of value-added capabilities have grown to unlock previously undiscovered capabilities and return on the company’s initial investment. Five significant trends have emerged that are driving the demand for even more capabilities from MDM technologies causing disruption in the market and potentially creating game-changing opportunities for companies deploying MDM solutions.

This article discusses these five disruptive MDM trends and the technology requirements resulting from market changes. The trends include:

• Expanding beyond entity resolution to include relationship resolution
• Creating data stewardship processes and technologies
• Integrating business rules engine and workflow products
• Externalizing enterprise data security and visibility
• Interoperability of metadata across applications and data stores

Expanding Beyond Entity Resolution to Include Relationship Resolution

As MDM evolves, organizations are looking for significantly better ways to understand the relationships between individuals; individuals and households; and individuals and corporate entities; informal groups and organizations. The market evolution and regulatory requirements are creating demand for MDM technologies that understand relationships between entities. For example, the Basel II Committee, which recommends standards and best practices in banking, suggests that corporations, partnerships, foundations and other organization types should look behind the institution and identify principals (and their family members and partners) who have control over the business and its assets. When a company is a subsidiary of another company, the principals at the parent company may be the individuals that need to be identified.

Even though the statements above are specific to banks, the mandate of understanding party relationships and hierarchies is quite generic and spans horizontally across all industry verticals, government agencies and organizations. Businesses are striving to determine who their best customers are, how to estimate the risk-adjusted values of these relationships, and what they should offer to strengthen these relationships. The goals for businesses are to increase their share of a customer’s wallet, enable more and better cross-sell and up-sell opportunities, and increase customer retention rates, all while gaining better control, and limiting, or even eliminating, some relationships that are risky or not cost effective. In addition, government agencies need a deeper understanding of relationships between entities to prevent terrorist threats, money laundering, and other criminal activities and unwanted events.

Many companies are already devoting resources to understanding these relationships, but are doing so through manual means with limited efficiency, scale or consistency. To help companies better understand relationships, MDM vendors will enhance their offerings with advanced algorithms capable of recognizing relationships and hierarchies in data.

­­­­­­­­­­­­­­­­­­­­­­­­­­

However, even with advanced computer algorithms, not all decisions can be made systematically. Human review of party records and potential relationships will be required when technologies cannot establish the identity or relationship with the required level of confidence or when data issues impacting identity or relationships identification are algorithmically discovered. For this reason, data stewardship processes and technologies integrated with MDM technologies is the second disruptive trend.

Creating Data Stewardship Solutions

Most companies today have informal, de facto data stewardship processes and resolve data quality issues in a reactive way when circumstances arise, such as complaints from customers or business users or failures in automated processes when data errors occur. Formal processes and employees with day-to-day responsibilities and performance metrics for data stewardship are more the exception than the rule.

As MDM solutions evolve and mature, there will be an increasing need for human review and analysis of data. Dedicated data stewards will be needed to review and evaluate findings daily or at more regular intervals to proactively assess and validate identified entities, relationships and hierarchies. MDM technologies will also need to proactively identify data issues and conditions of interest and generate configurable issue queues that can be directed, prioritized, reviewed, escalated and resolved by individuals with data stewardship responsibilities.

Depending on the tasks being performed, and the required accuracy and timeliness of the entity identification and relationship results, the number of dedicated data steward resources will vary. For example, when MDM focuses on marketing activities, it will typically require less accuracy and timeliness and fewer resources. However, when MDM is used in situations where even a few incorrect or untimely results can cause significant legal, safety or financial consequences, the highest accuracy levels and adequate resources will be of utmost importance.

At present, data stewardship is considered mainly a back-office function. A more collaborative approach, enabled by new friendlier technologies and user interfaces, might push some of the data stewardship functions to the outer edge of the company to include the front office. This will lead to more proactive data stewardship, which will ultimately improve data quality and governance.  

Data stewardship processes and technologies will help to ensure accuracy and completeness of information, but how can this new and powerful information be leveraged to achieve even more business benefits? The answer lies in a third disruptive trend – the integration of MDM technologies with business rules engine and workflow products.

Integrating MDM with Business Processes and Workflows

Instead of having built-in business rules and workflow processes, MDM offerings need to integrate easily with existing business rules engine and workflow technologies.

Many industries face complex supply chain challenges that require near-real-time synchronization of master data across participating systems. Both “upstream” processes (such as client on-boarding and account opening) and “downstream” processes (including data quality issue resolution, data stewardship, and data alert configuration, generation, and processing) will benefit greatly from the integration of MDM software with business rules engine and workflow products. This integration will require development of and compliance with interoperability standards, which will enable enterprises to more easily leverage the benefits of MDM to improve business processes and operations, and achieve other business advantages. The integration of MDM with business rules engine and workflow technologies will also significantly enhance the collaborative capabilities and qualities of the data stewardship process.

Once data management has been integrated with business rules engines and workflows and the data is accessible across the entire enterprise, security of enterprise data becomes an even greater requirement and challenge. In order to achieve seamless end-to-end enterprise data security, MDM applications need to ensure compliance with company and regulatory policies. The best way for this to be accomplished, and the fourth disruptive trend, is for MDM solutions to consume existing enterprise security policies owned by enterprise security management applications and systems such as LDAP (Lightweight Directory Access Protocol).

Externalizing Security and Data Visibility

Enterprises want to eliminate the data management issues that are created when stovepipe applications and systems have responsibility for determining who in the enterprise can read or change which data, or who are authorized to exercise which actions. Enterprise data access controls are best handled through a company's dedicated enterprise data security and visibility systems, rather than being redundantly owned by multiple individual applications, including MDM solutions. MDM technologies need to be able to work with external systems, e.g. LDAP, and be able to consume their authorization, eligibility, data visibility and security rules as needed. Open MDM products capable of sending data to and receiving data from a single enterprise-wide data security and visibility product will help improve the quality and robustness of data security and visibility solutions and reduce their cost.

While securing critical data is obviously a top enterprise priority, being able to read and understand data across multiple applications and systems is also a critical MDM success factor. Creating an environment where metadata interoperates across various applications and systems within the MDM ecosystem is the final disruptive trend.

Interoperability of Metadata Throughout the Enterprise

To create an MDM system, companies utilize multiple technologies including extract, transform and load (ETL) and data quality tools, data matching and linking engines, real-time synchronization tools and other required components. Unfortunately, tool-specific metadata typically use different and often proprietary standards. The result is interoperability problems across each of the components as data cannot be exchanged by tools and applications without significant analysis and development efforts, which greatly impacts project delivery timelines and the quality of integrated solutions. Today enterprises spend millions of dollars to integrate vendor technologies and custom-built solutions.

Although extremely challenging, unifying metadata and/or creating a common standard or layer that transforms metadata from multiple applications and data sources into a single format is critical. Enterprises will benefit greatly from better metadata integration and interoperability based on the creation and adoption of new metadata interoperability standards. Metadata interoperability standards will enable businesses to more rapidly create new business systems, based on these more agile data architectures, and respond faster to changing business demands.

MDM: The Next Phase

As MDM extends beyond single entities to include relationship and hierarchy management, robust data stewardship support, integration with business processes, and sound integration practices, the value of MDM technologies to companies deploying it will increase dramatically. As with all technology evolution, these changes will create new demands and result in market disruption. Companies that are aware of how to leverage the change and the advanced capabilities it delivers to their organizations will benefit greatly from the new game-changing options.

About the author

Dr. Lawrence Dubov is a recognized expert in master data management and customer data integration with a proven track record of successful large-scale project implementations. He is senior director of sales consulting and architecture at Initiate Systems, Inc. Dr. Dubov has been in information technology consulting for twenty years and has delivered complex business driven technology solutions to financial services, banking, insurance, and pharmaceutical verticals with a primary focus on CDI, MDM, CRM, data warehousing and operational data stores. He is a co-author of a definitive book “Master Data Management and Customer Data Integration for a Global Enterprise”, McGraw-Hill, 2007.