Every organization runs on data. For an organization to run well, its data has to be complete, accurate and delivered on time.
Integration connects data sources to data targets and delivers data to people, applications and devices. But what about data accuracy and completeness? To achieve those goals, you need data management and data governance.
Ten years ago, many companies embarked on ambitious master data management projects, hoping to identify sources and formats for their data fields, and to make data consistent across the organization. More often than not, these projects failed to deliver the comprehensive data quality they promised. As a result, many IT organizations today are understandably wary of any project promising comprehensive data quality.
Fortunately, IT organizations can achieve accuracy and completeness without undertaking vast, all-or-nothing data management initiatives. Instead, they can build data management into data integration projects as they go.
What we’ve seen working with our clients at Slalom is that data quality can be tackled incrementally in new integrations and data migration projects using Boomi’s integration platform as a service (iPaaS). The result is improved data quality and data governance. These projects, being highly focused, tend to stick within their budgets, and their user communities are grateful for tangible, readily apparent improvements to data accuracy and completeness.
To help your organization achieve these same improvements, here are seven best practices for building data management and data governance into data integration.
1. Don’t try to tackle data management on its own.
People still remember those failed grandiose data management projects from a decade ago. And they tend to be suspicious of projects that promise a lot. So they’ll argue against funding a comprehensive data management project and recommend funding other projects instead.
“IT organizations can achieve accuracy and completeness by building data management into data integration projects as they go.”
A better approach is to make data management part of another IT project. For example, if an organization is going to replace its on-premise CRM system with Salesforce, then IT could set the goal of improving the quality of CRM data as part of the data migration process. Then the project would involve not only adopting Salesforce as the company’s CRM but also ensuring that the data residing in Salesforce is more accurate and complete than the company’s current CRM data. That kind of project scope makes sense to most people and is likely to gain support.
So, whenever possible, make data management part of a larger project that involves integrating or transforming data.
The Boomi platform makes it easy to combine these two types of work. You can use Boomi’s iPaaS to integrate, migrate and transform data, and then use Boomi DataHub to implement data governance rules for the data being migrated and transformed. All this work is handled by a single platform, making it easier to implement and manage.
2. Start small: Tackle one type of data or data used for one operation or with one application.
This best practice follows from the previous one. Don’t try to implement data quality for all your data at once. Instead, focus on a single type of data — such as customer data — or on data for one application, such as Salesforce or NetSuite or Workday. Tackle that small project, and the organization will see that data management is achievable. Then they’ll trust your team when you tackle the next project and the next.
It’s important to be able to demonstrate success, especially early on and especially if data management has been a challenge for your organization in the past. Demonstrating that you can put a process in place for cleaning data, resolving discrepancies, and making life easier for end users will help you win the support of multiple departments before you have to tackle more challenging areas of data management — areas that could involve difficult decisions and higher costs.
Changing data structures and source systems is never free, so it’s important for people to have bought into the idea of data management and data governance before you take on big changes that will affect lots of people or mission-critical operations.
3. Identify goals for your data management project, and set up metrics for tracking progress.
We all want data to be complete and accurate. But how do you measure completeness and accuracy?
It’s important to establish measurable goals for your data management project.
Ask yourself: What would constitute data quality for the type of data you’ve decided to manage? If it’s customer data, you might want to make sure it’s complete. For example, all customer addresses are current. If the customer has multiple sites, they’re all listed somewhere in your CRM system, and the primary site is clearly identified. You might also want to ensure that communications with all sites associated with the customer are logged in the CRM system and are up to date.
In most cases, those goals cover not just the data itself but also the processes that are going to ensure that your data is complete and accurate. So set metrics for capturing your data quality goals. You want to be able to measure whether people are logging into whatever tool you’re using for data management. Are they responding to queries they receive? Are they resolving discrepancies? What percentage of discrepancies are being clear in a given week? What are the trends that show how your data management processes are doing over time?
If you establish accountability for data quality, and that accountability becomes reported and widely visible, you’ll find people participating more willingly. Most people would rather fix a problem the first time than be asked why they haven’t fixed it the third time.
Monitor data management in whatever tool you’re using to measure other business results. For example, if you’re monitoring Key Performance Indicators (KPIs) in Tableau or QlikView, use that tool for monitoring data quality, as well. Data quality should be a metric that you always have in mind.
4. Build data management into the integrations connecting your applications and services.
An organization’s data is no longer centralized in a single ERP application as it might have been 10 or 20 years ago. Today that data is distributed across on-premise systems and multiple cloud platforms and applications. This diversification isn’t going away. We can expect most organizations to shift more of their work to the cloud over time. We can also expect cloud services and applications to continue evolving.
What does this mean for data management? First, data needs to be federated into multiple systems. And because the same data has multiple uses, you might have different data stewards for different elements within the same data record. Trying to tackle all this at once as a monolithic system simply isn’t practical. Instead, it’s better to have data management tightly coupled into integration processes designed to address data governance requirements for specific types of data in specific applications.
You need a solution that can be small or tailored and actually adopted by business users without too much overhead in the process.
5. Recognize that data management is an ongoing process. Data will change.
Data is always in flux, so think of data management and data governance as ongoing work, not a short-term project that will be done once and then forgotten.
Because data management is ongoing, it’s best to:
- Set your team’s expectations about their involvement in data management practices over the long term. To a greater or lesser degree, people’s jobs will change in some way. That change might entail nothing more than responding to an email to clear up some data discrepancies or it could be attending a quarterly meeting to review different sorts of data types. But make sure people know that the project will continue to involve them, even if only for a few moments each day.
- Set the organization’s expectations about budget and timing, too. Some amount of time and money will need to be invested in data management on an ongoing basis.
- Select a data management platform that makes it easy to include stakeholders in ongoing work, without requiring them to learn new technical skills or adopt cumbersome processes.
6. Involve business people in your data management project from the start.
Most of the time when there are inconsistencies between data records or between how different departments are using the same data field, the issue is a business problem, not a technology problem. To resolve the problem, you need to involve business people. You can’t keep the project within the IT organization.
We recommend that you include business stakeholders in data management from the start, including the initial planning stages of the project. Make use of business people’s expertise about what data they need and how they use it. They’ll also be able to tell you what improving data quality will mean to them – how improving data quality will affect business processes and business results. Be sure to build their goals into the project.
When you design processes for monitoring and managing data quality over time, involve business people there, too. Figure out who can act as an authority or data field for each record or field, and find a way that’s convenient for them to contribute their expertise.
This leads us to our final best practice.
7. Build workflow automation into your data management process.
If you want people to be able to resolve data discrepancies and data omissions quickly and easily, then you need to find an easy way to slip data management tasks into their daily work. If you require people to log into a special application or portal, you’ll probably have only limited success. But if you use workflow automation to bring data management practices into people’s everyday work, you’ll do fine.
For several of our clients, we’re using workflow automation software – Boomi Flow – that’s tightly coupled with our integration platform. When an integration process flags a data discrepancy, the record is put into quarantine, and an email is sent automatically to the data steward or subject matter expert for that record. The email is short and to the point. A business user can see at once what the discrepancy is and resolve it with just a few clicks. We hide all the business logic and complexity supporting the process behind the scenes.
Workflow automation like this enables us to bring data management to business users, rather than requiring them to adopt new IT tools or new procedures. Because the process is easy and straightforward, it’s highly likely to be adopted, making our data management program overall a success.
Conclusion
By following these 7 best practices, you and your organization can improve data quality without needing to embark on huge and hugely expensive data management projects. The key to success, as these best practices make clear, have as much to do with people skills and project management as they do with technology. In summary:
- Take an agile approach, building data management into the data integration and data migration projects you were going to tackle next.
- Involve business stakeholders from the start, and let them have a voice in identifying a clear outcome for the project.
- Use a technology platform like Boomi that combines data integration, data management, and workflow automation capabilities, so you can build data quality and data management into the connections you’re building for applications, people, data and devices.
- Remember that people will need to work differently (but not harder) after you’re done, as data management becomes part of everyday jobs.
Want to learn more about ensuring data quality for your digital enterprise? Please contact a Boomi integration expert or reach out to our partner, Slalom.