Data governance that works

As tools like generative AI become increasingly mainstream, the quality and accessibility of enterprise data has become more important than ever before. Many organizations are rethinking their data governance programs as a result. But according to Kevin Lewis, a leading data analytics expert at AWS, 90% of organizations make the same mistake when they start a data governance program.  

"Most organizations have a generic goal to make data easier for people to find or to improve data quality generally," Lewis says. "But it's a big mistake to focus on data governance in isolation rather than starting your data governance journey by identifying business initiatives that will prove transformational for the company." 

Lewis sat down with ZDNET Editor-In-Chief Jason Hiner to talk about how to build a successful data governance program, the role of data governance in technology innovation, and how emerging technologies like generative AI are changing the way we work.

(The following interview has been edited for length and clarity.)

-

JH: Kevin, let's start with what you do at Amazon.

KL: As a part of AWS Professional Services, I work with customers to help them achieve their data analytics program goals. I get involved in data strategy, business alignment, data governance, and organizational structure. 

JH: How would you define data governance? 

KL: Data governance helps organizations accelerate innovation with data and data-driven decisions by making it easy for the right people and applications to securely and safely find, access, and share the right data when they need it. I like to think of it as simply making sure your data is in a condition that can meet the needs of your targeted business initiatives.

A key element in data governance success is finding the balance between access and control that enables innovation – and the balancing point is different for each organization. When organizations exercise too much control, the data gets locked up in silos and users are not able to access the data when they need it. This not only stifles creativity, but also leads to the creation of shadow IT systems that leave data out of date, and unsecured. On the other hand, when organizations provide too much access, data ends up in applications and data stores that increase the risk of data proliferation and leakage. 

JH: What are some of the common pitfalls that companies have to avoid when it comes to data governance?

KL: If you do a Google search right now for "How do I get started with data governance?" you'll see advice all over the place saying "make a business case for data governance." And this is the exact mistake that I see 90% of businesses make. 

Every major business initiative needs data to be successful. But there is a profound difference between finding issues that data governance can solve versus setting up data governance as what I like to call a "load-bearing wall." Instead of chasing the value of data governance on its own, your goal should be to position data governance as a load-bearing wall underneath the most important, vetted, and funded business initiatives of the company. If your governance strategy is supporting initiatives that the CEO and the entire executive team already care about, you're going to be much more successful.

For example, let's think about conditions-based maintenance for a manufacturing company. This company is trying to implement a program to proactively understand what maintenance is going to be needed on their machines based on sensor readings – things like temperature, vibration, and maintenance schedules. Once I have identified this program as my business initiative, I can work backwards and ask specific questions: "What are the most important data quality issues to be concerned about for this initiative? What are the most important data sources for this initiative that I need to be aware of?"

Contrast that with making a case for data governance on its own, where you're thinking more about making data easier for people to find or improving data quality generally. You might create some data sources that are easier to find, but to what end? Addressing general concerns will probably not produce value for something as specific as this machine maintenance initiative – and may not, in fact, produce near-term value for any of the company's business initiatives at all.  

JH: Are you saying that organizations have to work backward from customer requirements to address these pitfalls? 

KL: We want to be careful with the word "requirements." It's working backwards from requirements, but doing so in a very specific way. For example, if I work backwards from requirements, I could just go and ask end users what kind of problems they'd like me to solve. Maybe I'll find someone who has a spreadsheet that they work on every week, a sales spreadsheet, and it's very challenging for them to organize all that information and resolve data quality issues. Maybe they even have an executive who would give us the thumbs up that this is something that's good to target.

Asking someone what their requirements are is very different from working backwards from a vetted business initiative that's going to be transformational for the company. There are lots of problems within an organization that you can help to resolve. The best ones have been vetted by the executive team for major funding, and those are the ones I want to work backwards from.

JH: What's a good model for data governance?

KL: Once we've found our targeted business initiatives and the data is ready to meet the needs of those initiatives, there are three major governance pillars we want to address for that data: understand, curate, and protect.

First, we want to understand the data. That means having a catalog of data that we can analyze and explain. We need to be able to profile the data, to look for anomalies, to understand the lineage of that data, and so on. We also want to curate the data, or make it ready for our particular initiatives. We want to be able to manage the quality of the data, integrate it from a variety of sources across domains, and so on. And we want to protect the data, making sure we comply with regulations and manage the life cycle of the data as it ages. More importantly, we need to enable the right people to get to the right data when they need it. AWS has tools, including Amazon DataZone and AWS Glue, to help companies do all of this. 

It's really tempting to attack these issues one by one and to support each individually. But in each pillar, there are so many possible actions that we can take. This is why it's better to work backwards from business initiatives.

JH: How can a company build a data governance roadmap?

Building a data governance roadmap is taking all of these things we've talked about and making a formalized plan. We advise companies not to build a standalone data governance roadmap. Instead, fold data governance into your comprehensive data strategy roadmap. Within that roadmap we have comprehensive workstreams for the business initiatives you're targeting. We're asking things like, "What are the data domains that are needed for these initiatives and use cases?" Then we can begin to plan the data governance capabilities to make sure that your data is ready to support those initiatives. In parallel, you can get into things like architecture, security, and the organization's operating model.

In short, we make sure we understand the business initiatives, we make sure we understand the targeted applications and use cases, and then cascade all of our priorities from there and line them up in their own work streams. 

JH: What's the best way to distribute the responsibilities for data governance?

KL: This is a great question. The balancing act that companies need to get right is the distribution of responsibilities between centralization and decentralization.

Historically, a lot of data analytics programs have been centralized into a team that serves the needs of the community. In more recent years, companies have begun to distribute responsibilities so that different business areas are responsible for managing their own data – but they've been swinging way too far in that direction. Companies have been excessively decentralizing, such that the various distributed teams are responsible for too much of their own data management.

We advocate getting a good balance. It's important to empower distributed teams to manage and use their own data and share it as they see fit with other business areas. But it's also important for a central team to facilitate threading data together across domains. If you want to have an integrated view of data across the business, someone has to be responsible for doing that. A lot of companies are running into trouble today because they've excessively distributed responsibilities, and no one is responsible for this integrated view. 

JH: We have to talk about the hottest topic in tech, which is generative AI. How does data governance relate to this new leap forward in AI?

KL: All of the things that we've talked about related to data governance apply to generative AI, although, of course, there are additional considerations. Just as it's not a good idea to try to reinvent your overall data strategy for generative AI, it isn't a good idea to reinvent your data governance with this single focus, either. However, because of the interest in generative AI right now, you can use that as an impetus to drive data governance programs and position them to support a variety of initiatives.

JH: Kevin, do you have any final advice for organizations as they're looking to improve their data governance initiatives?

KL: Find someone else's funded business initiative to support. If you can support someone else's funded business initiative with elements of data governance and show near-term business value, you'll be off and running. You don't need to get married to that one initiative – but you do need to pick one or two to start. 

After you support multiple initiatives, you can look for commonalities and start to build foundations for your data governance program that can be widely used. With each successive initiative, you want to both have near-term business value and be contributing to the maturity of your data governance program.