By Jeremy Hughes

You can’t manage what you don’t measure.

Businesses who don’t measure the quality of their data waste time and energy on problems that are perceived rather than real.

Bad data quality can lead to inefficiencies, excessive costs, compliance risks and customer dissatisfaction.

Good data quality provides businesses with the facts to inform decision making and removes assumptions, emotions and politics.

Jeremy Hughes
Company-X co-founder and director Jeremy Hughes says bad data quality leads to inefficiencies.

There’s a number of ways in which data quality can be poor. Data quality issues are often not known until you pull all of your data together and measure the dimensions of accuracy, completeness, consistency, timeliness, usability, relevance, and uniqueness.

When you pull a lot of data together for the first time you discover the quality is variable. There can be a lot of inconsistency within organisations. Different regions, offices and staff can lead to variations in data quality. As can changes in staff and business processes. You don’t know that your data is inconsistent until you pull it all into one place.

The first thing that jumps out is although you’ve collected all of this data into a data lake you are unable to draw any reliable conclusions. You pull the big lever to produce your first overall benchmarking comparative reports and there’s a big sigh from the crowd. Instead of cheers, it’s like, “Oh, how can it be? We think we sold one million widgets but according to this report, the most amount of sales of any one thing is 50,000.”

As you drill in further you discover they have all been coded differently and that’s why nothing matches up.

Poor data quality leads to a distorted view of reality.

Pooling data shines the light on results and gets everybody’s attention.

You can’t improve what you don’t measure.

On one project we heard a lot of rhetoric that the client had poor data quality but we needed to quantify that. Is data quality really poor and is it poor everywhere? Or is data quality just poor in a few places? The evidence was quite anecdotal so we built a set of 63 metrics which quantified the data quality across the important data and built easy to use dashboards so that people could see where they needed to put their effort and investigate further.

I talked about this with a different client, got some data, put it together, and found data quality problems.

Does this apply to all businesses and organisations? I would challenge anyone to suggest that it doesn’t. If you are producing any sort of reports or dashboards that you are making decisions on you should consider and confirm the quality of the data that it is based on.

Company-X has helped many clients in New Zealand and overseas improve their data quality, and make better data-driven decisions as a result.

Once a business or organisation publishes metrics you shine the light on realistic data quality assessment, and also send a signal that data quality is important. That immediately leads to a change in culture around data quality, which inevitably leads to business improvement.

Jeremy Hughes is a co-founder and director of software specialist Company-X and is based in Hamilton, New Zealand.