Recently in Data Audit Category

Ask First, Fix Later

| No Comments | No TrackBacks
By Elliot King

Elliot King
Like the Boston Red Sox breaking their fans' hearts, almost inevitably (stress on the almost) you will discover that some percentage of your data is wrong. The realization that you have data quality problems may come about for few reasons: 1) you've looked under the hood of your data systems by conducting a data assessment or 2) a data audit revealed that the data you have is not what you think you have.

Or a problem may have percolated to the surface. Perhaps a direct mail campaign failed to yield the anticipated results or customer service representatives find themselves with incorrect information during critical interactions. So what do you do then?

With most rude awakenings, people want to act right way. After all, the data is broken, so let's get it fixed. With data quality, however, the impulse to act immediately may be a mistake. Indeed, the first question to ask is, does it really matter? The sad fact is that we live in a world of inaccurate and incomplete data.

Data sets will never be perfect. Inaccurate data may have little or no impact on ongoing processes and the investment required to remediate the data may be more than the return better data will provide. Identifying the impact of the data quality is essential. Have the problems resulted in lost revenue? Has customer service been compromised? Have the issues driven up costs? And so on.

Once the impact of the problem has been isolated, the next step is to better understand the nature and scope of the problem. What are the processes through which incorrect or poor data is entering the system? As most data professionals know, often data problems have more ways into your system than a freeway has on-ramps. Can the sources of incorrect data even be fixed? If they can, how much investment will be required and how much improvement can be expected? Finally, what will be the expected return on investment?

Though it seems a little counter-intuitive and perhaps even a little uncomfortable, the first step after data quality issues are discovered is to think. You may not want to act at all.

Why You Need a Data Audit

| No Comments | No TrackBacks
Elliot King
Everybody makes mistakes and those mistakes have consequences. As enterprises rely more heavily on data to make decisions and drive processes, the quality of that data becomes more critical to the overall success of the organization.

In many cases, the impact of bad data is not too hard to identify--direct marketing campaigns with high numbers of email bounce-backs or undeliverable mail; marketing efforts with poor response rates because the offering has not been correctly tailored to address the "wants and needs" of the customers; prospects who turn away because their names are spelled wrong or other information is incorrect.

With the growing need to integrate data from so many sources, both internal and external as well as the dynamic nature of business itself, the chances of introducing mistakes into critical databases is growing exponentially. It's not really a question if there are mistakes in your databases; the question is how many mistakes are there and what kind of negative impact will they have on your business?

The answer to those questions can be determined by conducting a data audit. A data audit is just what the name implies--a systematic look at data to insure that it is what it is supposed to be.

Conducting a data audit is pretty straightforward. First, determine which records and which fields should be examined. If customer records are to be audited, fields such as names and addresses perhaps; buying history, payment history or other critical fields may be included. The field selected should be those associated with a specific process or activity.

Then assess the fields to determine which do not conform to the data dictionary and the business rules that govern them. A wide range of errors may surface. Values may be missing. Values (such as dates) may fall outside of a specific range. Fields may be incomplete or formatted incorrectly.

The next step is to determine the source of the error; how the errors can be rectified (if, in fact, they need to be rectified); and how mistakes can be minimized in the future. Among the most common sources of errors are faulty data conversions and inconsistencies in integrating data from different sources. Not every error always has to be fixed or the underlying process changed. In most situations, there will always be some errors in the data that can be tolerated because they do not have a crippling impact on the business activity being supported.

How often should data audits be conducted? Not surprisingly, the answer is elastic. Data audits should be conducted often enough that faulty data does not impede reaching your business goals.