Solving the “Big Data” Problem: Data Quality is Essential

In December 2011, The International Data Corporation (IDC) predicted that the management of Big Data would earn its place as a “must-have” competency in 2012. And that’s exactly how the industry shaped up. By the end of 2012 alone, the volume of digital content was expected to grow to 2.7 zettabytes (ZB), up 48 percent from 2011. More than 90 percent of that information, which includes images, videos, MP3 files, and files based on social media and Web-enabled workloads, is unstructured.

Big Data is becoming increasingly too large and complex and, as a result, has become difficult for businesses to work with using simple management tools. In most cases, data sets have grown so large that it has become problematic for enterprises to capture the data, store it, search for it, share it and analyze it. As Big Data continues to grow, it’s that much more important for businesses to analyze the quality of the data itself to make sure it’s actually fit for intended uses in operations, including decision making and planning.

The quality of data is at a pivotal point because, for the first time ever, businesses are beginning to look at data from the top down rather than from the bottom up. While enterprises still find data migration and data management to be important from an IT viewpoint, they are now beginning to look at it from an information governance strategy perspective. This means that data now encompasses an entire set of multi-disciplinary structures, policies, procedures, processes and controls, all of which are implemented to manage information and data at an enterprise level. This type of data management supports an organization’s immediate and future regulatory, legal, risk, environmental and operational requirements.

Part of this increased attention to data quality means that, if they haven’t already, businesses should put a strong emphasis on the need to eliminate erroneous data, duplications, etc., in addition to the more difficult task of analyzing how the data will function in an end-to-end business process. This will help companies run smoothly and adhere to customer orders more accurately.

Today, poor data quality is wasting nearly 16 percent of an average company’s budget, according to research from Experian QAS. In fact, 90 percent of companies questioned in the Experian survey admitted to wasting departmental budgets, whether through duplicated mailings, lost contacts or missed sales opportunities, because of inaccurate data. Furthermore, departments including marketing, sales, operations and customer service report wasting 15 percent of their budget on average. And in IT and data management the number increased to 18 percent.

It’s important to note that there is a clear distinction between understanding traditional data quality and the ability to analyze how that data will actually run as part of an effective business process. Rather than looking at data simply as a tool, businesses will succeed and grow as they refocus and manage the data from a business process perspective. The key is for companies to see that there are areas where data migration, management and governance can be made easier through proper data quality measures.

Businesses and IT users looking to unlock the value of their data assets need a trusted data migration and information governance partner. Only then, whether monitoring ongoing information quality or executing the most complex migration and governance initiatives, will they begin to alleviate their Big Data problem -“ and maybe even the entire industry’s Big Data problem as a whole.

Rex Ahlstrom, chief strategy officer of BackOffice Associates, is a technology-centric serial entrepreneur with more than 22 years of experience. He served as vice president of Operations at SAP, following SAP’s purchase of his company, SOALogix. Ahlstrom led SOALogix as its CEO over a seven-year span. Prior to starting SOALogix, he was the CEO of MSI Software and has held various management roles with organizations including Defense Systems Inc., TeleSciences and Westinghouse. He can be reached at [email protected].