Less is more, claim the British. In der Beschränkung zeigt sich der Meister, the Germans say. When it comes to the usefulness of data, we should keep these sayings in mind, as these could be vital to succeed (or perhaps even survive) in the unfolding data-driven economy.
Making effective use of data is a struggle for many organizations. Over the years, IT structures have evolved into a complex collection of systems that barely support processes, let alone satisfy informational demand. This is an important part of the reason that many organizations are unable to supply reliable and unambiguous data to feed algorithms and make decisions. Data management programmes aim to solve this problem albeit with sometimes disappointing results. The programmes often come down to 'repair after the fact' and therefore risk being a one-off. Creating and maintaining high quality data requires more work than mere patching. When the basics aren't right, it's like filling a bucket full of holes.
As hard as it is to be successful in data management, there is another frequently overlooked factor in this respect: the fact that approximately 80% of organizational data is stored in an unstructured form. This enormous amount of data piles up in organizations, and just keeps growing. It is a vital source for insights, yet control over all this information remains very limited and the majority of the pile is in fact – simply put – garbage. The challenge is to remove the garbage and only keep the data that is relevant. Because less is in fact more.
Business leaders cannot afford not to take up this challenge. For starters because the cost of (hosting) this data pile is unnecessarily high and will only keep growing. Another factor is that old data needs to be removed because of laws and regulations (compliance). But by far the most important argument is that polluted unstructured data leads to bad decision making. Imagine hiring someone based on the wrong CV. And then accidently sending that new employee a permanent contract instead of a temporary one. In a data-driven economy, this is not acceptable.
The good news? There is a solution to clean up this enormous data pile in a controlled and responsible way. With modern techniques (indexing tooling) we can remove redundant, obsolete and trivial (ROT) data and apply classification logic to identify valuable data. These insights are added as metadata1 to create structure in what once was an enormous unmanageable pile of data. Over the years, we have developed and tested a 5 level data retention funnel that offers a proven approach. In every layer of the funnel, a little more ROT data is removed.
It's not unusual that organizations fear to lose valuable data. A good approach to overcome this is to store data in a hidden form for a limited period of time. End users can request specific data during this grace period, which would then be made available to them if required for appropriate reasons. Making data available by going through the steps of the data retention funnel through the indexing tool is important, to make sure data is correctly enriched with metadata. At the end of the funnel, only valuable data remains.
The bad news? This is hard work. There is no doubt that a structured data cleansing approach helps organizations to improve the management of unstructured data, thereby enabling better decision making. But the professionals who tackle this are the ones who do the heavy lifting. It's hard and unglamorous work cleansing the bits and the (tera)bytes. It's in fact not a very sexy job. But somebody's got to do it. It's extremely rewarding to witness how it contributes to new opportunities. And it's an excellent way to live up to the promise of a popular Dutch saying: Opgeruimd staat netjes.
1 Metadata is data about data, it provides information about a specific data object's content. Some examples are the author, creation date or document class (e.g. a CV or contract).
Please contact Simone Jeurissen, (020) 656 4089 or by email.