Extracting value from data? Avoid these pitfalls

Integrating and unlocking resources reliably and long-term is a challenge for many organizations. In many organizations, this involves dozens of internal systems and external sources, each with a different level of quality and topicality. Of course, a qualitative assessment is made of the source systems to be integrated, but you also have to continuously monitor whether the source is still providing the correct data. In many organizations, you can bet that something changes in this landscape each week. So continuous attention to integration and monitoring is essential for success.  

When integrating multiple sources, the first step to take is to record which term has which meaning (master data management). Because a certain term having different meanings depending on the system is extremely common. For example, is the price inclusive or exclusive of VAT? Is a customer seen as a person listed in the CRM system, as someone who has placed an order or as someone who has paid?  

The next question is: which system and which record contains the correct information? After all, a customer being entered into the CRM system three times, with three different records, is not uncommon. And on top of everything else, it turns out that the record in the ERP system also contains a different address, because the invoice address and delivery address do not match. Or sources suddenly produce a different type of data is also possible. For example, a system that returns turnover, but suddenly does so in the local currency.  

If you're already able to centrally unlock the right data, of the right quality and with the right timeliness, the next step is to extract value from that data. A common mistake to make is data scientists enthusiastically starting to developing models, but they lack the domain expertise to properly interpret the data. For example, regular mistakes involve the interpretation of cause and effect. This is because an algorithm is very good at finding connections, but not at identifying a root cause. And before you know it, you draw the conclusion: the weather will be nice, because we are selling more ice cream.  

Bias also creeps into the models all too easily (unconscious prejudice). The Dutch benefits scandal is perhaps the best-known example. The recent warning from Aleid Wolfsen, chair of the Dutch Data Protection Authority (AP), can be viewed in that same light. During an investigation into various government organizations, the AP found many discriminatory algorithms. This is often due to an unbalanced data set, a non-neutral training method or unconscious biases when developing the logic.  

Companies also often underestimate what is involved in setting up governance. After all, you will have to continuously monitor the performance of your models, because circumstances change and therefore the models will have to change along with them. Fortunately, a study by Conclusion demonstrates that more and more organizations have properly allocated data ownership. But that alone is not enough to ensure that data quality and model quality remain at the desired level.  

Finally, if you're going to use automated decision-making, you will need to ensure that you organize 24x7 support on the systems. When applying a data-driven approach was still something for people who worked in the office and made reports, a system being down for a few hours during the weekend because of an update was not a problem. But if you carry out an automated fraud test when accepting insurance policies that can be taken out online, you obviously cannot accept customers over the weekend who represent far too high a risk. Before starting your data project, think about how you will organize management and support of the data platform.  

 In this blog, we have described the most common mistakes that organizations make when they start applying data-driven working. Do you want to know the right approach? Read the white paper ‘In nine steps to data integration and a data-driven organization’.