Sander Klous | 8 June 2018
Imagine you are a parent whose child is applying to get into school. You'd want your child to have the best possible chance of being accepted at the school of your choice, right? Now, what if you knew that the decision of where to place your child was being made by artificial intelligence (AI). Would you trust an algorithm to have your child's best interests at heart?
The Wild West approach to data and analytics needs to evolve into a mature process, and quickly, for companies to maintain trust in how they do business.
That's exactly the scenario parents in one of the major Dutch cities have encountered ever since the school system embraced AI to create a more equitable and evenly distributed student allocation system. The algorithm has been designed to prevent the oversubscription of popular schools while providing the overall best result for all children - but how can you prove it works accurately and ethically?
It was a challenge we embraced when we were asked to create a model of assurance that would give peace of mind to administrators and parents alike that the algorithm was functioning fairly so all affected could have faith in the system.
Of course trust has long been a defining factor in an organization's success or failure - underpinning reputation, customer satisfaction, loyalty and shareholder value. Increasingly though, with the widespread adoption of data analysis in general and more specifically of AI throughout business, machines and algorithms have become a significant part of the trust equation.
That poses serious questions for all organizations because, in the technological rush to gain a competitive advantage through AI, companies might be prepared to take on higher levels of risk even though the data and algorithms they depend on are becoming more and more complex and opaque. This could lead to cases where for example, executives are being asked to make major decisions based on the output of an algorithm that they didn't create and don't fully understand.
The Wild West approach to data and analytics needs to evolve into a mature process, and quickly, for companies to maintain trust in how they do business. So far that trust is in pretty short supply even within companies themselves. According to KPMG's recent Guardians of Trust report - a survey of 2,200 global information technology (IT) and business decision makers involved in strategy for data initiatives - just 35 percent had a high level of trust in their own organization's analytics.1
For AI to be truly transformative we must have confidence in how it functions. That's why a comprehensive assurance model for AI is so important - one that builds trust through guaranteeing that algorithms are reliable, the system is cyber secure, IT processes and controls are properly implemented, appropriate data management is in place and that there is a governance structure that understands the ethics of machine learning. That understanding is subsequently included in the management of a wide range of organizational risks, like the potential impact of failure on financial results or reputation.
When you consider this assurance model, auditing AI is not all that different from auditing financial statements. The same principles and good practices apply - such as the three lines of defense and the impact of potential mistakes (materiality). And just as with financial statements public interest should be the highest priority of the auditor as well as a far-reaching willingness to be transparent and to cooperate closely with national and international regulatory bodies. As always, the auditor is accountable to the general public, as well as to regulators and the corporate sector.
Ultimately, the governance of machines shouldn't be fundamentally different from the governance of humans and it should be integrated into the structure of the entire enterprise. That way, hopefully those affected by AI decision-making will have as much belief in the system as the Dutch parents whose children will have a more equitable chance of school selection thanks to an independently audited algorithm.
For more insight on data-related topics, please visit our data-driven technologies article series page.