Trust in Machine Learning
I enjoy reading the newsletter that Andrew Ng releases every week. Called The Batch, it has the latest news of what’s happening in the AI field, both in industry and academia. Even better, in each letter, Andrew gives his personal take on important points to consider when we execute an AI project. Previous issues have covered topics such as managing change, model robustness and the challenges of small data. The issues are concrete and the conclusions are actionable. They’re great at portraying what working in AI is like in real life.
December’s first issue in particular covered something I’ve had to deal with in my daily work — getting end users to trust an AI product. Building trust has a recurring theme in all my projects, so having a way to approach trust was useful.
Explainability
Explain how the algorithm works. If we can understand how decisions result from our model, we can decide whether or not to trust the algorithm.
It’s not quite true that deep neural networks or Random Forests are black boxes. Although they might not be as straightforward to understand as rule-based systems, but there are still techniques available. To go back to basics, even pulling up examples of correct and incorrect classifications can go a long way to helping an end-user understand how an AI comes to its predictions.
Boundary inputs
Clearly state where AI can work and when it won’t. Andrew gives an example from the medical domain. X-ray classifiers should state conditions such as the resolution of pixels a model expects before it can function accurately. From my own experience, we once made clear that our model would perform well on short and well-structured articles written in the style of reports. Give the model long-form and discursive articles, and the accuracy may not be the same. Caveats like these are warning labels that need to be stated clearly.
Gradual rollout
With gradual rollout, AI does not make fully automated decisions from the start. Instead, only after gathering more data over time and fine-tuning our models do we start to build more automation into the system.
Discussing what gradual rollout looks like is a conversation that’s good to have early in a project. Andrew talks about gradual rollout in the context of getting a human to “handhold” an algorithm before it’s ready to fully function in a task, for example filtering forum posts. However, I’ve also seen other benefits of gradual rollout: having multiple “release cycles” is how an AI pipeline refined and improved into a real product. Ideally, each cycle introduces an incremental improvement so that, little by little, the system evolves progressively and manageably.
I’ve had first hand experience with these techniques. Andrew also mentions other ways of building trust in algorithms. There is testing systems the way clinical trials test new drugs — through randomised clinical trials. There are also audits where third parties can test for known sources of bias such as race and gender. In a more real-time sense, there are also monitors and alarms we can place to receive alerts if anything goes wrong.
“Trust isn’t just about convincing others that our solution works. I use techniques like these because I find it at least as important to convince myself that a solution works, before I ask a customer to rely on it.”
