How do you view the role of trust in autonomy? Why is trust so important?
- Does it function reliably? When there are consequences to failure, a single error (as deemed by humans) is enough to wipe out the trust built up by tens — or even thousands — of successes. Technology is often a convenience more than a necessity. We automate tasks not because we are incapable of doing them ourselves, but because it would be tedious to do so, and would require substantial human capital (or risk). We are only willing to take the route of convenience and efficiency if we trust that the technology will work at least as well as the human efforts it is replacing.
- Do I understand it? A critical component to achieving trust is to enable people to build a mental model of an autonomous system’s behavior. The mental model need not be representative of the true underlying algorithms, as long as it can predict the future actions of the system. Having a model allows a user to quickly gain confidence that the model is being followed, while without it, every action will be unexpected and unexplainable, causing the user to question if the system is working properly. While people can learn to trust systems that they do not understand, the reliability threshold becomes substantially higher.
What is the state of trust in technology today? What impact does this have on innovations?
In today’s tech-saturated world, where we can hold conversations with the AI in our pockets, there is not just a growing trust, but a growing expectation, that our technology can do everything we imagine. It seems like every month, AI manages to beat humans at another task, to the point that we’ve come to expect it and now wait impatiently for technology to take the next task off our plate. This expectation boosts innovation, as failure to innovate will rapidly leave a company behind its competitors.
Does the level of trust in technology depend on the type of product? For instance, trusting your email versus trusting an autonomous vehicle?
Absolutely — trust is inversely proportional to risk. We trusted industrial automation decades ago because the only risk was to the product being produced. Then we began to trust automated systems with our money — fraud detection, risk assessment and even trading. Soon we will trust autonomous systems with protecting our lives, as we let them drive us on highways, through traffic, and near drivers more hot-headed than the computers in our own cars.
