What role does trust play in the testing process? Why is testing important in building trust?
Testing builds confidence that not only do we say something will work, we have shown that it will work, and we’ve shown that again and again in a specified set of scenarios. Testing provides that ground truth information.
How do you build trust in the testing process itself?
Our goal is to constantly bring the testing process closer to reality. It’s important to communicate its teachings and its limitations constantly to developers and the product team and to be clear about gaps we’ve identified and how we are working to close them, now and in the future. This helps the rest of the company understand which areas of performance are understood and which will require additional effort going forward.
When we think of testing, we think about the results we get from it and what we learn. What role does trust play in learning?
When learning we essentially trust that the tested scenarios are representative of the use case of the actual system. We trust that the failures, mitigation strategies, and environmental properties are sufficiently accurate to be of value.
What is the role of testing systems in helping us learn?
We need systems around testing to learn at efficient rate. Without systems in place you’re essentially testing at random. If you test at random, you’ll eventually find useful pieces of information, but testing targeted scenarios lets you find that information orders of magnitude more quickly than you would otherwise. And build upon that learning to create a product that has maximum impact on mission.
