Communicating quality gaps, holding space for good testing, and writing automation are some of the ways that testers contribute to software teams. According to Maaret Pyhäjärvi, we need to think about testing, not testers. Collaboration and having conversations between team members can result in valuable impact that changes the product and the experiences of our users.
Maaret Pyhäjärvi gave a keynote about the impact of testers and testing in software teams at HUSTEF 2023.
A way that testers contribute to software teams is by identifying and communicating the quality gap, the concrete examples of things missing between what we mean we want, what we say we want, and what we have, Pyhäjärvi said. Some people call this bug reporting, but that misses the truly valuable part, choosing to have a conversation, and choosing which conversation is timely and relevant, she added.
Testers can hold space for good testing in teams. Instead of pointing at problems, pairing and ensembling in particular can help software developers discover they can do testing, as Pyhäjärvi explained in an example:
I was working with a developer who was testing their own feature, and one day I decided not to say what to test and do. They looked at me and said "You’d want me to click here" and I said nothing, they clicked things choosing what they thought I would do, and found problems they had been missing.
She did not realize that sometimes we should do less to let people do more, and just point out the pattern, Pyhäjärvi mentioned. Realizing that she has more impact - in the moment and in the long run - through guiding developers to discovery, rather than discovering for them, is an unusual but valuable contribution, she added.
Writing automation that stays behind when you leave, and does the work, is also a valuable contribution of testers in software teams, as Pyhäjärvi explained:
Building for ownership, building for time when I am no longer around to test myself, and ensuring my contribution is not just getting bugs fixed by finding them, but also in keeping track of past lessons in an executable documentation format, with test automation.
Pyhäjärvi mentioned that through testing, we can change the product, change the experiences of our users, and become better testers and programmers:
I like to think it’s about testing, not testers, and we need the courage to bring conversations that create and support change.
Very few of our meaningful impacts come from a single person, Pyhäjärvi argued. It takes a village, or a pair and a ladder effect where you no longer know exactly whose idea fed the final implemented idea. Collaboration creates great results, Pyhäjärvi said. Collaborative credit is acknowledging all parties, not just the one who spoke the final form of the idea, she concluded.
InfoQ interviewed Maaret Pyhäjärvi about testing productivity and sharing stories.
InfoQ: How can we measure the productivity of testing?
Maaret Pyhäjärvi: The work of testing is like having a white paper with invisible ink; no one can tell you in advance exactly what you will need to discover. I use all kinds of approximations in my projects. After I have tested a little, doing a tour or survey of the application, I usually guess how many bugs are hidden and set that as a budget that I work against, and track our progress in uncovering those. Working against a plan, this allows me to update the end goal as I learn, but also show steps of progress. Similarly, but with more effort, I create coverage outlines.
I have to say though, that lack of productivity with testers shows up as noise (bothering others) and as silence (missing relevant results). Testing provides information at best. At worst, testing provides information people don’t want or care for. I call that kind of information noise, and noise bothers others.
InfoQ: Why is it important that testers share their stories?
Pyhäjärvi: Bugs aren’t relevant without their meaning. Meaning is conveyed with a story.
Improvements over time, teaching our colleagues something, learning from our colleagues - these are impacts best shared with stories.
InfoQ: What’s the testing story that you would like to share?
Pyhäjärvi: In one project, it took a bit of effort over the two years to turn away from planning test automation to implementing tiny bits of it, and to realize that time spent warning about the limitations of automation is time taken away from reaching success with it.
I used to say automation could not do everything. I explained the limitations of automation again and again. And I did less automation. Then I chose differently. Every time I felt like warning of its limitations, I chose to add some valuable work to it. As a result, I got useful and improved automation. It is still not perfect and needs more thinking from me, but if I had chosen to spend time on selling the idea that it could go wrong, I would not have learned how it could go right.
I think of this as a story about the choices that we have, but don’t recognize we have, since pushing through in times of big changes seems safer than changing things. I don’t think we would have done a good job testing without centering exploratory testing and documenting selected lessons in test automation.