WTANZ 21 – Test Reporting: Selling the Story

WTANZ 21 – Test Reporting: Selling the Story

Date: Saturday 31st May, 2014

Time: 2:00 PM AEST (click here to check time in your timezone)

Facilitator: Dean Mackenzie

Attendees: Ahmed, Anurag, Teri Charles, Erin Donnell, Kim Engel, Jari Mikael Laakso, Richard Robinson, Tarun Sharma, Uma Shankar Gupta

In one comparison, test reporting has been likened to journalism, particularly in that they can share similarities in the way that “news” (i.e. test results, bugs and the like) is communicated to stakeholders. But test reporting can be done in almost countless different ways and styles. With this in mind, how the comparison (to journalism or ) help us improve our test reporting? Are there tips or techniques that we can borrow that could increase the chances of our results being heard?

The goal of WTANZ 21 was to look at test reporting, and the different styles in which testers do it. The application under test was “Remember the Milk”, a list-keeping and organisational app available in web or mobile formats. Participants were encouraged to try a different range of reporting styles as they tested.

After some initial questions around the scope and purpose of the testing, the weekend testers set to the task in pairs, groups and singles. A number of bugs were found with the app, including several around the sign-up / log-in processes. Discussion around the bugs found continued throughout the session, and it was noted the nature of the bugs could help determine what would ultimately be reported to stakeholders.

A range of styles were used to document the participants’ results, from spreadsheets to mind maps to Google Docs. Other discussion points included:-

  • Constraints with testing (e.g. time available, collaborative purposes, structure of task) can influence the style of test reporting used
  • Questions asked early in the piece can play a large part in shaping a test report
  • Test notes can play other roles beside the format of a test report

The full transcript for this session can be found here.

About the Author

Dean is a software tester living in sunny Brisbane, Australia. Having fallen into testing around seven years ago, he subsequently fell into the context-driving approach several years later. He enjoys studying new ideas (especially around testing or learning), and finds there just isn't enough time to look at all of the good ones. Dean blogs intermittently at www.yesbroken.com and can be found under @deanamackenzie on Twitter.