WTEU-49 – How do you deliver bad news?

WTEU-49 – How do you deliver bad news?

Date: Sunday 21st September 2014
Time: 3.30pm – 5.30pm British Summer Time (4.30pm – 6.30pm Central European Time)
Facilitators: Amy Phillips, Neil Studd
Participants: Dan Ashby, Jolanda Klijn-Gooiker, Srinivas Kadiyala, Jignesh Nayi, Aleksandar Simic, Trisha Agarwal, Ritika Gulati, Sheetal Kamathan

When we discover and report bugs in our everyday work, that’s only part of the story. It’s also critical that we are able to communicate information about these bugs (or any other indicators which can be used to assess product quality) to the decision-makers within the business, to help them to make informed decisions. However, sometimes these messages can be badly received; testers can be perceived as “troublemakers” when raising release-blocking issues, or managers may have other motivations for wanting a product to release regardless. We may need to use tact and diplomacy to communicate information which leaders don’t want to hear.

In this session, we offered a bug hunt, with a twist! As well as searching for issues within a defined piece of functionality, we also asked attendees to consider how they would report these issues to the business, and then brought in our “product owner” (acted by Neil!) to see how our team would handle him. Would the product owner be receptive to their feedback? Could our group justify their discoveries, fight for the issues which they felt were most important, and find compromises for deadlocked situations?

We based the session around an imaginary scenario within the development of the Evernote note-taking platform. Although the scenario that we created (and particularly Neil’s highly-strung product owner) were entirely fictional, the feature we were testing is real (and live), and is something that Neil reported having previously caused him frustration when trying to use it.

Testing Time

We announced that we were testing the below user story, using the live Evernote website, but imagining this was a pre-release development environment.

EVER-123: Create tables in Evernote Web
As a skilled creator of advanced structured documents
I want to be able to create tables within my Evernote web notes
So that I can present data in a format which can be easily understood

Acceptance criteria:

  • Users can specify number of rows (1 or many)
  • Users can specify number of columns (1 or many)
  • Users can specify preferred table width
  • Users can change the style/formatting of cells
  • It must be possible to create multiple tables in 1 document
  • Adding tables should be really fast and easy

Our group immediately began asking questions about the user story, which was great to see. We’d placed some traps in the story definition, and also in the product owner’s persona, and it was fascinating to watch as the group began to uncover each of these. Our product owner was being presented as somebody who was over-worked and spread between too many teams, and this was reiterated throughout the chat, with lots of messages about “they’re too busy right now” and, at one point, the PO diverted Dan’s call when he tried to ask a question!

First up was the main testing exercise, to explore the tables functionality within Evernote and evaluate to what extent it met our fictional acceptance criteria. Attendees followed their energy, spoiled for choice with the issues that could be found, and were quickly finding a vast array of issues. Although the feature only had a single visible touchpoint (an “Insert Table” toolbar button, which spawned a dialog containing 4 inputs), we uncovered issues including poorly-enforced boundary conditions, browser compatibility problems, difficulties on mobile platforms, and even a browser crash when pushing all available constraints to their limits. Everyone quickly had a very long list of issues. With our testing time drawing to a conclusion, we addressed the biggest elephant in the room – there’s no support for modifying tables (adding/removing/resizing rows and columns). Dan mentioned that he had used the CRUD heuristic (which we had previously discussed in WTEU-47) to identify this.

Discussions with the Product Owner

With the product owner’s arrival imminent, we discussed what techniques and strategies we would use, to communicate our concerns. Jignesh suggested that we should get the PO to review and decide upon each issue in turn (an excellent idea which is usually highly worthwhile, particularly for contentious issues, and if your team includes bug triaging within the “Done criteria” for their user stories). We also asked everyone to put themselves in the product owner’s shoes: what matters to them? What are they trying to achieve? Trisha, Jignesh and Jolanda all perceptively identified: releasing software, to deadlines, influenced by the information given to them by their teams. As we were soon to see, that’s almost exactly what was going on inside the head of today’s product owner!

And so to Neil’s acting debut. Neil’s product owner persona (who we’d already outlined as overworked and stretched too thin during earlier chat) came blustering in, late and distracted, with only limited recollection of what was being reviewed. Sometimes this was less than subtle…

[21/09/2014 16:47:59] Product Owner: SSSSH DAN I'M TALKING TO JOLANDA!
[21/09/2014 16:48:04] Product Owner: WAIT A MINUTE!

It was the group’s job to provide clarity and conviction in what they had found, which they did admirably, rallying to make the feature’s shortcomings known to the product owner, who (as we later found out) had other external factors which were driving him to release the feature regardless. The team did well to swerve around the various traps that I placed; questions/statements such as “Is it ready to release yet?”, “Does it work?”, “Nobody is going to do that” and “It’s only one button” were skilfully responded-to; check out the Session Transcript (within the section titled “TALKING TO THE PRODUCT OWNER”) to see just how well the group did here.

At the end of 20 minutes, the product owner (who had to dash to another meeting!) had been left in no doubt that the testers felt that the feature was lacking in key areas, particularly when compared to similar products, and with numerous bugs discovered in a relatively short timescale. The PO, desperate for a next-day release, was convinced to call a triage meeting with team leaders, to discuss how they could feasibly achieve that goal. It was a definite win for our team.


With the product owner gone, the participants debriefed on how they felt their discussions had gone, and what they could conclude were the product owner’s motivations. Unsurprisingly, everybody noticed that the PO was stressed and concerned with slipping timescales. Aleksander commented that it was harder to interact with a fictional persona, because normal real-world interactions are guided by our previous experiences with that person, and knowing how much they were willing to give and take. Jolanda made reference to de Bono’s “Six Thinking Hats” for considering how the deliverer and recipient of a message are interacting with each other.

Neil revealed that the product owner had a specific reason why a release tomorrow was so critical; if pushed, the PO would have revealed that (in our fictional universe) the product was going head-to-head with rivals in an online awards ceremony, and that the judges were specifically looking for tables support. Furthermore, the product owner’s annual bonus had a multiplier which activated if Evernote took home the award! Although this was rather exaggerated (and it’s unlikely that the PO would’ve ever revealed that last piece of information), it was a reminder that there are often other factors at-play when decisions are being made, and that if you can persuade people to reveal these, it can be easier to achieve compromise and understanding.

As a footnote, the folks at Evernote are well aware of the issues within this functionality. On their help forums, the topic “Support for Tables in Evernote Web” contains users reporting similar issues immediately following the implementation of the feature, and the recently-released Evernote 5.6 Desktop Beta for Mac contains revised table support which addresses most of these issues. During our debrief, Amy suggested that we could revisit this feature in a later session once the new version goes live; I think that’s a fantastic idea!

Further Reading:

  • Session Transcript
  • Dan Ashby’s WTEU-49 Mind Map
  • Satir Interaction Model  – We didn’t get time to discuss this, but Virginia Satir’s model of how our brains receive and process communications is invaluable in understanding the various ingredients of an interaction.
  • Simon Knight: My NVC Journey – Article which discusses the Satir model and various other aspects of Non-Violent Communication (NVC) which can aid our workplace interactions.
  • We’d love to add your reports/links here. Please ping us on Twitter, Skype or email if you’d like to share!

About the Author

Neil is a tester from the United Kingdom who has been testing desktop, mobile and web applications for the past ten years, working in a range of agile roles for organisations as varied as Oracle and Last.fm. In his spare time, he participates in freelance and beta testing projects, as a way of learning and developing new approaches to testing.