WTEU-54 – Exploratory Testing in an Agile Context

WTEU-54 – Exploratory Testing in an Agile Context

(Image credit: Matt Archer, Why you should resist overlaying traditional testing processes onto a sprint)

Date: Sunday 22nd February 2015
Time: 3.30pm – 5.30pm GMT
Facilitator: Neil Studd
Participants: Matt Archer, Roman Baranov, Dean Barnes, Dan Billing, Kai Bischoff, Christian Field, Claire Goss, Stephen Janaway, Emma Keaveny, Jolanda Klijn-Gooiker, Christian Kram, Christian Legget, Amy Phillips, Irina Săvescu, Aleksandar Simic, Toby Sinclair, Rafał Szypulewski, Akanksha Talwar, Neil Taylor, Amit, Raghu

In this month’s Weekend Testing Europe session, we focused on discussing some of the challenges typically associated with testing in an agile context. We began with a debate on the differences between “Agile” and “agile”, and how you can be one without being the other. Several participants championed the use of lean or kanban processes.

Talking more generally about working in fast-moving iterative development teams, we looked at the challenges that this can produce. Our group put a positive spin on things: Stephen Janaway pointed out that there was “no time to (pointlessly) write test cases”, while Dan Billing mentioned that agile tends to foster teamwork and collaboration – “not much room for a lone wolf”. Emma Keaveny gave an example of continuous integration, whereby brand new builds are available within minutes of developers making a fix; this began to move the conversation onto the focus of today’s topic.

In an agile environment, rapidly-shifting prioritisation (and occasional “dogpiling” sessions) can mean that you’re suddenly asked to test something of which you didn’t have prior knowledge. Although theoretically the content of a sprint shouldn’t change after it’s begun, real life is rarely so straightforward! The group discussed what their response would be if they were told, “Test this now!”

Thankfully, the WTEU-54 group were reluctant to being steamrollered by such a request. We brainstormed some high-level retorts:

  • “What does it do?” (Stephen Janaway)
  • “What’s it supposed to do?” (Christian Kram)
  • “Can you demo it to me?” (Matt Archer)
  • “When does it need to be done?” (Emma Keaveny)
  • “What do you mean by ‘test’?” (Neil Studd)
  • “Do I need to be the one to test it?” (Toby Sinclair)
  • “Why now? Why is this more important than what I’m already doing?” (Neil Studd)
  • “Let me play / learn about it” (Kai Bischoff)

Kai touched on two important points. Getting hands-on with a product can be a great way of discovering what it does, and although some people questioned the use of the word “play”, experimentation is a core part of learning. We also discussed some other ways in which we could learn about the software, including pairing with developers, or talking with stakeholders.

The centrepiece of the session was a live 20-minute exploratory testing challenge, which was hosted by renowned agile testing practitioner Matt Archer. At its core, Matt’s exercise appeared to be simply a fun bug-hunting task, but there was much more to it than that!

We won’t spoil the exercise for you, in case you get a chance to participate in one of Matt’s workshop classes. (Though you can always read the session transcript if you want to find out!) Suffice to say, the exercise secretly included some focus-tracking technology, which allowed Matt to produce a graph showing where each participant spent their time during the exercise: when they were reading requirements (red), when they were interacting with the application-under-test (green), and when they were recording notes (blue). Neil Studd’s histogram is below; you can view all of the other participants’ graphs in the “Further Reading” section.

Before everyone saw their results, we discussed whether people had deliberately chosen to take a particular approach in their 20 minutes of testing. Many people gave their responses in straightforward terms, e.g. “I read the requirements, then made some notes, then I did some testing, then I finished off my notes” – but when Matt produced each person’s graph, it became obvious that our working patterns were rarely that simple!

As Matt told the group, exploratory testing is often summarised as “simultaneous test design, execution and reporting”, but how do you bring that to life and show what that means in the real world? Through this exercise, we got to see a little inside our own individual working patterns, learning how each person’s exploratory cycle works.

Thanks to Matt for providing the exercise for this session, and compiling the graphs for each attendee – we’ve now each got our own unique exploratory testing DNA strand!

The transcript below is well worth a read, as there was a lot of interesting domain-specific discussion regarding this exercise.

Further Reading:

About the Author

Neil is a tester from the United Kingdom who has been testing desktop, mobile and web applications for the past ten years, working in a range of agile roles for organisations as varied as Oracle and Last.fm. In his spare time, he participates in freelance and beta testing projects, as a way of learning and developing new approaches to testing.