WTEU-61 – Designing a Data Set for Testing

WTEU-61 – Designing a Data Set for Testing

Date: Sunday 20th September 2015
Time: 3.30pm – 5.30pm BST
Facilitators: Amy Phillips
Attendees: Ajay Balamurugadas, Aleksandar Simic, Christian Legget, Kapil Saxena, Kiran Alisam, Loren Dunlop, Neil Taylor, Raghu, Sathish, Selvanathan Kapilan, Trisha agarwal, Оксана

This month’s session was all about test data. Our opening discussion focused on what test data is. There were lots of great answers, including, ‘verifiable’, ‘Appropriate, Affordable’, and ‘unique’.

The importance of test purpose was highlighted. With Neil pointing out the importance of understanding your users when designing test data.

We spent some time testing, and trying to design test data for three applications:
Application 1 – https://www.mindmup.com/
Application 2 – http://www.gamefools.com/onlinegames/free/zorktrilogy.html
Application 3 – http://hackyourselffirst.troyhunt.com/

Not all of these applications have a nice web interface to aid test data input. Zork in particular needed test exploration (ok, game playing!) to understand the test data needed.

Neil Taylor made a great point about test data being “designed for the purpose of learning about system.
they can also be designed to set the system into a state that will allowed us to performed specific test”. This seemed particulary relevant to systems like Zork’.

Throughout the session several tools for generating test data were shared (all linked in the ‘Tools’ section below).

Kapil shared a nice bit of wisdom on generating test data:
“I have an important observation about test data that I mentioned earlier; it has the power to kill your testing (sometimes). Means a test with same data over a period of time is of no/less value; hence we must update the test for data. But then it incurs cost. so what’s the solution? Well, for all those who use scripted testing in some prog language, I figured out a way. try to generate random data (within permissible limits) inside your scripts. This means every time a test case is run it will generate unique data and your tests will have an edge.”

We concluded that you will need at least some system knowledge to design a good data set. Until you have that knowledge exploratory testing and test oracles can help provide test data.

Understanding the purpose of your test will help decide which data is relevant.

You can read the full session transcript here.

Follow us on Twitter for more session updates.

Tools
http://www.satisfice.com/tools.shtml
http://www.generatedata.com/
Many more test data generators are listed at http://c2.com/cgi/wiki?TestDataGenerator
https://github.com/minimaxir/big-list-of-naughty-strings

Posts shared during session:
https://thetestdoctor.wordpress.com/2013/11/19/gaming-playing-to-be-a-better-tester/

About the Author

Amy has spent the last ten years testing in a variety of development environments for companies including Royal Mail, The Guardian, and Yahoo! She is currently test lead at Songkick and enjoying the challenges of mobile and web testing. You can find Amy on Twitter (@itjustbroke) or via testingthemind.wordpress.com