Date: 27th February 2010
Time: 15:30 – 18:00 GMT
Product: Brain Workshop, a “Dual N-Back” brain training game intended to improve working memory and fluid intelligence.
Mission: Explore Brain Workshop using the FCC CUTS VIDS touring heuristic and report on your findings. The goal is to practise using tours to learn interesting quality related information about a new application quickly, and to produce a short report on what you learned.
Testers: Anne-Marie Charrett, Tony Bruce, Markus Deibel, Jaswinder Kaur Nagi, Anna Baik, Amit Kulkarni joined for the discussion.
Anna Baik facilitated the discussion afterwards, asking each tester about their experience.
Testers this week chose to split the FCC CUTS VIDS heuristic, and picked a tour or two each to perform.
Anne-Marie Charrett started the discussion afterwards. She had chosen to perform the structure tour. She had known about tours before, but never used them so this was fun. She started out by checking the readme file to get some understanding of the language used, and also had a look at some of the bugs fixed, discovering that there had been lots of issues with sound and memory (triggering some discussion on ways to test for how lack of memory affects an app). She spent most of the time looking at the actual code and the structure of how the app is created. The structure tour was good for generating test possibilities – instead of looking at it from a user perspective, Anne-Marie understood more on how the application was formed, which would enable her to test at a much lower level than she’d previously intended. For instance, using a blank config.ini file on startup. Especially if working in a team situation, where most testers might go for the user interface approach, this testing would provide a different angle.
Tony Bruce had chosen the feature tour, but by the time he had the app launched and usable he had very little time left. He had audio problems, although he wasn’t sure if the app itself had caused them. He’d initially started testing on a Ubuntu netbook, but as it wasn’t possible to size the app, the options were cut off the bottom of the screen. Moving onto a WinXP machine next, he found that the zip version and the installed version crashed when he started them – though after a reboot, he was able to test. He found the app annoying to use – there were lots of options but using it was actually annoying.
Jaswinder Kaur Nagi was next. She’d also had time constraints, but in her case caused by a flaky network connection which meant she wasn’t able to access the internet for parts of the session. She had chosen the user tour, and chose a novice with less computer exposure, and a small child as the user personas. She observed that the tutorial should have been inbuilt, rather than online as anyone without an internet connection wouldn’t be able to play the game without reading the tutorial. The daily progress graph didn’t show anything, and you have to read the tutorial to see how the graph works. Overall she did like the game, although the interface was not attractive enough. But she’d still try to play it to see if it really helps.
Markus Deibel finished up the discussion. He’d chosen the Claims and the Configuration tours, and presented the information he’d gathered about the application from them. The data was present on a single html page, which helped him to extract the most valuable information. He noted a number of tests that could be performed as a result of this information – for instance, tests on different OSs as the app claims to run on XP, Vista, MacOS and Linux, checking if the runtime changes to configuration were stored in the config.ini file. He’d also found some defects during his searching.
“He’d initially started testing on a Ubuntu netbook, but as it wasn’t possible to size the app, the options were cut off the bottom of the screen. Moving onto a WinXP machine next, he found that the zip version and the installed version crashed when he started them – though after a reboot, he was able to test.”
An important gloss here. One definition of testing is “gathering information with the intention of informing a decision.” Here Tony found that the app didn’t fit the screen, that it didn’t adjust itself to fit on the screen, that he couldn’t adjust it, that two diferent versions crashed on startup, and that there was a workaround. That is significant information that could easily inform several decisions about the product, or so it seems to me.
Testers, an important lesson in self defense: you’re testing as soon as you start thinking about tests (that is, designing them) and acting on those ideas. The actions that you hoped to perform and the ones you were able to perform might not be the same, but the activity is still testing. In fact, the difference between them is a test result too. That’s particularly important when someone insinuates that, since you couldn’t get your “test cases” “done”, you must not have been testing.
—Michael B.
Hi Michael,
Thank you! I hadn’t spotted that, but I fell into a trap there… Given the last few months of “no green ticks appearing next to test scripts, but much investigation and activity”, I really should have seen that one.
Here’s another trap I’ve been falling into when using the application we tested. I’ve found that when playing the Brain Workshop games, quite often there will be no match made for several rounds – so that means I might spend quite a while without having hit a key to indicate a match. I’ve noticed I’m more likely to hit an invalid match after a string of no matches – it’s as if I feel I’m not “doing” anything unless I hit a key. Often I know as I’m hitting it that it’s wrong, there isn’t a match – but I feel my finger twitch on that key anyway, no “work” for a while so must press!
Perhaps I can see if Brain Workshop will train me out of feeling that seeing something, and noting it as “no action needed right now”, is not also valid work. Rationally, I know it is also work, but I clearly don’t feel it enough yet.
Anna
I added this weeks chat transcript on bug repository. Thanks for the great session.