WTA-68: Mobile Website Testing

WTA-68:  Mobile Website Testing

WTA-68:  Mobile Website Testing

Date:  1/9/16

Time:  1pm EST/10am PST – 3pm EST/12noon PST (check your own time zone for start and end times)

Facilitator:  Jean Ann Harrison

Attendees: Akanksha Talwar, Amir Chan, Bhanumathi Mahesh, Brendan Connolly, Dawn Code, P Mc, Satyajit Malugu, Shirin Panna, Sree Bandla, Swapna Koppu, Trisha Chetani, Agarwal, Vishali Manivannan, Michael Larsen and Jean Ann Harrison as WTA

As websites are being overhauled for viewing on mobile devices, are we checking a normally viewed website viewed on a device?  What should you consider important to tests?  Can we automate all these tests?  Yes, and we can talk about what tests can and should be automated and those that can be done faster manually.

For this weekend testing session, we’re going to focus on the type of user who uses the Weather Channel website on  mobile devices.

This means we’ll be using the browser on the device and not downloading any apps.

1.  What is the first thing any user expects to be able to view?

2.  What kind of view is the user expecting?   Fill the screen?  Scrolling?

3.  Is the font readable? Or does the user need to do something to make the website readable?

4.  Can the user view the videos?  Can the user pause the video and rotate the device and continue viewing the video?

5.  Is the website easy to access the various pages?  Think about the size of fingers or is a stylus needed?

6.  Is it easy to find the latest news?  How to navigate to the news page?

7.  How is the response time?  How is the painting of different pages?

8.  Is there a balance of content and ads?


Experience Report:

The general consensus is that a site that is rendered in a mobile browser needs to be readable and relatively quick to navigate. When sites have mobile specific views, that is often attained, but when sites don’t, they can be cumbersome to navigate and to read. In the Weather Channel example, I tested this in Mobile Safari and the net result was the site displaying in its entirety on my small screen. The need to pinch and swipe to view items as well as select them is often a challenging experience. What’s more, the connection can often go down while moving (an experience I deal with often as I am on trains going from home to work and back).

CNN.com has a more mobile responsive site, and has made design tweaks to accommodate mobile devices, but even there, the layout often feels inconsistent. CNN has a number of affiliate sites, and each of the affiliate sites has a different look and feel. The time it takes to navigate, get back to the home site, or divert to getting back and going to the home page all take significant time, which in turn  diminishes the usefulness of the site(s).

Several features of the mobile site require external apps to function (the Live TV option needs a specific app, which causes the user to need to dedicate space and process time). Additionally, the layout and the time to get everything rendered with graphics and images can vary considerably depending on what has been loaded.

A key takeaway is the fact that Usability and User Experience factor highly in Mobile site testing, and what works well for some might be intolerable for others. We need to be cautious when we use ourselves as the arbiters of what is usable. Persona based testing can help, but even that can have its limits, since we cary our own biases and preferences even when we try to represent someone else.

Full chat transcript can be seen here.

About the Author

I’m a software tester working with Socialtext in Palo Alto, CA. I have worked in a number of different fields and in a number of different capacities. I started my testing career in March of 1991. I am co-founder and primary facilitator for Weekend Testing Americas. I am a black-belt in the Miagi-do School of Software Testing, a member and Teacher in the Association for Software Testing, and the producer of Software Test Professionals' "This Week in Software Testing" podcast (now on hiatus).