Saturday, September 06, 2008

a test technique

Very recently I was part of a bug hunt: an effort to find as many defects as possible in a UI in a short period of time.

I am not an outstanding UI tester. I have worked with testers whose eye for detail, line, color, consistency, work flow, etc. etc. make them really outstanding testers to have examining the front end. But I've spent most of my career on green-screen apps, test automation, environment maintenance, stuff like that. I lack the really expert artist's eye that great UI testers have.

That said, I was on a roll on this particular bug hunt, finding a lot of nice, sophisticated bugs. It was classic ET: decide a test approach, determine your next test based on the results of your last test; and when you stop finding bugs, change your test approach.

I'd followed all the logical paths I could think of; I'd tried really big inputs and really small inputs; I'd tried permissions errors, navigation errors, and checking error messages themselves. I tried some crazy stuff. I started to run out of reasonable test approaches. Toward the end, when I was getting tired and finding fewer and fewer bugs, I hit on a neat test technique that found a really cool bug:

Here's what I did: I sort of unfocused my eyes, and addressed the UI spacially instead of logically. Moving from top left to bottom right, I manipulated everything I could find to manipulate. Then I went across the bottom; across the top; up and down in lanes; just clicking and typing and choosing whatever was located next to the last thing I manipulated.

I found this great bug this way: there was a very small link in the footer of a particular area on a page that had exactly the same label as a much larger link elsewhere on the page. Even though we had four expert testers closely examining the application, no one had discovered that the little link in the footer was wired up completely differently (and totally wrong) than the big link with exactly the same label. Until I unfocused my eyes and considered everything on the page as equal and logically unrelated, except by space. Because I approached the UI spacially instead of logically, I found the little piece that was wired up completely wrong.

2 comments:

Michael said...

Yes; this is a great example of a defocusing heuristic. We've been talking about this in the Rapid Software Testing course for a couple of years now. It's nice to have a blog post like this to point to as an example. Note that Blink tests are also examples of a defocusing heuristic.

TestWithUs said...
This comment has been removed by a blog administrator.