Skip to main content

tests to prevent bugs: coral snakes and king snakes



I was presented with a PNG screen capture of an application some time ago and asked to brainstorm test ideas based on the static representation of a limited section of the application UI.

I think I acquitted myself reasonably well at the time, but I've been thinking about the situation since then, because there is an interesting aspect to this particular application: if this software has defects, the repercussions of such defects could have a drastic effect on the physical health of users.

Assume that this application exists in order to tell the difference between snakes, so as to warn people if they are dealing with one that is poisonous or not. This is a very far-fetched analogy, but it serves to make my point. Right now, the development is in early stages, and the software is able to tell the difference between rattlesnakes, cobras, and coral snakes. Tests for those are easy: rattlesnakes have rattles, cobras have hoods, coral snakes are red, yellow, and black.

But someday we're going to want to add more snakes, like a scarlet king snake. Before we go on, take a look.



I don't have any requirement right now to add a scarlet king snake to the system, nor will I know any time soon if there ever will be such a requirement. However, the risk of confusing the two snakes is so incredibly high that I feel compelled to mitigate that risk with a test.

The appropriate test, of course, is to add a record for the scarlet king snake anyway, and to make sure regardless of the state of development of the app that the app never, ever confuses the coral snake record with the scarlet king snake record.

But the snakes are incredibly similar. How to design the test? We have to do enough research to find the 100% distinguishing feature:
Red next to Black - is safe for Jack;
Red next to Yellow - will kill any fellow.
I put a test into my regression suite saying that regardless of any other criteria (which could change anytime: my app is under development) if red-is-next-to-yellow: always choose "coral snake"; if red-is-next-to-black: always choose "scarlet king snake".

I'd probably want to add a couple of other bellwether tests as well, for instance, to distinguish between cottonmouth water moccasins and other water snakes.

If these tests ever break, then my users' health is at risk, and my company is at risk for serious litigation.

Comments

Popular posts from this blog

Reviewing "Context Driven Approach to Automation in Testing"

I recently had occasion to read the "Context Driven Approach to Automation in Testing". As a professional software tester with extensive experience in test automation at the user interface (both UI and API) for the last decade or more for organizations such as Thoughtworks, Wikipedia, Salesforce, and others, I found it a nostalgic mixture of FUD (Fear, Uncertainty, Doubt), propaganda, ignorance and obfuscation. 

It was weirdly nostalgic for me: take away the obfuscatory modern propaganda terminology and it could be an artifact directly out of the test automation landscape circa 1998 when vendors, in the absence of any competition, foisted broken tools like WinRunner and SilkTest on gullible customers, when Open Source was exotic, when the World Wide Web was novel. Times have changed since 1998, but the CDT approach to test automation has not changed with it. I'd like to point out the deficiencies in this document as a warning to people who might be tempted to take it se…

Watir is What You Use Instead When Local Conditions Make Automated Browser Testing Otherwise Difficult.

I spent last weekend in Toronto talking to Titus Fortner, Jeff "Cheezy" Morgan, Bret Pettichord, and a number of other experts involved with the Watir project. There are a few things you should know:

The primary audience and target user group for Watir is people who use programming languages other than Ruby, and also people who do little or no programming at all. Let's say that again:

The most important audience for Watir is not Ruby programmers 
Let's talk about "local conditions":

it may be that the language in which you work does not support Selenium
I have been involved with Watir since the very beginning, but I started using modern Watir with the Wikimedia Foundation to test Wikipedia software. The main language of Wikipedia is PHP, in which Selenium is not fully supported, and in which automated testing in general is difficult. Watir/Ruby was a great choice to do browser testing.  At the time we started the project, there were no selenium bindings for …

Open letter to the Association for Software Testing

To the Association for Software Testing:

Considering the discussion in the software testing community with regard to my blog post "Test is a Ghetto", I ask the Board of the AST  to release a statement regarding the relationship of the AST with Keith Klain and Per Scholas, particularly in regard to the lawsuit for fraud filed by Doran Jones (PDF download link) .

The AST has a Code of Ethics  and I also ask the AST Board to release a public statement on whether the AST would consider creating an Ethics Committee similar to, or as a part of the recently created Committee on Standards and Professional Practices.

The yearly election for the Board of the AST happens in just a few weeks, and I hope that the candidates for the Board and the voting members of the Association for Software Testing will consider these requests with the gravity they deserve.