Skip to main content

developer-testers and tester-developers

After hearing about a number of system-functional-testing adventures at AWTA, Carl Erickson proposed that there are "developer-testers" and that there are also "tester-developers". In context, the difference was clear, and it's worth talking about the two approaches. (Assuming that I did in fact understand what Carl meant.)

Developer-testers are those developers who have become test-infected. They discovered the power of TDD and BDD, and extended that enthusiasm into the wider arena of system or functional tests. There is a lot of this sort of work going on in the Ruby world and in the agile world. For examples, look at some of Carl's publications on his site at atomicobject.com, or talk to anyone in the Chicago Ruby Brigade about Marcel Molina's presentation last winter on functional tests in Rails and 37signals. (Follow the link, the number of examples is gigantic!) The unit-test mentality of rigor and repeatability and close control survives in these developer-tester system tests.

This is a good thing.
However...

Tester-developers are those testers who have discovered the power of programming to extend and augment their ability to test systems. Sometimes they have been production programmers, but even those who have not been production-code developers have almost always at least read some production code, and have a good grasp of how software is designed.

Tester-developers frequently use interpreted languages instead of compiled languages, for the sake of power. They come up with crazy ideas like finding, creating and using random data. They look at complex or "impossible" simulation problems and create the simulation in 100 lines of Ruby or Perl or Python. They turn problems inside-out and upside-down.

Tester-developers frequently make mistakes, but they attempt so many things that mostly you don't notice.

Tester-developers are often called either "test automators" or "toolsmiths". Some of us have started to suspect that there are situations where these terms are inadequate or misleading.

Tester-developers frequently (as in my own case) excel at craft but lack discipline.

In particular, there comes a time in the toolsmith's career when he needs an interface for testability. Or he needs looser coupling of the code to make his own work easier. Or he needs information about a private API he would like to use. Sometimes, if the toolsmith wants these things, he is the only one in a position to make them happen.

At that time, the tester-developer must be in a position to negotiate not in his own language of crazy ideas and attempts that may fail, but in the developer-tester's language of well-designed, carefully-executed implementation.

Likewise, the developer-tester will have come to realize that regardless of how disciplined his code is, there are probably situations in the world that will cause Bad Things to happen in his system. Often, he suspects that such things are possible, but doesn't have the perspective to recognize the exact problem.

At some point a developer-tester will probably find himself looking for someone to run his code and suggest reasonable ways in which it might fail. Of course, this is what all good testers do, but tester-developers have particular skill at this level, because they understand, at least in a general sense, how the code was written. Here are some reasonable criticisms that a tester-developer might make to a developer-tester:

That input should be a Struct, because if it's an ArrayOfString, there is a strong chance that the user will get the arguments in the wrong order.

This stored procedure contains too many functions. Either move some of the business logic back into the code, or break the procedure into smaller parts, because maintaining this in the future will become difficult. Also, I would like to invoke parts of this function for test purposes without having to exercise the entire set.

Consider adding logging to this aspect of the system, because critical information is being handled here, and if something goes wrong, having that information will be important.

"Developer-testers" and "tester-developers": The more I look at it, the more I like it.