Tuesday, January 16, 2007

developer-testers and tester-developers

After hearing about a number of system-functional-testing adventures at AWTA, Carl Erickson proposed that there are "developer-testers" and that there are also "tester-developers". In context, the difference was clear, and it's worth talking about the two approaches. (Assuming that I did in fact understand what Carl meant.)

Developer-testers are those developers who have become test-infected. They discovered the power of TDD and BDD, and extended that enthusiasm into the wider arena of system or functional tests. There is a lot of this sort of work going on in the Ruby world and in the agile world. For examples, look at some of Carl's publications on his site at atomicobject.com, or talk to anyone in the Chicago Ruby Brigade about Marcel Molina's presentation last winter on functional tests in Rails and 37signals. (Follow the link, the number of examples is gigantic!) The unit-test mentality of rigor and repeatability and close control survives in these developer-tester system tests.

This is a good thing.
However...

Tester-developers are those testers who have discovered the power of programming to extend and augment their ability to test systems. Sometimes they have been production programmers, but even those who have not been production-code developers have almost always at least read some production code, and have a good grasp of how software is designed.

Tester-developers frequently use interpreted languages instead of compiled languages, for the sake of power. They come up with crazy ideas like finding, creating and using random data. They look at complex or "impossible" simulation problems and create the simulation in 100 lines of Ruby or Perl or Python. They turn problems inside-out and upside-down.

Tester-developers frequently make mistakes, but they attempt so many things that mostly you don't notice.

Tester-developers are often called either "test automators" or "toolsmiths". Some of us have started to suspect that there are situations where these terms are inadequate or misleading.

Tester-developers frequently (as in my own case) excel at craft but lack discipline.

In particular, there comes a time in the toolsmith's career when he needs an interface for testability. Or he needs looser coupling of the code to make his own work easier. Or he needs information about a private API he would like to use. Sometimes, if the toolsmith wants these things, he is the only one in a position to make them happen.

At that time, the tester-developer must be in a position to negotiate not in his own language of crazy ideas and attempts that may fail, but in the developer-tester's language of well-designed, carefully-executed implementation.

Likewise, the developer-tester will have come to realize that regardless of how disciplined his code is, there are probably situations in the world that will cause Bad Things to happen in his system. Often, he suspects that such things are possible, but doesn't have the perspective to recognize the exact problem.

At some point a developer-tester will probably find himself looking for someone to run his code and suggest reasonable ways in which it might fail. Of course, this is what all good testers do, but tester-developers have particular skill at this level, because they understand, at least in a general sense, how the code was written. Here are some reasonable criticisms that a tester-developer might make to a developer-tester:

That input should be a Struct, because if it's an ArrayOfString, there is a strong chance that the user will get the arguments in the wrong order.

This stored procedure contains too many functions. Either move some of the business logic back into the code, or break the procedure into smaller parts, because maintaining this in the future will become difficult. Also, I would like to invoke parts of this function for test purposes without having to exercise the entire set.

Consider adding logging to this aspect of the system, because critical information is being handled here, and if something goes wrong, having that information will be important.

"Developer-testers" and "tester-developers": The more I look at it, the more I like it.

4 comments:

Dave said...

Nice. The overlap idea is important for us to explore, while at the same time understanding that a supremely test-infected developer differs from a code-infected tester, as far as their passion is concerned. Great developers must learn the ways of great testers in order to think properly when devising and applying tests. Great testers must learn the ways of great developers in order to apply their testing mind-set. One does not replace or supersede the other; but we've sure got a lot to learn from one another!
/Dave

butlimous said...
This comment has been removed by a blog administrator.
独孤客 said...
This comment has been removed by a blog administrator.
DrummerDaveF said...

I've worked with people who fit into both camps. I'd like to see more Developer-Testers around, as they tend to care more about the quality of their code, produce more elegant code and less garbage code.

The Tester-Developer is a rarer breed, and the reason is the complexity of the tools, the proprietary scripting languages, the fragile nature of the scripts, and those combined resulted in the tests having a high cost of ownership (time invested) even if the tool was free. While the intention is good, the tools are letting them down. This time expensive script creation and maintenance can make them targets for PMs keen on reducing time expensive tasks/people.

Tools like LiquidTest that are more robust with a simpler UI that create tests in popular languages like JUnit, Ruby, and C# are helping Tester-Developer and the Developer-Tester join forces.

I know of workplaces where both of these character types are using the one tool to achieve their goals, reducing the time it takes to create the test scripts, and the time needed to maintain them.

While the two camps still learn from each other, they can both put the learnings from the other into use without having to translate the knowledge into their own tool set.