Skip to main content

looking back, looking forward

Looking Back

The world of software testing today is radically different than it was on this day a decade ago. On New Year's Day 2000 I had been a dedicated software tester for about three years. I was a leader on a team testing an application that provided location information to the dispatchers who handle 911 calls. I was intensely interested in the most progressive thinking about software testing available, because when we released a bug to production, someone could die because of it.

I remained interested in the most progressive thinking about software testing throughout the decade. Looking closely, we owe a vast debt to three people: James Bach, Bret Pettichord, and Brian Marick. If they didn't supply every breakthrough idea in software testing in the last decade, one of them was nearby when it happened.

There was a shot across the bow in 1996 when Bach published "Test Automation Snake Oil". This would be the first of a relentless assault on proprietary test tool vendors and the intellectual bankruptcy of the approach to testing that such vendors were selling.

In 2001 Pettichord published the incendiary "Hey Vendors, Give Us Real Scripting Languages". These articles and others that followed were the beginnings of the big push not only for open source test tools, but also for encouraging testers to increase their skills to make the best use of such tools. The tools themselves were still some years away, but the conversation started here.

Meanwhile, not long before that in 2000, Bach and his brother Jonathan had published "Session Based Test Management" in a magazine called Software Testing and Quality Engineering, known today as Better Software. The ideas fueling Exploratory Testing had been around for some time at this point, but skepticism was widespread that ET was a worthwhile test approach. SBTM was the first treatment of a large-scale measurable ET process. To my mind, this single article turned the tide in making ET a valid and acceptable practice.

Automation and ET are the touchpoints for how we work today. But at this time we had no guidance on when and how to use them. Then 2002 saw the publication of Lessons Learned in Software Testing by Cem Kaner, Bach, and Pettichord. A lot of us had read Dr. Kaner's earlier work, and Beizer, and quite a number of other books on software testing, but we found that those practices mapped poorly to our own experience. While Lessons Learned has gotten a lot of valid criticism over the years, that hodgepodge of techniques and approaches and theoretical frameworks was the first accurate description of what it was like to be a tester on the ground early in the 21st century.

One of the tenets of Lessons Learned is the idea of "no best practices". Shortly after the publication of Lessons Learned, Pettichord began defining the Four Schools of Software Testing, of which the Context-Driven school is fueled by ET and reinforced by Lessons Learned. Context-Driven testing is alive and well around the world today.

At this point we understood ET fairly well and the Context-Driven school of software testing was far and away the best match for our actual day-to-day working experience. But the lack of open-source test tools was galling. Dedicated testers were falling farther and farther behind developers in terms of the power of their tools.

In 2003 Pettichord and Brian Marick began teaching their 'Scripting for Testers' course. Marick had written a small time-clock application, and they used a library called "iec" created by Chris Morris to drive Internet Explorer and automate some tests of the application.

I downloaded SfT a number of times but could never get it to work properly. I begged several managers to let me attend SfT but I never got to go in those early days. Later I would teach SfT myself on three different occasions, at the STARWest conference, the STAREast conference, and at Agile2006.

In 2004 a programmer named Paul Rogers took the SfT code base and rewrote it from scratch. Pettichord and Rogers released the rewritten SfT as Web Application Testing in Ruby aka Watir. I was the first user of Watir. Watir was the first open source web-testing tool robust enough for actual production work. I was responsible for many of the early enhancement requests, particularly for frames and iframes. To my knowledge, Watir was the first tool in history with robust support for frames.

In the meantime, Marick had been one of the authors of the Agile Manifesto, a document that would eventually change our entire approach to software development.

Also, for all of the decade so far, Marick had been the technical editor at STQE/Better Software magazine. We would devour every issue because Marick was publishing more of the most innovative, inspiring, and mind-bending stuff about software development than any other source. Marick would often solicit material for the magazine via his blog. He published my first article in Better Software in 2004 when I answered one.

Pettichord had been hosting a small peer conference called the Austin Workshop on Test Automation for some time, and 2005 was the first to feature Watir prominently. I attended that AWTA, where I met a number of people with whom I would later work on various other projects. I would also attend AWTA in 2007, one of the formative experiences of my last several years.

Pettichord went to work for Thoughtworks at about this time, where I would follow him around 2006. There we met Jason Huggins, who was developing an in-house time-and-attendance application that had an innovative test harness. Pettichord would be instrumental in getting Huggins' test harness released by Thoughtworks as open source. The project was called Selenium.

Marick left the editorship of Better Software, Pettichord now works in private industry, and Bach has turned a lot of his effort to his memoir in recent times. But without the work of these three in this last decade, the world of software testing would be a very different place.

Looking Forward

I can't imagine what the next decade will bring for software testing. But I'll make a prediction: women will be the biggest innovators in the field.

Just as Bach's "Snake Oil" article was an indicator of things to come, I think Lisa Crispin's and Janet Gregory's
Agile Testing book points the way to our future. Now in multiple printings and multiple languages, the book is a comprehensive statement about the state of the practice today, and has some hints about where we'll be going.

In recent months I've been part of a community of software testers who write publicly on the subject of software testing. A remarkable number of them are women, and a remarkable number are doing really original, ground-breaking work. Here are some women that will shape the next decade of software testing:

Marlena Compton is working with the visualization of large sets of data and with social networks.

Catherine Powell
should publish her first piece, on software architecture, very soon.

Lanette Creamer has created a number of innovative approaches to software testing. She'll be making the rounds of conferences this year.

Elisabeth Hendrickson is an international superstar. She has just opened her own 'agile practice space' called Agilistry.

Marisa Seal and I have taken over Gojko Adzic's WebTest project that provides a portable, platform-independent bridge between Selenium and Fitnesse. We intend to rename the project Selenesse. Marisa recently became a committer on the Fitnesse project.


Rahul said…
This comment has been removed by a blog administrator.
Chris, great post. We really should record a podcast on early days of Watir. :)
Tim said…
Really interesting stuff. It's great to hear about the 'forefathers' of open source testing. Thanks for the background info.
Bret said…
Janet Gregory, co-author of Agile Testing, is another woman to add to your list.
GeePawHill said…
Chris, thanks for a first-rate summary, both forward and back. I work with testers all the time, but I don't follow the literature (and the action) the way a pro does. You should do this twice a year in a newsletter: "The Testing World Summarized For Developers". :)
QualityPoint said…
This comment has been removed by a blog administrator.
marekj said…
+7 on that "The Testing World Summarized For Developers" suggestion from GeePawHill.
dorla said…
Really great ideas. I like every example. Just might have to try these...
more template
Lisa said…
I'm thrilled to be mentioned in this same post with so many of the testing leaders I greatly admire! This is such an interesting look back - so much has happened in 10 years!
Harendra verma said…
This comment has been removed by a blog administrator.
With all respects to you: This looks like the Looking Back, Looking Forward of the West. Curious to know if you considered East as well?
Chris McMahon said…
Hi Pradeep, yes, this was a personal look at the last decade, just my own view of my own experience. It's a narrow view even of what's happened in the West, and of what's happened in the East I am mostly ignorant.
Harendra verma said…
This comment has been removed by a blog administrator.
hs said…
This comment has been removed by a blog administrator.
ggbbgg127 said…
This comment has been removed by a blog administrator.
TestWithUs said…
This comment has been removed by a blog administrator.

Popular posts from this blog

Reviewing "Context Driven Approach to Automation in Testing"

I recently had occasion to read the "Context Driven Approach to Automation in Testing". As a professional software tester with extensive experience in test automation at the user interface (both UI and API) for the last decade or more for organizations such as Thoughtworks, Wikipedia, Salesforce, and others, I found it a nostalgic mixture of FUD (Fear, Uncertainty, Doubt), propaganda, ignorance and obfuscation. 

It was weirdly nostalgic for me: take away the obfuscatory modern propaganda terminology and it could be an artifact directly out of the test automation landscape circa 1998 when vendors, in the absence of any competition, foisted broken tools like WinRunner and SilkTest on gullible customers, when Open Source was exotic, when the World Wide Web was novel. Times have changed since 1998, but the CDT approach to test automation has not changed with it. I'd like to point out the deficiencies in this document as a warning to people who might be tempted to take it se…

Watir is What You Use Instead When Local Conditions Make Automated Browser Testing Otherwise Difficult.

I spent last weekend in Toronto talking to Titus Fortner, Jeff "Cheezy" Morgan, Bret Pettichord, and a number of other experts involved with the Watir project. There are a few things you should know:

The primary audience and target user group for Watir is people who use programming languages other than Ruby, and also people who do little or no programming at all. Let's say that again:

The most important audience for Watir is not Ruby programmers 
Let's talk about "local conditions":

it may be that the language in which you work does not support Selenium
I have been involved with Watir since the very beginning, but I started using modern Watir with the Wikimedia Foundation to test Wikipedia software. The main language of Wikipedia is PHP, in which Selenium is not fully supported, and in which automated testing in general is difficult. Watir/Ruby was a great choice to do browser testing.  At the time we started the project, there were no selenium bindings for …

Open letter to the Association for Software Testing

To the Association for Software Testing:

Considering the discussion in the software testing community with regard to my blog post "Test is a Ghetto", I ask the Board of the AST  to release a statement regarding the relationship of the AST with Keith Klain and Per Scholas, particularly in regard to the lawsuit for fraud filed by Doran Jones (PDF download link) .

The AST has a Code of Ethics  and I also ask the AST Board to release a public statement on whether the AST would consider creating an Ethics Committee similar to, or as a part of the recently created Committee on Standards and Professional Practices.

The yearly election for the Board of the AST happens in just a few weeks, and I hope that the candidates for the Board and the voting members of the Association for Software Testing will consider these requests with the gravity they deserve.