Friday, October 29, 2010

ignoring certification; with numbers

All of the questions about tester certification were answered many years ago. They exist and they cannot be made to unexist. The only remaining question on the subject is: how many tester certifications can be sold? And the answer to that question doesn't matter to anybody except the people selling the certifications.

A while ago on the writing-about-testing mail list we did a little exercise to come up with some back-of-the-napkin estimates about the number of software testers in the world. We used US Department of Labor data and also some other public information about software employment worldwide. We also had access to some privileged information about magazine subscriptions. In addition, a number of us have done serious work in social networking, and we have some analytical tools from that work to help estimate. Using all of that, we came up with a pretty consistent estimate that there are probably around 300,000 software testers in the US, and maybe 3 million in the whole world.

That is a pretty small market in which to sell tester certifications.

Elisabeth Hendrickson recently did a fascinating analysis of QA/testing job ads. According to her data, it is a good bet that 80% of the people doing modern software testing work in the US have programming skills of one sort or another.

Jason Huggins of Sauce Labs has been tracking job ads that mention browser automation tools. Jason notes a remarkable recent increase in the demand for Selenium skill. You can see the trend for some popular automation tools at The QTP vs. Selenium trend in job ads is fascinating, but looking closely, this graph indicates a general across-the-board increase in demand for technical skills in traditional UI-based software testing.

Finally, sorry I don't have a link handy, but I have seen a number of reports of a radical increase in the rate of adoption of agile practices among US companies of every size and description. And the agile whole-team approach to software development makes dedicated, siloed traditional V&V test departments irrelevant.

The existing tester certifications simply do not apply to this sort of work. Certification is becoming more and more useless to US testers, and to their employers as well.

I feel like I am pretty plugged in to the world-wide tester community and the world-wide agile community, and anecdotal evidence suggests that indeed, the market for tester certification in the US is very small. Again, this is anecdotal evidence, but the hot spots for certification seem to be the UK and Australia/New Zealand, possibly areas of Southeast Asia, possibly areas of Eastern Europe. Once more with the anecdotal evidence, but I would suggest that in political climates that favor a high degree of regulation of business practices, certification will be more popular.

So if we eliminate from our worldwide tester population of 3 million the majority of US testers and a significant fraction of the rest of the world as potential buyers of a tester certification, that leaves a pretty tiny market for tester certification.

I think we can say with some confidence that professional tester certification can safely be ignored by the vast majority of software testers. That said, if you are required to get a certification, or if you just want to get a certification, go ahead and do it. It won't hurt you, and at the very least, you'll learn how software was tested in 1996.

I think we can also safely say that any supposed controversy surrounding tester certification is overblown and can also be ignored.

Which suggests one more interesting question: if the supposed controversy over certification really is as trivial as these statistics indicate, then why does so much of the testosphere spend so much time agonizing over it?

I have a cynical answer to that, but I'll keep it to myself.

Update: made the links nice

Monday, October 25, 2010

Call for Participation: Second Writing About Testing peer conference

The Second Writing About Testing Conference: Frontiers for Software Testing

I am pleased to announce the call for papers for the Second Writing About Testing Conference to be held May 13 and 14 2011 in Durango Colorado.

For more information about the original conference and the Writing About Testing mail list please see:

Writing About Testing is a peer conference for those interested in influencing the public discourse on software testing and software development by writing and speaking on those subjects. The discussion revolves around blogging, giving presentations at conferences and user groups, and writing for established media outlets, both for pay and for other reasons. There will very likely be representatives from established media outlets attending. Having software writers and publishers talking to each other face to face is a unique aspect of WAT.

For the first WAT conference we asked only that participants be interested and have had already written about software. The second WAT will be a little different.

For the second WAT we ask that applicants propose a talk (informal is fine, no slides required) of 30-45 minutes about some subject critical to their work as a tester or developer, or designer or business analyst, but which is not generally recognized as being part of such work. A list of suggestions for such talks is below. The purpose of this is to expand the practice of software beyond the current artificial boundaries of recognized software activity.

To that end, presentations on these subjects are not welcome, unless the presentation has obviously unusual aspects:

  • test heuristics/mnemonics
  • exploratory testing
  • classic test automation (record-and-play, automation pyramid, etc. Information on unusual approaches to test automation is welcome.)
  • Scrum/Lean/Context-Driven/whatever. In general, anything involving capital letters is not welcome.
  • certification (for or against)

In a nutshell: don't bore the other attendees with stuff that has been discussed to death for years. This conference is to discuss frontiers.

Given that the presentations will be on unusual subjects, there will be a minimal requirement for having published previous work. Any publicly available source of writing, for instance a blog, would qualify an attendee. Applicants with no publicly available writing at all will not be considered. Attendees are encouraged to write with ambition and daring after the conference ends.

Attendance will be limited to about fifteen people. There will be a nominal fee of $50.00 per person to help cover room rental, and lunch will be provided. A discounted rate at a convenient bed and breakfast hotel is available.

Durango has a lot to offer visitors. Conference attendees may wish to arrive early or stay late to take advantage of the nearby attractions: beautiful mountains to the north and east, desert sandstone canyons to the west and south. The steam train from Durango to Silverton and back is a fantastic experience, as is soaking in the hot springs nearby. Within a short drive are Mesa Verde National Park, Canyons of the Ancients National Monument, Monument Valley and the Navajo Reservation, Great Sand Dunes National Park, etc. Local opportunities for hiking, biking, and boating abound.

There are direct flights to Durango from Denver and from Phoenix. Many attendees will likely come from the Denver area, so carpooling from there may be possible.

To submit a proposal, either send a message to me at christopher dot mcmahon on gmail, or join the writing-about-testing mail list at and submit your proposal there.

The deadline for submissions is Jan 1.
Invitations to the conference to be sent Feb 1.
The conference itself will be May 13/14.

Here are some possible frontier subjects for presentations:

System Administration

There is a surge of interest in recent times in a concept called "DevOps". DevOps proposes an alliance among software developers and system administrators in order to create the best possible production environment experiences. Testers need to be a part of that conversation.

Data Visualization

Not only our applications, but the whole world around us generates incomprehensible amounts of data, and the only way to make sense of it all is to render that data in a visual or tactile fashion. Testers need to understand these technologies in the service of their work.


Good test automation today happens at every level. A single framework may exercise the user interface, call REST or SOAP APIs, and reach into a database, all in the course of a single test suite. Myriad tools for such testing exist, and knowing how to get such tools to talk to each other for a particular purpose is becoming a critical skill for testers.

User Experience

Great strides have happened in user experience work in the last few years, and there are exciting advances on the horizon. Testers have largely ignored the conversation happening among user experience experts.

Web Design

New javascript libraries like jquery are making things possible in browsers that were unimaginable just a few years ago. Flex, Flash, and Adobe Air are making huge inroads into application UI design. HTML5 is on the horizon and looming. Testers need to know what is happening in this arena, and largely do not.

Web Services (REST/SOAP)

Twitter, Facebook, and the bleeding edge of web applications are no longer about the UI. Today it is all about the APIs, and the third party applications that use those APIs to bring killer experiences to users. Testers need to know how web services and APIs work.

Environments/VMs/Cloud Computing

Managing test environments has always been challenging. New cloud computing services in some ways make such work more challenging, but the reward is a vastly simplified process for provisioning test environments. This work needs public exposure.

Agile Methods

Agile methods work, but even today, no one knows exactly why. The explanations we have are frequently facile and often abused. Testers could be the ones to provide the well-considered explanations for the effectiveness of agile methods.

Process Work/Quality Assurance

QA has a bad reputation in the testing community that it does not deserve. I have said before on and in Beautiful Testing that QA is not evil, that it is work that still needs doing, and that often testers are in a good position to provide quality assurance. Bring back real discussion about Quality Assurance.

Aesthetics/Artistic Performance

There is a wealth of knowledge available from disciplines within the Liberal Arts that apply directly to software development. Testers can help bring that knowledge over to the world of software development.

Sunday, October 03, 2010

One Year Writing for

I generally do not post links to pay-wall or registration-wall sites, but today I am sincerely happy and proud to publish a link to

For each of the last twelve months, I have written at least two 1000-word articles for SSQ. In the last year, SSQ has published nearly forty individual pieces of mine, a book-size body of work.

I am sure some of those articles are better than others, but I wrote every one to the best of my ability with all sincerity, and I truly believe that every one of those articles contains at least one interesting idea intended to help people working in software testing and software development.

I would particularly like to thank my editors at SSQ, at first Jan Stafford, later Yvette Francino. The SSQ editorial staff is professional and efficient. Both Jan and Yvette have given me an enormous amount of freedom and encouragement over the last year, and it has been a real pleasure working with them both. I especially appreciate their tolerance on the few occasions when I pushed that freedom to the limit.

I would also like to thank the Writing About Testing mail list. A great number of people on that list have been immensely helpful, freely giving comments and constructive criticism, providing new ideas, and just being generally smart and encouraging human beings. In the very near future we will be announcing the CFP for the second Writing About Testing conference in the spring, which should bring even more new ideas and new voices into the public discussion of software development and testing.

Finally, I want to thank Matt Heusser specifically. Matt introduced me to SSQ a year ago. I would never have had this opportunity if it were not for his generosity.