Chris McMahon's Blog

Today stickyminds published my article with expanded descriptions of the "10 Frontiers for Software Testing" that I suggested as starting points for those interested in attending the second Writing About Testing conference .  Since I announced the CFP for the first WAT conference in October 2009, I have published several dozen articles on software and software testing.  (I actually lost count: it is well over thirty but fewer than fifty individual pieces.) My friend Charley Baker asked me recently where I get the ideas for so many articles.   It is an interesting question, and worth answering: The most important source of ideas is simply everyday work.  As I go about doing my job, it happens fairly often that a situation crops up that I think would be of general interest to the community of software testers and developers.  So I write it down and I make it public.  Articles about bugs, bug reports, test design, architecture, workflow, telecommu...

I took a poll of those interested so far in attending the second Writing About Testing peer conference May 13 and 14 and found that nine people are very seriously considering attending.  This is what they are thinking of discussing: Lisa Crispin (CO)  new and emerging approaches since the publication of Agile Testing Alan Page (WA) a new approach to test design, also personas for tester career roles Marlena Compton   (Australia) ongoing research in visualization of software project data Dawn Cannan   (NC) executable specifications within larger testing projects Sylvia Killinen (CO) practicing software craftsmanship Elena Yatzeck (IL) implementing DSLs for use by non-programmers Shmuel Gershon (Israel) diverse approaches to writing about testing using personas Charley Baker (CO) large-scale, Enterprise automation systems, open source Marisa Burt (CO) EAI in Enterprise systems UPDATED: Zeger Van Hese (Belgium) critical theory, etc. Frank Cohen (...

I read with interest Matt Archer's blog post entitled " How test automation with Selenium or Watir can fail " He shows a couple of examples that, while perfectly valid, are poor sorts of tests to execute at the UI level.  Conveniently, Mr. Archer's tests are so well documented that it is possible to point to exactly to where the smells come in.  The test in the first example describes a UI element that has only two possible states: either "two-man lift" or "one-man lift", depending on the weight of an item.  In a well-designed test for a well-designed UI, it should be possible for the test to validate that only those two states are available to the user of the UI, and then stop.   But Mr. Archer's test actually reaches out to the system under test in order to open a file whose contents may be variable or even arbitrary, iterates over the contents of the file, and attempts to check that the proper value is displayed in the UI based on the ...

The last Weekend Testers (Australia/New Zealand) was brilliant. Let me urge you to read Marlena Compton's report and the transcript of the session . This sort of practical implementation of critical theory is long overdue in the testing community, and the WTANZ crew did a great job of using a well-known theoretical tool to analyze and dissect some real problems in some real work. Compare what WTANZ did with Zeger Van Hese's recent demonstration of deconstruction . This sort of work, bringing reputable and sophisticated critical theory to bear on actual testing and QA activity, is a field wide open, barely explored, and long overdue.  May we see much more of it soon.

I noticed (thanks Twitterverse) that there was an interview with Rex Black over on the UTest blog .  In that interview he reveals a very interesting number: "...the ISTQB has issued over 160,000 certifications in the last ten years." Using the numbers from my previous post :  if we assume that there are about 3,000,000 software testers in the world right now, and if we issued 160,000 certifications right now, that would mean about 5 certifications for every 100 software testers.    I would be willing to bet that there were about the same number of testers ten years ago:  Y2K was just over and the value of dedicated testers had been shown.   But as Alan Page and others have noted, there is a lot of turnover, a lot of churn, among those practicing software testing.  So my numbers start to get a little sketchy here, I don't have anything to back them, so consider this a thought experiment:  as noted above, let's say that there were abou...

I bought this recently at a guerilla art show: Here it is hanging in my office: The poster caught my eye because I've loved the Alice books all my life and I re-read them often. I am especially fond of the Tenniel illustrations, and the one for Jabberwocky is a favorite. The poster also caught my eye because of the odd and interesting typeface. The story behind that typeface is fascinating. I asked the artist to send me that story in email so that I could have it written down: The story is this: Just south of where I grew up (near Green Bay, WI) is the Hamilton Wood Type Museum. A while back, I visited armed with a few sheets of printmaking paper with the goal of printing some or all of the Jabberwocky poem from some original wood type. During the course of the 19th and 20th century Hamilton made wood type for advertisements and headlines and circuses and had gone on to accumulate wood type from other manufacturers who had given in to market pressures or the eventual obso...

Before I do this, two things up front: for one thing, I am a crappy programmer. I read code far better than I write it, and I read non-OO code better than I read OO code. Also, I am writing as someone who knows a lot about Quality Assurance and testing, and very little about the hands-on day-to-day work of modern programming. So here goes: As a QA guy, I know this: long before Scrum and XP and the Agile Manifesto, people working in Computer Science and software development knew three things about software quality: having an institutional process for code review always improves software quality; having automated unit tests (for some value of "unit") with code coverage improves software quality; and having frequent, fast builds in a useful feedback loop improves software quality. Sorry, I don't have references handy, I read most of this stuff about a decade ago. Maybe some heavy CS people could leave references in the comments. The XP practices simply institutionali...

All of the questions about tester certification were answered many years ago. They exist and they cannot be made to unexist. The only remaining question on the subject is: how many tester certifications can be sold? And the answer to that question doesn't matter to anybody except the people selling the certifications. A while ago on the writing-about-testing mail list we did a little exercise to come up with some back-of-the-napkin estimates about the number of software testers in the world. We used US Department of Labor data and also some other public information about software employment worldwide. We also had access to some privileged information about magazine subscriptions. In addition, a number of us have done serious work in social networking, and we have some analytical tools from that work to help estimate. Using all of that, we came up with a pretty consistent estimate that there are probably around 300,000 software testers in the US, and maybe 3 million in the w...

The Second Writing About Testing Conference: Frontiers for Software Testing I am pleased to announce the call for papers for the Second Writing About Testing Conference to be held May 13 and 14 2011 in Durango Colorado. For more information about the original conference and the Writing About Testing mail list please see: http://chrismcmahonsblog.blogspot.com/2010/02/writing-about-testing-listconf-update.html http://chrismcmahonsblog.blogspot.com/2009/10/cfp-peer-conference-writing-about.html Writing About Testing is a peer conference for those interested in influencing the public discourse on software testing and software development by writing and speaking on those subjects. The discussion revolves around blogging, giving presentations at conferences and user groups, and writing for established media outlets, both for pay and for other reasons. There will very likely be representatives from established media outlets attending. Having software writers and publishers talki...

I generally do not post links to pay-wall or registration-wall sites, but today I am sincerely happy and proud to publish a link to SearchSoftwareQuality.com . For each of the last twelve months, I have written at least two 1000-word articles for SSQ. In the last year, SSQ has published nearly forty individual pieces of mine, a book-size body of work. I am sure some of those articles are better than others, but I wrote every one to the best of my ability with all sincerity, and I truly believe that every one of those articles contains at least one interesting idea intended to help people working in software testing and software development. I would particularly like to thank my editors at SSQ, at first Jan Stafford, later Yvette Francino. The SSQ editorial staff is professional and efficient. Both Jan and Yvette have given me an enormous amount of freedom and encouragement over the last year, and it has been a real pleasure working with them both. I especially appreciate their tole...

I've been neglecting my blog, mostly because I have been doing a whole lot of professional freelance writing on the subject of software dev and test, and really enjoying it a lot. A few months ago I also submitted a piece to the Mountain Gazette , one of my favorite magazines, available for free around the West. They always publish really good writing. Mountain Gazette was soliciting pieces on the subject "My Favorite Mountain". To their surprise, they got more than 200 submissions, of which they could only publish 11. I submitted a piece, it was rejected, but I don't mind, I've been reading the issue, and there are some really great essays. So since it isn't going to appear anywhere else, I figured I would publish it here: ---- I don't have very far to go to get to my favorite mountain. I go out my front door and take a right, and I walk about a mile through my neighborhood of mostly middle-class houses, some Victorian, some like mine vintage 1930s-40s...

On May 20 and 21 some of the brightest people in the field of software testing met in Durango Colorado for the first ever Writing About Testing conference. We participated in a diverse set of activities: formal presentations, ad-hoc demonstrations, collaborative exercises, lightning talks, and informal discussions of topics of interest that ranged from the role of media, to finding the time to write. I started my software testing career in the bad old days of the mid-1990s. Both Open Source software tools and agile methods were highly controversial at the time. And while many of us were doing amazing and innovative work, the entrenched culture of software development was highly skeptical that what we were accomplishing was valid, or even sane. I think there is a real danger of a return to those days, and I wanted to create a community where people working out on the edges of software creation could hone their ideas in a supportive community, and from what I saw at w-a-t, that commu...

For a number of years I've been writing about treating great software development as a very specialized subspecies of the performing arts. Some time ago I reviewed a piece of writing from a software person inspired by the concept of artistic software, but who had no background in the arts at all. It showed: the most egregious error was that instead of using the term "performing arts", this person used the term "performance art". The rest of the piece was earnest but the author's lack of expertise (in art, not in software) was painfully obvious. The performing arts are music, theater, and dance. Performance art, on the other hand, can be dangerous stuff. But artistic software development is only a minor representative of a number of new concepts in the field bubbling madly just behind the zeitgeist. For example, methods of harnessing immense amount of data in order to make them comprehensible to human beings are about to change all of our lives, both in...

Depending on how you define the term, I have been on at least five and as many as seven agile software teams. Two were brilliant; two were poisonous; the rest were just flaky. A big part of the poison stems from not understanding how to do agile estimation. This is part of a message that showed up on a mail list I lurk on: I'm working with a team that does great work. They are skilled and work well together. They also average about 50% or less in meeting their sprint commitments. And don't seem to mind. "There's a lot to do we just didn't get to it all." "We'll do that in the next sprint." "Yeah, that's not working yet." These are the kinds of statements during the sprint or in the retrospectives. How do I help this team look at the problem to solve it, instead of just living with it? Since the list name has the word "Scrum" in it, I will assume this person is a Scrum Master. The first misunderstanding here is to know t...

I never read business books, I mean I NEVER read business books. But after Marlena Compton read my chapter in Beautiful Testing , she recommended that I read Artful Making by Rob Austin and Lee Devin, subtitled "What Managers Need to Know About How Artists Work". I've been writing about creating artistic software for some time now, but with a copyright of 2003, this book pre-dates my endeavors and I was surprised not to have heard of it. Austin and Devin are professors at Harvard, Austin of business and software, Devin of theater. Early in the book they recount how they began the conversation that led to writing the book: We were surprised to discover common patterns and structures in our separate domains... Some recent ideas and methods in software development, especially in the so-called "agile" community, seemed almost identical to theater methods. As this became more obvious, an idea dawned on business professor Rob: These artists are much better at th...

I am really starting to dislike the term "User Experience", but I'll get back to that. In the mid-90s I was a bass player in the acoustic music scene in the South, living in Atlanta. If you happen to know Atlanta, to give you some perspective, my band opened New Years Eve at Eddie's Attic in 1994, and headlined New Years Eve in 1995 and 1996. Eddie's Attic was and still is one of the most important and influential clubs on the acoustic music circuit in the South. Also on that circuit is a club in Nashville called The Bluebird Cafe. The Bluebird is interesting because it enforces a strict no-talking policy. If you talk to your companions at all during a performance at the Bluebird, you are asked to leave the room. At one time back in the 90s there was an intense discussion among people on the acoustic music scene as to whether Eddie's should implement a no-talking policy like the Bluebird's. As far as I could tell, the musicians who advocated the most ...

The writing-about-testing mail list began in September 2009, and has already played a part in a remarkable number of achievements: The following writers appeared in print for the first time or contracted to be published in print for the first time: Abby Fichtner Dawn Cannan Catherine Powell Lanette Creamer Parimala Shankaraiah Lanette Creamer and Matt Heusser collaborated on an article about test automation in waterfall and agile projects. Alan Page, Matt Heusser, and Marlena Compton published the "Code Coverage Cage Match" collaboration. http://www.stpcollaborative.com/knowledge/538-heusser-v-page-code-coverage-cage-match Matt Heusser started a book project "Testers at Work". Adam Goucher signed a contract for a book on Selenium. Fiona Charles edited the Women of Influence issue of Software Test and Performance, with contributions from many list members. http://www.stpcollaborative.com/magazine/year/2010 Yvette Francino landed an editorial position at SearchSoftwar...

Looking Back The world of software testing today is radically different than it was on this day a decade ago. On New Year's Day 2000 I had been a dedicated software tester for about three years. I was a leader on a team testing an application that provided location information to the dispatchers who handle 911 calls. I was intensely interested in the most progressive thinking about software testing available, because when we released a bug to production, someone could die because of it. I remained interested in the most progressive thinking about software testing throughout the decade. Looking closely, we owe a vast debt to three people: James Bach , Bret Pettichord , and Brian Marick . If they didn't supply every breakthrough idea in software testing in the last decade, one of them was nearby when it happened. There was a shot across the bow in 1996 when Bach published " Test Automation Snake Oil ". This would be the first of a relentless assault on proprietary...