Tuesday, May 29, 2012

Join the Wikipedia/OpenHatch.org test event 9 June

Wikipedia allows users to leave feedback on each article.  Experienced Wikipedians analyze this feedback in myriad different ways to improve the Wikipedia user experience and to improve the encyclopedia itself. 

The Wikimedia Foundation has been creating a new Article Feedback system, and on Saturday 9 June from 10AM-noon Pacific time, WMF invites testers and anyone else interested to participate in a "shakedown cruise" to test a near-final version of the new Article Feedback system before the system is rolled out to all of Wikipedia.

Following on May's successful test event with Weekend Testing Americas, WMF is teaming up with the Open Source fans at OpenHatch.org for this event.  I am hoping that having experienced exploratory testers plus people interested in improving Wikipedia articles will be a killer combination of expertise and interest to shake out any final issues in this critical aspect of the Wikipedia experience.

Note:  anyone who shows up on 9 June will have access to some of the Article Feedback features, but if you create an account on Wikipedia no later than 4 June and also make at least ten edits to Wikipedia, your account will be "autoconfirmed" and you will have access to many more Article Feedback features than a casual user would.  (Feel free to edit for instance the Wikipedia article on your home town, or an article on software testing, or dice games, or if you're shy, just update your User page and your Talk page a few times, those edits count toward the ten also.)

This is the announcement and there is an optional sign up sheet. The Article Feedback Test Plan has all the details


Monday, May 14, 2012

Testing (automation or whatever) 101: ask a good question.

I tried to do A, and I really don't understand the response I got, X.  Does this make sense?

I know it should be possible to do A, but I tried it and X happened.  What sort of conditions would cause that?

I tried to do A, and X, Y, and Z happened.  X makes sense, but I don't understand Y, what's going on here?

It doesn't really matter whether you're asking about automation or any other kind of testing.  The tricky part is that before asking the question, you had better be pretty familiar with A, and you had better be able to report X, Y, and Z in a reasonable way. 

I have a corollary, and I have a (counter) example.

I have seen any number of people in the software world complain about testers who submit bad bug reports.  I'm sure it's true, I've seen the evidence, and it boggles my mind.  A good bug report will explain A and explain X, and a great bug report will phrase the issue in terms of a question. 

Not long ago I got an email from someone asking about a little script I wrote some time ago.  He asked me to give it to him.  I have not replied. I was astonished.  For one thing, a cursory google search would turn up the 30 lines of code in question.  But even worse than that:  why don't you WRITE IT YOURSELF? 

It's quite possible my script no longer works.  It's quite possible that there are better ways to accomplish what the script does than what I wrote.  But I absolutely refuse to copy'n'paste 30 lines of code in an email response.

Eric Raymond (if you don't know that name, google it) wrote an essay a long time ago How To Ask Questions The Smart Way.  I'm guessing that many readers of my blog are not familiar with it.  This is a travesty. 

NB: the last time I pointed a software tester to Raymond's essay, I was accused of misanthropy and worse.  Testing might be dead.

Sunday, May 06, 2012

Testing Summit at Telerik


I attended the Test Summit peer conference this weekend at the invitation of Jim Holmes of Telerik.

It was outstanding, as such peer conferences tend to be, and I and others will be posting a lot of information as a result of what went on there. 

But I want to talk about the conference itself. 

Software testing has a long history of peer conferences, starting (to the best of my knowledge) with the Los Altos Workshop on Software Testing (LAWST).  Bret Pettichord borrowed the LAWST format for his Austin Workshop on Test Automation (AWTA) in the mid-2000s, and I borrowed from AWST for my Writing About Testing (WAT) conferences in 2010 and 2011.   I think other examples exist.  The format has gotten looser over the years.  LOTS looser, as we find that motivated participants are pretty good at self-organization.

As far as I am aware, the Telerik Test Summit (TTS?) is the first such software testing peer conference created and sponsored by a commercial company.   I think this is important, and I think TTS has important implications.

As a general rule, I will not endorse or support commercial companies, even the ones I work for.  I don't even link from blog posts to my own writing for articles that are behind a registration-wall or pay-wall.  Now that I work for the Wikimedia Foundation, I have even more motivation to be as impartial as I possibly can. 

And yet I accepted the invitation from Telerik and Jim easily.  I'd like to explain that. 

I knew everyone in the room at TTS, and many of the attendees are good friends.  I had never met Jim Holmes. 

But I knew his work.  Jim created the CodeMash conference, which is notable not only because of its radical growth in popularity over the last several years, but also because of its reputation for friendliness and hospitality.   I'm not involved in CodeMash and I've never attended it, but I know a lot of people who have, and Jim has a stellar reputation among my friends and acquaintances. 

I didn't go to TTS because Telerik invited me;  I went because Jim Holmes has an unimpeachable reputation for integrity and hospitality. 

We were not pitched. We were not marketed to. There were no Telerik tools being discussed in the room.  We were given a tour of the Telerik office in Austin;  some attendees chose to pair up with Telerik staff to see what they were working on, some did not.   I consider this just hospitality. 

TTS should be a lesson to other companies.  Many of the best people working in software will give away their best ideas and even their best work freely.   But the moment such people suspect they are being used or co-opted for some purpose other than trying to move the cause forward, we will turn on you with a vengeance you can not imagine. 

So my advice to any companies who might want to sponsor their own peer conferences in the future:  first, get yourself a Jim Holmes; second, listen to what he has to say; then, do what he tells you.  That's probably harder than you imagine.





Tuesday, May 01, 2012

Weekend Testing for Wikipedia May 5


On Saturday May 5 at 10:00 AM Pacific Time the Weekend Testing Americas will be investigating the new release of Mediawiki on Wikimedia Foundation sites before WMF rolls out the new version to all of Wikipedia.

I am really excited about this project, I do hope you will consider joining in.

Details of how to join are on the official Weekend Testing site
1. Add “weekendtestersamericas” to your Skype contacts if you haven’t already.

2. Fifteen minutes prior to the start of the session, please message “weekendtestingamericas” and ask to be added to the chat session. Once we see you, we will add you to the session.

The test plan is here


See you on Saturday!