Skip to main content

Open browser test automation at WMF


Almost a year ago I started working as QA Lead for the Wikimedia Foundation.  Among other things, I had a mandate to create an automated browser testing practice at WMF.

From the start I wanted this project to be a world class, completely open, reference implementation of such a project, using the best and most modern tools and practices I could find.  I wanted this to be a project that anyone could read, anyone could run, and to which anyone could contribute.  I wanted this to be an industry standard implementation of a well-designed, well-implemented, working browser test automation project.

Around 2006 my career had veered off into a test automation approach that, while valid and useful in certain circumstances, would be inappropriate for the WMF project.  And in the years since 2006, the tools and practices that were immature at the time had grown into mature, stable, powerful projects of their own. 

I set out to educate myself about the details of the cutting edge of browser test automation in every language.  I visited Austin TX twice and San Francisco several times in the past year to discuss approaches to the project in person with experts on the subject. 

WMF hired Željko Filipin in October 2012 specifically for the browser test automation project, and just this week we opened the curtain and turned on the lights.  The WMF browser test automation project is in Ruby, using

* page_object gem
* watir-webdriver
* selenium-webdriver
* Cucumber
* RSpec
* rake
* Jenkins integration

The initial announcement is here.
A first pass at technical documentation is here, including instructions to run the tests locally if you want to see them in action on your own machine.
Some community concerns are here.
The main code base is managed in gerrit but there is a more accessible read-only mirror on github.
Our Jenkins instance is running on the Cloudbees service  using Sauce Labs  hosts, and the current test results are visible here.
If you would like to follow the project or contribute to it yourself, we are starting a community group you can join.  Feel free to add yourself to the page here  to follow the project or contribute to it.

Many people helped me along the way to making this project what it is.  Hopefully I haven't left out anyone, feel free to remind me if I did!

Jeff Morgan for creating the page-object gem and for answering tons of my questions
Jari Bakken, especially for this presentation about webdriver in Ruby.
Brahma Ghosh
Charley Baker
Alister Scott and his Watirmelon blog
Marlena Compton  and Matt Brandt  for discussing their experience at Mozilla WebQA with me.
Bret Pettichord for hosting  the Test Automation Bazaar
Jim Holmes of Telerik for hosting the Telerik Test Summit

And especially Željko for actually building the actual project! (And for kicking my ass  along the way, I still have a lot to learn.)