Skip to main content

Posts

TOAD Goes To 11

TOAD is Testing, Observability And Devops. We think these three things are related. What would happen if we took each of the three aspects of TOAD and put strong emphasis on each in turn? What happens if we take our testing effort as far as it can go, and "dial it to 11"? Observability to 11? Devops to 11? The meaning of "dial to 11" will be different for different organizations: it might mean hiring staff, or investing in tools, or even just emphasizing the mission more than before.

If we dial testing to 11, I think two things will result. For one thing, the number of observable incidents in production will likely go down, because the emphasis on testing means that more problems will be found and fixed before deploying to prod. I also think that rate of deployments (Devops) could potentially increase, because the decrease in observable incidents will make deployments safer. So: increase the T in TOAD to decrease the O and increase the D.

With testing at 11, now…
Recent posts

TOAD: RRRAR! Rollbacks, Replays, Reverts, and Regressions

TOAD is

Testing
Observability
And
DevOps

My last blog post  was about why TOAD ideas are important and how TOAD ideas interact with each other. But even the most excellent TOAD systems occasionally release bugs to production. This post describes how to understand these problems and how to address them when they happen. And there is an experience report at the end!

Most of the time we learn about problems in the production system because the system is observable. We see a performance problem, or a data problem, or a space problem, and we can use observability tools to narrow down and ultimately fix the cause(s) of the observed problem. There is also an entire class of problems that cannot be discovered with typical observability tools, but only by testing the system. Given that we find such problems in production, what can we do about it?
Rollback and Replay The most drastic response to a production problem is to "roll back" the system to the last known good state by means …

TOAD FTW! Evaluating a Test Suite: A TOAD Thought Experiment

TOAD is Testing
Observability
AND
DevOps
The AND is important!

Disclaimer: In my career I have done most of the things I describe below, but never tied them all into one project. The following should be possible:
Testing a System Suppose we have a software system of reasonable complexity. Suppose our system is comprised of a front end with a user interface (UI) and a back end, thus a client and a server. The front end and the back end communicate via an application programming interface (API) of some sort. This is a common architecture of many software systems.

Suppose further that our front end and our back end are well-designed. They have unit tests, and meet whatever definition of quality you would want to apply.

Because our system is reasonably complex, we want to have a suite of end-to-end tests that exercises the entire software-and-data stack,  that demonstrates that the users can do the things they need to do, and that the front end and the back end are communicatin…

My Remote Retrospective Process

Possibly the most important aspect of an agile process is the retrospective. A retrospective usually happens in a team meeting, generally at the end of every agile iteration or sprint. While there are any number of ways to run retrospectives, the object is to discuss three questions:

     What is working well?
     What is not working well?
     What should we change?

The first question tends to be the easiest to answer, and it is tempting to just skip it, but that is dangerous. It is every bit as important to celebrate the team's success as it is to grapple with the team's issues and problems. Positive reinforcement is more powerful than negative reinforcement.

The second question also tends to be easy to answer in teams where the members feel safe to discuss work honestly. It is important to surface problems so that the whole team is aware of the current issues and why those issues affect the work.

The third question tends to be hard to answer. There is an entire …

Why I like Cucumber (beyond BDD)

There is a sentiment in software development that if you do not have a working BDD practice then Cucumber is just unnecessary overhead. I understand this position, but I disagree, based on my own experience and my own practice. I find Cucumber is particularly valuable in automated browser tests.

I've been using Cucumber for automated browser tests at work just about every day for the last six years or so. At the same time, I have never worked with a full-on BDD team. Beyond BDD, here are three aspects of Cucumber I find particularly valuable:

Cucumber creates a low barrier to entry for anyone at any time to contribute and understand the project.Cucumber's Given/When/Then syntax provides a design guide particularly well suited for browser tests, especially using "When" in a particular way.When a test fails, Cucumber provides a plain-English description of what the test does that may not be immediately apparent from the code or the nature of the failure.
I tend to write b…

Watir: the first five years

Some of the Watir community has been discussing the history of the project. Here I try to set down some notable things that I remember about that time.

We have to start the story of Watir with Ruby. The first English documentation for Ruby was published in 2000, and it garnered a lot of interest, particularly as it was so amenable to creating Domain Specific Languages (DSLs), which caught the attention of a number of people working in testing, and in the Agile world. Python 2.0 also came out in 2000, and the dominant scripting language at the time was Perl.

Of all the browsers available at that time, only Internet Explorer exposed an API for automating browser actions, via a COM interface. In 2001, Chris Morris published a Ruby library that exploited IE's COM interface called WTR, for Web Testing in Ruby.

At that time, Open Source software was often seen as inferior or even downright suspicious, and was poorly understood by most businesses. Test automation products were exclusively …

Long Term Remote Pair Programming a Complex Project

This is the story of a really great project I did while working for Salesforce.org. I have done a significant amount of remote pair programming over the last ten years, but this project was extraordinary in a number of ways. For one thing, it was a really complicated problem that demanded a technically advanced solution. For another thing, it took almost an entire year to finish-- one hour per week.
The Problem I will try to give you the background in a way such that your eyes don't glaze over: in order to work with data in a Salesforce instance via the API, you address "Objects" and "Fields".  (These are actually tables in a database that may be addressed by a poor and crippled version of SQL.) For example, here is a description of the Account object whose first field is AccountNumber.

If you are a developer on the Salesforce platform, you can add your own Field to the Account object, but you have to append '__c' to it, like "MyField__c". Yo…