Thursday, July 14, 2016

Open letter to "CDT Test Automation" reviewers


To:

Tim Western
Alan Page
Keith Klain
Ben Simo
Paul Holland
Alan Richardson
Christin Wiedemann
Albert Gareev
Noah Sussman
Joseph Quaratella

Apropos of my criticism of "Context Driven Approach to Automation in Testing" (I reviewed version 1.04), I ask you to join me in condemning publicly both the tone and the substance of that paper.

If you do support the paper, I ask you to do so publicly.

And regardless of your view, I request that you ask the authors of the paper bearing your names to remove that paper from public view as well as to remove the copy that Keith Klain hosts here.  For the reasons I pointed out, this paper is an impediment to reasonable discussion and it has no place in the modern discourse about test automation.

6 comments:

Cem Kaner said...

Chris:
I don't think the paper was any good, but I think the content and the tone provide honest reflections of the views and analysis of some people who are seen as leaders in the field.

I don't think the paper should be taken down. I think it is part of their record. And if they eventually see it as poor work, they should say so. But even then, I don't take down my mistakes, I own them. I expect that from anyone else who wants my respect in the field.

Joris said...

Hi Chris,

Since we are apparently now looking to rid the field of software testing of any material that impedes any reasonable discussion, I was wondering who will lead the purification committee and who will be in it? I have some items that I really really want to add to the black list. Where can I submit them?

Best regards,

Joris

Chris McMahon said...

Posted on behalf of Alan Richardson, transcribed directly from what I received before Blogger removed it for what seems like login problems:

Alan Richardson has left a new comment on your post "Open letter to "CDT Test Automation" reviewers":

# "I ask you to join me in condemning publicly both the tone and the substance of that paper."

No.

"Condemn - to express complete disapproval of."

I don't think so.

James and Michael asked me to review the paper. I read it. I wrote comments.
I sent the comments back to James and Michael. Some of my comments they took on board
some of them they chose not to (or the paper does not exhibit all the type of changes I described).
That is their right. They wrote the paper. It is theirs.

I'll point out some features of the paper that I agree with. The simple fact that I agree with some of it, means I can't condemn it:

* "There are many wonderful ways tools can be used to help software testing."
* Tools applied incorrectly add waste
* "Let's break down testing further..." section
* "Let your context drive your tooling".
* Case 1 and Case 2 are better than I remember them

Edit: I wrote more words here, but had to edit them out to fit the max comment length of this blog. Mainly I'm editing out longer examples of things I thought were good, and some I thought were bad, but it doesn't actually matter what I thought. People can read it and make up their own mind.


All of the above was written to say:

* I can't condemn it, because there are parts I agree with
* The parts I disagree with, I can ignore, and I don't have to condemn
* I gave comments to James and Michael, they may have reworked some of the text because of it, some of it they may not have. Its their paper, that's their choice.

I don't seem to take as much offence with the paper that you do. I don't feel that strongly about it.


# "If you do support the paper, I ask you to do so publicly."

I think there are useful parts in this paper. I think some parts I disagree with.

As with everything you read. Read it and make up your own mind.



# And regardless of your view, I request that you ask the authors of the paper bearing your names to remove that paper from public view as well as to remove the copy that Keith Klain hosts here. For the reasons I pointed out, this paper is an impediment to reasonable discussion and it has no place in the modern discourse about test automation.

No. Absolutely not.

You are asking me to engage in, and condone, censorship.

No.

The fact that you asked for this. Is the reason I responded at length.

"this paper is an impediment to reasonable discussion".

Hardly.

You pointed out what you thought was wrong with the paper. I see nothing in your comments that demonstrate "this paper is an impediment to reasonable discussion". On the contrary, having pointed out the flaws, you can then go on to say what you think people can do instead. That is a prompt to reasonable discussion, not an impediment.

"it has no place in the modern discourse about test automation". People are entitled to describe their experiences of automating software. Even if they describe situations that they failed in the process.

Clearly you feel strongly about the paper.

I hope you create, or link to material, that offers alternative choices and demonstrates alternative views, which might open up options for your readers. Material that you do consider to have a place in the modern discourse of test automation, because I like to keep learning, and keep expanding my options.

Jason said...

Chris,

My thoughts weren't directly asked for, but for whatever they're worth:

- As I made clear in my comments on your review of the Bach/Bolton paper, I think it's a steaming pile that is likely only being taken seriously because of who the authors are. I don't think it reflects a competent view of test automation at all, and unless they're willing to reconsider their views, I do not take Bach and Bolton seriously on the topic of test automation as a result of that paper.

- I would very much like to see/hear the responses from all the folks who reviewed the paper. It's possible I'll learn something, but even if not, I would appreciate more publicity behind the review process. Thank you for being willing to call out those folks by name.

- I have no interest at all in seeing the paper removed, and I'm not sure you asking for that really helps your cause. You seem concerned about, among other things, incompetence being tolerated and/or embraced within CDT. Doesn't having that paper as an example help you with your case? Aside from strategic considerations, I'm in favor of people writing whatever they feel like writing, and distributing it as widely as they would like, no matter how stupid or harmful their words might be. As long as those harmed by the words enjoy the same freedom to add their feedback, of course. I would view removal of the Bach/Bolton paper quite similarly to Bach (apparently) deleting some of his abusive tweets - scrubbing the evidence. I don't think that really helps anybody.

Aside from my partial disagreement, thank you for adding your voice and stating your concerns boldly.

Chris McMahon said...

Jason,

The paper is not only incompetent, it is a cartoon. The language is so over the top you can almost picture those dastardly test automation people twirling their evil mustaches.

When I announced the review on Twitter, Bach and Bolton could not even agree between themselves whether it was intended to be example or counterexample. This led to Bach deleting tweets demonstrating his failure to understand the basic issues involved.

It seems clear to me that this paper is merely propaganda. It has no reasonable information about test automation in it at all, it's just a hit piece that's dragged on in one form or another for twenty years.

And I submit that having those names listed prominently as reviewers of the paper lend this piece of propaganda a legitimacy that it otherwise would not command.

I don't really care to demonstrate incompetence from CDT. Rather, as a professional software tester who can reasonably claim to be an expert in test automation, this paper is simply an unnecessary blight on work I care about deeply, and about which I have strong opinions.

I wrote the open letter to the reviewers to see who if anyone objected to having their names associated with such a blatant and unhelpful bit of propaganda.

I disagree that having the reviewers request the paper be removed constitutes censorship. As I said, this cartoonish piece of propaganda has no credible use in the modern discourse on test automation, and overcoming the obvious objections to it would be easier if it were simply removed.

Jason said...

Chris,

Appreciate the response. Nodding towards Dr. Kaner, I'd probably still rather see Bach/Bolton publicly own up to the folly and propaganda in their paper (an outcome that doesn't seem terribly likely, I'll admit) than see the paper removed, but I think I understand where you're coming from quite a bit better than I did before. I think it's perfectly valid for the reviewers to ask their names to be removed from the paper. I'm also inclined to think that concerned software testing professionals asking for the paper to be removed, even if I disagree with it, does not constitute censorship. To me, censorship is a person or body in authority removing speech regardless of whether or not they were asked to do so.

Keep on keeping on. Your voice and passion for software testing is needed and appreciated.