Monday, November 24, 2008

artistic testing

I like to point out that historical, well-vetted aesthetic frameworks are useful for evaluating software.

But what if similar aesthetic frameworks could not only validate software itself but also *predict* software test techniques?

For example, let's examine overarching schools of painting: first came realism, where painters attempted to represent as much as possible what appears to the human eye. We can test realistic scenarios, we do this all the time. I would suggest that most bugs are not discovered under what we understand to be realistic circumstances.

Then came expressionism, where painters attempted to express things not seen. Likewise, test techniques moved on to persona testing (imagining the use of software by peculiar users) and soap-opera testing. Such techniques expose significant bugs.

Then came abstract expressionism. Think of Jackson Pollack. Are there abstract expressionist test techniques? Of course there are.

I would really like to hear of any other explorations along these lines...

Saturday, November 22, 2008

two agile antipatterns

Both of these have been crossing my path with increasing frequency in recent months (not at work, but in conversations with others whom one would think would know better):

1) Stories are scheduled work for features. Stories should not include:
  • testing
  • refactoring
  • documentation
  • undefined customer support
  • calling your mom
The problem with writing stories for stuff other than scheduled work on features is that you completely undermine the concept of velocity.

Velocity is the number of story points accomplished per iteration. If you assign story points to activities other than scheduled work on features, you cheat the business of accurate information about release schedules. (Although it might make you feel better to be accomplishing so many points each iteration, if you are not delivering features, you fail.)

2) Doing it "by the book".

Part of the agile culture is to have retrospectives at the end of each iteration. Each retrospective is an opportunity to change your process.

So if you're doing 2-week iterations, at the end of a year, you've had 25 or so opportunities to change your process. At the end of two years, 50 process changes.

If after all of these retrospectives, you are doing your work the same way you were at the beginning, you fail.

Wednesday, November 12, 2008

Finally joined Twitter

I'm chris_mcmahon there if you want to follow me.

Wednesday, October 08, 2008

testers as navigators

Not long ago Audrey Tang started working for Socialtext. It has been a remarkable experience, as Audrey is mostly fixing bugs based on bug reports submitted by our testers.

Audrey is in Taiwan and another of our testers is working from Bangladesh (although he is usually in Vancouver BC). Today they needed to hash out some details of a bug report so they shared a VNC session on a machine hosted in Palo Alto CA. It took only a couple of minutes to arrive at an agreement that the bug was fixed.

Audrey's comment in IRC really struck us all: " having an excellent qa team is like driving with a first-grade GPS navigation system :)"

We have worked very hard to create consistent, readable, executable bug reports for real, visible bugs. Having a developer as good as Audrey validate that work means a lot.

Saturday, September 06, 2008

a test technique

Very recently I was part of a bug hunt: an effort to find as many defects as possible in a UI in a short period of time.

I am not an outstanding UI tester. I have worked with testers whose eye for detail, line, color, consistency, work flow, etc. etc. make them really outstanding testers to have examining the front end. But I've spent most of my career on green-screen apps, test automation, environment maintenance, stuff like that. I lack the really expert artist's eye that great UI testers have.

That said, I was on a roll on this particular bug hunt, finding a lot of nice, sophisticated bugs. It was classic ET: decide a test approach, determine your next test based on the results of your last test; and when you stop finding bugs, change your test approach.

I'd followed all the logical paths I could think of; I'd tried really big inputs and really small inputs; I'd tried permissions errors, navigation errors, and checking error messages themselves. I tried some crazy stuff. I started to run out of reasonable test approaches. Toward the end, when I was getting tired and finding fewer and fewer bugs, I hit on a neat test technique that found a really cool bug:

Here's what I did: I sort of unfocused my eyes, and addressed the UI spacially instead of logically. Moving from top left to bottom right, I manipulated everything I could find to manipulate. Then I went across the bottom; across the top; up and down in lanes; just clicking and typing and choosing whatever was located next to the last thing I manipulated.

I found this great bug this way: there was a very small link in the footer of a particular area on a page that had exactly the same label as a much larger link elsewhere on the page. Even though we had four expert testers closely examining the application, no one had discovered that the little link in the footer was wired up completely differently (and totally wrong) than the big link with exactly the same label. Until I unfocused my eyes and considered everything on the page as equal and logically unrelated, except by space. Because I approached the UI spacially instead of logically, I found the little piece that was wired up completely wrong.

Monday, June 23, 2008

technical investment

My colleague Matt Heusser is doing this workshop on technical debt soon.

I posted a little meditation on techdebt as impedance mismatch, but that seems to me to be trivially true, facile.

Since I made The Software Artists public, though, I've been fielding a lot of questions on the software-test mail list that has me rethinking my position on technical debt. So here is what I *really* think:

Technical debt doesn't really exist. At least, it doesn't exist for high-performing software teams. It's a useful concept for those encumbered by poor programming and poor testing, but the idea ceases to be useful about the time the code base becomes manageable.

I have a graph and and I have a metaphor that both explain my position.

I'll just describe the graph because I'm too lazy to hunt down some free software to draw it. This graph describes a couple of different projects I've been involved in, for a couple of different employers.

The critical line on the graph is "regression bugs released to production". Or you could title it "predictable bugs released to production" if you wanted to be more inclusive, but I like the regression idea best.

Another line on the graph is "features released to production". This is where we make the product better.

A third line on the graph is "test coverage". I don't really care about the distinction between unit tests, integration tests, acceptance tests, or whatever. Just stipulate that it is possible to achieve some significant automated test coverage of the product.

Over time, the graph of a productive, high-performance team creating software will likely show:

Early on, the number of regression bugs released to production will be a small number, then will go to zero. At Socialtext, we have a suite of Selenium-based regression tests. About a year ago, we rapidly increased the number of test steps from around 1000 to around 4000. The number of regression bugs released to production went from 'a few' at 1000 test steps to zero at 4000 test steps. That line on the graph has remained at zero, and the test suite now has about 7000 test steps.

The number of features added to the software will rise significantly. Good developers working well, releasing to production every two weeks: you get a lot of features.

Test coverage (and by implication, refactoring): this is the interesting part. Test coverage rises at a much smaller rate than the rate at which features are added. Picture the difference: as the number of features rises dramatically, and the amount of test coverage rises less dramatically, the amount of technical debt increases.

Yet the number of regression bugs released to production remains zero.

This works because that area of the graph between test coverage and features released doesn't really represent debt. We know of course that the number of *possible* tests for any reasonable feature far exceed the number of *necessary* tests for that feature. The only line that matters to us the 'defects released' line. We choose our tests to minimize defects, not to maximize coverage. Rather than go for feature-for-feature coverage, we weave a web of tests that is likely to catch any regression errors.

We INVEST in automated tests (and refactoring, etc.) to MINIMIZE RISK. I have to emphasize again that this can really only be done with a skilled and disciplined team in a reasonably well-designed code base. In a less-expert environment, the idea of technical debt might be more useful.

So now to the metaphor:

I have a friend who is a successful artist and gallery owner here in our small town in the West. She started in sculpture more than 25 years ago, moved to oils, and recently has fallen for acrylics and is doing some of the best work of her career. She tends toward landscapes both real and imagined, although she sometimes takes for a subject flowers or saxophones.

She's a Western female painter of landscapes and flowers: why is she not as successful as Georgia O'Keefe?

If we had been talking about software, the answer would probably involve 'technical debt'. But we can see on the face of it that such an answer is silly. There is no such thing as technical debt in a practice or in a performance.

However, the concept of technical *investment* does clearly apply in both cases. Georgia O'Keefe was an expert and shaped her career in a certain way. And she had a lot of luck. My friend Karyn is also an expert and shaped her career in a certain way. And she had her own share of luck, also.

The careers of both of these successful women show a lot of investment, but different kinds of investment in different areas, different people, different times, with different results.

The important thing is not that Ms. Gabaldon is less famous than Ms. O'Keefe: the important thing is that both succeeded in their work.

So where would the idea of technical debt actually be useful? Consider the team encumbered by poor programming and poor testing. If the number of predictable defects released to production on your project is high, consider studying the literature of software development and of software testing in order to bring the skills of the team up to a level where technical investment makes more sense than technical debt.

Saturday, May 31, 2008

The Software Artists: Citations for Part Three

Citations For the section “Practicing, Rehearsing,
and Performing Software”

Programmer, poet and guitarist Richard Gabriel's quote is from
his proposal for a Master of Fine Arts in Software:
http://www.dreamsongs.com/MFASoftware.html

Apple CEO Steve Jobs' quote is widely noted, but is documented
in context by Andy Hertzfeld here:
http://www.folklore.org/StoryView.py?story=Pirate_Flag.txt

The practice/rehearse/perform rubric was first presented on the
author's blog in September 2007:
http://chrismcmahonsblog.blogspot.com/2007/09/become-better-software-artist.html

The title of this paper comes from the author's blog post of May
2007 in response to Jonathan Kohl's recommendation of theater
and musical experience for software testing:
http://chrismcmahonsblog.blogspot.com/2007/05/here-come-software-artists.html

A performance by Ella Fitzgerald and Count Basie available on
YouTube was a strong influence on this paper, cited in response
to another of Brian Marick's thoughtful blog posts:
http://chrismcmahonsblog.blogspot.com/2007/05/example-of-analogy-monks-vs-music.html

Luke Closs of Socialtext is a performing juggler and an excellent
agile developer. His talk “Juggling for Geeks” is a fine
performance: http://revver.com/video/317034/juggling-for-geeks-alpha/

The author performed the jazz standard “These Foolish Things”
on a $25.00 green ukulele at the Developer-Tester/Tester-
Developer Summit at Google in Mountain View in February
2007, to demonstrate (as good software testers know) that is is
possible to produce excellent results even with inferior tools.
http://googletesting.blogspot.com/2007/03/developer-testertester-developer-summit.html

The Software Artists: Practicing, Rehearsing, and Performing Software

Software development is a performance exhibiting
skills developed by an individual—often in groups of teams...
-Richard P. Gabriel

Real artists ship.
-Steve Jobs


Since the language of art criticism provides useful tools for
evaluating software, and the practice of teaching music provides
useful approaches to teaching software, it is reasonable to think
that the practice of artistic performance should provide useful
ways to approach creating software.

Only one way to build software has ever been acknowledged:
software is designed, coded, and tested. Sometimes the
design/code/test cycle is done a single time, sometimes
design/code/test is done iteratively or incrementally, sometimes,
as in Test Driven Development, the steps are moved around, so
that the cycle goes test/design/code.

But this is not how human beings create a performance. Instead
of design/code/test, artists have a cycle of
practice/rehearse/perform.

Since human beings have been creating performances since
prehistoric times, it is worthwhile to see if creating software might
benefit from the practices of artists.

Practice

Before an artist is prepared to perform, the artist must have a
certain set of skills. Even the most naive or improvisational artist
will have practiced some aspects of their craft before performing.
Practice is where the artist acquires and improves the basic skills
of the craft. A musician will use practice to learn scales and
chords and rhythms, an actor will practice lines and passages, a
painter will practice brush strokes and coloring.

By practice the artist acquires not only the fundamental
knowledge of the craft, but also masters the necessary tools for
that craft. A musician will learn an instrument, a painter will
learn brushes and pigments and canvases, an actor will learn
speech and movement.

When practicing for software, the software creator will learn
applications and programming languages, design patterns and
editors. The software creator will also learn operating systems
and network protocols, the basic frameworks on which software
is created.

When practicing for a particular software performance, a software
creator might use a “spike” in the agile sense, or a simple
prototype of some feature or features to be developed.
Regression tests could be considered as a tool for practice,
because such tests reinforce good software behavior.

Rehearse

As Dr. Gabriel says, software is created by groups. But whether
solo or in a group, artists rehearse before they perform. Rehearsal
is where artists behave as if they were performing. In rehearsal,
the various aspects of the performance are worked out, and the
performers examine the boundaries of the performance to make
certain that what is expected of them is in fact not only possible,
but will also be attractive to an audience.

Integration testing and functional testing might be tools of
software rehearsal, also beta versions and “dogfooding”, using the
software internally or among a small group before releasing the
software. User experience testing might be part of rehearsal.
Pair-programming could be a tool of software rehearsal, as could
code inspection.

Investigation is a key part of rehearsal. After practicing, artists
still need to know where misunderstandings exist, where aspects
of the performance may break down, where skills may not be
adequate to the demands of the performance.

Iteration is a key part of rehearsal. Rehearsal often involves a
close examination of the performance looking for ambiguity and
wrong expression. Upon discovering such issues, the small part
of the work exhibiting the problem is reworked until the
performers agree on the proper way to conduct the performance.

Listening to a symphony orchestra rehearse is enlightening. A
major symphony orchestra is comprised of the most
accomplished musicians on Earth. These musicians have
dedicated their lives to practice and performance. A symphony
orchestra is also an extremely expensive operation.

In a symphony rehearsal, the conductor or music director will
have the orchestra play only a few bars from passages that the
conductor believes to be problematic in some way. Based on
only a few seconds or minutes of playing, the conductor will
adjust the performance of various players and sections of the
orchestra in a very short period of time. These adjustments are
understood to also apply to the rest of the performance.
Some things about artistic performance could greatly improve
software performance:

  • For an artistic performance, everyone on the project has a role on the stage. The best agile development projects emulate this behavior, since everyone on the team is always actively involved in creating the software.
  • If rehearsals for an artistic performance are poor, the performers may retire to practice more before continuing with rehearsal. The notion of a software “spike” is very much like this, when the software creators put their regular work aside in order to learn something new about a relevant aspect of creating the software.
  • Performers are rarely micro-managed. Successful artistic projects and successful software projects assume that the performers have a certain level of competence, and remove people who do not have that level of competence.
  • The value of practice and rehearsal is measured qualitatively not quantitatively. For both software and art, the creators worry first about the excellence of the performance, and only later about the number of tickets or copies of the performance to be sold.
Perform

Serious artists perform their art for an audience. Serious software
creators release software for their users.

Performance is where value is realized, where expertise is on
display, where practice and rehearsal pay off: for the performers,
the creators, the investors, and for the audience.

Maybe it is time for the software development practice to begin
referring not to “the users” but to “the audience”.

To the Studio

Artists' studios, rehearsal studios, movie studios, and recording
studios are busy places. They are full of the artists' tools, toys,
and works in progress, and they are arranged for the convenience
and inspiration of the artists.

A factory floor is arranged for the convenience of managing the
product.

This tension between the artist and the manufacturer is not new.
For instance, Andy Warhol's studio was called The Factory.
Many movies and photographs of The Factory exist. The irony is
clear.

It is notable that cubicle farms, departments, and isolated offices
are giving way to bullpens, open spaces, war rooms, co-working
space, home offices, even coffeehouses and brewpubs. The
correlation to artists' studios is clear.

It is notable that a small number of software creators are already
being treated like artists. Linus Torvalds, David Heinemeier
Hansson, and Richard Stallman all come to mind.

Perhaps in the future the software artists will practice and
rehearse in their software studios so that they can release software
performances to their audiences. The artists and the audiences
and everyone else involved in these software performances will
understand the value of these performances, and the feedback
between artist and audience will increase the value of these
performances. People have been doing exactly this in art for
thousands of years.

The Software Artists: Citations for Part Two

Citations for the section “Pedagogy and Practice in
Software and in Guitar”

Some of Joel Spolsky's writing about working with young
software creators is at these links:
http://www.joelonsoftware.com/articles/CollegeAdvice.html
http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html
http://www.joelonsoftware.com/items/2008/01/08.html

Thoughtworks University had been called “Thoughtworks Boot
Camp” until the author had to explain the term upon crossing an
international border in 2005. A description of Thoughtworks
University is here:
http://www.thoughtworks.com/work-for-us/TWU.html

Some representative citations from IEEE publications about poor
software education are:
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/8994/28540/01276501.pdf
http://stevemcconnell.com/ieeesoftware/eic12.htm
http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/mags/so/&toc=comp/mags/so/2002/05/s5toc.xml&DOI=10.1109/MS.2002.1032848

The citation from CrossTalk is here:
http://www.stsc.hill.af.mil/CrossTalk/2008/01/0801DewarSchonb erg.html

Samples of the discussion of the apprentice/journeyman/master
model in software are at these links:
http://butunclebob.com/ArticleS.UncleBob.TheProgrammingDoj o
http://www.mcbreen.ab.ca/SoftwareCraftsmanship/

Of particular interest is Dave Hoover's account of hearing Bob
Martin's presentation at Agile2007:
http://redsquirrel.com/cgi-bin/dave

The developers at Atomic Object also believe that software is a
craft:
http://www.atomicobject.com/pages/Software+as+a+Craft

Brian Marick's blog post about skill, discipline, ease, and joy is
here:
http://www.exampler.com/blog/2007/05/16/six-years-later-what-the-agile-manifesto-left-out/

This interview with Robert Fripp was very helpful:
http://emusician.com/em_spotlight/fripp_philosophy_guitar_craft/

Eric Tamm's biography of Robert Fripp has a wealth of
information. Chapters 10 an 11 concern Guitar Craft, and were
very helpful:
http://www.progressiveears.com/frippbook/contents.htm

The Guitar Craft website is
http://www.guitarcraft.com/
The site behaves oddly sometimes though, so a direct link to “A
Preface to Guitar Craft is here:
http://www.guitarcraft.com/?np=5&&id=4

A number of Robert Fripp quotes are here:
http://www.brainyquote.com/quotes/authors/r/robert_fripp.html

The description of Bob Martin's ideas of “professionalism” come
from the abstract of his presentation at Agile2007:
http://www.agile2007.com/agile2007/index.php?page=sub/&id=649

The Software Artists: Pedagogy and Practic in Software and in Guitar

Very few people graduating with a college degree in Computer
Science or Information Technology are prepared to write
production code or to test production software.
Companies that hire recent college graduates often have special
training for such employees before they may work on actual
projects. Joel Spolsky has written extensively about how his
company Fog Creek trains young software workers. Publications
of the IEEE mention the issue frequently. The well-regarded
software consultancy Thoughtworks has “Thoughtworks
University”, a six-week “boot camp” style training event for new
hires hosted in Bangalore. Even the journal CrossTalk: The
Journal of Defense Software Engineering says:

It is our view that Computer Science (CS) education is
neglecting basic skills, in particular in the areas of
programming and formal methods.


The Craft of Software

Besides the mentorship approach Spolsky takes and the “boot
camp” approach Thoughtworks takes, Robert C. (Uncle Bob)
Martin, Pete McBreen, Dave Hoover, and others have for some
time advocated a master/journeyman/apprentice system for
teaching software, much like the ancient guild systems. Mr.
Hoover in particular reports notable success with this approach.
But using 15th century teaching techniques for a 21st century craft
seems unlikely to succeed on any reasonable scale without
considerable adjustment.

For one thing, both the in-house training systems and the guild
systems of pedagogy tend to keep effective practices secret.
Knowledge is passed from person to person directly, without
involvement from the community.

For another thing, craft alone is not sufficient to produce excellent
software or excellent software creators.

Brian Marick has a remarkable weblog post from May 2007
called “Six years later: What the Agile Manifesto left out”. In it
he cites four values in which software creators should invest:
skill; discipline; ease; and joy. Of this list, Marick seems to
equate “skill” with Martin's idea of “craft. “Ease” and “joy”
follow from the practice of skill and discipline. The most
remarkable aspect of this essay is that “discipline” is a new idea
in the public discourse of software practice.

Martin's idea of “craft” and Marick's idea of “discipline” exactly
echo the language of the teaching philosophy of the well-known
guitarist Robert Fripp.

Robert Fripp: Craft and Discipline

Those who do not know Robert Fripp's musical work may know
him as the creator of the new sounds for Windows Vista. He has
been at the forefront of technology in music since the late 1960s.
Since 1985 he has been teaching an approach to music and guitar
called Guitar Craft. Many of his students, for instance the
California Guitar Trio, are quite successful in their own right.

There are sanctioned Guitar Craft programs in Italy, Japan,
Mexico, North America, South America, Spain, and the UK.

The two axes of Fripp's teaching methods are Craft and
Discipline. Fripp uses these terms in a way very similar to the
way that Martin and Marick use them. But unlike Fripp's notable
success at teaching music over the last two decades, software has
no single successful long-running education project with a
consistent philosophy.

While a mature discussion of software craft exists, that discussion
has yet to demonstrate any repeatable successes. Martin
discusses software craft in conjunction with professionalism, but
the milieu in which “professionalism” is exercised is somewhat
limited: professionalism for Martin is “...about behavior, attitude,
and professional ethics”, and that “Writing clean code is a matter
of both skill and character. “

These are fine and noble sentiments, but ethics and character have
little to do with actually creating great software.

Brian Marick, in the same essay where he identifies the four
values, explicitly says that “...new Agile teams... are
floundering.” While his essay identifies the problem, it fails to
explain how to exercise skill, discipline, ease, and joy in the
service of creating software, except to pronounce that these
qualities are necessary.

By contrast, Guitar Craft has thousands of words from Robert
Fripp and others on how the interaction of Craft and Discipline
are necessary for performing this music on guitar:

The invisible side of craftsmanship is how we conduct
ourselves within the process of acquiring craft, and how we act
on behalf of the creative impulse expressing itself through music.
In time, this becomes a personal discipline. With discipline, we
are able to make choices and act upon them. That is, we become
effectual in music.
It is notable that when Fripp re-formed the band King Crimson in
1981, they chose to name their new album “Discipline”.

Like Martin, Fripp uses the apprentice/journeyman/master
metaphor, but adds an additional role of “genius”. Each role is
delineated in a number of ways:

The apprentice acquires skill.
The craftsman acquires sensitivity.
The master acquires vision.
The genius attains freedom.

The apprentice is noisy.
The craftsman shapes sound.
The master shapes silence.
The genius is silent.

The apprentice practices the craft of craft.
The craftsman practices the art of craft.
The master practices the craft of art.
The genius is artless.
Beyond the four roles, Guitar Craft also recognizes seven levels
of accomplishment. The details of the seven levels are not
relevant to this discussion, except to point out that the well-
delineated roles combined with levels of achievement that
correspond to the student's commitment to practicing Guitar Craft
provide a well-designed structure for communicating the
principles of Guitar Craft not only to the community itself, but
also to those outside the community who may be interested in
examining the practice of Guitar Craft. Software craft has no
such well-defined hierarchies of learning or commitment.

There are of course technical details of Guitar Craft, having to do
with posture, tuning, left- and right-hand techniques, etc. etc.
Such details are beyond the scope of this paper. Such details,
while important when playing the guitar, are not relevant to the
discussion of whether software craft can learn from Guitar Craft.

While it is not actually possible (or desirable) to separate the
philosophy of Guitar Craft from the actual practice of playing the
guitar, there is so much in the written history of Guitar Craft that
is provocative for “Software Craft” that mining the Guitar Craft
canon for software methodology could be a valuable exercise:

If an apprentice does not hear what a master hears, is
then that quality not present in the music? Yes and no.
In the world in which the apprentice lives: no.

...if I cannot manage to persuade the members (of the
group) of what I see to be the next course of action,
how do you expect the group to deal with the
expectations of thousands of people. It is not possible.

These are ten important principles for the practice of
craft:
Act from principle.
Begin where you are.
Define your aim simply, clearly, briefly.
Establish the possible and move gradually towards the
impossible.
Exercise commitment, and all the rules change.
Honor necessity. Honor sufficiency.
Offer no violence.
Suffer cheerfully.
Take our work seriously, but not solemnly.
You have to find a way of recognizing and describing
the quality inherent in the music. If you fail to do that,
then the piece itself is compromised.

The history of Guitar Craft demonstrates that the ideas of Craft
and Discipline together can create a successful international, self-
perpetuating teaching and performing regime. And the language
today's most skilled and prolific software creators that it seems
worthwhile to find out if ideas from Fripp's way of teaching
music might translate to the similarly creative process of teaching
software.

Sunday, May 18, 2008

The Software Artists: Index of Links to All Sections

Explanation
Introduction

The Value of Software
Citations for Part One

Pedagogy and Practice in Software and in Guitar
Citations for Part Two

Practicing, Rehearsing, and Performing Software
Citations for Part Three

The Software Artists: Citations for Part One

Many of the ideas in this paper were first presented at the Austin
Workshop on Test Automation in early 2007. The substance of
the talks appeared on the author's blog shortly afterward:
http://chrismcmahonsblog.blogspot.com/2007/01/craft-and- discipline-larks-tongues-in.html

Most of the New Criticism and Structuralist citations are from
Wikipedia, except:
Child Jr., William C. 2000 Monroe Beardsley's Three Criteria for
Aesthetic Value: A Neglected Resource in the Evaluation of
Recent Music. Journal of Aesthetic Education, Vol 34, No. 2
(Summer 2000), pp 49-63 doi:10.2307/3333576
The author did not know that unity/variety/intensity had been first
presented in music criticism until reading this article.

The author also referred to
Adams, Hazard, Searle, Leroy (eds.) 1986 Critical Theory Since
1965 University Presses of Florida/Florida State University Press
for background information.

Bruce Schneier on “security theater”:
http://www.schneier.com/blog/archives/2006/08/terrorism_secur. html

The Software Artists: The Value of Software


Previous: The Software Artists: Explanation
Previous: The Software Artists: Introduction


Philosophy of Art and the Value of Software


Manufactured goods generally have a clear relationship between
cost, price, and value. In software, as in art, the value of the work
is more often completely unrelated to cost and price. Operating
systems provide a great example: the Windows audience for the
most part must use Windows regardless of cost or price. The
Mac OSX audience generally chooses to use OSX regardless of
price, and often explicitly because of the aesthetic experience of
using OSX. Linux has no cost at all, and a price that varies
wildly, and it also has a dedicated audience.

The value of software, like the value of an artistic performance,
lies in the ability of the software to attract and keep an audience.
The software industry would benefit immensely by turning the
tools of artistic criticism on software.

The 20th century in particular saw a great proliferation of critical
theory of artistic work. Among the most important ideas were
The New Criticism and Structuralism. Both provide fine tools
for evaluating software.


The New Criticism: The Intentional Fallacy and Aesthetic Value

The New Criticism was the most important school of literary
criticism in the middle of the 20th century. An essay this length
can touch only lightly on some key ideas of New Criticism, but
those ideas turn out to be quite valuable.

The New Critics believe that once the authors finish their works,
they disappear from the milieu in which the work exists. New
Critics do not involve themselves in what the author intended to
create; they concern themselves only with the work as it exists.
This principle is called the “Intentional Fallacy”: what the author
intended has no part in the value of the author's work.

Creating software can be seen in a similar light. Once the
software is released, the team that created it cannot control how
the software is used, nor who uses it, nor what it is used for. The
value of the software must lie in the software itself and in how it
is used, not in how it was created or in what it was intended to do.

The most important tool of the New Critics is a technique called
“close reading”. From Wikipedia: “close reading describes the
careful, sustained interpretation of a brief passage of text. Such a
reading places great emphasis on the particular over the general,
paying close attention to individual words, syntax, and the order
in which sentences and ideas unfold as they are read.”

Creating software has its own kinds of close reading: code
review, unit tests, and feature tests are all ways in which software
creators emphasize the particular over the general. But one of the
most important results of close reading, and one of the most
important aspects of value to New Critics, is to identify and
examine the works for ambiguity. These ambiguities are
examined for their value: some kinds of ambiguity add to the
value of the work; some kinds of ambiguity detract from the
value of the work.

Software whose ambiguous features allow the user to do more
than the developers intended will be more valuable-- think of a
wiki. Software whose ambiguous features stop the user in his
tracks has much less value-- think of the Windows Vista security
regimen.

Aesthetic Value: Unity, Complexity, Intensity

Monroe Beardsley, who wrote “The Intentional Fallacy” along
with William K. Wimsatt, also proposed a way to measure the
value of a particular work. Beardsley states that the value of the
work resides in three criteria: unity, variety, and intensity.
Applying these criteria to software is an interesting exercise.
Software may be unified if all of its features support a central
activity or theme. Software may have variety if the features cover
a wide range of activity. Software may be intense if using the
software is a compelling experience.

Periodically examining a software project throughout the
development process using these criteria is a fascinating exercise.

Structuralism: Signs and Myths

Structuralism became popular later in the 20th century. Where
New Criticism seeks to examine particular works disconnected
from intent or affect, Structuralism is rooted in linguistics and
anthropology, and seeks to examine works in the context of their
language and of their culture.

Two aspects of Structuralism are of particular interest: the idea
that linguistic signs can be decomposed into the signifier and the
signified; and that works can have deep structures that reflect
cultural values that can be represented as myths.

Signs: Signifier and Signified

A linguistic sign is a speech act: a word, a sentence, a poem, a
book. It is not at all unreasonable to treat the use of a software
feature or a software user interface as a sign.

The signifier is the “sound pattern” of the sign. It is what
happens in the physical act of speaking, or in the personal act of
reciting to ourselves. Furthermore, signifiers are the tokens by
which people communicate. Signifiers are what people send each
other as they participate in culture.

The signified is the “concept or meaning” of the signifier. The
signified is the true act or true feeling or
physical/emotional/apprehended manifestation of the signifier.
With the ability to separate signifier and signified, the
Structuralists look for common deep elements and mythological
underpinnings of artistic and cultural works.
The value of any particular work, then, can be evaluated in
several ways. A work might have value to a Structuralist if:
  • Signs are rich and easily available
  • Signifiers in the work are rich and pleasant
  • Signified are deep and clearly delineated by their
  • The work reflects a rich understanding of the cultural mythology and milieu
Exercise: New Critical and Structuralist Evaluation of Browser Behavior

New Critics contribute the idea of close reading in order to
evaluate unity, variety, and complexity of a work, looking for
ambiguity in the service of a rich aesthetic experience.

Structuralists contribute the ability to separate signifier and
signified within a work, in order to evaluate the richness of both
the experience of the work itself and the extent to which the work
relates cultural mythology, either that of the work itself or that of
the consumer of the work.

As a thought experiment, use these tools to measure the value of
this behavior:

When Internet Explorer opens a web page that contains MPEG3
files (for instance
http://www.drummingstyles.com/Genres/Latin/Bossa-
Nova/index.html), it by default prevents the user from loading the
page, and pops up a warning message that reads

(IE6 and IE7) Do you want to allow software such as
ActiveX controls and plugins to run?

(IE6) A script is accessing some software (an ActiveX
control) on this page which has been marked safe for scripting.
Do you want to allow this?

(IE7) This website wants to run the following add-on:
'Windows Media Player' from 'Microsoft Corporation'. If you
trust the website and the add-on and want to allow it to run,
click here...
A New Critic can use close reading on this behavior to see if it
exposes any ambiguities. If those ambiguities contribute to the
unity, intensity, and variety of the experience, then the behavior is
valuable.

Since the only executable files on the page are mp3, the false
reference to ActiveX sends the reader down a false path. Since
there is no explanation for why an mp3 is considered to be
ActiveX, this falseness only detracts from both the unity and the
intensity of the experience.

Furthermore, since the software requires validation of behavior
that the reader requested anyway (the false “ActiveX control” is
in fact “safe for scripting”) this behavior can only detract from
the whole point of the work, which is to play the mp3 file while
reading the text of the page.

So a close reading of the the behavior in question in a New
Critical sense shows that the software reports false information.
Although the behavior could be construed as adding variety, it
does so for no good reason, and prevents the experience of the
work itself, which is to play the mp3 files while reading the page.

A Structuralist would have a different interpretation. The
references to ActiveX are a signifier, but it is unclear exactly what
is being signified, since the obvious signified (a real ActiveX
control) does not actually exist.

The Structuralist finds a clue in IE7, where under some
circumstances, when loading a page with mp3 files, warns “This
website wants to run the following add-on: 'Windows Media
Player' from 'Microsoft Corporation'. If you trust the website and
the add-on and want to allow it to run, click here...”

Microsoft itself is concerned about security, or at least about the
appearance of security. Perhaps the false warnings are intended
to warn about the possible insecurity of the Microsoft Windows
Media Player itself.

Looking at the wider culture, everywhere from airports to banks
to offices, there is a demonstrable trend toward what Bruce
Schneier calls “security theater”. Security theater is
“countermeasures that provide the feeling of security while doing
little or nothing to actually improve security”.

A Structuralist will see that the culture that produced this work
values “security theater” while providing only the appearance of
security, so the Structuralist will concede a certain value to this
Internet Explorer behavior that a New Critic would not.

Citations for part one


Sunday, May 11, 2008

The Software Artists: Introduction

The people who create software are not factory workers. Nor are they engineers, in the sense that engineering is the “practical application of the knowledge of pure sciences, as physics or chemistry”. But the software industry continues to treat software workers as if they were factory workers or construction workers. The software industry also attempts to value software as if it were a manufactured product.

But making software is a fundamentally creative process, more similar to performance than to manufacturing. Like art and music, software has an audience that is involved in a personal way with the software. And the people who create software are much more like performers than they are like construction workers.

If it is true that software is much more like art than it is like manufacturing, then the tools of artistic criticism should be useful for evaluating software.

It should also be possible to apply successful approaches to art or music pedagogy to software pedagogy.

Furthermore, it should be possible to manage software projects in the same way that artists manage performances, with better results than if the software projects were managed like manufacturing or engineering projects.

The Software Artists: Explanation

I wrote a paper some time ago to submit to the Conference of the Association for Software Testing, but the paper was not accepted for the program. I'm on the waiting list if another presenter drops out, which seems unlikely at this point.

I think it is likely that my paper was not accepted at CAST was because it is somewhat similar to a presentation from Jonathan Kohl and Michael Bolton, two of the best testers in the universe. I intend to publish my paper here on my blog in the hopes that Jonathan and Michael and other Software Artists will have access to as much relevant information as possible to support their position.

I think that one enormous reason that few people take software-as-artistic-practice seriously is because of a perceived lack of practical application: manufacturing and engineering provide venerable examples of measurement tools, education curricula, and market strategy-- assuming that you believe that software is an engineered and manufactured product. Those of us who believe that software is an artistic process have failed to provide compelling alternatives to the tools provided by engineering and manufacturing.

I set out to start to remedy that situation with this paper.

It is a long piece, so I'll publish it in several sections in the coming days, as I have time to get it right and looking good. I have extensive references also. My plan is to publish references as separate blog posts after each section, so that people who don't care can skip them and people who do care can relate them to recent posts.

Friday, April 18, 2008

technical debt as impedance mismatch

My colleague Matt Heusser will be hosting a peer conference on "technical debt" later in the summer. I've been thinking about the subject and realized that technical debt could be interpreted as a kind of impedance mismatch. Impedance mismatch happens in acoustics, electric current, and many other places. Here are a couple of examples that everyone will understand:

trying to fill a swimming pool with an eye dropper

trying to get a drop of water from a firehose

I can think of two software examples from my own career, one involving tools, the other involving skill.

An application I was testing the other day has two javascript prompts in a row. Selenium correctly recognized and dismissed the first prompt, but it is not able to see or manipulate the second one. As a result, I have an unfinished automated test, and technical debt in the form of selenium hacking, manual testing, and future test maintenance. This is the eyedropper-swimming-pool example: no matter how much effort I expend, there is no way I can overcome the nature of the problem at hand. I'm forced to improve my tools in order to overcome technical debt.

In my career I have been a tester on two different COBOL systems. One was very nicely designed and maintained, the other was a total nightmare. The developers of the nightmare system were very poor coders. COBOL makes it easy to do dumb things, and the codebase was full of dumb things from people who didn't know any better. This is the water-drop-from-a-firehose problem: gross efforts were accepted when fine manipulation was required. The developers have to improve their skills in order to overcome technical debt.

If this really is a valid comparison, it might even be measurable in the way that electric current and sound pressure levels are measurable. If not, it is still worth noting that having the appropriate tools and the appropriate level of skill in the project team will likely minimize technical debt.

Tuesday, March 18, 2008

advice on a selenium problem?

I thought I'd mention this here. If anyone has a theory, please leave a comment or find me somewhere...

I'm using Selenium to drive a Javascript WYSIWYG editor. I type some text into a largish textarea; save the page; open the page again in the editor.

Upon doing this, IE7 can properly read and parse the contents of the textarea, but Firefox thinks the textarea is empty.

I can set up an environment to reproduce this for an interested party.