Monday, April 23, 2007

but, but, but... that's black box testing!

Brian Marick has been thinking about test frameworks in Ruby.

It's nicely done, but the killer idea is in the next-to-last paragraph that says:

In this particular case, the app communicates with the outside world via XML over the network, FTP files, and a database, so there’s no need to link the fixtures into the code. (emphasis mine) If it worked through a GUI, some remote procedure call interface would have to be added, but I persist in thinking that’s not too much to ask.

This has been my approach to test automation for some time, but it's hard to get across the idea until you've done it and succeeded once or twice.

I'll go even further, and stipulate that there is always a way to slice the application such that a test automator can write fixtures without reference to the underlying source code, and still provide effective, efficient system-level testing.

Thursday, April 19, 2007

fine tools and beautiful machines

My new laptop for work arrived yesterday, and I've been spending the day getting it set up:

I've used Windows machines at work for my whole career, but I've always had Macs at home. It's going to be a real pleasure to use a Mac for work.

I had a birthday recently, and my wife completely surprised me with one of these, an Ibanez AG86:

It was on the wall at the local music store, and I would pull it down and play it every time I went in. A really well-made guitar, a pleasure to play, and a good value. Guitar is not my main instrument, but I've been motivated to try to cop a few Wes Montgomery tricks.

This computer and this guitar are both fine tools and are both beautiful machines. There is a time in every student's career when they have mastered enough skills that an excellent set of tools is required in order to advance. This seems to be that time.

Thursday, April 12, 2007

i've always wanted one of these



and now I have one of my very own.

RadView WebLOAD now open source

This is great news, and should really shake up the tools market. Not only is it open-source, it's released under GPL, which is a pretty radical move.

Open source load test tools have been kinda sorta OK for some time, but performance testing is the only case where commercial alternatives are pretty much universally acknowledged to be superior to open source tools. Building good performance test tools is hard. What RadView has done has instantly changed the whole scene.

I am no kind of performance test expert, but I've worked a little bit with the standard open source tools for doing that work. The next time I need one, I have no doubt that I'll reach for WebLOAD first to see how it works.

Thursday, April 05, 2007

make a change to Ruby

OK, I can't resist Pat Eyler's "Bloggin Contest"

I started programming in Ruby after having programmed in Perl for a while. There is not very much I miss about Perl, but there is one thing: in Perl, you can locate a subroutine anywhere in the script; in Ruby, any call to a method has to be located physically below the definition of the method. (Yes I know that there are reasons that Ruby's interpreter does it that way, but still...)

I write a lot of scripts that are run by non-technical non-programming users, like QA people and business analysts. The nice way to set up such scripts is to put all of the editable bits right up at the very top of the page, so that the users can ignore all of the programming guts below the important variables and such. This is easy in Perl. In Ruby, I am forced to either put all of the editable parts at the very bottom of the page, so the non-technical user has to scroll through a lot of intimidating code to get to the important stuff; or I have to distribute a separate library file and use 'require'.


Here's examples:

########################
#Perl
sub i_am_at_the_top {
print "I'm at the top\n";
}

i_am_at_the_top();
i_am_at_the_bottom();

sub i_am_at_the_bottom {
print "I'm at the bottom\n";
}
#########################
yields:
>perl -w subs.pl
I'm at the top
I'm at the bottom

#########################
#Ruby
def i_am_at_the_top
puts "I'm at the top"
end

i_am_at_the_top
i_am_at_the_bottom

def i_am_at_the_bottom
puts "I'm at the bottom"
end
#############################
yields:
>ruby defs.rb
I'm at the top
defs.rb:7: undefined local variable or method `i_am_at_the_bottom' for main:Object (NameError)
>Exit code: 1

If Ruby's interpreter would make all methods available for use at runtime, it would make communication with non-technical script users a lot more appealing.

Tuesday, April 03, 2007

it wouldn't hurt you to use the compiler, either

I surf the O'Reilly Network blogs pretty often, and this one from Chris Josephes caught my eye, especially the part where he says of system administrators
"I don’t expect them to be a full fledged C programmer, but I think it’s important to know how to build software. The candidates that have demonstrated these skills have also been more proficient in debugging, tracing system calls, and identifying performance problems ahead of time."
I think the same is true of testers.

If you are a tester who has never used a compiler (or even if it's been a while), I have an exercise for you to do the next time you need inspiration for a new test.

Find your local build person, or a sympathetic developer, and have them send you the compiler log(s) from building your particular application. Do text string searches in the log files for the terms "warning" and "error". Unless you are working in an extremely mature organization, I'll bet you find some warnings and errors from the compiler.

Those errors and warnings point to places where bugs are very likely to live.