Free Association

If you didn’t learn it that way, test-first isn’t the easiest way to program.

But it can be learned.

1. Write the code you’re going to write anyway.
2. Test your just-written code thoroughly.
3. Keep your tests and throw away all your code.
4. Now rewrite your code.

It takes extra time, but only the first time.

Given you’re writing test code anyway, test-first will pay you back the second or third time you write the code.

If you’re not writing test code, move along, nothing to see here.

A former freelancer sends a message to a programming email list, explaining his full time job doesn’t give him enough time to update his former client’s website. And it’s on an obsolete and soon-to-be-unsupported version of the web application framework (which framework is irrelevant).

The main gotcha: he didn’t write any test code.

Most likely, the client was unable or unwilling to pay for the test code when the site was first created.

Most likely, the client will not be interested in paying for test code now.

The irony here is that had the developer originally written test code, he would have much more confidence updating the site.

The client pays regardless: either someone comes and cleans it up, or the client pays for a new web site. At this point, not writing test code increases developer time as the new developer reverse engineers the existing code. Might as well test.

Test-driven development is often synonymous with test-first development, but perhaps that’s a little too restrictive. Everybody knows it’s not always possible to write test code first, thus the notion of a “spike.”

Here’s one way to do it:

1. (Best) Test first when you can.
2. (Almost as good) Test immediately after writing the code.
3. (Still pretty good) Fix anything broken before pushing to remote.
4. (Better than most) Push no uncovered code.

It’s worth noting that iterating into a spike by writing and continually rewriting tests results in a “covered spike.” It’s not test-first, but it’s hard to argue it’s not test-driven.

Taking test-first as an optimal strategy for development, we can redefine test-driven development as “push no uncovered code.”

Programmers who don’t write test-first code typically don’t write any test code at all.

The usual reason is that testing is hard and it takes too much time.

It’s true: writing software test code does take time, and writing test first code seemingly takes even more time.

But here’s the thing: everyone tests their code, if by no other technique than using visual feedback.

Visual feedback is manually operating the software and watching it with human eyeballs to “see” that it works correctly.

Typically this follows a code-compile-rewrite cycle. In the browser, it’s code-reload-rewrite.

It’s horribly time consuming.

Test first simply formalized this cycle.

Here’s the thing: just as it’s ridiculous to assume that the code will be written correctly on the first pass, its just as ridiculous to assume that tests supporting TDD will magically appear as the correct tests on the first pass.

Getting the tests right requires iterating the tests as the code is iterated.

The difference being that once the code does what it’s supposed to do, the test is in place.

Think of the tests in test-first as tools to help speed up learning about the problem.

When these tests take time, it means the problem is not well-understood.

I claim writing view specs or view tests is not difficult, not at all.

However, writing testable views is a lot harder.

And nobody bothers doing it.

Over time, good programmers build up a library of working code which gets ported from project to project.

Having a library of test code makes this even easier. Instead of messing around with the code, port the test code first. It’s much more fungible.

The received opinion in the software engineering test driven development community is “never test your framework.” This is good advice. And like most good advice, there are exceptional reasons why framework testing might be appropriate:

1. If the framework is undergoing rapid development and is not well-covered in tests, ensuring the framework acts as assumed before running your own tests might save a lot of debugging time later. This is especially true when the API is changing fast. This is critical if you have to patch your way around a bug in the framework.

2. When you don’t understand exactly how the framework behaves, a few tests will help clarify what’s going on. For example, a recently popular CSS framework “decided to be helpful and sneakilly [sic] stuffed jquery into the minified <framework>.js file. Everything worked fine except the ajax:whatever call backs when using remote => true.”

3. When the same test could be misconstrued as a framework test but is really an application test. For example, in Rails, it’s testing associations can be considered a framework test when employed between two models with a straightforward relationship. However, once a large number of models start interacting in complicated relationships, explicitly testing associations will ensure everything is connected correctly. Technically, possibly unnecessary. Practically, these tests provide early warning when associations are changed during development.

Where to test a framework

Some guidelines:

• If it’s mission critical, have a small, separate suite of tests which run independently.
• If it’s for clarity, or for watching something liable to change in the future (such as the ongoing webrat versus capybara matchers issues), drop a framework test right in with a particular spec.

There is a fine line here, easy to cross, but it’s also a line as much in the eye of any particular beholder. That is, when and where to add a few framework tests depends a lot on the developer’s opinion.

Also, I may change my opinion completely in the future, but this is what makes sense to me right now.

Here’s an interesting observation I’ve made as a result of reading (over the last 10 years) the pros and cons of test driven development (TDD). The discussion seems to eventually get this point:

• “TTD is a crock. I just don’t see the need to fully test everything. It’s a waste of my time.”
• “TDD is awesome. We found TDD really helped speed up our development and ship better product.”

Feel encouraged to draw your own conclusions.