(image from http://weepeixin.blogspot.com/2008/06/red-light.html)
What a lot of people miss is that the
Red part of the
TDD mantra "
Red,
Green, Refactor" serves as a test of the test (or spec for the
BDD faithful). In seeing a test fail prior to the addition of code and seeing the test subsequently pass after the addition of code you learn 2 things
- The test indeed tested something that actually changed with the addition of code
- The code added had something to do with that test changing
One really appreciates this when they've been bitten -- written a test w/o bothering to see it fail, and writing code that supposedly makes it pass, but in fact did not. What can happen, albeit infrequently, is that since the test might've passed w/o the addition of new code--- the code added had nothing to with the code passing, or the test didn't test what you thought it did --- both variations of "accidental programming." Unfortunately, you won't be alerted to this since you never had the red turn into green as a result of the new code. This will break the normally "debugger free" cycle of TDD and reduce your coding momentum. I've been bitten, so I'm particularly aware of this phenomenon. Simply making it habit to always see the
red, can avoid this.
I think side effects of not being able to "test the test" occurs more frequently test last, whether you are doing so by design (non TDD coding) or by necessity (adding tests to legacy code, working on code that is not easily TDD'd). In such situations, it's often hard, or not worth the ROI to find a way to make the test fail prior to writing the test. Consequently, writing tests last can often just serve as "documenting how the existing code works", rather than testing it.
(image from http://blogs.technet.com/seanearp/archive/2007/10/02/map-active-directory-in-visio.aspx)I fell prey to this effect while putting some tests on some LDAP/ActiveDirectory code that I had to explore to get working . Normally, I strive to TDD as much as possible, but I couldn't really see how to do so in this case. I did a spike, exercising the LDAP API until it got the results I needed, then proceeding to refactor and add tests to this "legacy code" (as defined by Michael Feathers).
The test in question itself was
wrong -- it didn't test what I thought it did, but since I never saw it fail, then pass, I didn't know it. Fortunately, I had a controller test (Rails app) that touched this code which failed and alerted me to the faulty test which also should've
failed (yay multi level test coverage).
I'm reminded of Micah Martin's
Ruby Kata and Sparring presentation at
RubyConf where Micah did an exercise of live coding w/critiques as a device to improve one's coding. He mostly did a by the book "red, green, refactor" TDD/BDD with Rspec. I had pointed out that he had written a test in which he never saw it fail, but went straight ahead wrote the implementation to see the
green. Now it was obvious that Micah had coded this problem before, that he pretty much knew what he was going code here, and that it was probably unlikely that there would be existing code to make this test pass, but none the less it's possible he might've written the wrong test, which would've sent him into a debugging cycle at some later point. When I brought this up, another attendee thought it was a spurious criticism, something along the lines that "we were judging his coding, not his adherence to TDD." I feel that person didn't get the significance of
Red in
Red,
Green,Refactor, but perhaps he's never been bitten.
Posts
