What's the impact of how long your tests take?

Preface: There has been a lot of talk lately about whether or not Test First is Dead.This is a complicated and tangled issue, so I want to simplify many aspects of this debate in this blog post to focus on a single aspect of TDD: 

Test Run Time.

So for this blog, let's ignore the distinction between unit, integration, acceptance tests, even manual tests. I'll lump them all together into a single box - 'tests' so we can focus on the 2nd benefit of Tests: Feedback

How fast is fast enough for you tests to run? There is a lot of discussion. minutes? seconds? fractions of a second?

Usually there is a very important step skipped in this discussion. People never seem to ask:

What does it matter how long they take?

Cycle Time

Have you ever needed to make 'just a small fix'? You change a line of code. Compile, save, deploy (maybe to your phone or websever) look at result. 

"Damn." That didn't actually fix it. You change the line again, Compile, save, deploy, look at the result... "Damn". 

4-5 tries later and it's fixed. less than 20 characters typed. Less than 60 seconds of work and yet the clock says it's an hour later than you started.

What is going on here?

Again a lean graph is helpful in mapping out and understanding this system. 

  The problem is for every 10 seconds of work you do, you have to wait 10 minutes to get your feedback. This is a highly infective way of working, which is why we don't do it that offend. 

Although this phoneme is well know enough to inspire xkcd comics

To discuss this part it is better to use the repeat notation to display the above lean graph

Work to Wait Ratio

Now, we tend to want high feedback but we also want a good work to wait ratio. I find that in practice we want at least a 3:1 work to wait ratio.

That means if it takes you 10 minutes to get feedback (test it out) then you will probably end up coding in 30 minute or longer chunks before running the code. If compiling and running takes you 3 minutes you'll work in at least 9 minute chunks.

This allows a lot of time for mistakes to fester and complexity to compound.

A lot of good Test Driven Developers I know like to run their test's at least 3 times a minute. That means 20 seconds per cycle, which means an absolute max of 5 seconds per run to achieve the minimum work to wait ratio.

But this changes it get to the 'very very quick' stage. If you are typing in word it underlines typpos  as you go. This might seem like a minor connivence over the 'Check Spelling' feature but, then again, when is the last time you remember running a spell check manually? 

It worth noting I know many legacy systems where the compile time alone can take 3 hours. That means if I come back from lunch and program for 1 hour, I will not be able to see the effects of my changes before I go home for the night.

Discovery 

One last thing to consider is the aspect of discovery when creating. When you are trying to be creative and discover 'the right thing' then the immediacy of feedback is very very important. This is why many people find themselves preferring things like paper, whiteboards,  wysiwyg editors when they are brainstorming. 

This is part of why you are seeing more things like continuous test runners and website monitors that automatically refresh on file change. 

You'll find that the longer the cycle times the more deliberate each action becomes. This drastically lowers the chance for a joyful discovery. Not many people played around with punchcards.

If you are interested in this aspect of feedback, I suggest you check out this talk [inventing on a principle] by Bret Victor. Who has though about this more than anyone else I know of.  

Test First vs Test After

There has been a lot of talk lately about whether or not Test First is Dead.
This is a complicated and tangled issue, so I want to simplify many aspects of this debate in this blog post to focus on a single aspect of TDD: time.

So for this blog, let's ignore the distinction between unit, integration, acceptance tests. I'll lump them all together into a single box  - 'tests'.

Let's focus on the speed aspects of Test First vs Test After.

Test After

To start, we'll look at the time it takes to write code using test after.
I'm going to use a lean graph to help map the differences between 'actual work' and 'non-work'. I get paid to write code, not tests, so I'll map code to work and tests to 'non-work'.

In this example, it took me 60 minutes to code a feature. Then more time to write the tests for that feature.

Now for the key question:

"Is it faster to not write the tests?"

In other words, which is true:
 a)  60  < 60 + X
 b)  60  > 60 + X

This is obviously  a)  60  < 60 + X  regardless of how long or quickly you can write the tests.
This doesn't mean that in the long term it isn't faster to write tests, but it does mean in the short term, if you write tests after, it will always take you longer.

so what about...

Test First

When you have a test before you start it can make it easier for you to write the code. This is because the Tests can aid with Specification & Feedback of the task at hand. This saves some time. Again, we'll assume for this example that it saves me 30 minutes while writing the code.

Now for the key question:

"Is it faster to not write the tests?"

In other words, which is true:
 a)  60  < 30 + X
 b)  60  > 30 + X

This is a more difficult question. It depends on how long it took to write the tests (what the value of X is). If X is less than 30 minutes it is faster. If X is longer than 30 minutes it is slower.

This means that the techniques, technologies & fluency of the programmer all affect the result of this equation. So it is not a clear cut answer & it will vary depending on the individual.

Conclusion

For me, it is often quicker to write the test. I have the advantage of years of fluency and tools like approval tests. However, the interesting point to me is while Test After means:  writing tests takes more time

Test  First means: writing tests may or may not save you time

I believe this possibility is worth exploring.

Intention and Indebt

Today I have been reading a lot about technical debt, and I keep seeing terms like

“… at this point they choose to take on some technical debt…”

I have never seen a programmer make this choice.

Which is not to say I have never seen a programmer take on technical debt.

Allow me to explain with another metaphor:

Weight.

I have never said, “I think I will eat this piece of cake to gain some weight”

I have eaten cake. I have not watched what I ate. I have not exercised afterward. As a result I have gained weight. But it was never an intentional choice. It was just the side effects my actions. It was NOT intentional.

Likewise, I have seen programmers:

• Use bad names
• Add lines to long methods
• Add an additional If block
• Add more methods to large classes
• Comment out a section of code
• Skip writing tests
• Skip refactoring
• Ignore extracting a common interface

But I have never seen a programmer  “take out a loan”. 

Technical debt isn’t like a mortgage, a large decision you make with a bunch of thought.  It’s more like a credit card or a bar tab. You just keep coding and coding, little by little, and then one day you realize you have a large amount of debt.

(Side note: I have quite often seen people realize that they are in debt and then actively decide not to pay off that debt; You could argue that this is a form of "intention" but deciding to keep your debt is not even close to deciding to take on debt.)

On the flip side, I see quite a lot of intention by people who say fit and slim. They tend to actively watch what they eat both quantity of food and type of food. They make a habit of exercise. As a result they tend to think that everyone else does it to. After all, you choose to eat that ice cream, right? Yes, but I didn’t choose to get fatter, I actually didn’t consider my weight at all when I ate that ice cream.

And that’s the point, when we analyze why people choose technical debt, we tend to be missing that point that the vast majority of people DON’T CHOOSE technical debt, it is a side effect of their actions, but never part of the decision.

Dependency Injection vs. Dependency Injection Framework

Yesterday, David Heinemeier Hansson wrote a blog: ‘Dependency injection is not a virtue’. There is a lot of things mixed together in this blog, all finally put together with the statement “I'm a Ruby programmer”. I have found this moves people into more of a sports team cheering mindset and does little to help keep conversation rationale and productive.

So let's take a step back, and start to unpack the meaning behind 'dependency injection'

In the article, DHH uses the ruby example of time, saying that

Time.stub(:now) { Time.new(2012, 12, 24) }
article.publish!
assert_equal 24, article.published_at.day

is preferable to

article.publish! Time.new(2012, 12, 24)
assert_equal 24, article.published_at.day

Let’s skip the opinion part of which is preferable, and start by pointing out that both of these actually are examples of dependency injections, albeit via different implementations [Monkey patch vs. parameter passing].

So I wanted to break down the first misconception my experience has shown to be common in the programming community; The confusion between Dependency Injection and Dependency Injection Frameworks.

disclaimer: as each framework is unique, this is a fairly blanket statement I am about to make. It may not apply to your framework.

Dependency Injection (Concept)

Dependency Injection (DI) is a general concept which addresses polymorphism. The idea is that you will have different implementations for things, and you need some way of switching which implementation you use. The programming world needed a label to refer to that concept and chose Dependency Injection. There are many many forms of Dependency Injection, and they offer different pro’s & con’s for different scenarios. Off the top of my head, I can think of quite a few methods for Dependency Injection:

• parameters
• inheritance
• mocks
• byte code manipulation
• Global Variables
• Factory Pattern
• Callbacks
• Inversion of control (IoC)
• Monkey Patching
• Dependency Injection Frameworks
• Reflection

There are many more and as programming evolves many more will be created.

Dependency Injection Framework (Post Compiler DI)

Dependency Injection Frameworks, on the other hand, have seemed to evolved to support a specific type of scenario. Namely

“How do I inject a polymorphic instances after compilation time?”

You might ask, why would I want to do this? There are many answers, the best being Plugins (You do not want to have to recompile your web browser to allow for a plugin to work). There are, of course, many more reasons.

With non-compiled languages, it get’s even weirder to think of “compile time” but the concept is still valid, especially if you are not at the top of the runtime stack.

Dependency Injection Frameworks usually achieve their DI via a combination of factories, reflection and some sort of runtime configuration setup (files and metadata are common).

and they can be a bit of a heavy handed solution if you are using them for DI when you aren’t really concerned about what happens after compilation time.

Was DHH talking about Dependency Injection Frameworks?

It is worth noting the DHH never actually mentions dependency injection frameworks. Although, there is definitely a lot of mention of them is the conversations afterwards. There is a lot more in this article that is worth talking about, but let’s save that for a different blog…

Better Carpools with Compossible Cars

Todays random thought is about how to make car pooling better.

The Trouble with Car Pools

This week I have to drive 60 miles to a client down in San Marcos. This is a long drive, I would appreciate both the gas advantages of a car pool, but also the time savings of being able to be productive while driving. However, to do this I need someone near by, who will drive to my house (or vice versa) then drive to my work, and then pick me up. This is a bunch of extra’s and lot’s of little things that if they go wrong will ruin my day. It’s just too much trouble.

 

but what if it didn’t have to be this way?

Image if our 2 cars could become 1, for a period, then split apart again. Then we could both drive to a common meet up, join up, ride together until we covered the common portion of our trip. spilt apart and not have to worry about getting stuck if one of us needs to stay late at work tonight.

Here’s a diagram of the idea:

The nice thing about this is not only do you get to share the burden of driving, but also gas mileage only goes up slightly with weight. From my short search, doubling your weight (2 cars) would only remove about 10% to your gas mileage. So a 34MPG Smart Car should get about 30 MPG combined with a second car.

Generic Type Information at Runtime in Java

I’ve been playing around with Hadoop lately, and it seems that this is a situation when the lack of runtime information about generics really hurts.

The bad news: java isn’t going to be changing that anytime soon.

The good news: you don’t need to wait for them to do it.

       Here’s how.

1. Define Useful Generic Base

The secret here is to add a method per generic type to expose it’s type. Here’s an example

public interface UsefulMap<In1, Out1>
{
  public Class<In1> getIn1Type();
  public Class<Out1> getOut1Type();
}

2. Implement

Now when you implement, you will have to provide the runtime information.

public class MyMap implements UsefulMap<Double, String>
{
  @Override
  public Class<Double> getIn1Type()
  {
    return Double.class;
  }
  @Override
  public Class<String> getOut1Type()
  {
    return String.class;
  }
}

The nice part is, if you forget or change it, it won’t compile.

  public Class<Double> getIn1Type()
  {
    return Integer.class; <- Type mismatch: cannot convert from 

                             Class<Integer> to Class<Double>
  }

Once you do this, there are a bunch of convenience functions you can write that will make your setup much easier & safer.

For example, you can now write

HadoopJob.Create()         

         .WithMapper(new MyMapper())

         .AndReducer(new MyReducer())

         .Start();

This (new) developers life

Today I discovered the podcast “This Developer’s Life”. It’s a very differently crafted podcast. Very true to the style of  "This American Life” that it’s name is borrowed from. It is more in a story telling venue, better production quality, not about API’s or even craftsmanship, but rather “common” themes to lives programmers.

I listened to Episode’s “Play” & “Problems”. They are nice, but it stirred in me an issue that first surfaced last year at Tech Ed South Africa, when I attended a presentation that presented this clip from the Movie “The social network” about being “wired in” and then stated this is the best way to be productive. I remembered nights or days with my headphones on, programming in the zone, but I could NOT remember the last time I actually programmed with headphones. I don’t work that way anymore.

“Play” also described this state, but mentioned that it was hard to get into with programming. They argued that racecars could instantly move you into it, but code was more haphazard about when you’d get into the zone.

“Problems” talked about what I consider to be the opposite effect;Bugs and the frustration and confusion they bring about. They also talked about the lack of satisfaction when the bug is finally found. Techniques to build up success, when to quit or undo.

The Old Me

I remember a time when this felt very familiar. In fact most of my younger years identified heavily with these anecdotes. I use to say “you have to love programming to be able to survive because bugs are so frustrating.” However, this whole thing was more of a nostalgic experience for me. In fact, I found myself shaking my head often, wanting to correct the process. For example, In the “Problems” episode, when they are talking about the bug introduced while programming in Sinatra, all I could think was “use Git Bisect”.

I’m not sure exactly when the changes started and I became proficient enough in my new skills and attained enough discipline that I stopped having these experiences. And I should note that they haven’t stopped completely, I still have relapses sometimes, but they are much much fewer and father apart.

But I do know why I have changed.

The New Me

Nowadays, I am almost constantly in the zone when I’m programming. I rarely fight with a bug for hours, and almost never if it’s an regression bug, in fact 10 minutes is a long time for a regression bug.

I am more disciplined than ever in my code, and part of that is focus, and part of that is small steps building up success.

I also am frequently in a state I would describe as “brain on fire” a state of hyper creativity that makes the old “in the zone” feel a bit stale.

The changes have come from from eXtreme programming. I started on this road almost a decade ago in 2002, and I have to search back to around that time to identify with the concepts and stories in the podcast.

There are many practices in eXtreme programming that contribute to the overall effect, and the combine upon each other to get the full result, but I want to highlight the most useful ones. I’ll do it in order of value for me personally.

Pair Programming

When they talk about the zone, pairing brings that almost instantly. I am amazed at how focused I can be when pairing with someone. Email, twitter, other people… it all fades into the background, and the code pops in front of you. Even if a interruption occurs it is quick & easy to regain that focus.

Learn more about pair programming, here & here.

Test Driven Development (TDD)

A clearly stated problem, constant feedback, and regression about twice every minute! TDD has removed the massive hell that bugs by breaking things into nice little steps, and catching you if you slip. Nowadays when something goes wrong I know about it 30 seconds later, and Ctrl+Z is usually the answer.

“The journey of a Million lines starts with a single Unit Test.”

Learn more about TDD here, here & watch a video demo here.

 

Incremental & Iterative Development

TDD sort of forces you to do small steps in a iterative fashion with constant feedback, but eXtreme programming doesn’t stop there. It uses this technique with user stories, refactoring, continuous integration, source control, retrospectives and many other practices. It’s extremely powerful. Do something small. No smaller. Now add some more.

How powerful is smaller? Let me show you an example from my own history

Books written – 0
Blog Posts – 25
Tweets - 2,621