Tuesday, March 22, 2016

The coordination tax of working alone

If you have team members working together on a project but separately on their own piece of the project then you need to coordinate that work.

There are lots of interestingly different ideas on how much 'cost' there is to this coordination. Ranging from it's a very low overhead, to it requires a full time manager who is the highest paid person on the team. Often these drastically different opinions are held on the same teams by the same people.

What I want to explore today however is what does the actual coordination looks like and what it would mean to get it right.

To start off with, there are a few types of coordination:

Multi-functional coordination

If someone is building the database and someone is building the UI those things need to talk to each other; usually through a middleware build by a third person. If they don't line up the costs are large indeed, ranging from "we need to redo this" to the far worse "Let's ship the wrong thing". 

The coordination includes:
  • knowing what each piece looks like
  • which functionality goes where
  • how to connect them
If you know the answers to all of this, then I believe you can make a fairly good assembly line fashion process. 

However, I would like to state categorically 
"If you have not yet built it, you don't know everything involved"

or as the great Woody Zuill said

"It's in the doing of the work that we discover the work that needs to be done"

And I don't mean done 'something like it'. If you've ever done a remodel on your house you will notice the estimates don't match the end result. It's not that this is the first remodel the contractor has ever done. Each situation has it own unique unknown unknowns.

Unfortunately, in software we are always doing something new because duplication is a copy and paste away.

Coordination of these types of tasks is expensive. You need an level of detail and level of oversight that is time consuming to obtain. If you don't get it right there is a lot of waste in the process that is silent.

Multi-person coordination

Even if you have people with the exact same skill set you will often have them working separately in an effort to maximize throughput. Here the coordination is in reducing duplication and keeping the code united. You don't want different people creating the same things or worst the same things in a different way where it's hard to realize they are even the same things. Ideally you want everyone producing unique work in the best possible way.

Perfect coordination 

I think perfect coordination is easiest to picture in a single entity. This entity is aware of the details and the larger picture. It has the skill sets for all aspects of the code. It incorporates new knowledge and processes  the moment it learns them. It has a memory of the previous work and knows when it should reuse pieces. It has high self awareness and doesn't have to spend a lot of time in meeting talking about work.

For many small projects this entity has been a single person.

I hypothesise that it is also the reason a Mob can be effective.


Sunday, March 6, 2016

Two people working on the same thing

One of the most common objections to Pair or Mob Programming is

"How can more than one people working on the same thing be productive?"

Let's put pairing that aside for a second though and look at common scenario:

Duplication in the average code base

You have a fairly big system, maybe 300,000 lines of code. How much of that is some form of duplication? Keep in mind that duplication is more than just copy and paste. It's anytime you need to change something in more than 1 place. This inconsistency is particularly insidious because it means new creative solutions for each one, plus a harder time identifying the problems later.

Note: I have seen cases of over 1500 instances of duplication in a single code base of this size. (If this seems familiar, hire me :-)

This duplication is one of the main causes of technical debt.

Obviously, you should remove this. You can see some techniques to do just that in our Practical Refactoring video but let's instead focus on the reason this duplication get created in the first place.

Keep in mind that often this duplication isn't written by the same single person. Often in is many people, over many months. Often they aren't even aware there is simular code somewhere else in the system. Who has time to just read over 300 thousand lines of code? [this is roughly 15 copies of the last Harry Potter Book] 

I doubt anyone would actively say "Please have a couple people write the same thing over 4-5 places in different ways". But, of course, no one is saying that. These areas of duplication are smaller than the tasks that are ever discussed. It's just something that happens but is unaccounted for. So let's account for it:

Time to do the Math

We need an amount of duplication which is a bit of a problem as there isn't a nice industry number for this. So I'll use a range from 5%-50% for the common scenario of 300,000 Lines of Code (LOC). 


300,000 * 0.05 = 15,000 LOC. 
15,000 LOC / (100 LOC/Day) = 150 days to create.


300,000 * 0.50 = 150,00 LOC.
150,000 LOC / (100 LOC/Day) = 1500 days / (300 day/year) = 5 years to create.

Meaning the average project has inadvertently budgeted between 1/2 man year to 5 man years to have 2 people work on the same thing for the purpose of making the code worse.

For which I have to ask:

"How can more than one people working on the same thing to be productive?"

Thursday, March 3, 2016

Defining 'Technical'

TL;DR: Technical can mean advanced, lingo and/or labs.

I'll like to a moment to get technical about the concept of technical talks...

It is often discussed before, during and after a conference about the 'right' amount of 'technical' talks. This is a very hard subject and in some of my recent discussion I started to get a glimpse into why:

"Technical is a poorly defined umbrella term."

As I started to break down what it means to be a technical talk, I started to see 3 axis emerge on aspects of a talk that can be considered technical.

The aspects of Technical

1) Lingo vs. Clear Words

"When my WCF backed by a RDMS wasn’t rendering in WPF I opened MSDN and said WTF"
If you are a .net developer this statement makes perfect sense, but everyone can understand
"When my program wasn’t connecting to the database I had to read the documentation online"

Pros: Lingo conveys large info in small words.
Cons: It isolates many people and makes them feel stupid while making the speaker look smart.

2) Advanced vs. Beginner

How many prerequisite do you need to attend this session?

If you hear the statement
"Then just apply your basic differential equations..."
you better have taken a few years of math. Whereas

"With just your basic arithmetic you can..."
is probably accessible to most people.

To be clear, I'm not trying to put a value judgement on the topic. Advanced topics don't inherently hold more value than beginner topics, but they do have more barriers to entry.

Pros: There are many great and valuable things in the advanced space.
Cons: Understanding the prerequisite most people have is very difficult and error prone. These sessions also have a limited audience by their very nature.

3) Labs vs. Concepts

Is this where the rubber meets the road or a 10,000 foot overview?

Will the presenter just mention that
"It’s saved in a settings configuration" or are we going to the command line and opening emacs?

It's worth mentioning that there is a fairly solid middle ground between concepts and labs that I would refer to as examples/demos. Like a lab where you are watching rather than doing.

Pros: Labs get to the real knowledge that concepts simply gloss over. For example: I understand the concept of flying a plane, but you don't want me to be your pilot.
Cons: Labs cover a much smaller space in a much larger time scale than concepts do. They are also very specific to a particular context that might not transfer to your own.

In the end we have a complicated 3D space looking something like this:
This is a very complicated space and most talks lie somewhere in between. However, I have found the most value from examining the 8 outermost points.

The 8 Extremes of Technical

1) Clear Beginner Concepts

This the least technical presentation.
That's not to demean the presentation. These talks for valuable and deliver a lot of (albeit shallow) understanding to a large audience.
This is the most common type of talk at most conferences.

2) Clear Advanced Concepts (GOOD)

This is technical in that you need a fair amount of knowledge to get into the conversation.
I have seen a fair amount of these talks at strangeloop. They are the kind of conversation where you understand every word but your head is still hurting.
These can also be the topics that open your eyes and stick with you for ages.

3) Advanced Concepts with lot’s of lingo

These are error prone. Like the above they make your head hurt but there is a good chance you didn't understand every word and you are too embarrassed to ask for clarification.
While Academic conferences seem to have a bunch of these the truth is they are everywhere.

4) Beginner Concepts with lingo (BAD)

I can see no use for this. Lingo implies a prerequisite for understanding but beginner states none is needed.
Obviously there is some use of this from a dark place where the speaker wants to feel superior or confuse the listener, but I believe more of this happens by accident because of
'the curse of knowledge'. Namely, once you internalize the lingo you forget that it is even is lingo; It's meaning just seems so clear.

5) Beginner Labs with lot’s of lingo (BAD)

See above, except because you are in the details the confusion is even more damaging.

6) Advanced Labs with lot’s of lingo

Interestingly, this seems to be a fairly good place for lingo. While I would still suggest brief definitions when introducing the terms (to avoid having people too embarrassed to ask) the efficiency that is provided by lingo is very beneficial here, you have people that have been in the space a while to fulfill their prereqs and you are using specific technology that needs names and labels.

7) Clear Advanced Labs 

Even better.

8) Clear Beginner Labs (GOOD) 

Beginner and Introductory talks are fantastic. They are heavily attended and help bring up new people (Doubles every 5 years means 50% of industry is less than 5 years of experience). Making these labs clear welcomes newbies and gives them a good experiences and solid learning. Labs help to make the learning concrete. I personally really enjoy these type of presentations and give many of them myself (so I'm a bit biased).
They do a have a couple pitfalls though.

  • They require more effort on the student. This is particularly bad if they are held right after lunch or competing with an easier session.
  • Sometimes you can't see the forest for the trees. Students will feel that they are missing out on the big important stuff because they are struggling with the details.
  • Experts like to discount them. "that's too basic" or "We've all seen that before" is the most common thing I hear from experienced people.

So those are my thoughts into everything that usually gets wrapped up in the term 'technical'.
Hopefully this blog post rates as a Clear, Advanced and somewhere in between Concepts/Lab.
If not, let me know in the comments.

Sunday, February 21, 2016

Don't use the greater than sign in programming

One simple thing that comes up time and time again is the use of the greater than sign as part of a conditional while programming. Removing it cleans up code, here's why:

Conditionals can be confusing

Let's say that I want to check that something is between 5 and 10. 
There are many ways I can do this

All of these mean the same thing... Wait, did I actually do all that right? Sorry, one of those is incorrect. Go ahead and find out which one, I'll wait...

If you remove the use of the greater than sign then only 2 options remain:
(x < 10 && 5 < x)
which is a stupid option because it implies 10 < 5 and
(5 <&& x < 10)

This is a nice way of expressing "x is between 5 and 10" because it is literally between 5 and 10.

It's also a nice way of expressing that "x is outside the limits of 5 and 10"
(x < 5 || 10 < x)

Again, this expresses it nicely because x is literally outside of 5 to 10.

Simple. Clear. Consistent.

This is such a nice way to express numbers I wonder why programming languages allow for the greater than sign ( > ) at all.

But why is this so expressive?

The Number line

Here's how you represent between 5 and 10 on a number line vs code:
number line of between 5 to 10
(5 <&& x < 10)

Here's how you represent outside of  5 and 10 on a number line vs code:
number line of outside 5 to 10
(x < 5 || 10 < x)

On a number line everything to the left is less than the numbers to the right, so these two ways of representing the relationship between things matches up.


This problem gets much worse as the conditional grows. For example
 number line of between -5 and -1 or between 2 and 4
((-5 <&& x < -1) || (2 <&& x < 4))

Has 15 other possible ways to be expressed if you  include the greater than sign and don't make your expressions conform to the number line.

Saturday, September 5, 2015

Mob Programming and Trivial problems

Whenever I hear about Mob Programming people always talk about how great it is for the hard problems.

But let’s explore a fictional example of some trivial work because I believe this is where things really start to shine.

At the Institute for Boring  & Monotonous work (IBMw) there was a team of 6 developers that whose large code base had all become destroyed due to a large disk crash. Fortunately, all of the code had been printed out before the destruction, but it had to be re-typed in. The estimated work was going to take the 6 programmers 8 months. It doesn’t get more trivial than this, IBMw was really living up to its name.

Nonetheless, Mob Programming was the new buzzword so this team said, “why not try it out?”. Here’s what happened.

Day 1 (Many eyes):

The team started typing in the pages. They quickly started to arrange focus so that a couple would be manning the physical paper and couple eying the projector to keep everything in tack. They were definitely slower with only one person typing, but they also realized that they weren’t making any typos. This is very important as the code had to be perfect or the whole system wouldn’t fit together again.

At the end of the day they retrospected as it is recommended for mobs. They came up with 2 big findings.
1)   quality is up
2)   some people can’t type
Idea for improvement: “Let’s spend 20 minutes tomorrow working on typing”

Day 2 (Countable Improvements)

The mob downloaded a typing tutor and started the day working on keyboarding. It didn’t make a huge difference, but  it was nice to know the slower members were working on it. Made it easier to be patient with them during the day.

But one part of the day went a bit off, something went wrong with the typing and they had to debug a bit. As they did someone counted the number of words. Paper: 304 Computer:303. Damn. Eventually they found the missing part.

At the daily retrospective, a couple of post-its showed up in the positive side with ‘word count helped debugging’

Day 3 (Checks)

They continued with the typing practice in the morning. A little progress but not much, some members had moved from 15 words a minute to 16.

The word count was really helpful so they started doing that with each part. It usually wasn’t wrong but there was something reassuring in knowing that the computer matched the paper so they kept doing it. It was extra helpful when it didn’t match. However programmers can be extra lazy and  someone complained that it sucked that they had to count the words on the computer as well. Everyone agreed, and then they suggested they write a small programmer to count the words for them. Easy enough; No more manual counting of the words.

The retrospective had a lot of sticky notes with “auto counter” written on them in the what went well section.

Day 4 ( Auto-enhancement)

It was great having the ability to know the word count when they matched up, but still a bit hard figuring out where the missing part was when it didn’t. So they would add improvements to the AutoCounter to allow for easier debugging. It was interesting to note that some of the best ideas came from the most frustrated people. Team members that never would have been able to implement those ideas on their own.

At one point, they just couldn’t get past one section.
“I know something is wrong”
“But the word count is the same, even the character count is the same”
“but look at it, it doesn’t feel the same”
“yes, but, oh the printed font is different. That’s why it feels different”

The retrospective had a bunch of “font caused frustration” in the negative side.

End of Week 1 (double check)

After typing practice the first thing suggested was to change the font to match the paper. This made the day a lot easier. They would hold the paper up and do a quick sight match. Still programmers are nothing it not lazy. The retrospective included “arm tired from holding up paper”

Week 2 (Seeing words)

The second week seem much easier than the 1st. They brought in a second projector to overlay the paper so it was even easier to see the differences, but this also got them thinking why do we need to count the photo? It was too hard to program up something to read the printouts, but it wasn't to hard to detect the words. They could figure out the boxes around words. It wasn't great but it helped.  Now WordCounter could also help a bit with the physical world.

They were still doing typing, but it wasn't helping much the slowest was up to 22 words per minute on Friday from 21 words per minute Thursday.

Oh, someone was sick and out for 2 days, didn’t really matter.

Week 3 (Word counter sees letters and bugs)

The team was able to get the boxes from words to letters. This really improved the speed of the checking and we are almost completely sure there are no bugs going in from typing now. Of course there were some bugs in the original code. We got in one argument over a weird compiler setting that has different effects on how a struct would end up in assembly code. It was rather goofy, but we added setting to Word Counter to notify us if this shows up again.

Typing still isn’t helping much, our slowest is barely making 26 words per minute.

Week 4 (Splitting Word Counter)

The team split up the Word Counter into a separate program that detects the patterns of common bugs. We only have 3 patterns so far but its pretty neat and we are removing some bad code because of it. We were also able to run this over the previous code so fixed some stuff from the first 3 weeks.

We also had an idea of how to detect the letters from the printouts, but it didn’t work.
On the failure side, typing is still not really improving. Slowly McTypesABit is only doing 30 words a minutes.

Month 2  (OCR & Linting)

This month we were able to add a lot to the bug finder. Up to 13 patterns. We’ve even shared the program with other teams to help them check their code. We also got a way to do a bit of automated reading of the characters from the printouts. It’s not very good (about 50%) but it gives us a starting point which is nice and we are doing good overall. Might even be a month or two early to finish this project.

On a side note: because of a 2 week vacation we have a new slowest typer! It’s very close though (40 wpm vs 39 wpm)

Month 3 (Done Done?)

The WordReader is still having problems (only 80% accurate) but it’s been helping us move really much faster. With the combination of BugFinder we are going to be done with the rewriting by the end of next month. The teams still has budget though and it looks like both WordReader and BugFinder might be useful outside of just helping us finish our original Task. We are looking into making them more robust…

Wednesday, April 15, 2015

Making Things Smaller

Often, when people talk about minimum viable product or story splitting we hear the question

"but how can I make this smaller?"

I'm going to show a very simple, powerful and yet often overlooked strategy for doing this.

Use smaller numbers

If your scenario has numbers in it make them smaller. You can do this in 2 ways depending on what you are trying to accomplish.

Minimum Viable Product

When you are trying to get to the smallest thing start your numbers near 0 or 1.

Here are some examples:
 "This report should do 5 things" -> "This report should do 1 thing"
 "Get data from 3 sources" -> "Get data from 1 source"
 "The UI changes these 8 things" -> "First we will change this 1 UI feature"
"We check in code every 8 hours" -> "We check in code at least once an hour"

Sometimes people 'hide' the numbers
 "Handle English, Spanish & French" = "Handle 3 languages" -> "Handle 1 Language"

Also, remember that programmers tend to think in terms of only 3 numbers: 0, 1 & many
 "Save directories" -> "Save many directories" -> "Save 1 Directory"

This also creates a very nice step from 1 to many of 1. In programmer that is:

"int postalCode = 90210"  -> "int[] postalCodes = {90210}"

Minimum Viable Change

When coaching you usually want to go the opposite direction so that the change is less noticeable and easier to absorb without conflict.

 "Our sprints are 4 weeks" -> "Our sprints are 3 weeks" -> "Our sprints are 2 weeks" -> "Our sprints our 1 week" ->  "We release weekly" -> "We release daily"

 "Our Build has 9 manual steps" -> "Our build has 8 manual steps"
Sometimes this shows up as negative numbers:
"No teams are writing tests" = "8 teams aren't writing tests" -> "7 Teams aren't writing tests" = "1 team started writing tests"

The Point is... if you start making a habit of looking for numbers in your stories and processes you will see them everywhere and then they are easier to reduce.

Monday, January 5, 2015

Women in Tech & Men in Zumba

I have been a long time supporter of women in tech but today I realized how little I truly understand their situation.

Today my girlfriend brought me along to a Zumba Gold class. This is a easier version of zumba and often is frequented by older (60+) women. We were the youngest there by far.

I was the only man.

I'm a good dancer. Proficient in swing, salsa, cumbia & merengue which is basically what Zumba is. Youth is also an advantage in most physical activity. This should be very easy for me. Also, everyone was both very nice, complementary and welcoming. 

I was the only man.

I felt very uncomfortable. I would not have been there if not with my girlfriend and can't imagine going there by myself. I wouldn't say that I enjoyed it. I wouldn't say that it was "my thing". I wish I could have turned invisible many times. But that was not the worst thing.

The worst thing was I had absolutely no idea what to do to make the situation more comfortable. Here I was, with all the self interest in the world to want to make things easier and no idea at all how to do it. 

I did gain some clear insight into what I did NOT want:
  • please, oh god please, do not single me out or call more attention to me
  • I felt very uncomfortable when people would come over to give me compliments
  • I did not enjoy the particularly extra girly dance moves
This makes me question some of the things I would do in regards to women in tech. Such as 
  • Make sure the women are recognized
  • Compliment women when they do good 
  • Anything remotely 'brogrammer' related (the mere thought of this now makes me cringe)

As uncomfortable as it was being the only man in zumba today, it was one day. It's not like it's my career... 

I can only imagine how many girls we lose the first few days of a programming class when they just feel like they don't fit in.