Wednesday, August 17, 2022

Practice vs. Competition

By Llewellyn Falco & Jacqueline Bilston

In any sport, you have practice and you have competitions. Practice helps you to improve, but what about competition? Is it just about ranking yourself against others? Or is it also a mechanism to improve?

We've done a lot of coderetreats, which are mainly about practice, but this February (2022), we did something different. We participated in a Lean Poker event, which is much more of a competition.

Everybody has a plan until they get punched in the mouth
--Mike Tyson

It turns out you can learn something from competition, but most of that learning doesn't come when you win, it comes after you lose and you reflect on what went wrong.

So what was the result? We got punched in the face (metaphorically)...

Competition Format

Lean Poker is a battle royale of poker bots. A new battle to the death commences every 30 seconds. The bots play poker until only 1 remains. The winning team gets 5 points added to their score on a global score board that determines the overall winner for the week. (The 2nd place player also gets 3 points).

Every team starts with a fully functioning bot in the language of their choice, but the only thing this bot does is fold.

Note: Lean Poker normally runs in a single day, but we decided to do it in one hour a day sessions over a 5 day week, to accommodate a remote global audience.

The Players

We had 6 teams comprised of a total of 22 players. All of them were specially invited, and the teams were balanced according to competencies, but the only two teams that matter are ours, so that's what we are going to tell you about :-)

Both of our teams chose a language where at least one team member was an expert (Ruby for one and Java for the other). Both of our teams chose to mob. Both of our teams followed most XP practices, including TDD.

... things started out the same ...

Session 1

Moments after the competition started, we felt the pressure. All of our bots were folding and losing. Most teams released their first change within 5 minutes.

After we had a bot that didn't just fold, we started to make it smarter. This involved crafting code and objects to represent our hands and help us decide on how much to bet. The bots were very basic, either betting on a good hand or folding on a bad hand.

The logic defining a good or bad hand was also rather basic. At this point our logic was pretty close, but that's where things started to diverge.

The Middle

Llewellyn's team

We decided that we needed better logic to figure out better hands. We could easily detect a pair, and 3 of a kind and 4 of a kind includes a pair so we got that for free. We thought we could differentiate ourselves by being able to detect a straight.

If you have only 5 cards ordered from low to high, a straight is easy to detect. [5, 6, 7, 8, 9] is a straight because each one is one more than the other. it's slightly harder with 7 cards [2, 5, 6, 6, 7, 8, 9], but we figured it out - eventually.

However, even though we were deploying consistently, we weren't doing better in the battles.

Then we realized that none of the games were even getting to the point of having 5 cards in play. Other bots were going all in on the initial two cards, so it was impossible to raise after the 1st round.

Everything we had written was waste.

Jacqueline's team

With a poker expert on our team, we realized the most important cards to assess were the initial 2 cards (the pocket cards). Our strategy focused on refining our bet based on these 2 cards, but we were still losing... Why?

Our algorithm seemed solid, but when we finally looked at the playbacks, we realized we were consistently being overbet. We would have a good hand and bet 80%, only to have another team bet 100%, and then we would fold.

This forced us to change our strategy to all-in or fold based on those first two cards.

When we did this, we started to improve.

The End

Llewellyn's team

We now had a bunch of code, but we weren't getting the results we wanted, and we realized we needed a better understanding of what was happening in the battles. It was late to start adding logging to our system, but it was better late than never. Sometimes our logs didn't seem to match what our code should have done, so we also started labeling each version so we could better match the production to the code.

This telemetry should have been one of the first things we did, but the other teams were already winning, and we didn't want to be left behind. Our early shortcuts ended up hurting us.

Jacqueline's team

Even in the short amount of time available we were able to write some fairly confusing code. We knew it did what we wanted, because we used TDD, but we had been skipping the refactoring step.

Everyone on the team wanted to refactor, and we didn't know why everyone else seemed to avoid it. Finally, we made time for a retrospective where we discovered that the pressure to release a better bot kept getting in the way. We decided to spend the last session cleaning the code.

Once we had finished refactoring, it became obvious and simple to improve the code. These improvements helped - we gained 30% on the competition, but by that point, it was too late.

After the Game

Neither of our teams won, but in our failures, there was a lot of learning. Some retrospectives lasted up to 6 hours after the final match. All of us agreed that we made stupid mistakes because of the pressure we were under to perform. All of us could relate to that pressure in our day-to-day jobs.

This pressure resulted in cutting corners and preferencing short term wins which created long term problems (even just 5 hours later). It also dramatically reduced the amount of experimentation and exploration.

Worst of all, we all knew better, but we still didn't act better.

Ironically, at work, it often seems that this pressure is constructed artificially in the form of deadlines to try to drive better performance.

An odd observation

One question that came up in the final retro was:

"What if 1 team was only allowed to release every 30 minutes?"

Everyone agreed this would put that team at a major disadvantage.

It's obvious in the game that this restriction would be crippling. Yet we normalize monthly deploys in the business world, and no one seems to complain.

Details

Just in case you'd like to see the details of our match

Tuesday, August 2, 2022

should_methods_be_named_like_this?

tl;dr: all method names that return a True/False setting should begin with the word is

The Problem

The readability of code is very important to me. Unfortunately English is a remarkably complex language. This leads to a lot of variations on how you can ask a True/False question.

Here are a few:

  • am I allowed to write to files
  • are files writable
  • can I write to this file
  • is this file writable
  • should file writes be allowed
  • was the file writable
  • were the files writable
  • will file writes be allowed
  • would the file allow modification

Consistency

The issue with even considering which of these is the better way of expressing this question in English is Consistency.

“A consistent experience is a better experience.” — Mark Eberman

Your API is the UX that programmers experience and a consistent UX is better, even when it's worse. Why? Because it lowers the cognitive load on a user. New things fit into your existing understanding, Things are where you expect them to be so you don't miss them and they are easy to discover. If I want to know what an object can do it is helpful to be able to ask it 3 general questions:

  1. is - What properties are True/False?
  2. get - What other properties can you give me?
  3. set - What properties can I change?

This convention has been decided decades ago. Furthermore, IDEs will often autocomplete in alphabetical order making things discoverable.

The answer and the problem

This means if you have a method that answers a True/False question, the method should begin with the word is.

But English can make this soooo ugly!
Yes
But...
... it doesn't matter :-(

Yes the present singular form of the to-be verb can make for an awkard method name, but consistency is more important.

> isThisMethodNameInTheCorrectForm()   
True

Wednesday, June 16, 2021

Hexploration



Over the past little while we've been exploring an alternative to the Game of Life with a hexagonal board. You can see the rules here. This led us to some super interesting discoveries that we'd like to share.

1. Verifying Sequences

The interesting part of the Game of Life is how it evolves over time. We wanted to write tests that could help us verify how the board changes frame to frame. You can do this by mapping time to space and creating a text file that prints out each frame one after the other, much like a comic book. Instead we decided to take advantage of the animated gif format, and play out the evolution much like a short film.

Here's an example of one of those calls displaying the initial board plus 33 additional frames:

verifySequence(board, 33, f -> board.advance())

which produces:

2. Exploratory Testing with Property-Based tests

Once we could verify sequences, we could start hunting for initial layouts that produce interesting sequences. There are lots of documented scenarios in a traditional square board (blinkers, guns and flyers, etc.), but we couldn't find any for hexagonal space. We decided to search on our own.

2A. Generating Random Boards

We took a page from property-based testing, and created a generator to randomly generate a board with a given number of live cells. We also printed out the board in a way that allowed us to capture and reproduce the ones we liked.

Here's an example of a board we reproduced.

new HexGameOfLife(_(2, 4), _(2, 6), _(1, 9), _(1, 3), _(5, 5), _(5, 1), _(3, 1))

This generated lots of results, and most of them were booooring.

2B. Filtering for "Interesting"

Taking another page from property-based testing, we created a filter to remove boring results. To do this, we had to ask

Q. What does interesting mean?
A. We can see cells in every frame.

Fortunately, coding this was quite simple.

  1. Generate a board
  2. Advance a turn, and check that cells still exist
  3. Repeat until you get to the end of the frames
  4. If you make it to the end, return, otherwise GOTO 1

With these steps, we found this board:

2C. The Blinker Search

In this process, we saw a simple blinker, but lost the board before we could capture it. We tried to get lucky again, but couldn't, so we decided to define what the properties of a blinker were, and then find one automatically.

Fortunately, coding this was quite simple.

  1. Generate a board
  2. Advance 12 turns, and check that the board is the same as the initial board.
  3. If it is, you have found something that repeats every 1,2,3,4,6, or 12 frames. Otherwise GOTO 1

In just under 300,000 randomized searches, this returned: 

If you would like to check out the full code, start here

Monday, May 3, 2021

What is an Agile Technical Coach?

What is an Agile Technical Coach? By Llewellyn Falco & Jacqueline Bilston


Note: We originally wrote this as an internal memo. It's not meant to be a complete definition, but we thought it might be helpful to others so we are sharing it here.

TL;DR: An agile technical coach writes code with programmers for the purpose of improving how the team programs. Common areas include:

  • Refactoring
  • Test Driven Development
  • DevOps
Overview

The focus of an agile technical coach is specifically on technical practices like refactoring, test driven development, and DevOps. This means that technical coaches are sitting with the programmers, writing code together. Their value is in the sustained behaviours the team continues using after they leave.

Although a coach is not a teacher (who works in practise problems), and a coach is not a consultant (whose main value is delivered while they are working with you), the roles sometimes overlap. A coach often wears the hat of a teacher in short bursts in order to enable a more productive coaching session while working with a team, and only rarely, if ever, acts as a consultant.

Agile is about being able to respond to change. There are two parts that work hand-in-hand to allow this to happen in software. One side comes from the management side of what and when to change. The other side comes from the technical side of having code that is easy to change. Technical Agile Coaches empower Agile Coaches to be able to respond in the ways they want.

Thursday, April 8, 2021

The 4 Benefits of Tests

By Llewellyn Falco & Jacqueline Bilston

I’ve found that when I'm creating tests they provide 4 categories of benefits. Having these categories helps me to see how to write better tests and helps me see what to improve when I am feeling a particular pain. These benefits tend to occur in chronological order, which is how I’m going to lay out the explanations.


Note: This article is about tests as artifacts, not testing as performance. There is another area of testing known as exploratory testing. It is a different practice and provides different benefits.

#1 Specification

What am I building?”


At the very beginning, before any code is even written, writing a test scenario can help you understand what it is you’re trying to build. These scenarios often work much better than requirements, as the specifics of a scenario will often surface hidden issues. Programming is often done between two or more parties. Sometimes it can feel that way even if you are doing it by yourself. Scenarios will surface misunderstandings where you thought you were in agreement; edge cases you weren’t considering; and logical errors before they are written.


Even if this is as far as you proceed with your tests, they will still bring value.


#2 Feedback


“Does it work?”


After you have a test scenario, you can immediately start getting feedback on whether or not it is completed. This feedback can come in many forms:


Programming by red: Compiler errors can guide you on what needs to be built next. Failing tests can guide you on what needs to be implemented differently.


Debugging / Logging / Inspecting: Running code and seeing the results is critical to understanding that you’ve built what you meant to. Just imagine if you shipped code that you never ran :-/ 


People don’t know what they want until they see what they don’t: Showing the results to a customer will bring insight into if you are building the right thing.


#3 Regression


“Does it still work?”


99 bugs in the code, 99 bugs in the code, take one down pass it around, 127 bugs in the code

Just because code worked yesterday does NOT mean it still works today. Rerunning your tests allows you to know that things still work. A suite of passing tests gives confidence that things still work. Better still, a single failing test proves that something doesn’t.


Note: In my experience, you won’t get good regression without automated tests.


#4 Granularity


Why did it break?”


Granularity can show up in many forms, but it’s always about figuring out how to fix what went wrong. I tend to see 2 main forms of it.


#1 Temporal Granularity: 

You just changed a line of code and it broke the build. That line is the problem. Maybe you just roll it back, maybe you fix it and roll it forward. Either way it’s fixed and it was easier than if you had changed 1,000 lines of code.


or...


5 people commit and the nightly build breaks, who caused it?

vs.

CI runs after each commit and Jonny's commit broke it.


or ...


The tests passed 2 minutes ago and now they are broken

vs.

The tests passed this morning and now they don't.


#2 Differential Granularity

Exception thrown’ doesn’t help anybody

Array index out of bounds’  is a little better.

Array[10] is out of bounds for length 10’ means you have an off-by-one error.

Array[-2] is out of bounds for length 10’ means you have calculation bug.

Array[two] is out of bounds for length 10’ means you have to verify user input .

Array[5] is out of bounds for length 0’ means you have to check if result came back empty.


Logs in JSON are queryable which helps you find issues faster.

Objects with useful toStrings help you understand the state of your system.

Diff tools help to find what changed between two outputs.

Giving meaningful output helps us understand the causes for failures and the potential fixes.


Saturday, November 21, 2020

The Point of 'No Return'

The Point of 'No Return'
Welcome to \new world

I believe that life is important...
...and way too much of it has been wasted on dealing with issues arising from inconsistencies between line endings.

A modest proposal:
In today's world most programs can handle the line ending \n correctly.
The 'No Return' movement seeks to make inconsistent lines ends a thing of the past.
There are 3 practices on the path to achieve this:
  1. By default never use \r\n line ending in anything that it outputs.
  2. Have a toggle to turn this off for people stuck in the past
  3. (optional) On reading, replace all '\r\n' with a '\n' on any file used by the program
But:

Don't most programs already have the ability to toggle?

Yes, but they tend to default to the version of the system. Meaning Windows perpetuates the line ending inconsistency problem.  If you have dealt with this issue (which you probably have) this is why.
Most people don't change the defaults. We need to start making inconsistent line endings a very rare exception

It's just two options, is it really such a big deal?

It shouldn't be. But I personally have spent weeks of my life dealing with this issue. If you were to total up the hours of all programmers it's lifetimes of people, and it isn't going away. Our current path is pass this problem on to the next generation. We need a path to end it.

But isn't there a reason for \r\n?

Yes, it was based on the way typewriters worked. We are still dealing with legacy from type writers.

Ok, you've convinced me, how do I signup?

Simple, copy the above logo and add it to your site, link to this article and add the 3 rules to your code base.



note: I originally wrote this in august of 2014. For some reason I never published this. I'm finally doing it in November 2020 because I'm still dealing with this issue....





Thursday, October 22, 2020

Online Remote Training: Lessons Learned


By Llewellyn Falco and Clare Macrae.


Here is what we learned about how to do effective online training, to help you run better courses yourselves, or to help you choose better courses to attend.


TL;DR: Take advantage of being able to spread classes out over time. When meeting virtually, don’t pretend that you can replicate what you do at in-person training. You will need to scale back and make accommodations for the added problems of remote training.


See it for yourself at our November 2020 class “Testing Legacy C++ Code effectively with Approval Tests

Introduction


Over the last 6 months, we have been intensively learning how to create online training with a similar level of quality to in-person training. In this post, we share what we’ve learned so far. 


Note: Skills vs knowledge. There are multiple types of training: many are focussed on transferring knowledge. But we believe that one of the big advantages of training is the ability to transfer skills. Teaching skills requires a lot of hands-on work from the participants. This is a different, and slower, process, but the one we are talking about today.


Benefits over in-person training

Fortunately, there are some advantages to online training, which can make them better than in-person courses. Let’s start with the positive.

Multiple Shorter Time Slots

If you remember back in school, we didn’t do a week of arithmetic, followed by a week of literature, followed by a week of physical education, followed by a week of art. This is by design. Humans learn better by doing a little bit every day. Part of this is how much we can learn in a given day, but the other part has to do with the need for periods of rest in between. Time in-between is where we can absorb and incorporate those skills into our being. Remember, exercise does not build muscles, it destroys muscles. It’s the rest afterwards where the muscles rebuild.


Spreading out a course of short periods on multiple days is never done during in-person training courses, because these typically involve the time and costs of hotels and planes. So instead we batch-up into a full day or a full week, with 8 to 40 hours of training. Sometimes we try to spin that as a good thing, with a title like “Boot Camp”.


We have found that, rather than teaching an 8-hour class in a single day, it is better to teach an 8-hour class, split up into 2 hours per week over 4 weeks. This yields much better results for the students. It not only gives them more time to absorb the information, and revisit it if needed. More importantly, they get to try out these skills during their normal activities.


Other bonuses are that it also makes it easier to maintain focus for the entire session, and even makes it easier to deal with time-zones. (I can easily wake up early for 2 hours every Monday without it disrupting my entire week. If I have to wake up early every day, my entire week is shot.)


This is true for both attendees and instructors.


Homework


There are lots of benefits to homework, in deepening an understanding. Unfortunately, it is not practical to assign homework after an 8 hour day, that has to be completed by the next day.


With short, weekly sessions, homework is now practical, useful and accessible.


We have found these useful types of homework:

  1. Repeat the exercises that we did in class, to make sure that you can understand them, and get them working on your own computer.
  2. Homework that expands your absorption, such as actively trying to use the techniques you are learning, in your day-to-day work.
  3. Supplemental reading and videos. Use this sparingly.


You cannot make homework essential, so we do not try to introduce new concepts via homework. This means that if a concept is introduced in supplemental reading or a video, we will always re-introduce the concept in class, as though it is the first time students are seeing it. This is important to not disadvantage students who don’t have the extra time to spare.


A powerful motivation for doing this week’s homework is the fact that other students did last week’s homework. Be aware that even with very motivated people, some will still not do the homework because other important things took priority. So you do not want to shame anyone for not doing the homework, but at the same time, you want to make visible that the other students are. We have found this to be a better motivator than your instructor telling you to do your homework or grading the homework.


To this end, these are the things that make it easier for the homework to be done:


  1. End each session with a clear and consistent assignment. It’s easier to do something if it’s clear what you have to do.
  2. Begin each session with a check-in, where the students can see that (usually) the majority of the other students are doing the homework.


Constraints

Of course it’s not all better. We have found some serious constraints when running online courses.

The need for video

 
One of the things we noticed is, as an instructor you are often helping people to accomplish tasks. Without video it’s almost impossible to tell the difference between “I’m not doing something because I have no idea where to start” and “I’m right at the edge, ready to jump, and i just need a little bit of encouragement to move”. These are huge extremes, and how you help an individual student at these moments greatly depends on where they are.


We have found it to be so important that we have made video a requirement for attending our courses.


Protocol: At the beginning of each course we state that, if for any reason you need to step away from your computer, that is OK. Simply turn off your camera and mute your mic, and we will assume that you are out of the room.

Smaller classes


While it is possible to run a lab with 20, 30 or sometimes 40 people in a room, we have found it very difficult to handle class sizes greater than 12 online. The issue here is the inability to split the instructor’s attention. If you put people into break-out rooms, you are effectively blind to everyone, or blind to everyone else but those in the room you are in. If you keep them all in the same room, you can’t do small-group work. This is not the case in the real world, where you can see the whole room, and the tables can just focus on themselves. Some of this can be mitigated by having an extra person co-facilitate, but this also complicates things. You need to have smaller classes.


Have backup options


“Anything that can go wrong will go wrong” (Murphy). Things go wrong all the time, even at in person events. But I’ve never been in a physical class where I just disappeared. Online, I have.


You can lose the internet, your computer can die, you can be region-blocked, the student’s computer and internet can die, servers and services can go down. What do you do when this happens?


The best advice we have is to have backup options. Two instructors means one of you can lose the internet (if you make the other one a co-host). Having multiple channels of communication (Google docs, email, Zoom, remote-desktop to shared computer) not only means that you have a way to communicate when something goes wrong, but also makes it easier to figure out what went wrong, when the inevitable happens. Also have a second internet connection (usually a phone). 


Checklist


Backup options don’t work if they are not set up and tested. We have found that we need to make a checklist of things to do before we start each class. There are usually only 4 or 5 items, and we usually won’t remember to do all of them without the checklist.

Practicalities

Allow time on both sides

In the real world, many instructors and students arrive early to the room, and sometimes they hang around after the class has finished. But because of the ability to instantly teleport into a Zoom call, we have a tendency to start an 11am call at 11am, and to book a 12 o’clock call right after. From the instructor’s point of view, we recommend that you open the call 10-15 minutes early, to allow people to arrive at leisure, and to ensure you can start on time. Also, clear your schedule for another 30-45 minutes after the class, to you to allow questions and discussion with students. 


We also recommend that our students don’t book anything for 30-60 minutes after the class. This isn’t so that they can have discussions with us (although that’s a benefit). It’s because they’re tired after the class, and they need a break before jumping into whatever their next thing is.

Breaks

An advantage of the real world is that while you are in class, the rest of your life is out-of-sight and out-of-mind. This is not true in the virtual world. The cognitive load we all feel from the rest of our household weighs on us, and we need small releases at regular intervals. For example, it is easier to tell a child “wait 10 minutes and I can take care of it” than “wait 2 hours”. We recommend a 5-10 minute break every hour, without fail.


When we are running breaks, we usually display a count-down timer in a web browser, so everyone knows when to return. Just Google “5 minute timer”.


Need for Activity

You are simply not as charismatic and engaging on video as you are in person. Remote calls need more activities, and shorter periods of you talking. This is good advice for skill-based training anyway, but is 10 times important when training remotely. We suggest breaking things down into small exercises, that you can start almost immediately. We try to use frequent changes in how the students are interacting, to introduce variety and hold engagement.


Not Seeking Volunteers

Asking for volunteers tends to unintentionally exclude a fair amount of your class, and you may find one or two voices tend to dominate. This is true in-person, but it’s significantly worse in virtual classes, where it is easier to hide, and harder to jump in.


We suggest using systematic methods of selection. For example:

  • Rotate in order: from a list
  • Randomly select: the key here is make the random selector visible. For example, assign everybody a number, and use an online random-number generator.
  • Mechanistic selection: break into groups of two, and the person with the shortest name will go first, and then you will rotate.
  • Call on people who have not yet contributed.

Requirements and Geographic Locations

It’s easy to think that now we are virtual, anyone from anywhere can attend a course. But this is not always true. Many software products and web sites are blocked or not available to certain geographical areas. Our best suggestion is to have a very clear list of requirements to attend the course, and let the attendees determine it for themselves.


For example, our current course lists this at the top:


Course Requirements

To attend this course, you will need to be able to use the following:

  1. Zoom, with a microphone and camera.
  2. AnyDesk
  3. Google Docs
  4. Basic understanding of C++ (creating strings, creating call functions, variable, loops, etc…)


Cloud-hosting and AWS

Having people work on virtual environments in the cloud is a huge help, for remote training. This is not as doable for in-person training, as reliable internet is not something that you can count on at a conference, but is something that you have to rely on, for people to attend a course virtually. To learn more about this, check out “How to setup a Windows AWS EC2 instance for remote mobbing and pairing

See for yourself


If you would like to experience how we are doing virtual training yourself, you can always sign up for our November 2020 classTesting Legacy C++ Code effectively with Approval Tests


Alternatively, if your company would like a private course, we offer them in Java, C#, C++, JavaScript and Python, on a variety of topics relating to Agile Software Development and Code Quality, such as Unit Testing, Refactoring and Legacy Code.


Contact us at: info@spunlabs.com and clare@claremacrae.co.uk