Part I:The Problem
I write a verification framework called ApprovalTests. It uses itself to test itself and is generally bug free. So when exploratory tester extraordinaire Maaret Pyhäjärvi wanted to use it as a test target we were both rather excited. I was excited to show off how good automated testing and TDD can be. She was excited to show off how much it still failed to cover.
She won.
To be clear, the stuff I tested was pretty solid. However, I was woefully incomplete on the system as a whole. A mere hour of testing discovered gaping holes in different environments, documentation and onboarding of new users, and usability issues with my API.
These are hard problems, many of them still haven't been solved and I wanted to talk about one in particular that I am still struggling with today:
Naming
I use something called reporters and annotations in ApprovalTests. It means you can write code like
[UseReporter(typeof(DiffReporter))]
The issue was in discoverability. If you start typing this you get very little help from your editor:
You get some other reporters [KDiff, Tortoise] but not many and these happen to be useless and DiffReporter will use them if they exist on your system anyways.
If you typed "Reporter" instead this goes away and the over 50 options will present themselves to you, but it's not intuitive to this and a few painful usability tests showed this to me over and over as I watch is silence and frustration.
Renaming can fix this
[OnFailure(typeof(ReportWithDiffTool))]
I ran an online poll, most people preferred this. 70% vs the 30% that preferred the previous version.
I think I'm in the 30%, I prefer my classes to have Noun names, but I might be partial because this is what I'm used to.
So I'm faced with changing a lot of the API.
The issue is:
How do I know which one is the right answer?
How do I know there *is* a right answer?
Part II: 2 Pepsis and the world of choice
Malcolm Gladwell did a great Ted Talk on choice. In it he talks about a taste test to find the perfect sweetness for Pepsi. There were 2 peaks, so they averaged them out, but this isn't the right answer, the right answer is there are 2 preferences for sweetness. Because of this as a culture we have changed from a single 'perfect' spaghetti sauce to an aisle of choices
Maybe we should have the same occur with API's?
Maybe the issue isn't to have either UseReporter or OnFailure. Maybe I need to have both.
This requires a bit of finesse. I don't yet have answers on how to version and package these solutions.
Should they be in separate nuget packages? Should they interplay with each other?
How to I balance having a clear way with choice paralysis.
I don't know the answers to these questions, but I am beginning to see that maybe there isn't the one perfect API...
