Java Packing

Recently I have started to have a large appreciation for java’s class loader and jar packages. The combination allowing for many great things that becomes troublesome and cumbersome in other languages. But, I wanted to talk about a few things I believe could be much better.

Embedded jars.
Jar files can have class files in them, even resources in them, but not other jar files. Why not? If I have a project that needs a.jar, b.jar, myproject.jar, it would be simpler and easier to jar up the a & b jars into my jar. I’m not saying this is always the best way to go, but it should be an option.

Monkey Patching.
Monkey Patching is the ability to hack out a solution over someone else nice code. Interpreted languages like ruby or php always get this, since you get the source, not a binary. Now there are lots of bad things about Monkey Patching, but it’s really useful when you need it. Now I understand that if you don’t have source, Monkey Patching is not so practical.

However 100% of the jar files I am currently using are Open Source. As such, there should be a standard way of constructing a Jar file so the source is included. Additionally, there should also be easy ways to recompile the jar.
Like: javac recompile a.jar
Again, the jars I’m using are open source, but the ability to easily modify that source is very complex in java. Also, debugging into those classes usually loses source. We need a simpler way to work with the source the jar was made from.

Distilling Minimum class files.
This would also be nice if you could use the jar file the same as regular source path.
For example, let’s say I have a class my1 which uses a1, and a1 uses b1.
(short hand: my1-> a1 -> b1 ) but my source tree has
my1.java
my2.java
a1.java
a2.java
b1.java
b2.java

if I compile my1.java and let javac know my source path,
it’s will follow the links and only give me

my1.class
a1.class
b1.class

Nice right, the minimum needed to run my class. Now think how nice it would be if you had jar files with source attached,
a1 & a2 are in a.jar
b1 & b2 are in b.jar

but still you could get the same results, a jar file with just the minimum needed to run your app.

Right now, there might be a class that i'd like to use, but i'll rewrite it, because i don't want to include the whole jar that it's packaged in. I can't just remove the class file, because it depends on other class files. So I rewrite rather that bloat.
Why should I ever have to make that choice?

Approval Tests (a pictures worth a 1000 tests)

Scenario 1 [Testing a User Interface]

Core Concept :

Component gui = createGui();

Approvals.approve(gui);

This creates a snap shot of the gui: gui.received.png

This passes if gui.received.png == gui.approved.png

If you like the result, simply rename gui.received.png -> gui.approved.png and the test will pass.

Could you explain that in a diagram?

yes.

Why is this awesome?

“a picture is worth a 1000 tests”. Ever find that your unit tests aren’t giving you enough security? Have you started making lots & lots of asserts? Ever find your tests test too many things and the maintenance required to change anything is actually reducing your agility?

Approval Testing gives you 100% lock down on your gui, yet changes simply require a click of a button.

Where are the approved images stored?

Approved images are stored in your testing folder and in your source control. This makes the test repeatable on different systems & you can actually browse a visual of how your gui’s have changed over time!

How does it work for a web app?

String html = createWebPage();

Approvals.approve(html);

Instead of an image, now a text file (page.received.html) is created.

Scenario 2 [Behavior Driven Development]

Core Concept :

Object myObject = createObject();

/*temp*/ Approvals.approve(myObject.toString());

doSomething1();

/*temp*/ Approvals.approve(myObject.toString());

doSomething2();

/*temp*/ Approvals.approve(myObject.toString());

doSomething3();

/*temp*/ Approvals.approve(myObject.toString());

doSomething4();

/*temp*/ Approvals.approve(myObject.toString());

doSomething5();

Approvals.approve(myObject.toString());

As the code is created, the approval will move down, so That the final code is

Object myObject = createObject();

doSomething1();

doSomething2();

doSomething3();

doSomething4();

doSomething5();

Approvals.approve(myObject.toString());

Why is this awesome?

Many times a unit test is scattered with spot testing to assure things are done correctly. This creates a barrier to readability, and can lead to unneeded code.

Approval tests reduce the need to spot check, making the test cleaner and simpler.

What if I don’t want to change my toString?

No problem, create a toOtherString(), or a ObjectWritter, or Inspect the object. Sometimes we even create a ObjectVisualizer when an image is a better way to show the state.

What if there’s a bug in my code?

The first thing to do is to try to walk through your code and get an idea of the state of the object, so you can see where it went wrong... if only you had a nice way of visualizing the state of your object.... oh wait! that’s the first thing you created! I guess it will be pretty easy to debug.

Scenario 3 [The lock down]

Core Concept :

Say I have a web page that displays a user profile. I want to refactor it, but I have no tests. However, I do have 300 users in my current database.

String allPages = getAllUserProfilePages();

Approvals.approve(allPages);

Why is this awesome?

2 lines of code to completely lock down a legacy process. That is the definition of awesome!

Won’t this test be kinda slow?

Yes. But after you are done safely refactoring, you can/should remove the tests.

Is this test data independent? Will it be repeatable tomorrow?

No. But after you are done safely refactoring, you can/should remove the tests.

Should I keep this test after the refactoring?

No!

Locking down legacy code.

So let’s look at the situation where we have a block of code that’s run, but not unit tested. There are many great examples that look like

someBlock()
{
...
o = calculateSomething(a,b);
....
}

now let’s say we wanted to refactor calculateSomething, but it’s dangerous to do that, since we aren’t sure the tests will assert against the changes we might make. Even if the code coverage is 100%, that doesn’t guarantee that the asserts are 100%. For example the test

testSomeBlock()
{
new SomeObject().someBlock();
}

would yield 100% coverage, and not provide any asserts against change.

So let’s look at a locking technique for this code.
1) ‘peel’ the function. Make calculateSomething(){ return calculateSomething1();}
2) Clone the function make calculateSomething2();
3) Add an in production code assert

calculateSomething(a,b)
{
result1 = calculateSomething1(a,b);
result2 = calculateSomething2(a,b);
assertEquals(result1, result2);
return result2;
}

4) now you can refactor calculateSomething2 with safety, even if the above execute only test above was your only test.
5) clean up, once you feel safe with the refactor, remove the calculateSomething1, and inline calculateSomething2.

Of course there need to be a few properties of calculateSomething(a,b), like it has a return value & it doesn’t cause problems by being run twice, etc...

Now this bring some added bonuses:
Case 1. image you have a big block of code you can do an automatic refactoring, like ‘extract method’ and then apply this technique to it.

Case 2. Theory based testing. the interesting thing about the calculateSomething2(a,b) == calculateSomething1(a,b) is you don’t care what a and b are, nor do you need pre-knowledge of their return value. This means you can write a random generator of a,b and blast calculateSomething with thousands of cases.

For example, i have a triangle class that calculates the angles given 3 points. then if i have a generator of point(x,y) i can write
for(int i = 0; i < 10000; i++)
{
Triangle t = new Triangle(generatePoint(),generatePoint(), generatePoint());
assertEquals(t.getAngles1(), t.getAngles2);
}
and WOW, i have massively locked down code in a matter of seconds.

Case 3. Code in production. So lets say I have a total mess of legacy code, in production. I can use this technique, but with a ‘soft’ logging fail. refactor, wait a week, and if nothing went wrong, remove old methoda. It’s not ideal, but ‘total mess of legacy code’ took us away form ideal in the beginning.

What the Inca’s taught me about Sustainable Pace.

I spent the last 2 weeks in Peru, during which I hiked the Inca trail to Machu Picchu. 26 miles of very steep up and down hiking between 10,000 and 14,000 feet: where the air has much less oxygen. Many of you who know me, might have guessed that I’m not quite ready for such activity…

The 2nd day we went straight up 4,000 feet, and by mid-day I was exhausted and found it very hard to breath. I wanted to continue, of course, so slept though lunch, drank a lot of electrolytes (not Gatorade, this stuff tasted awful) , took carbo-gel, everything they recommended. Still when we took off for the afternoon, I could barely go 40 feet without having to sit-down. Finally, the porters tried to lift me, but even that was too much. I asked for oxygen, but was told that every time they gave it people did better, then got worse. Finally my girlfriend and I decided to have me evacuated out.

However, as we looked around to tell the guides, they were 50 feet ahead, in their own conversation. Christina, my girlfriend, then had this suggestion, “try taking 4 steps, then stopping and catching your breath.” I tried it, four (small) steps then two breaths. It came to about a meter at a time, but I didn’t have to sit-down. When we reached our guides, they had the oxygen out, but we refused it, saying “this is working, let’s try it out”. I continued, without sitting down, like this for about 3 miles. Later, as it got steeper steps, I reduced my rate to 2 steps, then 2 breaths. Then remainder of the trip I used this method as we continued up and down between the mountain passes. Constantly monitoring my breathing, and varying my steps from as little as 2 steps, to as many as 20. My metrics became clear, If I had enough breath to talk, I was going too slow. If I couldn’t breathe through my nose (in other words, if I needed to breathe through my mouth) I was going too fast.

And in the end of the 4 days, I arrived at Machu Picchu, and the end of the line, but with the whole group.

Now, hiking 26 miles, 4 steps at a time gives you a lot of time to think, and I got to wondering about sustainable pace and agile programming. We give some lip service to sustainable pace, but mainly we define it as a 40 hour work week. Normal development methods end up in such trouble that they don’t normally come close to sustainable pace.

Also, on the trail, I knew exactly how much progress I was making. I knew if I was going forward. I could say, “look, I’m 3 feet closer to Machu Picchu than I was 30 seconds ago.” But most projects don’t have sufficient tests to realize if they are moving forward, backwards or in circles.

So I wanted to start a discussion about refining sustainable pace for agile programming. There are (I think) 3 areas of this discussion

1) Metrics of sustainable pace for the team and for individuals

2) Techniques for going faster, yet staying sustainable (like my hiking sticks)

3) Metrics for judging progress and speed of the pace

For now, I’d like to focus only on the 1st question, what metrics do you use to tell if you are going too slow, or going too fast?

How to rip audio books

So i listen to a lot of audio books, and here's how i create the mp3 of them.

Overview
we are going to create bite size files about 30 minutes each. then label them, and create the id tags. finally we'll attach a picture.

Make sure the book is UNABRIDGED

1) get a ipod - i prefer the nano. http://www.apple.com/ipodnano/

Ripping
2) download audiograbber & install. http://www.audiograbber.com-us.net/
3) download lame encoder & install. http://jthz.com/~lame/
4) set your encoding bitrate to 64k & stereo, this is a very high quality audio book.
5) select the last track and copy the end location, paste this into the end location for the 2nd track. To do this double click a track. select the Sectors>Last. copy.
6) select and copy the end location for a middle track (usually around 150,000)
7) paste the middle as the end of track 1 and beginning of track 2
8) rip
tips) i like setting the auto query of FreeDB, setting the disk to eject when finished, and not auto selecting all the tracks by default

Naming & Labeling.
the file should look like:
Keith Ferrazzi - Never eat alone [02x20] Don't Keep Score.mp3

usually i don't know the chapter names so they are blank.
9) The total number of tracks is #cds X 2 (so this book is 10 cd's)
10) The labeling schema looks like this - *author - *title [*trackx*totaltracks]*chapter.mp3
11) after the files are named, you need to set the idtags, so download jj mp3 renamer http://software.jensjj.dk/uk/welcome.html
12) add files
13) select files
14) first i convert from filename to ID3 Tag (both) using the format
*Artist - *Album [*Trackx*Comment].mp3
15) then i convert from filename to ID3 Tag again, but using the format
*Artist - *Title.mp3
16) Last, to add the photo add these files to itunes (drag and drop), select them, right click and pick info, then drag a photo from your web browser, i usually do a image lookup on google to find it.

And there you go...