You are asked to sort a collection of records. The records are stored in a file on the disk, and the size of the file is much bigger than available memory you can work with. Assume a record size is 10 bytes and file size is 1GB. you read and write data to/from disk in units of pages where each page is 1KB big. You have few (less than 10) available pages of physical memory. Assume you are given a method to compare two records.

Here is how I would approach this problem. It was just a quick direct writeup of the thoughts that popped up in my mind:

1. Easiest solution: use Unix’s sort tool, with -S option to specify max memory usage.
2. If this is not allowed, the second easiest solution is to implement a simple Merge sort, and use multiple passes with one temporary file until everything is sorted.
3. If speed is an issue, you can have a pre-pass before the merge sort that loads as much data junks into memory as possible, sorts this junk with Quicksort, and write each sorted junk back into the temporary file.
4. If you want to go even faster, you can go crazy with 10 passes at once and use a sorting network to merge these 10 streams. Here comes the point where you should benchmark before coding, because caching, harddisk properties etc. might have an enormous influence.

I believe this is a quite reasonable approach. It is important to have multiple solutions for a problem to select from, then you can choose the simplest one that works. This is the Do The Simplest Thing That Could Possibly Work (DTSTTCPW) – Principle. This approach is very important in test driven development, and in general a big part of the scientific method (see also Occams’ Razor). If the simplest solution is enough, that’s great! If this is not enough, you can try again with the second simplest solution. Repeat until the problem is solved

I have use this approach for a long time now, and its really productive. Never go for cool and interesting but complex solutions just because they are cool and interesting. These kind of solutions are just for your own ego. There are even 10 commandments for egoless programming.

1. A developer must write code that creates value.
2. A developer must make their code easy to maintain, except where such expenditure will conflict with the first law.
3. A developer must reduce their code to the smallest size possible, as long as such reduction does not conflict with the first two laws.

That’s about all there is to software development. Print this out, make copies for all your coworkers, and engrave it in stone plates so that our childrens children will still be able to live by these timeless wisdoms!

Lately I was busy adding tests that are able to verify if some code (e.g. an optimizer that uses random, like genetic algorithm, simulated annealing, …) produces the desired in e.g. 95% of the cases (Wikipedia has a nice practical example for confidence intervals).

Example Usage

Here is an example that tests the nonsense code Math.random() * 2.

ensure(new Experiment() {
    public double measure() {
        return Math.random() * 2;
    }
}).between(0.9, 1.1, 0.95).sample(10, 100);

The code most likely does not make much sense out of context like this, so here is an explanation of what it does:

Explanation

In that example we want to verify that the code returns values whose 95% confidence interval is between 0.9 and 1.1. At first take 10 samples to verify this. If the mean is not as expected between the interval, there is still the chance that this was just bad luck. We can take further samples (up to 1000) to rule bad luck out. When we have to take more than 1000 samples and the mean is still not in the specified still not met, there is a very high probability that the code does not produce the desired result, so we have to fail.

I hope the above description is somewhat understandable. It’s no piece of cake, and there is no way to cheat around the complexity involved with that kind of statistical tests. I have tried to make the interface as clean and simple as possible, suggestions are always welcome.

Open Source?

I am developing ensure4j partly at work, and partly in my free time. I would like to make it open source, but I need to get the OK from my boss first.

ensure4j is the the only implementation I know of that uses statistical tests that can be used with JUnit. If you know something similar than this, please post!

This are the goals I have for the redesign:

• Make the code expressive and as readable as possbile. This means the code should look like plain english.
• Usability is more important than flexibility.
• Allow for much more poweful checks.

Usability is very important. The good thing is that it is easy to analyze how JUnit is used (download some test driven developed open source projects, grep for assertTrue, and there you have all the cases where JUnit’s asserts were not good enough )

I am playing around with a few ideas, and this is the API I am currently after:

Assertions Example Usage

public void testInt() {
  int num = 4+1;
  ensure(num).is(5);
  ensure(num).either(4, 5, 6);
  ensure(num).between(3, 10);
}

public void testDouble() {
  double val = 4 + 0.4;
  // an epsilon is mandatory when comparing double
  ensure(val).is(4.4, 0.0001);
  ensure(val).between(4.3, 4.5);
}

public void testArray() {
  int[] a = { 1, 2, 3, 4, 5 };
  ensure(a).contains(3);
  ensure(a).size().either(5, 6);
  ensure(a).contains(2, 3, 4);
  ensure(a).contains().either(3, 10, 11);
  ensure(a).contains().neither(6, 7, 9);
  ensure(a).contains().any(3, 4);
  ensure(a).contains().all(3, 4, 5);
  ensure(a).isSorted();
  ensure(a).isUnique();
}

Starting point is always ensure(). You put in the actual value, then execute some operations that modify the following constraint check, then the last command is the constraint check. The last command is always the constraint check, so you can only do one check per line of code. That’s not as flexible as jMock, but it is simpler and more readable and good enough.

Technically all this is very doable. Probably quite a bit of work, and most likely requires a code generator since lots of code is almost the same but not reusable (e.g. arrays for primitive data types), but that’s not a problem. I have done a few simple feasibility studies and I don’t see a big problem with this interface.

Please tell me what you think about this!

Updates

From time to time I will update this page with progress reports, until I have something that is stable enough to give away.

2006-08-05

I have started developing this. So far I am very happy with it, its a fun project to implement I will probably create a project on sourceforge and opensource it, but I am not sure about the licence.

Presentation: “Beyond Test Driven Development: Behaviour Driven Development”

You can also view and download the video here.

What this is about?

In short, Behaviour Driven Development can be summarized into just one sentence:

Behaviour Driven Development is exactly what you are doing if you do Test Driven Development right.

That’s it? Yep, that’s it. Test driven Development is truely excellent in theory, but in practice it takes a long time to master. The people behind Behaviour Driven Development argue that this is due to bad naming. Language strongly influences how you think, and when you develop test-driven, you constantly talk about “tests”, “units”, “asserts” etc. If you are unexperienced this puts you in a wrong mindstate, because TDD it is not so much about testing. Here is why:

• When you talk about tests, you mostly think you need to validate how the code does something. That’s not what you should do. In fact you should not care at all how the code does someting, you should only care about what it does, In short, you want to know it the codes has a desired behaviour. So lets replace the word “test” with another catchy name spec. So from now on you are not writing tests to test something, but you are writing specifications for behaviours. Sweet.

• Everybody has a different idea what a unit is. Some think it’s a class, others think it is a tiny piece of code, others talk about an entity of code that somehow belongs together. This ambiguousness is bad, so let’s ditch it. Lets just talk about behaviours: This is independent of the structure of what you want to test, and clearly describes what you should be validating all along; if you talk about unit tests you are limiting your scope to whatever you think a unit represents. When thinking about behaviours you are free to validate what you really should.

• The syntax of the asserts can be cleaned up quite a bit. So in Ruby with rUnit you write assert_equals(expected, actual), but wouldn’t it be much nicer if you could write actual.should_be expected. In fact this is exactly how it works in rSpec.

What now?

Behaviour Driven Development is a very recent invention, and work around this has just started. I expect that after rSpec we will soon see jSpec, pSpec, and lots of other tools. I hope this will change peoples mindset and allows us to grok how software engineering should work more thoroughly.