Judging process

This page explains the procedure we used to determine the winners of the programming contest.

Errare Humanum Est

We were not quite as merciless as we had announced. We applied relaxed criteria on these three points:

We didn't fix any other kind of bug, however trivial. The line has to be drawn somewhere.

Elimination round

We tried to eliminate as many programs as possible using the correctness and time-limit criteria. Because the number of entries was overwhelming, we were eager to get rid of most of them.

The elimination round consisted of running the programs on the following input files (the file size and time limit are given in parentheses).

The elimination round reduced the competition to 57 entries, eliminating exactly 2/3 of the entries. The full table of results is available.

Performance round

The goal of this round was to pick a list of 10 finalists. We used the following algorithm:

The finalists are:

The full table of results is available.

Final round

Now that we are down to 10 programs, we can try larger time-limits, in order to give a good opportunity to sophisticated programs. We used the following input files:

We ranked the programs with the same algorithm as for the performance round, using the following list of files: 005, 006, 101, 102, 103, 201, 202, 203, 204. 100-hand.txt was not used for this round because it is the same file as 202-hand-again.txt (and it gives the same ranks). The final scores are:

The full table of results is available.

Final round for the lightning division

For lack of a better idea for the judges' special prize, we simply repeated the performance round and final round (with 6 finalists), restricted to lightning programs, with the following results:

The full table of results is available.