There are 140 teams and 171 competing programs [and 92 non-competing programs]. Lightning entries are emphasized, non-competing entries are [bracketed]. Non-competing entries are superseded by other entries of the same team, or submitted after the deadline.
Results are presented in the following table. There is one row per program and one column per input. Lightning entries are emphasized and the rows non-competing programs have a darker background.
One cell gives the result of a given program on a given input.
example | null | almost-empty | third-step | random | hand | icfp2000 | exhaustive | validate-big | validate-small | the-random-returns | ||||||||||||
13942 | 3 min | 0 | 3 min | 9 | 3 min | 5548 | 3 min | 13020 | 3 min | 4392 | 15 min | 64936 | 10 min | 1269376 | 30 min | 134881 | 15 min | 97483 | 15 min | 11385 | 15 min | |
001 | Bug, Out, Err | |||||||||||||||||||||
002 | 0 | 0.02 | 0 | 0.06 | 0 | 0.05 | 0 | 0.06 | 0 | 0.06 | 0 | 0.03 | 0 | 0.03 | 0 | 0.04 | 0 | 0.06 | 0 | 0.05 | 0 | 0.06 |
003 | 0 | 0.41 | 0 | 0.35 | 9 | 0.50 | Bug, Out, Err | |||||||||||||||
004 | 752 | 0.08 | 0 | 0.06 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
005 | 752 | 0.43 | 0 | 0.48 | 9 | 0.47 | 415 | 0.51 | 1846 | 0.58 | 128 | 1.35 | 4844 | 1.50 | 50346 | 4.44 | 30226 | 0.81 | 6252 | 4.29 | 1662 | 1.44 |
006 | 835 | 0.25 | 0 | 0.49 | 9 | 0.99 | Bug, Out, Err | |||||||||||||||
007 | 849 | 0.02 | 0 | 0.08 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
008 | 826 | 0.03 | 0 | 0.05 | 9 | 0.03 | 0 | 0.06 | 0 | 0.11 | 0 | 0.04 | 6616 | 0.19 | 0 | 2.45 | 0 | 0.32 | 0 | 0.26 | 0 | 0.09 |
009 | Bug, Out, Err | |||||||||||||||||||||
010 | 835 | 0.26 | 0 | 0.25 | 9 | 0.29 | Bug, Out, Err | |||||||||||||||
011 | 771 | 0.03 | 0 | 0.10 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
012 | 771 | 0.03 | 0 | 0.09 | 9 | 0.06 | Bug, Out, Err | |||||||||||||||
013 | 771 | 0.03 | 0 | 0.11 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
014 | 753 | 0.03 | 0 | 0.06 | 9 | 0.03 | 464 | 0.06 | Bug, Out, Err | |||||||||||||
015 | 0 | 0.49 | 0 | 0.06 | 0 | 0.04 | 0 | 0.26 | 0 | 0.54 | 0 | 0.19 | 0 | 2.32 | 0 | 46.46 | 0 | 5.05 | 0 | 3.60 | 0 | 0.49 |
016 | Bug, Out, Err | |||||||||||||||||||||
017 | 716 | 0.03 | 0 | 0.15 | 9 | 0.04 | Bug, Out, Err | |||||||||||||||
018 | 680 | 0.03 | 0 | 0.05 | 0 | 0.04 | 225 | 0.06 | Bug, Out, Err | |||||||||||||
019 | Bug, Out, Err | |||||||||||||||||||||
020 | Bug, Out, Err | |||||||||||||||||||||
021 | Bug, Out, Err | |||||||||||||||||||||
022 | 789 | 0.07 | 0 | 0.08 | 9 | 0.04 | 963 | 0.20 | 2418 | 0.22 | 497 | 0.06 | 5644 | 0.55 | 109174 | 21.01 | 43366 | 1.50 | 11040 | 1.15 | 2784 | 0.23 |
023 | Bug, Out, Err | |||||||||||||||||||||
024 | 842 | 0.25 | 0 | 0.28 | 9 | 0.29 | 823 | 0.33 | 0 | 1.38 | 487 | 1.42 | 5825 | 1.46 | 1002046 | 2.38 | 46327 | 2.10 | 10040 | 1.02 | 4634 | 1.42 |
025 | Bug, Out, Err | |||||||||||||||||||||
026 | Bug, Out, Err | |||||||||||||||||||||
027 | 849 | 0.83 | 0 | 0.34 | 9 | 0.40 | Bug, Out, Err | |||||||||||||||
028 | Bug, Out, Err | |||||||||||||||||||||
029 | 789 | 0.02 | 0 | 0.10 | 9 | 0.04 | 995 | 0.07 | 2499 | 0.08 | 529 | 0.04 | 5644 | 0.13 | 335522 | 3.86 | 44651 | 0.35 | 11277 | 0.22 | 3346 | 0.07 |
030 | 831 | 0.02 | 0 | 0.04 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
031 | 752 | 0.08 | 0 | 0.20 | 9 | 0.12 | 197 | 0.32 | 332 | 3.10 | 47 | 0.28 | 2192 | 3.10 | 24309 | 1107.52 | 12141 | 68.13 | 1012 | 40.60 | 699 | 0.96 |
032 | 752 | 0.04 | 0 | 0.07 | 9 | 0.08 | 197 | 0.13 | 332 | 1.56 | 47 | 0.09 | 2192 | 1.37 | 24309 | 833.84 | 12141 | 30.84 | 1012 | 19.14 | 699 | 0.51 |
033 | 788 | 0.08 | 0 | 0.05 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
034 | 753 | 0.23 | 0 | 0.08 | 9 | 0.07 | Bug, Out, Err | |||||||||||||||
035 | 753 | 0.43 | 0 | 0.45 | 9 | 0.45 | Bug, Out, Err | |||||||||||||||
036 | 681 | 0.06 | 0 | 0.08 | 9 | 0.08 | 322 | 0.47 | 1290 | 150.75 | 100 | 0.42 | 4375 | 0.35 | 34231 | 9.92 | 25604 | 871.14 | 5603 | 12.87 | 1121 | 5.84 |
037 | Bug, Out, Err | |||||||||||||||||||||
038 | 848 | 0.10 | 0 | 0.26 | 9 | 0.07 | Bug, Out, Err | |||||||||||||||
039 | 571 | 8.19 | 0 | 0.07 | 9 | 0.07 | 118 | 0.15 | 163 | 0.41 | 0 | 0.10 | 1332 | 5.31 | 279 | 27.08 | 10496 | 3.37 | 848 | 1.96 | 155 | 0.37 |
040 | 789 | 0.44 | 0 | 0.45 | 9 | 0.43 | Bug, Out, Err | |||||||||||||||
041 | 849 | 0.42 | 0 | 0.21 | 9 | 0.05 | Timeout | |||||||||||||||
042 | 0 | 0.01 | 0 | 0.02 | 0 | 0.04 | 0 | 0.05 | 0 | 0.04 | 0 | 0.03 | 0 | 0.04 | 0 | 0.22 | 0 | 0.07 | 0 | 0.05 | 0 | 0.05 |
043 | 848 | 0.09 | 0 | 0.23 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
044 | 849 | 177.03 | 0 | 177.08 | 9 | 177.04 | 658 | 177.11 | 192 | 177.14 | 324 | 897.05 | 5167 | 597.22 | 984307 | 1801.12 | 42284 | 897.32 | 6013 | 897.28 | 4784 | 897.13 |
045 | Bug, Out, Err | |||||||||||||||||||||
046 | 789 | 0.02 | 0 | 0.04 | 9 | 0.04 | 685 | 0.06 | 2344 | 0.06 | 219 | 0.08 | 5466 | 0.04 | 106628 | 0.25 | 37425 | 0.08 | 9439 | 0.05 | 2239 | 0.06 |
047 | 814 | 170.13 | 0 | 0.25 | 9 | 0.16 | 301 | 170.27 | 90 | 170.26 | 47 | 890.62 | 2891 | 590.27 | 369483 | 1793.01 | 18614 | 890.32 | 1008 | 890.31 | 118 | 890.27 |
048 | Bug, Out, Err | |||||||||||||||||||||
049 | 848 | 0.10 | 0 | 0.28 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
050 | 777 | 0.05 | 0 | 0.06 | 0 | 0.05 | Bug, Out, Err | |||||||||||||||
051 | 752 | 0.02 | 0 | 0.07 | 9 | 0.02 | 513 | 0.10 | 1888 | 0.12 | 226 | 0.21 | 4821 | 0.12 | 368076 | 27.51 | 32220 | 0.34 | 7382 | 0.25 | 2906 | 0.11 |
052 | 789 | 0.04 | 0 | 0.11 | 9 | 0.07 | 632 | 0.11 | 2308 | 0.09 | 296 | 0.12 | 5198 | 0.24 | 103795 | 1.32 | 38560 | 1.00 | 9215 | 1.24 | 2053 | 0.13 |
053 | 752 | 0.06 | 0 | 0.04 | 9 | 0.05 | 357 | 0.07 | 773 | 0.08 | 77 | 0.10 | 4518 | 0.31 | 47583 | 4.81 | 24758 | 0.55 | 5518 | 0.40 | 1077 | 0.08 |
054 | 717 | 0.72 | 0 | 0.04 | 9 | 0.03 | 527 | 0.28 | 1589 | 0.18 | 4679 | 50.72 | Timeout | |||||||||
055 | 848 | 0.07 | Bug, Out, Err | |||||||||||||||||||
056 | Bug, Out, Err | |||||||||||||||||||||
057 | Bug, Out, Err | |||||||||||||||||||||
058 | 745 | 0.28 | 0 | 0.22 | 9 | 0.13 | Bug, Out, Err | |||||||||||||||
059 | 752 | 0.05 | 0 | 0.07 | 9 | 0.04 | 320 | 0.11 | 1426 | 0.10 | 89 | 0.13 | 3944 | 0.24 | 50346 | 11.80 | 20956 | 0.53 | 2678 | 0.39 | 1483 | 0.15 |
060 | 0 | 0.03 | 0 | 0.15 | 0 | 0.03 | 0 | 0.26 | 0 | 0.26 | 0 | 0.27 | 0 | 0.25 | 0 | 0.36 | 0 | 0.07 | 0 | 0.14 | 0 | 0.27 |
061 | 0 | 175.21 | Bug, Out, Err | |||||||||||||||||||
062 | Timeout | |||||||||||||||||||||
063 | 680 | 0.28 | 0 | 0.09 | 0 | 0.04 | 197 | 0.19 | 80 | 0.31 | 47 | 0.18 | 2192 | 1.28 | 0 | 21.60 | 10521 | 2.44 | 895 | 1.79 | 87 | 0.28 |
064 | Bug, Out, Err | |||||||||||||||||||||
065 | 807 | 175.18 | 0 | 0.12 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
066 | 856 | 175.20 | 0 | 0.11 | 9 | 0.07 | Bug, Out, Err | |||||||||||||||
067 | 856 | 175.17 | 0 | 0.09 | 9 | 0.07 | Bug, Out, Err | |||||||||||||||
068 | 788 | 0.14 | 0 | 0.06 | 9 | 0.04 | Bug, Out, Err | |||||||||||||||
069 | 753 | 0.15 | 0 | 0.11 | 9 | 0.06 | 583 | 0.15 | 2180 | 0.17 | 247 | 0.13 | 4883 | 0.38 | 121038 | 6.37 | 37546 | 0.77 | 8212 | 0.69 | 1801 | 0.17 |
070 | 753 | 0.08 | 0 | 0.05 | 9 | 0.12 | Bug, Out, Err | |||||||||||||||
071 | 753 | 0.10 | 0 | 0.07 | 9 | 0.06 | Bug, Out, Err | |||||||||||||||
072 | 849 | 0.16 | 0 | 0.10 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
073 | Bug, Out, Err | |||||||||||||||||||||
074 | Bug, Out, Err | |||||||||||||||||||||
075 | 835 | 0.18 | 0 | 0.05 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
076 | 788 | 0.12 | 0 | 0.09 | 9 | 0.04 | 720 | 0.11 | 2093 | 0.12 | Bug, Out, Err | |||||||||||
077 | 877 | 5.98 | 0 | 0.12 | 9 | 0.10 | Timeout | |||||||||||||||
078 | 809 | 0.47 | Bug, Out, Err | |||||||||||||||||||
079 | 884 | 0.33 | 0 | 0.07 | 9 | 0.03 | 919 | 0.66 | 1295 | 1.21 | 652 | 0.24 | 7667 | 4.18 | 1005477 | 64.34 | 32309 | 34.85 | 8364 | 10.32 | 4461 | 0.72 |
080 | 772 | 0.13 | 0 | 0.05 | 9 | 0.04 | 789 | 0.09 | 80 | 0.11 | 344 | 0.10 | 2192 | 0.37 | 920566 | 2.21 | 40843 | 0.50 | 3445 | 0.48 | 3979 | 0.11 |
081 | 877 | 0.62 | 0 | 0.16 | 9 | 0.11 | Timeout | |||||||||||||||
082 | 849 | 0.57 | 0 | 0.24 | 9 | 0.18 | 777 | 1.03 | 0 | 3.73 | 441 | 0.59 | 5209 | 3.49 | 719772 | 91.55 | 40066 | 13.86 | 3828 | 13.05 | 1361 | 1.87 |
083 | Bug, Out, Err | |||||||||||||||||||||
084 | 780 | 0.18 | 0 | 0.14 | 9 | 0.08 | 1112 | 0.19 | Bug, Out, Err | |||||||||||||
085 | 877 | 178.03 | 0 | 0.06 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
086 | 789 | 0.12 | 0 | 0.04 | 9 | 0.01 | 691 | 0.14 | 2342 | 0.21 | 237 | 0.13 | 5380 | 0.45 | 97307 | 21.02 | 35667 | 2.25 | 9439 | 2.02 | 2205 | 0.17 |
087 | 828 | 150.10 | 0 | 0.11 | 9 | 0.13 | 819 | 150.11 | 192 | 150.13 | 353 | 870.12 | 5727 | 570.21 | 984323 | 1774.46 | 42517 | 870.29 | 5214 | 870.45 | 4261 | 870.13 |
088 | 877 | 0.18 | 0 | 0.13 | 9 | 0.09 | 1251 | 0.20 | 2522 | 0.23 | 6897 | 1.35 | Timeout | |||||||||
089 | 859 | 0.10 | 0 | 0.07 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
090 | 829 | 0.08 | 0 | 0.06 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
091 | 849 | 0.59 | 0 | 0.50 | 9 | 0.69 | 835 | 1.49 | 2259 | 0.72 | 6077 | 1.52 | Timeout | |||||||||
092 | 789 | 0.52 | 0 | 0.44 | 9 | 0.41 | 464 | 0.50 | 2054 | 0.56 | 163 | 0.52 | 5042 | 0.65 | 90644 | 4.64 | 34015 | 0.83 | 7804 | 0.77 | 1968 | 0.56 |
093 | 791 | 145.75 | 0 | 145.54 | 9 | 145.57 | 798 | 146.01 | 855 | 146.02 | 342 | 866.32 | 3437 | 566.24 | 625692 | 1766.96 | 27527 | 865.55 | 6955 | 865.82 | 665 | 865.80 |
094 | 839 | 1.17 | 0 | 0.49 | 9 | 0.49 | 741 | 0.99 | 0 | 1.36 | 468 | 1.05 | 5267 | 3.11 | 965511 | 29.85 | 0 | 4.04 | 0 | 3.03 | 0 | 1.38 |
095 | 849 | 0.27 | 0 | 0.14 | 9 | 0.09 | 842 | 0.43 | 0 | 0.30 | 380 | 0.41 | 5582 | 0.53 | 994153 | 2.36 | 42715 | 0.57 | 5602 | 0.61 | 3157 | 0.39 |
096 | 849 | 140.14 | 0 | 0.14 | 9 | 0.01 | 823 | 140.15 | 192 | 140.19 | 385 | 860.16 | 5811 | 560.18 | 984323 | 1760.62 | 44354 | 860.24 | 7255 | 860.24 | 4766 | 860.16 |
097 | 831 | 0.13 | 0 | 0.07 | 9 | 0.01 | 994 | 0.14 | Bug, Out, Err | |||||||||||||
098 | 877 | 173.47 | 0 | 0.05 | 0 | 0.01 | 1180 | 173.54 | 0 | 173.53 | 1004 | 894.04 | 7559 | 593.19 | 712422 | 1786.55 | 34970 | 893.33 | 0 | 893.97 | 0 | 893.64 |
099 | 848 | 0.37 | 0 | 0.07 | 9 | 0.04 | 876 | 0.14 | 2398 | 0.17 | 6017 | 1.85 | Bug, Out, Err | |||||||||
100 | 820 | 170.05 | 0 | 0.04 | 9 | 0.01 | 742 | 170.03 | 192 | 170.04 | 2432 | 590.06 | Timeout | |||||||||
101 | 789 | 1.21 | 0 | 0.15 | 0 | 1.07 | 919 | 5.27 | 2472 | 175.66 | 455 | 7.33 | 5542 | 596.93 | 241424 | 43.99 | 42556 | 895.95 | 10646 | 896.77 | 3110 | 894.75 |
102 | 717 | 1.96 | 0 | 0.89 | 9 | 0.07 | Bug, Out, Err | |||||||||||||||
103 | 849 | 0.05 | 0 | 0.05 | 9 | 0.04 | Bug, Out, Err | |||||||||||||||
104 | Bug, Out, Err | |||||||||||||||||||||
105 | Bug, Out, Err | |||||||||||||||||||||
106 | Bug, Out, Err | |||||||||||||||||||||
107 | 680 | 0.84 | 0 | 0.12 | 9 | 0.09 | Bug, Out, Err | |||||||||||||||
108 | 848 | 0.20 | 0 | 0.05 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
109 | 849 | 11.19 | 0 | 0.08 | 9 | 0.04 | 1004 | 11.91 | 692 | 13.33 | 538 | 11.72 | 6191 | 9.92 | 934202 | 22.11 | 48703 | 13.10 | 11305 | 14.89 | 5162 | 11.47 |
110 | 849 | 0.29 | 0 | 0.13 | 9 | 0.07 | 842 | 0.70 | 0 | 0.94 | 380 | 0.99 | 5582 | 1.20 | 994153 | 3.83 | 42715 | 1.55 | 5602 | 1.67 | 3157 | 0.67 |
111 | 849 | 0.23 | 0 | 0.19 | 9 | 0.13 | 1018 | 0.20 | Bug, Out, Err | |||||||||||||
112 | Bug, Out, Err | |||||||||||||||||||||
113 | 877 | 0.68 | 0 | 0.16 | 9 | 0.09 | Timeout | |||||||||||||||
114 | 793 | 0.12 | 0 | 0.08 | 9 | 0.03 | 0 | 160.53 | 0 | 175.09 | Bug, Out, Err | |||||||||||
115 | 851 | 0.35 | 0 | 0.07 | 9 | 0.05 | 1106 | 0.17 | Bug, Out, Err | |||||||||||||
116 | 851 | 0.38 | 0 | 0.08 | 9 | 0.05 | 1106 | 0.15 | Bug, Out, Err | |||||||||||||
117 | 849 | 135.11 | 0 | 0.10 | 9 | 0.01 | 1022 | 135.13 | 732 | 135.14 | 577 | 855.13 | 6485 | 555.15 | 1002845 | 1755.54 | 48636 | 855.19 | 11529 | 855.22 | 5057 | 855.14 |
118 | 831 | 0.28 | 0 | 0.25 | 9 | 0.04 | 1090 | 0.32 | Bug, Out, Err | |||||||||||||
119 | Bug, Out, Err | |||||||||||||||||||||
120 | 830 | 0.37 | 0 | 0.12 | 9 | 0.06 | 865 | 0.39 | 2418 | 0.36 | 5630 | 1.45 | Timeout | |||||||||
121 | 877 | 170.07 | 0 | 170.05 | 0 | 170.03 | 1401 | 170.08 | 2493 | 170.04 | 942 | 890.06 | 7121 | 590.06 | 1016633 | 1790.78 | 53846 | 890.07 | 16448 | 890.07 | 5683 | 890.04 |
122 | 849 | 0.41 | 0 | 0.12 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
123 | 0 | 0.18 | 0 | 0.10 | 0 | 0.07 | 0 | 0.36 | 0 | 0.35 | 0 | 0.37 | 0 | 0.36 | 0 | 0.42 | 0 | 0.38 | 0 | 0.37 | 0 | 0.36 |
124 | 849 | 0.48 | 0 | 0.11 | 9 | 0.08 | Bug, Out, Err | |||||||||||||||
125 | 849 | 0.15 | 0 | 0.06 | 9 | 0.03 | 1101 | 0.10 | 2892 | 0.32 | 6077 | 3.23 | Timeout | |||||||||
126 | 884 | 0.32 | 0 | 0.06 | 9 | 0.01 | 1458 | 0.51 | 2563 | 8.58 | 984 | 0.47 | 7715 | 3.64 | 1019912 | 158.07 | Timeout | |||||
127 | 789 | 0.07 | 0 | 0.14 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
128 | 680 | 0.24 | 0 | 0.09 | 9 | 0.06 | 322 | 0.23 | Bug, Out, Err | |||||||||||||
129 | 842 | 0.09 | 0 | 0.06 | 9 | 0.01 | 967 | 0.10 | 0 | 0.13 | 5778 | 0.19 | Bug, Out, Err | |||||||||
130 | 0 | 2.13 | 0 | 176.93 | 0 | 176.92 | 0 | 2.10 | 2593 | 2.11 | 617 | 2.12 | 6023 | 2.11 | 988324 | 4.33 | 0 | 2.48 | 12567 | 4.16 | 5776 | 2.18 |
131 | 0 | 2.10 | 0 | 176.93 | 0 | 176.92 | 0 | 2.13 | 2593 | 2.08 | 617 | 2.10 | 6023 | 2.12 | 988324 | 4.28 | 0 | 2.14 | 12567 | 4.13 | 5776 | 2.12 |
132 | 777 | 0.49 | 0 | 0.05 | 0 | 0.05 | Bug, Out, Err | |||||||||||||||
133 | 849 | 2.65 | 0 | 1.59 | 9 | 1.53 | 752 | 3.08 | 0 | 3.72 | 465 | 3.03 | 4786 | 8.01 | 966812 | 41.80 | 0 | 9.57 | 0 | 8.42 | 9 | 3.50 |
134 | 849 | 0.06 | 0 | 0.03 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
135 | 877 | 0.18 | 0 | 0.15 | 9 | 0.08 | 1265 | 0.14 | 2557 | 0.27 | 799 | 0.16 | 6897 | 2.02 | 989374 | 1718.33 | Bug, Out, Err | |||||
136 | 849 | 0.04 | 0 | 0.04 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
137 | 849 | 0.18 | 0 | 0.17 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
138 | 812 | 0.15 | 0 | 0.08 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
139 | 680 | 0.41 | Bug, Out, Err | |||||||||||||||||||
140 | 856 | 0.22 | 0 | 0.09 | 9 | 0.02 | 1228 | 0.25 | 2485 | 0.80 | 762 | 0.15 | 6835 | 1.81 | 1007040 | 257.16 | 51389 | 5.44 | 15343 | 4.94 | 5847 | 0.48 |
141 | 788 | 0.49 | 0 | 0.41 | 9 | 0.16 | 472 | 1.12 | 2018 | 1.27 | 184 | 1.10 | 5159 | 1.46 | 96706 | 12.29 | 32452 | 2.18 | 7423 | 1.82 | 1950 | 1.25 |
142 | 731 | 0.50 | 0 | 0.24 | 9 | 0.16 | Bug, Out, Err | |||||||||||||||
143 | 848 | 0.14 | 0 | 0.06 | 9 | 0.05 | Bug, Out, Err | |||||||||||||||
144 | 680 | 0.58 | 0 | 0.11 | 0 | 0.08 | Bug, Out, Err | |||||||||||||||
145 | 884 | 0.25 | 0 | 0.07 | 9 | 0.01 | 935 | 0.19 | 1295 | 0.59 | 650 | 0.15 | 7671 | 3.16 | 1005477 | 33.46 | 32271 | 20.93 | 8357 | 4.33 | 4461 | 0.45 |
146 | 831 | 9.27 | 0 | 0.15 | 9 | 0.09 | Bug, Out, Err | |||||||||||||||
147 | Bug, Out, Err | |||||||||||||||||||||
148 | 849 | 5.19 | 0 | 5.17 | 9 | 5.04 | 1014 | 5.21 | 2318 | 5.27 | 678 | 5.23 | 6077 | 5.29 | 970509 | 5.63 | 47097 | 8.73 | 10975 | 5.29 | 5330 | 5.23 |
149 | 849 | 0.08 | 0 | 0.04 | 9 | 0.02 | 1086 | 0.06 | 1646 | 0.11 | 620 | 0.06 | 5774 | 0.17 | 908615 | 2.21 | 48732 | 0.31 | 11334 | 0.30 | 5072 | 0.09 |
150 | 877 | 0.88 | 0 | 0.16 | 9 | 0.08 | 0 | 104.74 | Timeout | Bug, Out, Err | ||||||||||||
151 | 877 | 174.11 | 0 | 174.07 | 9 | 174.08 | Bug, Out, Err | |||||||||||||||
152 | 789 | 0.08 | 0 | 0.05 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
153 | 849 | 0.08 | 0 | 0.05 | 9 | 0.02 | 1332 | 0.14 | 1427 | 0.12 | 866 | 0.14 | 6401 | 0.11 | 981050 | 0.50 | 49191 | 0.18 | 11793 | 0.17 | 4967 | 0.14 |
154 | 789 | 0.05 | 0 | 0.04 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
155 | 877 | 1.71 | 0 | 0.11 | 9 | 0.04 | Bug, Out, Err | |||||||||||||||
156 | 884 | 0.07 | 0 | 0.05 | 9 | 0.02 | 1217 | 0.07 | 2650 | 0.11 | Bug, Out, Err | |||||||||||
157 | 680 | 0.43 | Bug, Out, Err | |||||||||||||||||||
158 | Bug, Out, Err | |||||||||||||||||||||
159 | 680 | 0.35 | 0 | 0.09 | 0 | 0.02 | 197 | 0.18 | 80 | 0.30 | 47 | 0.18 | 2192 | 1.32 | 0 | 21.56 | 10521 | 2.43 | 895 | 1.80 | 87 | 0.29 |
160 | 753 | 170.03 | 0 | 0.06 | 9 | 0.01 | 464 | 0.05 | 2053 | 170.04 | 128 | 0.05 | 4876 | 590.02 | 60268 | 109.73 | 33372 | 890.07 | 7095 | 890.07 | 1787 | 0.09 |
161 | 849 | 0.53 | 0 | 0.21 | 9 | 0.07 | 812 | 0.68 | 0 | 3.82 | 476 | 0.56 | 5958 | 3.11 | 976399 | 111.06 | 44686 | 12.50 | 8336 | 11.80 | 3230 | 1.95 |
162 | 849 | 0.40 | 0 | 0.10 | 9 | 0.03 | 1103 | 0.33 | Timeout | |||||||||||||
163 | 826 | 0.82 | 0 | 0.05 | 9 | 0.01 | 1285 | 2.47 | Bug, Out, Err | |||||||||||||
164 | 835 | 0.67 | 0 | 0.40 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
165 | Bug, Out, Err | |||||||||||||||||||||
166 | 877 | 0.08 | 0 | 0.06 | 9 | 0.01 | 1249 | 0.09 | 730 | 0.11 | 783 | 0.07 | 6924 | 0.14 | 1007111 | 3.86 | 51682 | 0.29 | 14448 | 0.27 | 4908 | 0.10 |
167 | 877 | 0.25 | 0 | 0.05 | 9 | 0.02 | 1190 | 0.16 | 2708 | 0.25 | 710 | 0.12 | 6863 | 25.36 | 929716 | 1546.09 | 50033 | 299.27 | 13306 | 193.82 | 5120 | 0.18 |
168 | 848 | 0.12 | Bug, Out, Err | |||||||||||||||||||
169 | 812 | 0.12 | 0 | 0.06 | 9 | 0.02 | 741 | 3.30 | Bug, Out, Err | |||||||||||||
170 | 849 | 0.13 | 0 | 0.06 | 9 | 0.01 | 821 | 0.09 | 2170 | 0.10 | Bug, Out, Err | |||||||||||
171 | 849 | 0.19 | 0 | 0.15 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
172 | 789 | 0.06 | 0 | 0.05 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
173 | Bug, Out, Err | |||||||||||||||||||||
174 | Bug, Out, Err | |||||||||||||||||||||
175 | Bug, Out, Err | |||||||||||||||||||||
176 | 877 | 0.19 | 0 | 0.13 | 9 | 0.01 | 1265 | 0.16 | 2557 | 0.24 | 799 | 0.15 | 6897 | 2.03 | 989374 | 1718.92 | Bug, Out, Err | |||||
177 | 877 | 0.11 | 0 | 0.07 | 9 | 0.01 | 1249 | 0.12 | 730 | 0.13 | 783 | 0.09 | 6924 | 0.14 | 1007111 | 3.85 | 51682 | 0.30 | 14448 | 0.27 | 4908 | 0.11 |
178 | 877 | 2.00 | 0 | 0.07 | 9 | 0.04 | Bug, Out, Err | |||||||||||||||
179 | Bug, Out, Err | |||||||||||||||||||||
180 | 870 | 0.07 | 0 | 0.05 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
181 | 849 | 0.12 | 0 | 0.07 | 9 | 0.02 | 1133 | 0.08 | 1607 | 0.12 | 486 | 0.07 | 5844 | 0.22 | 992202 | 2.58 | 47906 | 0.41 | 10675 | 0.38 | 5182 | 0.08 |
182 | 863 | 0.10 | 0 | 0.17 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
183 | 884 | 0.21 | 0 | 0.11 | 9 | 0.03 | 1135 | 0.10 | Bug, Out, Err | |||||||||||||
184 | 849 | 4.89 | 0 | 0.30 | 9 | 0.11 | 890 | 14.49 | 1719 | 171.47 | 424 | 13.18 | 6240 | 64.43 | 989241 | 1793.79 | 45796 | 717.13 | 8476 | 893.58 | 4254 | 227.22 |
185 | 753 | 0.11 | 0 | 0.04 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
186 | 849 | 0.09 | 0 | 0.06 | 9 | 0.02 | 1332 | 0.06 | 1427 | 0.12 | 866 | 0.07 | 6401 | 0.10 | 981050 | 0.39 | 49191 | 0.12 | 11793 | 0.11 | 4967 | 0.09 |
187 | 849 | 0.18 | 0 | 0.16 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
188 | 849 | 0.07 | 0 | 0.05 | 9 | 0.01 | 975 | 0.07 | Bug, Out, Err | |||||||||||||
189 | Bug, Out, Err | |||||||||||||||||||||
190 | -323 | 0.67 | ||||||||||||||||||||
191 | Bug, Out, Err | |||||||||||||||||||||
192 | 884 | 0.18 | 0 | 0.08 | 9 | 0.03 | 1135 | 0.14 | Bug, Out, Err | |||||||||||||
193 | 753 | 0.93 | 0 | 0.87 | 9 | 0.06 | 598 | 1.07 | 2197 | 0.93 | 209 | 0.90 | 4876 | 0.94 | 54677 | 2.82 | 34067 | 1.13 | 7664 | 1.13 | 2126 | 0.93 |
194 | 831 | 0.11 | 0 | 0.08 | 9 | 0.03 | 909 | 0.12 | 2512 | 0.20 | 443 | 0.12 | 6059 | 0.29 | 1006714 | 3.95 | 44517 | 0.84 | 7200 | 0.81 | 3068 | 0.20 |
195 | 680 | 0.08 | 0 | 0.05 | 0 | 0.01 | Bug, Out, Err | |||||||||||||||
196 | 848 | 2.62 | 0 | 0.13 | 9 | 0.08 | 906 | 0.59 | 1902 | 2.39 | 594 | 0.47 | 5852 | 83.78 | 223996 | 400.98 | 31920 | 8.81 | 6506 | 137.18 | 2994 | 1.13 |
197 | 717 | 0.07 | 0 | 0.03 | 9 | 0.01 | 464 | 0.07 | 1987 | 0.10 | 191 | 0.05 | 5105 | 0.15 | 75458 | 4.04 | 34989 | 0.66 | 7796 | 0.60 | 2031 | 0.08 |
198 | 789 | 4.45 | 0 | 1.24 | 9 | 0.91 | 615 | 2.38 | 2191 | 4.26 | 279 | 2.13 | 5466 | 17.69 | 106610 | 175.70 | 38645 | 32.14 | 9449 | 23.74 | 2037 | 3.32 |
199 | 681 | 0.44 | Bug, Out, Err | |||||||||||||||||||
200 | 848 | 4.14 | 0 | 0.10 | 9 | 0.03 | 906 | 0.64 | 1902 | 1.65 | 594 | 0.43 | 5852 | 94.83 | 223996 | 283.84 | 31920 | 10.23 | 6506 | 99.85 | 2994 | 0.93 |
201 | 849 | 0.16 | 0 | 0.15 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
202 | Bug, Out, Err | |||||||||||||||||||||
203 | 812 | 0.14 | 0 | 0.07 | 9 | 0.01 | 659 | 0.43 | Bug, Out, Err | |||||||||||||
204 | 884 | 173.13 | 0 | 0.07 | 0 | 0.01 | 1429 | 173.83 | 0 | 173.16 | 1004 | 894.06 | 7619 | 593.24 | 712422 | 1788.84 | 34970 | 893.59 | 0 | 893.65 | 0 | 893.87 |
205 | Bug, Out, Err | |||||||||||||||||||||
206 | 831 | 9.30 | 0 | 0.19 | 9 | 0.09 | Bug, Out, Err | |||||||||||||||
207 | 848 | 0.26 | 0 | 0.09 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
208 | 849 | 5.23 | 0 | 5.19 | 9 | 5.04 | 1090 | 5.22 | 2600 | 5.25 | 754 | 5.22 | 6077 | 5.27 | 1001935 | 5.66 | 48989 | 9.36 | 12815 | 5.29 | 5766 | 5.21 |
209 | 774 | 0.21 | 0 | 0.05 | 9 | 0.02 | 818 | 1.56 | 2439 | 10.81 | 5629 | 23.18 | Timeout | |||||||||
210 | 877 | 0.20 | 0 | 0.09 | 9 | 0.01 | 1265 | 0.17 | 2557 | 0.29 | 799 | 0.17 | 6897 | 2.03 | 965950 | 1711.66 | Bug, Out, Err | |||||
211 | 807 | 0.67 | 0 | 0.38 | 9 | 0.35 | Bug, Out, Err | |||||||||||||||
212 | 716 | 0.08 | 0 | 0.06 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
213 | 830 | 0.33 | 0 | 0.11 | 9 | 0.02 | 865 | 0.30 | 2418 | 0.37 | 5630 | 1.34 | Timeout | |||||||||
214 | 675 | 0.57 | 0 | 0.61 | 9 | 0.24 | 0 | 0.88 | Timeout | |||||||||||||
215 | Bug, Out, Err | |||||||||||||||||||||
216 | 835 | 19.02 | 0 | 0.10 | 9 | 0.09 | Bug, Out, Err | |||||||||||||||
217 | 786 | 0.15 | 0 | 0.08 | 9 | 0.07 | 628 | 0.16 | 0 | 0.15 | 176 | 0.14 | 1231 | 0.21 | 227344 | 1.09 | 31253 | 0.24 | 0 | 0.21 | 0 | 0.15 |
218 | 0 | 0.08 | 0 | 0.05 | 0 | 0.02 | 0 | 0.06 | 0 | 0.07 | 0 | 0.08 | 0 | 0.07 | 0 | 0.09 | 0 | 0.07 | 0 | 0.05 | 0 | 0.06 |
219 | Bug, Out, Err | |||||||||||||||||||||
220 | 854 | 155.90 | 0 | 0.26 | 9 | 0.13 | Timeout | |||||||||||||||
221 | 884 | 0.32 | 0 | 0.03 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
222 | 849 | 0.14 | 0 | 0.06 | 9 | 0.02 | 1117 | 0.13 | 1098 | 0.12 | 486 | 0.07 | 5873 | 0.24 | 934567 | 3.08 | 46288 | 0.42 | 8689 | 0.38 | 5093 | 0.09 |
223 | 877 | 1.14 | 0 | 0.17 | 9 | 0.05 | Timeout | |||||||||||||||
224 | Bug, Out, Err | |||||||||||||||||||||
225 | 681 | 2.12 | 0 | 0.12 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
226 | Bug, Out, Err | |||||||||||||||||||||
227 | 877 | 0.90 | 0 | 0.17 | 9 | 0.08 | 0 | 93.07 | Timeout | |||||||||||||
228 | 786 | 0.18 | 0 | 0.10 | 9 | 0.09 | 628 | 0.84 | 0 | 0.55 | 176 | 0.16 | 1231 | 0.24 | 227344 | 0.67 | 31253 | 0.26 | 0 | 0.20 | 0 | 0.14 |
229 | Timeout | |||||||||||||||||||||
230 | Bug, Out, Err | |||||||||||||||||||||
231 | 786 | 0.18 | 0 | 0.10 | 9 | 0.06 | 628 | 0.17 | 0 | 0.13 | 176 | 0.13 | 1231 | 0.24 | 227344 | 0.66 | 31253 | 0.27 | 0 | 0.19 | 0 | 0.14 |
232 | Bug, Out, Err | |||||||||||||||||||||
233 | Bug, Out, Err | |||||||||||||||||||||
234 | 729 | 0.09 | 0 | 0.04 | 9 | 0.01 | Bug, Out, Err | |||||||||||||||
235 | 870 | 0.45 | 0 | 0.07 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
236 | Bug, Out, Err | |||||||||||||||||||||
237 | 849 | 12.48 | 0 | 0.24 | 9 | 0.04 | 1009 | 17.80 | 1286 | 116.82 | 543 | 15.93 | 6691 | 126.31 | 999525 | 1770.91 | 50128 | 388.14 | 12853 | 365.17 | 5205 | 57.79 |
238 | 849 | 0.18 | 0 | 0.11 | 9 | 0.02 | 1132 | 0.21 | 2350 | 0.26 | 666 | 0.20 | 6828 | 6.01 | 995140 | 1793.07 | 49030 | 183.43 | 13054 | 65.91 | 5426 | 0.33 |
239 | Bug, Out, Err | |||||||||||||||||||||
240 | 680 | 0.05 | Bug, Out, Err | |||||||||||||||||||
241 | 849 | 4.78 | 0 | 0.39 | 9 | 0.42 | 752 | 4.45 | 1785 | 4.02 | 465 | 2.12 | 4786 | 7.57 | 966812 | 44.69 | 18249 | 9.76 | 5219 | 8.67 | 1542 | 2.69 |
242 | 752 | 174.07 | 0 | 174.07 | 9 | 174.06 | 527 | 174.09 | 1892 | 174.15 | 240 | 894.09 | 4793 | 594.08 | 50346 | 1794.12 | 32719 | 894.10 | 7031 | 894.09 | 1636 | 894.07 |
243 | 849 | 0.32 | 0 | 0.16 | 9 | 0.23 | 1004 | 0.39 | 2527 | 0.66 | 545 | 0.30 | 6457 | 0.45 | 930300 | 9.46 | 46703 | 0.66 | 11496 | 0.61 | 4928 | 0.39 |
244 | 789 | 0.11 | 0 | 0.05 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
245 | 793 | 0.14 | 0 | 0.09 | 9 | 0.03 | 0 | 173.09 | 0 | 173.10 | Bug, Out, Err | |||||||||||
246 | 793 | 0.16 | 0 | 0.09 | 9 | 0.03 | 0 | 160.12 | 0 | 175.28 | Bug, Out, Err | |||||||||||
247 | 884 | 170.10 | 0 | 170.04 | 9 | 170.01 | 1462 | 170.11 | 0 | 173.39 | 991 | 890.08 | 7704 | 590.27 | 0 | 1796.47 | 0 | 890.55 | 0 | 890.38 | 0 | 890.11 |
248 | 849 | 0.37 | 0 | 0.26 | 0 | 0.04 | 881 | 0.31 | 0 | 0.70 | 415 | 0.31 | 6555 | 0.59 | 999525 | 2.39 | 47666 | 0.79 | 10348 | 0.74 | 4416 | 0.32 |
249 | 884 | 0.82 | 0 | 0.23 | 0 | 0.03 | 1429 | 16.22 | 2136 | 88.26 | 958 | 12.30 | 7711 | 85.28 | 999525 | 1770.45 | 53610 | 248.10 | 10348 | 125.35 | 5869 | 42.78 |
250 | 753 | 0.08 | 0 | 0.11 | 9 | 0.02 | Bug, Out, Err | |||||||||||||||
251 | Bug, Out, Err | |||||||||||||||||||||
252 | Timeout | |||||||||||||||||||||
253 | Bug, Out, Err | |||||||||||||||||||||
254 | 877 | 0.10 | 0 | 0.03 | 9 | 0.03 | 1176 | 0.12 | 1259 | 0.13 | 708 | 0.10 | 6719 | 0.13 | 1009431 | 0.63 | 49599 | 0.20 | 12617 | 0.18 | 5363 | 0.10 |
255 | 848 | 0.26 | 0 | 0.08 | 9 | 0.03 | Bug, Out, Err | |||||||||||||||
256 | 804 | 0.12 | 0 | 0.08 | 9 | 0.01 | 872 | 0.14 | -172 | 1.11 | ||||||||||||
257 | 849 | 0.23 | 0 | 0.16 | 9 | 0.02 | 1132 | 0.26 | 2350 | 0.33 | 666 | 0.23 | 6828 | 4.36 | 995140 | 1322.61 | 49030 | 124.18 | 13054 | 48.42 | 5426 | 0.36 |
258 | Bug, Out, Err | |||||||||||||||||||||
259 | 849 | 7.66 | 0 | 0.09 | 9 | 0.02 | 1172 | 37.06 | 0 | 171.99 | 713 | 94.50 | 6506 | 216.18 | 0 | 1791.76 | 0 | 891.42 | 0 | 910.98 | 0 | 890.85 |
260 | 849 | 0.24 | 0 | 0.15 | 9 | 0.05 | 1080 | 0.19 | 931 | 0.58 | 614 | 0.20 | 6723 | 0.38 | 991514 | 4.40 | 49629 | 1.36 | 12231 | 0.90 | 4987 | 0.86 |
261 | 877 | 0.45 | 0 | 0.30 | 9 | 0.12 | Bug, Out, Err | |||||||||||||||
262 | 863 | 0.12 | 0 | 0.07 | 9 | 0.07 | 1129 | 0.24 | -769 | 0.78 | ||||||||||||
263 | 877 | 174.07 | 0 | 174.07 | 9 | 174.07 | 1189 | 174.10 | 2050 | 174.09 | 866 | 894.09 | 6613 | 594.12 | 1002493 | 1794.13 | 48483 | 894.12 | 12327 | 894.11 | 5789 | 894.11 |
114 [66] programs are eliminated.
[001], [003], [004], [006], 007, [009], [010], [011], [012], 013, 014, 016, 017, 018, 019, [020], 021, [023], 025, [026], 027, [028], 030, 033, 034, [035], 037, [038], 040, 041, [043], [045], 048, 049, 050, 054, 055, 056, 057, [058], [061], [062], [064], [065], [066], 067, [068], [070], [071], 072, [073], 074, 075, [076], [077], [078], 081, 083, 084, 085, [088], 089, 090, 091, [097], 099, 100, 102, [103], 104, 105, 106, 107, 108, 111, 112, 113, 114, [115], 116, 118, [119], [120], [122], [124], 125, 126, 127, 128, [129], 132, [134], [135], 136, [137], [138], [139], 142, [143], 144, [146], 147, [150], 151, [152], [154], [155], 156, 157, 158, 162, 163, 164, 165, 168, [169], 170, [171], 172, 173, [174], 175, [176], 178, 179, 180, 182, [183], 185, [187], 188, 189, 190, 191, 192, 195, 199, 201, 202, 203, 205, 206, [207], 209, 210, 211, 212, 213, 214, 215, 216, 219, 220, 221, 223, 224, 225, 226, 227, [229], 230, 232, 233, 234, 235, 236, 239, 240, 244, [245], 246, 250, [251], [252], [253], [255], [256], [258], [261], [262]
57 [26] programs are still correct.
002, [005], 008, 015, [022], 024, 029, 031, [032], 036, 039, 042, 044, 046, 047, 051, 052, 053, [059], 060, [063], [069], [079], 080, [082], 086, [087], 092, 093, [094], [095], 096, [098], 101, 109, 110, 117, 121, 123, [130], 131, [133], 140, 141, 145, [148], 149, [153], 159, 160, 161, [166], 167, 177, [181], 184, 186, 193, 194, 196, 197, 198, [200], 204, 208, [217], 218, 222, [228], 231, 237, 238, 241, 242, 243, 247, 248, 249, [254], [257], [259], [260], [263]