Agent Software Testing

Branch coverage

Release 2026.1Data of 2 November 2026

Branch map of bc

1,922 branches in 10 files

execute.c, 260 branchesexecute.cload.c, 77 branchesload.cmain.c, 60 branchesmain.cstorage.c, 240 branchesstorage.cutil.c, 160 branchesutil.cnumber.c, 448 branchesnumber.cbc.c, 240 branchesbc.cbc.y, 151 branchesbc.yscan.c, 242 branchesscan.cscan.l, 44 branchesscan.lexecute.c, 260 branchesexecute.cload.c, 77 branchesload.cmain.c, 60 branchesmain.cstorage.c, 240 branchesstorage.cutil.c, 160 branchesutil.cnumber.c, 448 branchesnumber.cbc.c, 240 branchesbc.cbc.y, 151 branchesbc.yscan.c, 242 branchesscan.cscan.l, 44 branchesscan.l
1 tool 1962 tools 7843 tools 644 tools 101all 5 458Unreached 319

Each cell is one branch of bc, in source order, file by file. Colour counts the tools that reach it among the 5 that ran on bc; 319 branches (16.6%) are reached by none.

Open the page of bc

Median coverage on each program

A dot is the median over the runs of one entry; its line runs from the lowest to the highest run.

  • KLEE
  • SymTuner
  • AFL++
  • SymCC
  • SymSan
  • ConcoLLMic
  • Agolic

Median final branch coverage of each entry on each program, with the lowest and highest run
ProgramMedian final branch coverage, 0 to 100 percent
bc1,922 branches
  • Agolic KLEE, GPT-5.5: 75.4 percent, 3 runs, from 75.1 to 80.2 percent
  • ConcoLLMic, GPT-5.5: 69.4 percent, 3 runs, from 68.0 to 69.7 percent
  • KLEE fixed options, 6 cores: 22.4 percent, 3 runs, from 20.5 to 25.0 percent
  • KLEE uniform options, 6 cores: 27.4 percent, 3 runs, from 26.3 to 28.9 percent
  • SymTuner, 6 cores: 27.9 percent, 3 runs, from 27.6 to 28.0 percent
Confetti632 branches
  • Agolic KLEE, GPT-5.5: 50.0 percent, 3 runs, from 49.8 to 50.5 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 51.1 percent, 3 runs, from 49.8 to 51.1 percent
  • ConcoLLMic, GPT-5.5: 49.3 percent, 2 runs, from 49.2 to 49.4 percent
  • ConcoLLMic, DeepSeek V4.1 Flash: 49.5 percent, 3 runs, from 48.6 to 49.7 percent
  • KLEE fixed options, 6 cores: 32.3 percent, 3 runs, from 24.4 to 32.3 percent
  • KLEE uniform options, 6 cores: 40.7 percent, 3 runs, from 36.6 to 40.8 percent
  • SymTuner, 6 cores: 41.3 percent, 3 runs, from 40.2 to 43.0 percent
  • AFL++, 1 core: 51.1 percent, 3 runs, from 48.6 to 51.1 percent
  • SymCC + AFL++, 1 core: 50.6 percent, 3 runs, from 50.6 to 50.6 percent
  • SymSan + AFL++, 1 core: 51.1 percent, 3 runs, from 51.1 to 51.1 percent
diff4,369 branches
  • Agolic KLEE, GPT-5.5: 47.4 percent, 3 runs, from 43.6 to 49.1 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 55.1 percent, 3 runs, from 51.0 to 57.7 percent
  • KLEE fixed options, 6 cores: 21.4 percent, 3 runs, from 17.9 to 22.5 percent
  • KLEE uniform options, 6 cores: 31.9 percent, 3 runs, from 28.3 to 42.8 percent
  • SymTuner, 6 cores: 30.7 percent, 3 runs, from 26.5 to 34.7 percent
du7,331 branches
  • Agolic KLEE, GPT-5.5: 22.7 percent, 3 runs, from 22.7 to 23.7 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 24.1 percent, 3 runs, from 24.0 to 25.0 percent
  • KLEE fixed options, 6 cores: 14.5 percent, 3 runs, from 14.2 to 14.5 percent
  • KLEE uniform options, 6 cores: 18.0 percent, 3 runs, from 17.8 to 18.3 percent
  • SymTuner, 6 cores: 18.0 percent, 3 runs, from 18.0 to 19.5 percent
gawk20,356 branches
  • Agolic KLEE, GPT-5.5: 37.5 percent, 3 runs, from 33.1 to 39.3 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 55.7 percent, 3 runs, from 52.7 to 55.8 percent
  • KLEE fixed options, 6 cores: 10.1 percent, 3 runs, from 10.0 to 10.4 percent
  • KLEE uniform options, 6 cores: 23.8 percent, 3 runs, from 22.4 to 25.0 percent
  • SymTuner, 6 cores: 24.1 percent, 3 runs, from 24.0 to 24.8 percent
gcal15,632 branches
  • Agolic KLEE, GPT-5.5: 41.9 percent, 3 runs, from 40.3 to 47.4 percent
  • KLEE fixed options, 6 cores: 22.6 percent, 3 runs, from 22.5 to 22.9 percent
  • KLEE uniform options, 6 cores: 43.4 percent, 3 runs, from 41.5 to 44.0 percent
  • SymTuner, 6 cores: 44.5 percent, 3 runs, from 42.4 to 50.1 percent
grep4,806 branches
  • Agolic KLEE, GPT-5.5: 55.8 percent, 3 runs, from 52.5 to 57.8 percent
  • KLEE fixed options, 6 cores: 36.2 percent, 3 runs, from 35.6 to 43.5 percent
  • KLEE uniform options, 6 cores: 47.8 percent, 3 runs, from 44.2 to 49.2 percent
  • SymTuner, 6 cores: 47.0 percent, 3 runs, from 44.5 to 47.9 percent
gzip3,083 branches
  • Agolic KLEE, GPT-5.5: 50.8 percent, 3 runs, from 50.7 to 51.0 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 53.1 percent, 3 runs, from 52.7 to 53.9 percent
  • KLEE fixed options, 6 cores: 35.6 percent, 3 runs, from 35.3 to 35.7 percent
  • KLEE uniform options, 6 cores: 46.4 percent, 3 runs, from 46.0 to 46.5 percent
  • SymTuner, 6 cores: 47.1 percent, 3 runs, from 46.5 to 47.4 percent
jq18,280 branches
  • Agolic KLEE, GPT-5.5: 30.5 percent, 3 runs, from 28.3 to 36.2 percent
  • KLEE fixed options, 6 cores: 8.6 percent, 3 runs, from 8.5 to 8.8 percent
  • KLEE uniform options, 6 cores: 13.4 percent, 3 runs, from 13.3 to 13.5 percent
  • SymTuner, 6 cores: 13.2 percent, 3 runs, from 13.2 to 13.5 percent
Krep1,231 branches
  • Agolic KLEE, GPT-5.5: 67.2 percent, 3 runs, from 65.4 to 67.8 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 66.0 percent, 3 runs, from 65.7 to 71.2 percent
  • ConcoLLMic, GPT-5.5: 72.7 percent, 3 runs, from 72.5 to 72.9 percent
  • ConcoLLMic, DeepSeek V4.1 Flash: 71.4 percent, 3 runs, from 70.3 to 73.4 percent
  • KLEE fixed options, 6 cores: 47.1 percent, 3 runs, from 46.5 to 47.8 percent
  • KLEE uniform options, 6 cores: 55.0 percent, 3 runs, from 53.9 to 55.3 percent
  • SymTuner, 6 cores: 55.9 percent, 3 runs, from 54.3 to 59.3 percent
  • AFL++, 1 core: 16.0 percent, 3 runs, from 16.0 to 16.0 percent
  • SymCC + AFL++, 1 core: 16.0 percent, 3 runs, from 15.9 to 16.6 percent
  • SymSan + AFL++, 1 core: 15.9 percent, 3 runs, from 15.9 to 16.5 percent
Libmatheval591 branches
  • Agolic KLEE, GPT-5.5: 56.9 percent, 3 runs, from 56.9 to 56.9 percent
  • ConcoLLMic, GPT-5.5: 56.5 percent, 2 runs, from 56.3 to 56.7 percent
  • KLEE fixed options, 6 cores: 24.7 percent, 3 runs, from 24.7 to 25.2 percent
  • KLEE uniform options, 6 cores: 44.5 percent, 3 runs, from 44.5 to 44.5 percent
  • SymTuner, 6 cores: 44.7 percent, 3 runs, from 44.5 to 44.7 percent
  • AFL++, 1 core: 58.0 percent, 3 runs, from 58.0 to 58.0 percent
  • SymCC + AFL++, 1 core: 45.9 percent, 3 runs, from 45.5 to 51.3 percent
  • SymSan + AFL++, 1 core: 50.1 percent, 3 runs, from 48.4 to 50.9 percent
LibYAML4,275 branches
  • Agolic KLEE, GPT-5.5: 63.7 percent, 3 runs, from 62.8 to 65.6 percent
  • ConcoLLMic, GPT-5.5: 42.4 percent, 3 runs, from 39.0 to 43.0 percent
  • KLEE fixed options, 6 cores: 41.0 percent, 3 runs, from 38.3 to 46.9 percent
  • KLEE uniform options, 6 cores: 52.5 percent, 3 runs, from 50.4 to 54.8 percent
  • SymTuner, 6 cores: 54.6 percent, 3 runs, from 53.1 to 55.5 percent
  • AFL++, 1 core: 68.7 percent, 3 runs, from 68.2 to 68.7 percent
  • SymCC + AFL++, 1 core: 60.0 percent, 3 runs, from 59.5 to 60.2 percent
  • SymSan + AFL++, 1 core: 67.4 percent, 3 runs, from 67.3 to 68.9 percent
ls4,441 branches
  • Agolic KLEE, GPT-5.5: 47.4 percent, 3 runs, from 46.0 to 47.4 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 58.0 percent, 3 runs, from 44.9 to 60.5 percent
  • KLEE fixed options, 6 cores: 28.8 percent, 3 runs, from 28.7 to 29.9 percent
  • KLEE uniform options, 6 cores: 39.6 percent, 3 runs, from 39.4 to 39.9 percent
  • SymTuner, 6 cores: 39.9 percent, 3 runs, from 39.6 to 40.1 percent
Oggenc1,138 branches
  • Agolic KLEE, GPT-5.5: 71.0 percent, 3 runs, from 69.4 to 71.2 percent
  • ConcoLLMic, GPT-5.5: 57.8 percent, 3 runs, from 56.7 to 62.2 percent
  • KLEE fixed options, 6 cores: 17.7 percent, 3 runs, from 17.6 to 18.3 percent
  • KLEE uniform options, 6 cores: 29.6 percent, 3 runs, from 29.6 to 29.6 percent
  • SymTuner, 6 cores: 29.5 percent, 3 runs, from 29.5 to 29.5 percent
  • AFL++, 1 core: 15.7 percent, 3 runs, from 15.7 to 15.8 percent
  • SymCC + AFL++, 1 core: 15.9 percent, 3 runs, from 15.9 to 15.9 percent
  • SymSan + AFL++, 1 core: 16.0 percent, 3 runs, from 15.8 to 16.7 percent
sed6,958 branches
  • Agolic KLEE, GPT-5.5: 55.3 percent, 3 runs, from 54.7 to 56.6 percent
  • KLEE fixed options, 6 cores: 17.9 percent, 3 runs, from 17.7 to 19.2 percent
  • KLEE uniform options, 6 cores: 42.0 percent, 3 runs, from 41.5 to 42.2 percent
  • SymTuner, 6 cores: 42.4 percent, 3 runs, from 41.5 to 45.2 percent
WOFF211,412 branches
  • Agolic KLEE, GPT-5.5: 16.0 percent, 3 runs, from 14.1 to 16.1 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 16.1 percent, 3 runs, from 15.2 to 16.4 percent
  • ConcoLLMic, GPT-5.5: 15.4 percent, 1 run, from 15.4 to 15.4 percent
  • ConcoLLMic, DeepSeek V4.1 Flash: 16.5 percent, 3 runs, from 15.4 to 16.8 percent
  • KLEE fixed options, 6 cores: 1.4 percent, 3 runs, from 1.4 to 1.4 percent
  • KLEE uniform options, 6 cores: 7.6 percent, 3 runs, from 7.1 to 8.1 percent
  • SymTuner, 6 cores: 7.9 percent, 3 runs, from 7.7 to 8.5 percent
  • AFL++, 1 core: 17.3 percent, 3 runs, from 16.6 to 17.4 percent
  • SymCC + AFL++, 1 core: 15.8 percent, 3 runs, from 14.4 to 16.5 percent
  • SymSan + AFL++, 1 core: 15.6 percent, 3 runs, from 14.4 to 16.0 percent
xmlwf6,293 branches
  • Agolic KLEE, GPT-5.5: 61.5 percent, 3 runs, from 58.8 to 64.7 percent
  • Agolic KLEE, DeepSeek V4.1 Flash: 66.6 percent, 3 runs, from 66.0 to 66.9 percent
  • KLEE fixed options, 6 cores: 21.6 percent, 3 runs, from 21.4 to 22.0 percent
  • KLEE uniform options, 6 cores: 31.9 percent, 3 runs, from 30.7 to 32.1 percent
  • SymTuner, 6 cores: 32.8 percent, 3 runs, from 32.4 to 33.0 percent
xorriso54,607 branches
  • Agolic KLEE, GPT-5.5: 10.7 percent, 3 runs, from 9.5 to 22.2 percent
  • KLEE fixed options, 6 cores: 6.4 percent, 3 runs, from 5.0 to 6.4 percent
  • KLEE uniform options, 6 cores: 6.5 percent, 3 runs, from 6.1 to 6.6 percent
  • SymTuner, 6 cores: 8.3 percent, 3 runs, from 6.0 to 8.5 percent

A filled dot and a ring are two entries of one tool, such as two models.

How a number is made

  1. Test cases in

    A tool hands in the test cases that its runs generated.

  2. One pinned replay

    One gcov build of each program, pinned, replays the test cases.

  3. Branch sets out

    Each run leaves the set of branches it reached.