Oggenc

1.4.3

Terms

Run: One campaign of one tool on one program.
Cores: The parallel processes of one campaign.
Budget: The longest a campaign may run.
Fixed track: The command line is given and the tool supplies only the input. Method
Open track: The tool also chooses arguments, files and environment. Method
Median: The middle value over a tool's runs; the median of two runs is their mean.
standard: Conventional gcov replay of every retained test case, same counting code and denominators. Method
Unreached: A branch that none of the main-group tools that ran on the program reaches (4 to 12 tools).
Union of runs: The branches that at least one of a tool's runs reaches, each counted once.
Output tokens: Model output of a whole campaign, reasoning included, as self-reported in the release.
Sanitiser signature: The sanitiser, the error kind and the innermost frame in the program's own source; two can share a cause.
Language model: A system in which a language model does the central reasoning of a classical technique, such as concolic execution.
Symbolic: Runs the program on symbolic inputs and asks a constraint solver for concrete inputs that reach new paths.
Fuzzer: Coverage-guided fuzzing: mutates inputs and keeps those that reach new code.
Hybrid: A fuzzer paired with a concolic executor; each supplies the other with inputs.
Main: Shown by default: the tools a reader is most likely to compare.
Ablation: Variants of the agent that switch off or change one part, to show what that part adds.
Seeding: Runs of a tool that start from seeds produced by another system.
Control: Systems that run without the agent, as a reference for the main group.
Release: A dated snapshot of the data, named like 2026.1. It never changes; corrections go into the next release.
Superseded: An entry that a newer one replaces. It stays in the release and shows under all versions.
Withdrawn: An entry taken out of the comparison after it was published.

Release 2026.1

oggenc, from vorbis-tools, encodes audio into the Ogg Vorbis format.

Branches: 1,13826.0% unreached (9 tools)
Language: C
Inputs: command-line arguments, files
Fixed track: {input}
Tools with runs: 149 main
Runs: 4848 with branch sets

Branch map

1 tool 202 to 4 4905 to 6 1537 to 8 66all 9 113Unreached 296

Final coverage

Final coverage of every tool on Oggenc
Tool	Median	Min to max	Share (median)	Runs	Output tokens
Main group, 9 tools
Agolic KLEE, GPT-5.5agent, open track, six cores, version 1.0.1+2407fd07	808	790–810	71.0%	3	366,443
Agolic AFL++, DeepSeek V4.1 Flashagent, open track, six cores, version 1.0.1+8a86838e	812	807–819	71.4%	3	485,177
ConcoLLMic, GPT-5.5language model, open track, version 9634e686+f3fd7c36	658	645–708	57.8%	3	533,239
KLEE fixed options, 6 coressymbolic, open track, six cores, version efa1e052	201	200–208	17.7%	3	not applicable
KLEE uniform options, 6 coressymbolic, open track, six cores, version efa1e052	337	337–337	29.6%	3	not applicable
SymTuner, 6 coressymbolic, open track, six cores, version 71806b45	336	336–336	29.5%	3	not applicable
AFL++, 1 core fixed trackfuzzer, fixed track, one core, version 5.02c	179	179–180	15.7%	3	not applicable
SymCC + AFL++, 1 core fixed trackhybrid, fixed track, one core, version 07c8895f+06a515fa	181	181–181	15.9%	3	not applicable
SymSan + AFL++, 1 core fixed trackhybrid, fixed track, one core, version 2937d1c0+499bf218	182	180–190	16.0%	3	not applicable
Ablation group, 1 tool
Agolic KLEE, GPT-5.4 supersededagent, open track, six cores, version unreleased+cfd2c75f	799	779–803	70.2%	3	not reported
Control group, 4 tools
AFL++ rebuilt targets fixed trackfuzzer, fixed track, version 5.02c	182	182–187	16.0%	3	not applicable
KLEE fixed options, 1 coresymbolic, open track, one core, version efa1e052	170	166–183	14.9%	5	not applicable
KLEE uniform options, 1 coresymbolic, open track, one core, version efa1e052	309	305–330	27.2%	5	not applicable
SymTuner, 1 coresymbolic, open track, one core, version 71806b45	305	302–315	26.8%	5	not applicable

finals.csv in the results repository

Branches reached by one tool only

Branches reached by each main-group tool and by it only, on Oggenc
Tool	Runs in union	Branches reached	Share (union)	Reached by this tool only
Agolic KLEE, GPT-5.5	3	834	73.3%	14
Agolic AFL++, DeepSeek V4.1 Flash	3	821	72.1%	5
ConcoLLMic, GPT-5.5	3	734	64.5%	1
KLEE fixed options, 6 cores	3	210	18.5%	0
KLEE uniform options, 6 cores	3	337	29.6%	0
SymTuner, 6 cores	3	337	29.6%	0
AFL++, 1 core	3	180	15.8%	0
SymCC + AFL++, 1 core	3	181	15.9%	0
SymSan + AFL++, 1 core	3	190	16.7%	0

Sanitiser findings

Signatures in each tool’s final test cases.

Distinct sanitiser signatures of each tool on Oggenc
Tool	Runs replayed	AddressSanitizerkind, signatures	UndefinedBehaviorSanitizerkind, signatures
Agolic KLEE, GPT-5.5	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range6 misaligned-access1 null-pointer-use1 signed-integer-overflow1 value-outside-the-range-of-its-type2
Agolic AFL++, DeepSeek V4.1 Flash	3	segv1 calloc-overflow1	left-shift-of-a-negative-value1 left-shift-out-of-range6 misaligned-access1 null-pointer-use1 signed-integer-overflow4 value-outside-the-range-of-its-type4
ConcoLLMic, GPT-5.5	3	segv1	left-shift-of-a-negative-value1 misaligned-access1 null-pointer-use1 value-outside-the-range-of-its-type1
KLEE fixed options, 6 cores	3	segv1	left-shift-out-of-range2 null-pointer-use1 value-outside-the-range-of-its-type1
KLEE uniform options, 6 cores	3	segv1	left-shift-out-of-range5 misaligned-access1 null-pointer-use1 value-outside-the-range-of-its-type2
SymTuner, 6 cores	3	segv1	left-shift-out-of-range5 misaligned-access1 null-pointer-use1 value-outside-the-range-of-its-type2
AFL++, 1 core	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range3 null-pointer-use1
SymCC + AFL++, 1 core	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range3 null-pointer-use1
SymSan + AFL++, 1 core	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range3 null-pointer-use1
AFL++ rebuilt targets	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range3 null-pointer-use1
KLEE fixed options, 1 core	5	segv1	left-shift-out-of-range1 null-pointer-use1 value-outside-the-range-of-its-type1
KLEE uniform options, 1 core	5	segv1	left-shift-out-of-range5 misaligned-access1 null-pointer-use1 value-outside-the-range-of-its-type2
SymTuner, 1 core	5	segv1	left-shift-out-of-range5 null-pointer-use1 value-outside-the-range-of-its-type2
Agolic KLEE, GPT-5.4	3	segv1	left-shift-of-a-negative-value1 left-shift-out-of-range4 misaligned-access1 null-pointer-use1 signed-integer-overflow1 value-outside-the-range-of-its-type2

Counts are of untriaged signatures. † marks fewer than three runs.

Oggenc

Branch map

Coverage over time

Final coverage

Branches reached by one tool only

Sanitiser findings