Agent Software Testing

KLEE

A symbolic execution engine for LLVM bitcode. The entries run it under its symbolic-argument harness, with fixed options or with options sampled uniformly from SymTuner's parameter space.

Terms
Release
A dated snapshot of the data, named like 2026.1. It never changes; corrections go into the next release.
Entry
A tool in one version and configuration, with a model, a track and a budget. One row of the data.
Superseded
An entry that a newer one replaces. It stays in the release and shows under all versions.
Withdrawn
An entry taken out of the comparison after it was published.
All tools

Project: github.com/klee/klee

Reference: www.usenix.org/conference/osdi-08/klee-unassisted-and-automatic-generation-high-coverage-tests-complex-systems

Versions

Every entry of KLEE
EntryVersionModelConfigurationSubmittedFirst releasePrograms coveredStatus
KLEE fixed options, 6 coresefa1e052–fixed options, default search; symbolic-argument harness; six workers2026-10-162026.118 of 18current
KLEE uniform options, 6 coresefa1e052–options sampled uniformly from SymTuner's parameter space; symbolic-argument harness; six workers2026-10-162026.118 of 18current
KLEE fixed options, 1 coreefa1e052–fixed options, default search; symbolic-argument harness; one worker2026-10-162026.118 of 18current
KLEE uniform options, 1 coreefa1e052–options sampled uniformly from SymTuner's parameter space; symbolic-argument harness; one worker2026-10-162026.118 of 18current

Median coverage by version

Median final branch coverage of every entry of KLEE by program
ProgramKLEE fixed options6 coresefa1e0522026-10-16KLEE uniform options6 coresefa1e0522026-10-16
bc22.4%, 3 runs27.4%, 3 runs
Confetti32.3%, 3 runs40.7%, 3 runs
diff21.4%, 3 runs31.9%, 3 runs
du14.5%, 3 runs18.0%, 3 runs
gawk10.1%, 3 runs23.8%, 3 runs
gcal22.6%, 3 runs43.4%, 3 runs
grep36.2%, 3 runs47.8%, 3 runs
gzip35.6%, 3 runs46.4%, 3 runs
jq8.6%, 3 runs13.4%, 3 runs
Krep47.1%, 3 runs55.0%, 3 runs
Libmatheval24.7%, 3 runs44.5%, 3 runs
LibYAML41.0%, 3 runs52.5%, 3 runs
ls28.8%, 3 runs39.6%, 3 runs
Oggenc17.7%, 3 runs29.6%, 3 runs
sed17.9%, 3 runs42.0%, 3 runs
WOFF21.4%, 3 runs7.6%, 3 runs
xmlwf21.6%, 3 runs31.9%, 3 runs
xorriso6.4%, 3 runs6.5%, 3 runs