- Coverage
- Each tool’s test cases are replayed and counted on the branches of one pinned gcov build per program. Every replay runs in a container that sees only its own processes.
- Runs
- † marks fewer than three runs. A run lasts six hours by default (other budgets labelled). Tables show medians.
- Tracks
- On the fixed track the tool supplies only the input to a fixed command line; on the open track it also chooses arguments, files and environment. Results on different tracks do not compare; budgets and cores change what a result means.
- Statistics
- A12 follows section 9 of SPEC.md.
- Limits
- A crash or timeout writes no coverage. A model may have seen the programs. Of 28 entries, 14 are variants of the maintainers’ agent, Agolic. One more starts from its seeds. The maintainers ran, configured or ported every baseline. Budgets, cores and tokens are self-reported. Coverage counts the branches that test cases reach. A sanitiser signature is an untriaged report.
- Specification
- SPEC.md in the
agent-software-testing repository defines the replay and governs where this page differs.