Agent Software Testing

Releases

A release is a dated snapshot of the data.

Terms
Release
A dated snapshot of the data, named like 2026.1. It never changes; corrections go into the next release.
Entry
A tool in one version and configuration, with a model, a track and a budget. One row of the data.
Superseded
An entry that a newer one replaces. It stays in the release and shows under all versions.
Withdrawn
An entry taken out of the comparison after it was published.
RELEASE.json in the results repository
The releases
ReleaseDateEntriesToolsProgramsRunsAddedSupersededWithdrawn
2026.12 November 2026287188482810

Added: entries new in this release. Superseded and withdrawn: entries with that status in this release.

Measured with

Specification
0.1-draft
Replay image
0617e1767f9b99be3abd23e20f127f7e33f405cefdd2fb6452e06ade5643c580
Runner
commonreplay 0.1.4.dev0+fbc76f4c7308c1cf671094924003d1a40677eda66e224d096e6e18b6ed68affe
Profile
standard