Track A · First experiment planned within 90 days
Do outcome verifiers reduce false completion?
- Second question
- How should pass@k and pass^k be reported together?
- First experiment
- Identical workflows run with and without authoritative state checks; measure the false-success rate across repeated trials.
- Output
- A failure taxonomy, a verifier implementation, and a public evaluation method with its limits.
- What we will not claim
- A result on one task family is not a result on all tasks. We state the task distribution with every number.
Publications from this track
Nothing published yet. The first evaluation method is due within 90 days of October 2026. Each report will be reviewed by someone who did not build the system.
Work with the programme
We run experiments on customer workflows only with written permission and on permitted data. If you have a workflow and a question, write to research@chainfore.com.
Research collaboration →