Open

Bring any domain where experts can judge the work: law, finance, biology, materials, long-form writing. Score the complete deliverable, not a quiz about it. The best model finishes under a fifth of long, realistic professional tasks, and stories lose quality chapter by chapter.

Benchmarks to know

• Legal Research ↗Vals AI

  Multi-source US legal research across eight practice areas; an answer counts only if every rubric item is met.

• GDPx ↗SemiAnalysis

  206 long professional tasks across 11 domains; the best model completes under a fifth, and most score zero in finance and law.

• Long Creative Writing ↗EQ-Bench

  An eight-chapter story from a bare prompt; models drift into repetition and stock phrasing as it goes on.

• Access to Justice ↗Vals AI

  Models explain the law well but rarely point people to the local help they need.

• BioMysteryBench ↗Anthropic, on Vals AI

  Real biology data investigations, where a failing model gives a confident wrong answer instead of saying it is unsure.

• BallotBench ↗Thoughtful Lab

  Agents forecast and called all 863 races of the 2026 midterms; prediction markets were better calibrated than every one.

• First Proof ↗First Proof Foundation

  Ten unpublished research results, proofs graded blind by 30 mathematicians; right answers often come badly argued and poorly cited.

• AccountingBench ↗Penrose

  Close a real SaaS company's books for a year; top models match a CPA early, then drift and revert to cash accounting.

• ForecastBench ↗Forecasting Research Institute

  Real questions about the future, scored once they resolve; the best AI only just matches superforecasters.

Bring an eval in this track, or in any other, on Saturday, December 5, 2026 at Cornell University.