Issue Spotting
Identify the legal issues a Canadian appellate court had to decide from the facts of the decision — testing issue recognition.
Judge-scored column, strict since 2026-10-05: a record counts when every key issue is identified; additional issues are not penalized. The judge matches the model's issues to the key issue by issue; the score counts the records where every key issue was found. Two diagnostics are shown beside each model: the exact-set-match rate (every key issue found and no extra items) and the lenient pass rate (set F1 ≥ 0.5).
† the model's shared thinking and answer token budget ran out before it wrote any answer.‡ the provider's safety filter blocked the answer, or the answer came back empty with no recorded finish reason.These answers stay in the denominator and are scored 0; the count is disclosed per cell. Empty-output policy: an answer that came back empty is never removed from its cell; it counts as not correct against the frozen record count, and the number of such answers and their cause are published with the cell.
- Cohere Command A+: † 28 answers hit the token budget before any answer token; scored 0
- Gemini 3.1 Pro: ‡ 8 answers blocked by the provider's safety filter (empty output); scored 0