What will be the best score (5/5 reliability) on ZeroBench by December 31st 2025?
2
Ṁ72
2026
21%
0 - 10
13%
11 - 20
8%
21 - 30
8%
31 - 40
8%
41 - 50
8%
51 - 60
8%
61 - 70
8%
71 - 80
8%
81 - 90
9%
91 - 100

ZeroBench is a benchmark for visual reasoning, introduced by Roberts et al. in "ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models" (https://arxiv.org/abs/2502.09696).

This market will use the variant of the benchmark frozen one week after the initial release (following the public benchmark red-teaming stage to identify flawed/ambiguous questions).

The temperature used for the 5/5 reliability evaluation will be the default setting provided by each LLM API provider. In cases where this default is ambiguous to determine, we will default to a temperature of 0.7.

Get
Ṁ1,000
and
S3.00
Sort by:

As of March 28th 2025, Gemini 2.5 Pro is the new SotA: https://x.com/JRobertsAI/status/1905577784300183653

3% pass@1

5% pass@5

1% 5/5 reliability

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules