Will a text model achieve 100% performance on the MMLU in five years?
Basic
3
Ṁ402028
28%
chance
1D
1W
1M
ALL
This video predicts that a model will achieve 100% performance on the Massive Multitask Language Understanding Benchmark.
This will resolve YES if a model is listed as having achieved 100 percent performance on the MMLU on papers with code, in a technical document, or published article by 3:00 PM EST June 28, 2028. This will resolve NO if a model is not listed as ahving achieved 100 percent performance on the MMLU.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
By 2025 will there be a competitive large language model with >50% of the total training data generated from a large language model?
75% chance
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
58% chance
Who will have the best Text-to-Image Model at the end of 2025 (as decided by the Artificial Analysis Leaderboard)?
Will an open-source text-to-music LLM model be able to create an entire album as convincing as a human by end of 2025?
26% chance
Who will have the best Text-to-Video Model at the end of 2025 (as decided by the Artificial Analysis Leaderboard)?
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
68% chance
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance
Will a team consisting primarily of ML models complete MIT Mystery Hunt by 2030?
45% chance