When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
Basic
7
Ṁ3062032
4%
5%
Before 2023
11%
2023-2023
17%
2024-2025
25%
2025-2026
18%
2026-2027
17%
2027-2032
1.6%
2032-never
10@k - unlimited samples may be generated, but at most 10 code samples may be submitted per problem.
I will post my own responses and pick whichever one of those turns out to be correct.
CodeContests dataset: https://github.com/deepmind/code_contests
Close date updated to 2050-01-01 11:59 pm
Apr 8, 4:16pm: Options are written as [a, b), so 2024-2025 means occurring after the start of 2024 and before the start of 2025.
Close date updated to 2032-01-01 11:59 pm
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
@ValeryCherepanov Yeah the before 2023 option seems like free money to me, I think the market is just old and people don't have it on their radar.
Related questions
Related questions
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
81% chance
User's Codeforces rating reaches Candidate Master (1900+) by February 2025
3% chance
Will an AI achieve >80% performance on the FrontierMath benchmark before 2027?
81% chance
Will GPT-5 reach a 1000 rating on Codeforces?
95% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
68% chance
Top score on codeforces by an AI model at the end of 2025
By when will SAT solvers be 10x faster (than 2023 on same hardware)?
When will an AI model be better than me at competitive programming?
Will an AI model achieve superhuman ELO on Codeforces by the 31 December 2025?
61% chance
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
26% chance