Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
Basic
3
Ṁ75Jan 1
25%
chance
1D
1W
1M
ALL
Will any company release a pure Mamaba LLM that scores over 82 points on the standard full MMLU (5-shot) using https://github.com/EleutherAI/lm-evaluation-harness with open source weights?
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
9% chance
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
37% chance
🐕 OpenLLMs: Will Any Open Source LLM on the HuggingFace OpenLLM Leaderboard Significantly Gain in Avg Score by YE 2024?
49% chance
Will any open-source model achieve GPT-4 level performance on MMLU through 2024?
83% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
79% chance
Will there be an open source LLM as good as GPT4 by the end of 2024?
68% chance
What will be the maximum context length of any OpenAI LLM by EOY 2024?
Will an open-source LLM beat or match GPT-4 by the end of 2024?
81% chance
What will be true of OpenAI's best LLM by EOY 2025?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance