Will xAI release an LLM with BIG-Bench score as good as GPT-4 Turbo before the end of 2024? | Manifold

Will xAI release an LLM with BIG-Bench score as good as GPT-4 Turbo before the end of 2024?

Plus

50

Ṁ36k

Dec 31

55%

chance

1D

1W

1M

ALL

xAI is Elon Musk's new AI startup, founded in Mar 2023. It's rumored that they have already bought thousands of GPUs. Musk probably has the capital and charisma to poach top engineers. https://www.theverge.com/2023/4/14/23684005/elon-musk-new-ai-company-x

This question is managed and resolved by Manifold.

Get

1,000

and

3.00

Sort by:

@Ryannn what is the criterion by which you are going to resolve this question? The leaderboard for BIG-Bench Lite hasn't been updated in a long while, and does not list either model. The BigCodeBench Leaderboard (hard) is regularly updated, it currently lists an un-specified model "Grok-Beta". Does the BigCodeBench leaderboard suffice for market resolution?

reposted

Cool question!

bought Ṁ100 YES

Well they claim Grok 2 beats gpt 4 turbo on mmlu. We will need to wait for API access to do an evaluation on big bench.

I think the question should specify the GPT-4 version.

@HannesLynchburg agreed

some clarity would be good. I've been assuming gpt4's release version.

@HannesLynchburg GPT-4 Turbo

https://www.ft.com/content/19cce4b5-b2dd-4c1e-a109-b500e504dbb6

Subscribe to read | Financial Times

News, analysis and comment from the Financial Times, the worldʼs leading global business publication

@RyanKidd Paywalled, what does it say?

Related questions

Will xAI develop a more capable LLM than GPT-5 by 2026

China will make a LLM approximately as good or better than GPT4 before 2025

Will there be an open source LLM as good as GPT4 by June 2024?

Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?

Will an uncensored open-source LLM model comparable to 2023 GPT4 be available to the public by the end of 2025?

Will an open-source LLM beat or match GPT-4 by the end of 2024?

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

Will GPT-4-Turbo be ranked in the top 20 on the Chatbot Arena Leaderboard at the end of 2025?

How much time will pass between an LLM being released that beats GPT4 and the next OpenAI LLM being released? (+ANSWERS)

When will an open-source LLM be released with a better performance than GPT-4?

Related questions

Will xAI develop a more capable LLM than GPT-5 by 2026

Will an open-source LLM beat or match GPT-4 by the end of 2024?

China will make a LLM approximately as good or better than GPT4 before 2025

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

Will there be an open source LLM as good as GPT4 by June 2024?

Will GPT-4-Turbo be ranked in the top 20 on the Chatbot Arena Leaderboard at the end of 2025?

Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?

How much time will pass between an LLM being released that beats GPT4 and the next OpenAI LLM being released? (+ANSWERS)

Will an uncensored open-source LLM model comparable to 2023 GPT4 be available to the public by the end of 2025?

When will an open-source LLM be released with a better performance than GPT-4?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules