Which organization's large language model (LLM) will be ranked first as of 15 November 2024, according to the Large Model Systems Organization (LMSYS Org)?

Started May 10, 2024 05:00PM UTC
Closed Nov 15, 2024 08:01AM UTC
Challenges
Tags

Technology companies and organizations are racing to develop the best performing chatbots (The Verge). The question will be suspended on 14 November 2024 and the outcome determined using the ranks as reported by LMSYS Org at approximately 5:00PM ET on 15 November 2024 (LMSYS, see "Rank* (UB)"). As of 8 May 2024, OpenAI was ranked first with its "GPT-4-Turbo-2024-04-09" model with an Arena Elo of 1258. For more information on how the leaderboard is constructed, see (LMSYS Blog). In the event of a tie for first place by LLMs of different organizations, the LLM with the higher Arena Elo score will be considered first, followed by the "Votes" total.

Confused? Check our FAQ or ask us for help. To learn more about Good Judgment and Superforecasting, click here.

To learn more about how you can become a Superforecaster, see hereFor other posts from our Insights blog, click here.


The question closed "Google" with a closing date of 15 November 2024.

See our FAQ to learn about how we resolve questions and how scores are calculated.

Possible Answer Correct? Final Crowd Forecast
Anthropic 0%
Google 25%
Meta 0%
OpenAI 75%
Another organization 0%

Crowd Forecast Profile

Participation Level
Number of Forecasters 39
Average for questions older than 6 months: 187
Number of Forecasts 218
Average for questions older than 6 months: 542
Accuracy
Participants in this question vs. all forecasters better than average

Most Accurate

Relative Brier Score

1.
-0.924906
2.
-0.532848
3.
-0.464407
4.
-0.464335
5.
-0.436178

Recent Consensus, Probability Over Time

Files
Tip: Mention someone by typing @username