deploy: 7410080

lm-sys · May 2, 2024 · 74dd289 · 74dd289
1 parent 5120d63
commit 74dd289
Show file tree

Hide file tree

Showing 89 changed files with 51 additions and 52 deletions.
diff --git a/404/index.html b/404/index.html
diff --git a/_next/data/UDKcIifEoEcTa4oi0k9Ur/blog.json b/_next/data/UDKcIifEoEcTa4oi0k9Ur/blog.json
diff --git a/_next/data/UDKcIifEoEcTa4oi0k9Ur/blog/2024-05-02-kaggle-competition.json b/_next/data/UDKcIifEoEcTa4oi0k9Ur/blog/2024-05-02-kaggle-competition.json
diff --git a/_next/data/UDKcIifEoEcTa4oi0k9Ur/about.json → _next/data/pMmg9QK3XeClKusCgm1fz/about.json b/_next/data/UDKcIifEoEcTa4oi0k9Ur/about.json → _next/data/pMmg9QK3XeClKusCgm1fz/about.json
diff --git a/_next/data/pMmg9QK3XeClKusCgm1fz/blog.json b/_next/data/pMmg9QK3XeClKusCgm1fz/blog.json
diff --git a/...oEcTa4oi0k9Ur/blog/2023-03-30-vicuna.json → ...XeClKusCgm1fz/blog/2023-03-30-vicuna.json b/...oEcTa4oi0k9Ur/blog/2023-03-30-vicuna.json → ...XeClKusCgm1fz/blog/2023-03-30-vicuna.json
diff --git a/...EoEcTa4oi0k9Ur/blog/2023-05-03-arena.json → ...3XeClKusCgm1fz/blog/2023-05-03-arena.json b/...EoEcTa4oi0k9Ur/blog/2023-05-03-arena.json → ...3XeClKusCgm1fz/blog/2023-05-03-arena.json
diff --git a/...4oi0k9Ur/blog/2023-05-10-leaderboard.json → ...usCgm1fz/blog/2023-05-10-leaderboard.json b/...4oi0k9Ur/blog/2023-05-10-leaderboard.json → ...usCgm1fz/blog/2023-05-10-leaderboard.json
diff --git a/...4oi0k9Ur/blog/2023-05-25-leaderboard.json → ...usCgm1fz/blog/2023-05-25-leaderboard.json b/...4oi0k9Ur/blog/2023-05-25-leaderboard.json → ...usCgm1fz/blog/2023-05-25-leaderboard.json
diff --git a/...a4oi0k9Ur/blog/2023-06-09-api-server.json → ...KusCgm1fz/blog/2023-06-09-api-server.json b/...a4oi0k9Ur/blog/2023-06-09-api-server.json → ...KusCgm1fz/blog/2023-06-09-api-server.json
diff --git a/...4oi0k9Ur/blog/2023-06-22-leaderboard.json → ...usCgm1fz/blog/2023-06-22-leaderboard.json b/...4oi0k9Ur/blog/2023-06-22-leaderboard.json → ...usCgm1fz/blog/2023-06-22-leaderboard.json
diff --git a/...cTa4oi0k9Ur/blog/2023-06-29-longchat.json → ...ClKusCgm1fz/blog/2023-06-29-longchat.json b/...cTa4oi0k9Ur/blog/2023-06-29-longchat.json → ...ClKusCgm1fz/blog/2023-06-29-longchat.json
diff --git a/...EcTa4oi0k9Ur/blog/2023-07-20-dataset.json → ...eClKusCgm1fz/blog/2023-07-20-dataset.json b/...EcTa4oi0k9Ur/blog/2023-07-20-dataset.json → ...eClKusCgm1fz/blog/2023-07-20-dataset.json
diff --git a/...Ta4oi0k9Ur/blog/2023-10-30-toxicchat.json → ...lKusCgm1fz/blog/2023-10-30-toxicchat.json b/...Ta4oi0k9Ur/blog/2023-10-30-toxicchat.json → ...lKusCgm1fz/blog/2023-10-30-toxicchat.json
diff --git a/...r/blog/2023-11-14-llm-decontaminator.json → ...z/blog/2023-11-14-llm-decontaminator.json b/...r/blog/2023-11-14-llm-decontaminator.json → ...z/blog/2023-11-14-llm-decontaminator.json
diff --git a/...EoEcTa4oi0k9Ur/blog/2023-11-15-slora.json → ...3XeClKusCgm1fz/blog/2023-11-15-slora.json b/...EoEcTa4oi0k9Ur/blog/2023-11-15-slora.json → ...3XeClKusCgm1fz/blog/2023-11-15-slora.json
diff --git a/...r/blog/2023-11-21-lookahead-decoding.json → ...z/blog/2023-11-21-lookahead-decoding.json b/...r/blog/2023-11-21-lookahead-decoding.json → ...z/blog/2023-11-21-lookahead-decoding.json
diff --git a/...4oi0k9Ur/blog/2023-12-07-leaderboard.json → ...usCgm1fz/blog/2023-12-07-leaderboard.json b/...4oi0k9Ur/blog/2023-12-07-leaderboard.json → ...usCgm1fz/blog/2023-12-07-leaderboard.json
diff --git a/...oEcTa4oi0k9Ur/blog/2024-01-17-sglang.json → ...XeClKusCgm1fz/blog/2024-01-17-sglang.json b/...oEcTa4oi0k9Ur/blog/2024-01-17-sglang.json → ...XeClKusCgm1fz/blog/2024-01-17-sglang.json
diff --git a/...0k9Ur/blog/2024-02-05-compressed-fsm.json → ...gm1fz/blog/2024-02-05-compressed-fsm.json b/...0k9Ur/blog/2024-02-05-compressed-fsm.json → ...gm1fz/blog/2024-02-05-compressed-fsm.json
diff --git a/...oEcTa4oi0k9Ur/blog/2024-03-01-policy.json → ...XeClKusCgm1fz/blog/2024-03-01-policy.json b/...oEcTa4oi0k9Ur/blog/2024-03-01-policy.json → ...XeClKusCgm1fz/blog/2024-03-01-policy.json
diff --git a/...a4oi0k9Ur/blog/2024-04-19-arena-hard.json → ...KusCgm1fz/blog/2024-04-19-arena-hard.json b/...a4oi0k9Ur/blog/2024-04-19-arena-hard.json → ...KusCgm1fz/blog/2024-04-19-arena-hard.json
diff --git a/_next/data/pMmg9QK3XeClKusCgm1fz/blog/2024-05-02-kaggle-competition.json b/_next/data/pMmg9QK3XeClKusCgm1fz/blog/2024-05-02-kaggle-competition.json
@@ -0,0 +1 @@
+{"pageProps":{"frontmatter":{"title":"LMSYS Kaggle Competition – Predicting Human Preference with $100,000 in Prizes","author":"LMSYS Arena Team","date":"May 2, 2024","previewImg":"/images/blog/kaggle_competition/thumb_4x.png"},"content":"\nLMSYS and Kaggle are launching a human preference prediction competition! You are challenged to predict which responses users will prefer in head-to-head battles between Large Language Models (LLMs). You'll work with a dataset from the [Chatbot Arena](https://chat.lmsys.org), containing conversations and user preferences across various LLMs. By developing a model that accurately predicts human preferences, you'll contribute to improving chatbot performance and alignment with user expectations. The training dataset includes over 55,000 real-world user and LLM conversations and user preferences, with personally identifiable information removed. Your solution submission will be tested on a hidden test set of 25,000 samples.\nThe dataset includes real-world conversations with over 70 state-of-the-art LLMs, such as GPT-4, Claude 2, Llama 2, and Mistral models. [Click here to join the competition.](https://www.kaggle.com/competitions/lmsys-chatbot-arena/overview)\n\n<img src=\"/images/blog/kaggle_competition/header_4x.png\" style=\"width: 60%; max-width: 60%; margin-left: auto; margin-right: auto; margin-top: 0px; margin-bottom: 0px\"></img>\n\nCurrent LLM benchmarks often fail to capture real-world LLM usage, resulting in a discrepancy between model performance and user satisfaction. Platforms like Chatbot Arena allow users to submit questions and vote on preferred responses; however, the potential of this data has been largely untapped in developing models that predict and optimize for user preferences at scale. Predicting user preferences is essential for creating human-aligned conversational AI that delivers a satisfying user experience. Successful models could enable language models to dynamically adapt their output based on individual preferences across different contexts and use cases. Moreover, this competition aims to uncover the factors that drive user preferences beyond objective correctness. Many user questions are open-ended, and we have already found a correlation between user preference and subjective qualities like conversationality. This could also be one of the best testbeds for reward modeling in your RLHF algorithms.\n\nThe competition will run until August 5th, **with a total prize of $100,000**, featuring a $25,000 prize for 1st place, 20,000 prizes for 2nd through 4th places, and a 15,000 prize for 5th place. This is your opportunity to contribute to the advancement of human-aligned language models while gaining valuable insights into human preferences and decision-making. These insights could provide value to both the computer science and psychology communities, shedding light on the factors that shape human preferences in conversational AI.\n","slug":"2024-05-02-kaggle-competition"},"__N_SSG":true}
diff --git a/...data/UDKcIifEoEcTa4oi0k9Ur/donations.json → ...data/pMmg9QK3XeClKusCgm1fz/donations.json b/...data/UDKcIifEoEcTa4oi0k9Ur/donations.json → ...data/pMmg9QK3XeClKusCgm1fz/donations.json
diff --git a/...ta/UDKcIifEoEcTa4oi0k9Ur/vicuna_eval.json → ...ta/pMmg9QK3XeClKusCgm1fz/vicuna_eval.json b/...ta/UDKcIifEoEcTa4oi0k9Ur/vicuna_eval.json → ...ta/pMmg9QK3XeClKusCgm1fz/vicuna_eval.json
diff --git a/...c/UDKcIifEoEcTa4oi0k9Ur/_buildManifest.js → ...c/pMmg9QK3XeClKusCgm1fz/_buildManifest.js b/...c/UDKcIifEoEcTa4oi0k9Ur/_buildManifest.js → ...c/pMmg9QK3XeClKusCgm1fz/_buildManifest.js
diff --git a/...cIifEoEcTa4oi0k9Ur/_middlewareManifest.js → ...g9QK3XeClKusCgm1fz/_middlewareManifest.js b/...cIifEoEcTa4oi0k9Ur/_middlewareManifest.js → ...g9QK3XeClKusCgm1fz/_middlewareManifest.js
diff --git a/...tic/UDKcIifEoEcTa4oi0k9Ur/_ssgManifest.js → ...tic/pMmg9QK3XeClKusCgm1fz/_ssgManifest.js b/...tic/UDKcIifEoEcTa4oi0k9Ur/_ssgManifest.js → ...tic/pMmg9QK3XeClKusCgm1fz/_ssgManifest.js
diff --git a/about/index.html b/about/index.html
diff --git a/blog/2023-03-30-vicuna/index.html b/blog/2023-03-30-vicuna/index.html
diff --git a/blog/2023-05-03-arena/index.html b/blog/2023-05-03-arena/index.html
diff --git a/blog/2023-05-10-leaderboard/index.html b/blog/2023-05-10-leaderboard/index.html
diff --git a/blog/2023-05-25-leaderboard/index.html b/blog/2023-05-25-leaderboard/index.html
diff --git a/blog/2023-06-09-api-server/index.html b/blog/2023-06-09-api-server/index.html
diff --git a/blog/2023-06-22-leaderboard/index.html b/blog/2023-06-22-leaderboard/index.html
diff --git a/blog/2023-06-29-longchat/index.html b/blog/2023-06-29-longchat/index.html
diff --git a/blog/2023-07-20-dataset/index.html b/blog/2023-07-20-dataset/index.html
diff --git a/blog/2023-10-30-toxicchat/index.html b/blog/2023-10-30-toxicchat/index.html
diff --git a/blog/2023-11-14-llm-decontaminator/index.html b/blog/2023-11-14-llm-decontaminator/index.html
diff --git a/blog/2023-11-15-slora/index.html b/blog/2023-11-15-slora/index.html
diff --git a/blog/2023-11-21-lookahead-decoding/index.html b/blog/2023-11-21-lookahead-decoding/index.html
diff --git a/blog/2023-12-07-leaderboard/index.html b/blog/2023-12-07-leaderboard/index.html
diff --git a/blog/2024-01-17-sglang/index.html b/blog/2024-01-17-sglang/index.html
diff --git a/blog/2024-02-05-compressed-fsm/index.html b/blog/2024-02-05-compressed-fsm/index.html
diff --git a/blog/2024-03-01-policy/index.html b/blog/2024-03-01-policy/index.html
diff --git a/blog/2024-04-19-arena-hard/index.html b/blog/2024-04-19-arena-hard/index.html
diff --git a/blog/2024-05-02-kaggle-competition/index.html b/blog/2024-05-02-kaggle-competition/index.html
diff --git a/blog/index.html b/blog/index.html
diff --git a/donations/index.html b/donations/index.html
diff --git a/images/blog/kaggle_competition/header_4x.png b/images/blog/kaggle_competition/header_4x.png
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-10.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-10.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1080.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1080.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1200.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1200.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-128.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-128.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-16.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-16.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1920.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-1920.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-2048.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-2048.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-256.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-256.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-32.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-32.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-384.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-384.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-3840.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-3840.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-48.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-48.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-64.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-64.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-640.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-640.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-750.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-750.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-828.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-828.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-96.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/header_4x-opt-96.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-10.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-10.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1080.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1080.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1200.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1200.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-128.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-128.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-16.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-16.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1920.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-1920.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-2048.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-2048.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-256.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-256.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-32.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-32.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-384.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-384.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-3840.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-3840.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-48.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-48.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-64.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-64.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-640.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-640.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-750.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-750.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-828.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-828.WEBP
diff --git a/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-96.WEBP b/images/blog/kaggle_competition/nextImageExportOptimizer/thumb_4x-opt-96.WEBP
diff --git a/images/blog/kaggle_competition/thumb_4x.png b/images/blog/kaggle_competition/thumb_4x.png
diff --git a/index.html b/index.html
diff --git a/projects/index.html b/projects/index.html
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		{"pageProps":{"frontmatter":{"title":"LMSYS Kaggle Competition – Predicting Human Preference with $100,000 in Prizes","author":"LMSYS Arena Team","date":"May 2, 2024","previewImg":"/images/blog/kaggle_competition/thumb_4x.png"},"content":"\nLMSYS and Kaggle are launching a human preference prediction competition! You are challenged to predict which responses users will prefer in head-to-head battles between Large Language Models (LLMs). You'll work with a dataset from the [Chatbot Arena](https://chat.lmsys.org), containing conversations and user preferences across various LLMs. By developing a model that accurately predicts human preferences, you'll contribute to improving chatbot performance and alignment with user expectations. The training dataset includes over 55,000 real-world user and LLM conversations and user preferences, with personally identifiable information removed. Your solution submission will be tested on a hidden test set of 25,000 samples.\nThe dataset includes real-world conversations with over 70 state-of-the-art LLMs, such as GPT-4, Claude 2, Llama 2, and Mistral models. [Click here to join the competition.](https://www.kaggle.com/competitions/lmsys-chatbot-arena/overview)\n\n<img src=\"/images/blog/kaggle_competition/header_4x.png\" style=\"width: 60%; max-width: 60%; margin-left: auto; margin-right: auto; margin-top: 0px; margin-bottom: 0px\"></img>\n\nCurrent LLM benchmarks often fail to capture real-world LLM usage, resulting in a discrepancy between model performance and user satisfaction. Platforms like Chatbot Arena allow users to submit questions and vote on preferred responses; however, the potential of this data has been largely untapped in developing models that predict and optimize for user preferences at scale. Predicting user preferences is essential for creating human-aligned conversational AI that delivers a satisfying user experience. Successful models could enable language models to dynamically adapt their output based on individual preferences across different contexts and use cases. Moreover, this competition aims to uncover the factors that drive user preferences beyond objective correctness. Many user questions are open-ended, and we have already found a correlation between user preference and subjective qualities like conversationality. This could also be one of the best testbeds for reward modeling in your RLHF algorithms.\n\nThe competition will run until August 5th, with a total prize of $100,000, featuring a $25,000 prize for 1st place, 20,000 prizes for 2nd through 4th places, and a 15,000 prize for 5th place. This is your opportunity to contribute to the advancement of human-aligned language models while gaining valuable insights into human preferences and decision-making. These insights could provide value to both the computer science and psychology communities, shedding light on the factors that shape human preferences in conversational AI.\n","slug":"2024-05-02-kaggle-competition"},"__N_SSG":true}