{"updated":"2026-10-31","source":"https://benchdata.branolabs.com","license":"CC BY 4.0 (this directory's own data; each leaderboard's data keeps its own licence, see licence per entry)","note":"Where each LLM leaderboard publishes its raw data, its licence or terms (verbatim quote + source URL), submission rules and the newest date read from the source (last commit, dataset lastModified or a stated date). No benchmark scores. null means not published / not found, not 'no'. Not legal advice.","count":40,"leaderboards":[{"slug":"aider-polyglot","name":"Aider LLM Leaderboards (polyglot)","operator":"Aider (Paul Gauthier)","url":"https://aider.chat/docs/leaderboards/","page":"https://benchdata.branolabs.com/l/aider-polyglot","category":"Coding","test_runner":"own_runs","summary":"Code-editing benchmark across languages. Results are kept as a YAML file in the aider repository.","raw_data":[{"kind":"GitHub file","url":"https://github.com/Aider-AI/aider/blob/main/aider/website/_data/polyglot_leaderboard.yml","note":"YAML file behind the leaderboard table.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"attribution","scope":"repo","note":null,"evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/Aider-AI/aider/main/LICENSE.txt","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":"The page has a \"Contributing results\" section.","evidence":null,"more":null},"freshness":[{"date":"2025-10-04","method":"Last commit touching aider/website/_data/polyglot_leaderboard.yml","url":"https://github.com/Aider-AI/aider/commits/HEAD/aider/website/_data/polyglot_leaderboard.yml","quote":null},{"date":"2025-11-20","method":"Date stated on the page","url":"https://aider.chat/docs/leaderboards/","quote":"last updated November 20, 2025","quote_verified":true}],"last_updated":"2025-11-20","status":"stale","last_checked":"2026-10-31"},{"slug":"alpacaeval","name":"AlpacaEval","operator":"Stanford Tatsu Lab","url":"https://tatsu-lab.github.io/alpaca_eval/","page":"https://benchdata.branolabs.com/l/alpacaeval","category":"General chat","test_runner":"submissions","summary":"LLM-judged instruction following with length-controlled win rates. Leaderboard files are in the repo.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/tatsu-lab/alpaca_eval","note":"Leaderboard CSVs and model outputs in the repo.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0 (repo)","verdict":"attribution","scope":"repo","note":"GitHub reports Apache-2.0 for the repo that holds the leaderboard files.","evidence":null,"more":null},"submissions":{"status":"open","note":null,"evidence":{"quote":"Contributing a model First, you'll need to add a model config definition","source":"https://raw.githubusercontent.com/tatsu-lab/alpaca_eval/main/README.md","quote_verified":true},"more":null},"freshness":[{"date":"2025-08-09","method":"Last commit to the GitHub repo","url":"https://github.com/tatsu-lab/alpaca_eval/commits","quote":null}],"last_updated":"2025-08-09","status":"stale","last_checked":"2026-10-31"},{"slug":"arc-prize","name":"ARC Prize leaderboard (ARC-AGI)","operator":"ARC Prize Foundation","url":"https://arcprize.org/leaderboard","page":"https://benchdata.branolabs.com/l/arc-prize","category":"Reasoning","test_runner":"own_runs","summary":"ARC-AGI-1/2/3 results tested under the ARC Prize Verified policy, plus Kaggle competition entries.","raw_data":null,"raw_data_note":"Task sets are on GitHub; we found no result download.","licence":{"label":"No licence found for results","verdict":"no_licence","scope":"site","note":"ARC-AGI-2 task repo is Apache-2.0; no licence statement found for leaderboard results.","evidence":null,"more":null},"submissions":{"status":"restricted","note":"Results are tested by the foundation under its published testing policy.","evidence":{"quote":"This document describes how ARC Prize Foundation tests AI systems, how we select what to test, and how we publish results.","source":"https://arcprize.org/policy","quote_verified":true},"more":null},"freshness":[],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"artificial-analysis","name":"Artificial Analysis","operator":"Artificial Analysis","url":"https://artificialanalysis.ai/","page":"https://benchdata.branolabs.com/l/artificial-analysis","category":"General / multi-benchmark","test_runner":"own_runs","summary":"Independent model and provider evaluations (intelligence, speed, price). Data is available through an API with attribution.","raw_data":[{"kind":"API","url":"https://artificialanalysis.ai/data-api/docs","note":"Data API; attribution required on all tiers.","evidence":{"quote":"Use of the API requires attribution across all tiers.","source":"https://artificialanalysis.ai/data-api/docs","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"Proprietary terms","verdict":"prohibited","scope":"Data Platform Terms","note":null,"evidence":{"quote":"Customer shall not use the Data to develop, operate, or improve any product or service made available to third parties whose primary purpose is benchmarking, ranking, comparison, competitive intelligence, or model/provider selection guidance, without Company's prior written consent.","source":"https://artificialanalysiscdn.com/legal/ProDataPlatformTerms.pdf","quote_verified":true},"more":{"quote":"Use of the API is also subject to our Terms of Use and Data Platform Terms","source":"https://artificialanalysis.ai/data-api/docs","quote_verified":true}},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"benchleader","name":"BenchLeader","operator":"BenchLeader","url":"https://www.benchleader.com/","page":"https://benchdata.branolabs.com/l/benchleader","category":"Aggregator","test_runner":"aggregator","summary":"Daily-rebuilt index over 252 benchmarks from 44 publishers, with JSON and CSV of top configurations.","raw_data":[{"kind":"JSON / CSV","url":"https://www.benchleader.com/data/leaderboard.json","note":"Also /data/leaderboard.csv.","evidence":{"quote":"The day's top configurations are published as JSON and CSV .","source":"https://www.benchleader.com/about","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"CC BY 4.0 (own figures)","verdict":"attribution","scope":"own computed figures","note":null,"evidence":{"quote":"BenchLeader's own computed figures (the index, interval, category scores and ranks) are licensed CC BY 4.0","source":"https://www.benchleader.com/about","quote_verified":true},"more":{"quote":"prices, speeds and the underlying benchmark results are not relicensed and stay under their publishers' own terms","source":"https://www.benchleader.com/about","quote_verified":true}},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":null,"method":"Date stated on the page","url":"https://www.benchleader.com/about","quote":"BenchLeader is rebuilt daily at 22:03 UTC.","quote_verified":true}],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"benchlm","name":"BenchLM","operator":"BenchLM.ai","url":"https://benchlm.ai/","page":"https://benchdata.branolabs.com/l/benchlm","category":"Aggregator","test_runner":"aggregator","summary":"Aggregates scores from many publisher leaderboards; offers JSON downloads, an API and MCP.","raw_data":[{"kind":"JSON download / API","url":"https://benchlm.ai/data","note":"Website JSON files plus a query API.","evidence":null}],"raw_data_note":null,"licence":{"label":"CC BY-NC 4.0","verdict":"noncommercial","scope":"website exports","note":null,"evidence":{"quote":"The website JSON downloads and benchlm.ai/api/data exports are published under CC BY-NC 4.0","source":"https://benchlm.ai/data","quote_verified":true},"more":{"quote":"Underlying publisher licenses and notices continue to apply.","source":"https://benchlm.ai/data","quote_verified":true}},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"bfcl","name":"Berkeley Function Calling Leaderboard (BFCL)","operator":"UC Berkeley (Gorilla)","url":"https://gorilla.cs.berkeley.edu/leaderboard.html","page":"https://benchdata.branolabs.com/l/bfcl","category":"Tool use","test_runner":"own_runs","summary":"Function and tool calling accuracy, including multi-turn. Evaluation code and test data are in the Gorilla repository.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard","note":"Evaluation code and test data; we did not find a published score file.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"unclear","scope":"repo","note":"Apache-2.0 covers the Gorilla repo. The leaderboard page states no licence for the scores.","evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/ShishirPatil/gorilla/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-04-12","method":"Date stated on the page","url":"https://gorilla.cs.berkeley.edu/leaderboard.html","quote":"Last Updated: 2026-04-12","quote_verified":true},{"date":"2026-03-23","method":"Last commit touching berkeley-function-call-leaderboard","url":"https://github.com/ShishirPatil/gorilla/commits/HEAD/berkeley-function-call-leaderboard","quote":null}],"last_updated":"2026-04-12","status":"stale","last_checked":"2026-10-31"},{"slug":"bigcodebench","name":"BigCodeBench","operator":"BigCode project","url":"https://bigcode-bench.github.io/","page":"https://benchdata.branolabs.com/l/bigcodebench","category":"Coding","test_runner":"own_runs","summary":"Practical coding tasks with library calls. The GitHub repository is archived.","raw_data":[{"kind":"HF space","url":"https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard","note":"Leaderboard space.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0 (code)","verdict":"unclear","scope":"repo","note":"GitHub reports Apache-2.0 for the code repo; no licence statement for the results.","evidence":null,"more":null},"submissions":{"status":"closed","note":"GitHub marks bigcode-project/bigcodebench as archived (read-only).","evidence":null,"more":null},"freshness":[{"date":"2025-02-04","method":"Hugging Face lastModified of Space bigcode/bigcodebench-leaderboard","url":"https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard","quote":null},{"date":"2025-10-15","method":"Last commit to the GitHub repo","url":"https://github.com/bigcode-project/bigcodebench/commits","quote":null}],"last_updated":"2025-10-15","status":"archived","last_checked":"2026-10-31"},{"slug":"epoch-ai-benchmarking-hub","name":"Epoch AI Benchmarking Hub","operator":"Epoch AI","url":"https://epoch.ai/benchmarks","page":"https://benchdata.branolabs.com/l/epoch-ai-benchmarking-hub","category":"General / multi-benchmark","test_runner":"own_runs","summary":"Database of benchmark results, partly run by Epoch AI and partly collected from external projects.","raw_data":[{"kind":"ZIP download","url":"https://epoch.ai/benchmarks/use-this-data","note":"Full benchmark data ZIP.","evidence":{"quote":"LLM benchmark data ZIP , Updated Oct. 31, 2026","source":"https://epoch.ai/benchmarks/use-this-data","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"CC BY 4.0","verdict":"attribution","scope":"data","note":null,"evidence":{"quote":"Epoch AI's data is free to use, distribute, and reproduce provided the source and authors are credited under the Creative Commons Attribution license","source":"https://epoch.ai/benchmarks/use-this-data","quote_verified":true},"more":{"quote":"This hub also includes data sourced from external projects, which retains its original licensing.","source":"https://epoch.ai/benchmarks/use-this-data","quote_verified":true}},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-31","method":"Date stated on the page","url":"https://epoch.ai/benchmarks/use-this-data","quote":"Updated Oct. 31, 2026","quote_verified":true}],"last_updated":"2026-10-31","status":"active","last_checked":"2026-10-31"},{"slug":"eq-bench","name":"EQ-Bench","operator":"Samuel Paech","url":"https://eqbench.com/","page":"https://benchdata.branolabs.com/l/eq-bench","category":"Emotional intelligence / writing","test_runner":"own_runs","summary":"Emotional intelligence, creative and longform writing, judging and game benchmarks.","raw_data":null,"raw_data_note":"Benchmark code is on GitHub (EQ-bench/eqbench3); no result download found.","licence":{"label":"MIT","verdict":"unclear","scope":"repo","note":"MIT covers the legacy EQ-Bench code repo, not clearly the leaderboard results.","evidence":{"quote":"MIT License Copyright (c) 2023 Samuel Paech","source":"https://raw.githubusercontent.com/EQ-bench/EQ-Bench/main_v2_4/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"restricted","note":null,"evidence":{"quote":"At this time we only accept submissions of open weight models that are available to everyone via HuggingFace.","source":"https://eqbench.com/about.html","quote_verified":true},"more":null},"freshness":[{"date":"2026-05-31","method":"Last commit to the GitHub repo","url":"https://github.com/EQ-bench/eqbench3/commits","quote":null}],"last_updated":"2026-05-31","status":"stale","last_checked":"2026-10-31"},{"slug":"evalplus","name":"EvalPlus (HumanEval+ / MBPP+)","operator":"EvalPlus team","url":"https://evalplus.github.io/leaderboard.html","page":"https://benchdata.branolabs.com/l/evalplus","category":"Coding","test_runner":"own_runs","summary":"Stricter test suites for HumanEval and MBPP.","raw_data":null,"raw_data_note":"No result download found.","licence":{"label":"Apache-2.0 (code)","verdict":"unclear","scope":"repo","note":"GitHub reports Apache-2.0 for the code repo; no licence statement for the results.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2025-10-02","method":"Last commit to the GitHub repo","url":"https://github.com/evalplus/evalplus/commits","quote":null}],"last_updated":"2025-10-02","status":"stale","last_checked":"2026-10-31"},{"slug":"gaia","name":"GAIA","operator":"GAIA team (Meta, Hugging Face)","url":"https://huggingface.co/spaces/gaia-benchmark/leaderboard","page":"https://benchdata.branolabs.com/l/gaia","category":"Agents / tool use","test_runner":"submissions","summary":"General AI assistant tasks. Public results are a Hugging Face dataset.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/gaia-benchmark/results_public","note":"Public submission results.","evidence":null}],"raw_data_note":null,"licence":{"label":"No licence on the dataset","verdict":"no_licence","scope":"dataset","note":"The results_public dataset card has no licence field.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-31","method":"Hugging Face lastModified of dataset gaia-benchmark/results_public","url":"https://huggingface.co/datasets/gaia-benchmark/results_public","quote":null}],"last_updated":"2026-10-31","status":"active","last_checked":"2026-10-31"},{"slug":"helm","name":"HELM","operator":"Stanford CRFM","url":"https://crfm.stanford.edu/helm/","page":"https://benchdata.branolabs.com/l/helm","category":"General / multi-benchmark","test_runner":"own_runs","summary":"Holistic evaluation suites (Capabilities, Safety, Lite, MMLU and more). Full raw results sit in a public cloud bucket.","raw_data":[{"kind":"Public GCS bucket","url":"https://crfm-helm.readthedocs.io/en/latest/downloading_raw_results/","note":"gs://crfm-helm-public, public unauthenticated access.","evidence":{"quote":"All of HELM's raw result data is stored in Google Cloud Storage (GCS) in the public crfm-helm-public bucket.","source":"https://crfm-helm.readthedocs.io/en/latest/downloading_raw_results/","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"unclear","scope":"repo","note":"Apache-2.0 covers the HELM code repo. We found no separate licence statement for the bucket data.","evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/stanford-crfm/helm/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-06-05","method":"Last commit to the GitHub repo","url":"https://github.com/stanford-crfm/helm/commits","quote":null}],"last_updated":"2026-06-05","status":"stale","last_checked":"2026-10-31"},{"slug":"humanitys-last-exam","name":"Humanity's Last Exam (HLE)","operator":"Center for AI Safety and Scale AI","url":"https://lastexam.ai/","page":"https://benchdata.branolabs.com/l/humanitys-last-exam","category":"Knowledge / reasoning","test_runner":"own_runs","summary":"Expert-written questions at the frontier of knowledge; HLE-Rolling and HLE-Diamond variants.","raw_data":null,"raw_data_note":"The question set is on Hugging Face (cais/hle); no result download found.","licence":{"label":"MIT (code)","verdict":"unclear","scope":"repo","note":"GitHub reports MIT for the code repo; no licence statement for leaderboard results.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-09-22","method":"Date stated on the page","url":"https://lastexam.ai/","quote":"[09/22/2026]: HLE-Diamond is released.","quote_verified":true}],"last_updated":"2026-09-22","status":"active","last_checked":"2026-10-31"},{"slug":"k2-vendor-verifier","name":"K2 Vendor Verifier","operator":"Moonshot AI","url":"https://github.com/MoonshotAI/K2-Vendor-Verifier","page":"https://benchdata.branolabs.com/l/k2-vendor-verifier","category":"Provider accuracy","test_runner":"own_runs","summary":"Tool-call accuracy of Kimi K2 across API vendors, published in the README.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/MoonshotAI/K2-Vendor-Verifier","note":"Results tables in the README.","evidence":{"quote":"we launch K2 Vendor Verifier to monitor and enhance the quality of all K2 APIs","source":"https://raw.githubusercontent.com/MoonshotAI/K2-Vendor-Verifier/main/README.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"No licence","verdict":"no_licence","scope":"repo","note":"GitHub reports no licence for the repo.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-02-14","method":"Last commit to the GitHub repo","url":"https://github.com/MoonshotAI/K2-Vendor-Verifier/commits","quote":null}],"last_updated":"2026-02-14","status":"stale","last_checked":"2026-10-31"},{"slug":"kagi-llm-benchmark","name":"Kagi LLM Benchmarking Project","operator":"Kagi Inc.","url":"https://help.kagi.com/kagi/ai/llm-benchmark.html","page":"https://benchdata.branolabs.com/l/kagi-llm-benchmark","category":"Reasoning","test_runner":"own_runs","summary":"Kagi's private, changing task set testing reasoning, coding and instruction following.","raw_data":null,"raw_data_note":"Tasks are kept private by design.","licence":{"label":"No licence found","verdict":"no_licence","scope":"site","note":"No data licence found on the page.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-02-13","method":"Date stated on the page","url":"https://help.kagi.com/kagi/ai/llm-benchmark.html","quote":"Last task list revision: February 13th, 2026","quote_verified":true}],"last_updated":"2026-02-13","status":"stale","last_checked":"2026-10-31"},{"slug":"livebench","name":"LiveBench","operator":"LiveBench team / Abacus AI","url":"https://livebench.ai/","page":"https://benchdata.branolabs.com/l/livebench","category":"General / multi-benchmark","test_runner":"own_runs","summary":"Contamination-limited benchmark with regularly released question sets. Model answers and judgments are on Hugging Face.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/livebench/model_judgment","note":"Model judgments.","evidence":{"quote":"Also available are the [model answers](https://huggingface.co/datasets/livebench/model_answer) and the [model judgments]","source":"https://raw.githubusercontent.com/LiveBench/LiveBench/main/README.md","quote_verified":true}},{"kind":"HF dataset","url":"https://huggingface.co/datasets/livebench/model_answer","note":"Model answers.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0 (copied from FastChat)","verdict":"unclear","scope":"repo","note":"The repo LICENSE is the Apache 2.0 text copied from FastChat; GitHub reports NOASSERTION. The HF result datasets carry no licence field.","evidence":{"quote":"The original LICENSE from https://github.com/lm-sys/FastChat is copied below.","source":"https://raw.githubusercontent.com/LiveBench/LiveBench/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-25","method":"Date stated on the page","url":"https://livebench.ai/","quote":"Last updated 25 October 2026","quote_verified":true}],"last_updated":"2026-10-25","status":"active","last_checked":"2026-10-31"},{"slug":"livecodebench","name":"LiveCodeBench","operator":"LiveCodeBench team","url":"https://livecodebench.github.io/leaderboard.html","page":"https://benchdata.branolabs.com/l/livecodebench","category":"Coding","test_runner":"own_runs","summary":"Coding problems collected over time to limit contamination; the leaderboard uses a selectable time window.","raw_data":null,"raw_data_note":"Problem sets are on Hugging Face; we found no published per-model result file.","licence":{"label":"MIT","verdict":"unclear","scope":"repo","note":"GitHub reports MIT for the code repo; no licence statement for leaderboard results.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2025-07-16","method":"Last commit to the GitHub repo","url":"https://github.com/LiveCodeBench/LiveCodeBench/commits","quote":null}],"last_updated":"2025-07-16","status":"stale","last_checked":"2026-10-31"},{"slug":"confabulations","name":"LLM Confabulation Leaderboard","operator":"Lech Mazur","url":"https://github.com/lechmazur/confabulations","page":"https://benchdata.branolabs.com/l/confabulations","category":"Hallucination","test_runner":"own_runs","summary":"How often models give non-existent answers to misleading RAG questions.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/lechmazur/confabulations","note":"Results in the README.","evidence":null}],"raw_data_note":null,"licence":{"label":"No licence","verdict":"no_licence","scope":"repo","note":"GitHub reports no licence for the repo.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2025-08-07","method":"Last commit to the GitHub repo","url":"https://github.com/lechmazur/confabulations/commits","quote":null}],"last_updated":"2025-08-07","status":"stale","last_checked":"2026-10-31"},{"slug":"llm-stats","name":"LLM Stats","operator":"llm-stats.com","url":"https://llm-stats.com/","page":"https://benchdata.branolabs.com/l/llm-stats","category":"Aggregator","test_runner":"aggregator","summary":"Model stats, benchmark results and prices collected in one place.","raw_data":null,"raw_data_note":"No bulk download confirmed.","licence":{"label":"Reuse with attribution (site terms)","verdict":"attribution","scope":"site terms","note":null,"evidence":{"quote":"You may view, copy, modify, share, and republish the content — including for commercial purposes — without needing to ask for permission.","source":"https://llm-stats.com/terms","quote_verified":true},"more":{"quote":"The only requirement is attribution: when you reuse our materials, credit llm-stats.com as the source in a place visible to your users","source":"https://llm-stats.com/terms","quote_verified":true}},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"lmarena","name":"LMArena (Arena) leaderboard","operator":"LMArena","url":"https://arena.ai/leaderboard","page":"https://benchdata.branolabs.com/l/lmarena","category":"General chat","test_runner":"votes","summary":"Crowd-voted pairwise comparisons; historical leaderboard snapshots are published as a Hugging Face dataset.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset","note":"Historical snapshots of the Arena leaderboard, one subset per arena, splits full and latest.","evidence":{"quote":"Historical snapshots of the [Arena](https://arena.ai/leaderboard) leaderboard.","source":"https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset/raw/main/README.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"CC BY 4.0","verdict":"attribution","scope":"dataset","note":null,"evidence":{"quote":"license: cc-by-4.0","source":"https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset/raw/main/README.md","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-30","method":"Hugging Face lastModified of dataset lmarena-ai/leaderboard-dataset","url":"https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset","quote":null}],"last_updated":"2026-10-30","status":"active","last_checked":"2026-10-31"},{"slug":"matharena","name":"MathArena","operator":"SRI Lab, ETH Zurich","url":"https://matharena.ai/","page":"https://benchdata.branolabs.com/l/matharena","category":"Math","test_runner":"own_runs","summary":"Models evaluated on new math competitions right after they happen. Model outputs are published as Hugging Face datasets.","raw_data":[{"kind":"HF datasets","url":"https://huggingface.co/MathArena","note":"Per-competition *_outputs datasets with saved model outputs.","evidence":{"quote":"Saved model outputs are published in the corresponding","source":"https://raw.githubusercontent.com/eth-sri/matharena/main/README.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"MIT","verdict":"unclear","scope":"repo","note":"MIT covers the code repo. Licences of the output datasets vary per dataset (for example MathArena/aime_2025 is CC BY-NC-SA 4.0).","evidence":{"quote":"MIT License Copyright (c) 2024 SRI Lab, ETH Zurich","source":"https://raw.githubusercontent.com/eth-sri/matharena/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-19","method":"Most recently modified MathArena dataset (MathArena/cmimc_2025_outputs)","url":"https://huggingface.co/MathArena","quote":null}],"last_updated":"2026-10-19","status":"active","last_checked":"2026-10-31"},{"slug":"mmmu","name":"MMMU / MMMU-Pro","operator":"MMMU team","url":"https://mmmu-benchmark.github.io/#leaderboard","page":"https://benchdata.branolabs.com/l/mmmu","category":"Multimodal","test_runner":"submissions","summary":"College-level multimodal questions across disciplines. The leaderboard is a table on the project site.","raw_data":null,"raw_data_note":"No result download found; questions are HF datasets (MMMU/MMMU, MMMU/MMMU_Pro).","licence":{"label":"Site template CC BY-SA 4.0","verdict":"unclear","scope":"website","note":"The CC BY-SA notice refers to the website (adapted from Nerfies and MathVista), not clearly the results.","evidence":{"quote":"licensed under a Creative Commons Attribution-ShareAlike 4.0 International License","source":"https://mmmu-benchmark.github.io/","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":"The README strikes through the EvalAI evaluation-server notice.","evidence":null,"more":null},"freshness":[{"date":"2025-09-05","method":"Date stated on the page","url":"https://mmmu-benchmark.github.io/","quote":"Last updated: 09/05/2025","quote_verified":true}],"last_updated":"2025-09-05","status":"stale","last_checked":"2026-10-31"},{"slug":"mteb","name":"MTEB Leaderboard","operator":"MTEB (embeddings-benchmark)","url":"https://huggingface.co/spaces/mteb/leaderboard","page":"https://benchdata.branolabs.com/l/mteb","category":"Embeddings","test_runner":"submissions","summary":"Text embedding benchmark. All results live in the embeddings-benchmark/results repo and the mteb/results dataset.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/embeddings-benchmark/results","note":"Result files.","evidence":null},{"kind":"HF dataset","url":"https://huggingface.co/datasets/mteb/results","note":"Mirror of the results.","evidence":null}],"raw_data_note":null,"licence":{"label":"CC0 1.0","verdict":"cc0","scope":"results repo","note":null,"evidence":{"quote":"Creative Commons Legal Code CC0 1.0 Universal","source":"https://raw.githubusercontent.com/embeddings-benchmark/results/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"open","note":"Results are added by pull request with a model implementation.","evidence":{"quote":"this is no longer possible as we want to ensure that we can match the results with the model implementation.","source":"https://raw.githubusercontent.com/embeddings-benchmark/results/main/README.md","quote_verified":true},"more":null},"freshness":[{"date":"2026-10-30","method":"Last commit to the GitHub repo","url":"https://github.com/embeddings-benchmark/results/commits","quote":null}],"last_updated":"2026-10-30","status":"active","last_checked":"2026-10-31"},{"slug":"nyt-connections","name":"NYT Connections (Extended)","operator":"Lech Mazur","url":"https://github.com/lechmazur/nyt-connections","page":"https://benchdata.branolabs.com/l/nyt-connections","category":"Reasoning","test_runner":"own_runs","summary":"940 NYT Connections puzzles with extra trick words.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/lechmazur/nyt-connections","note":"Results in the README.","evidence":null}],"raw_data_note":null,"licence":{"label":"No licence","verdict":"no_licence","scope":"repo","note":"GitHub reports no licence for the repo.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-23","method":"Last commit to the GitHub repo","url":"https://github.com/lechmazur/nyt-connections/commits","quote":null}],"last_updated":"2026-10-23","status":"active","last_checked":"2026-10-31"},{"slug":"open-llm-leaderboard","name":"Open LLM Leaderboard","operator":"Hugging Face","url":"https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard","page":"https://benchdata.branolabs.com/l/open-llm-leaderboard","category":"Open-weight models","test_runner":"submissions","summary":"Hugging Face's evaluation of open-weight models on its research cluster. Officially retired; results remain online.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/open-llm-leaderboard/contents","note":"Aggregated leaderboard contents.","evidence":null}],"raw_data_note":null,"licence":{"label":"No licence on the dataset","verdict":"no_licence","scope":"dataset","note":"The dataset card has no licence field. The Space code is tagged apache-2.0, which covers the app, not clearly the results.","evidence":null,"more":null},"submissions":{"status":"closed","note":null,"evidence":{"quote":"the leaderboard is officially retiring!","source":"https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1135","quote_verified":true},"more":null},"freshness":[{"date":"2025-03-20","method":"Hugging Face lastModified of dataset open-llm-leaderboard/contents","url":"https://huggingface.co/datasets/open-llm-leaderboard/contents","quote":null}],"last_updated":"2025-03-20","status":"retired","last_checked":"2026-10-31"},{"slug":"openrouter-rankings","name":"OpenRouter rankings","operator":"OpenRouter, Inc.","url":"https://openrouter.ai/rankings","page":"https://benchdata.branolabs.com/l/openrouter-rankings","category":"Usage","test_runner":"usage","summary":"Models ranked by tokens processed through the OpenRouter API, not by benchmark scores.","raw_data":null,"raw_data_note":"No public download of the rankings found.","licence":{"label":"No licence found","verdict":"no_licence","scope":"site","note":"The Terms of Service cover use of the service; we found no licence for the rankings data.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-30","method":"Date stated on the page","url":"https://openrouter.ai/rankings","quote":"Usage data through Oct 30, 2026","quote_verified":true}],"last_updated":"2026-10-30","status":"active","last_checked":"2026-10-31"},{"slug":"open-vlm-leaderboard","name":"OpenVLM Leaderboard (OpenCompass)","operator":"OpenCompass (Shanghai AI Lab)","url":"https://huggingface.co/spaces/opencompass/open_vlm_leaderboard","page":"https://benchdata.branolabs.com/l/open-vlm-leaderboard","category":"Multimodal","test_runner":"own_runs","summary":"Vision-language model results produced with VLMEvalKit.","raw_data":[{"kind":"JSON download","url":"http://opencompass.openxlab.space/assets/OpenVLM.json","note":"All detailed results as one JSON file.","evidence":{"quote":"The performance numbers on our official multi-modal leaderboards can be downloaded from here!","source":"https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/README.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"unclear","scope":"repo","note":"Apache-2.0 covers VLMEvalKit; the JSON download has no licence statement.","evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-03-11","method":"Hugging Face lastModified of Space opencompass/open_vlm_leaderboard","url":"https://huggingface.co/spaces/opencompass/open_vlm_leaderboard","quote":null}],"last_updated":"2026-03-11","status":"stale","last_checked":"2026-10-31"},{"slug":"osworld","name":"OSWorld / OSWorld-Verified","operator":"XLANG Lab (HKU)","url":"https://os-world.github.io/","page":"https://benchdata.branolabs.com/l/osworld","category":"Computer use agents","test_runner":"own_runs","summary":"Computer-use agents completing desktop tasks. OSWorld-Verified results are run by the maintainers.","raw_data":null,"raw_data_note":"We found no downloadable result file. The site returned HTTP 500 to our fetches on 2026-10-31.","licence":{"label":"Apache-2.0","verdict":"unclear","scope":"repo","note":"Apache-2.0 covers the OSWorld code repo, not clearly the leaderboard results.","evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/xlang-ai/OSWorld/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-24","method":"Last commit to the GitHub repo","url":"https://github.com/xlang-ai/OSWorld/commits","quote":null}],"last_updated":"2026-10-24","status":"active","last_checked":"2026-10-31"},{"slug":"quantdiff","name":"QuantDiff","operator":"QuantDiff","url":"https://quantdiff.com/leaderboard/","page":"https://benchdata.branolabs.com/l/quantdiff","category":"Quantization","test_runner":"own_runs","summary":"Measured quality and speed of quantized GGUF and MLX files against their full-precision originals.","raw_data":[{"kind":"CSV / JSON export","url":"https://quantdiff.com/data/export/README.md","note":"quantdiff-quants.csv, quantdiff-quants.json, quantdiff-full.json.","evidence":{"quote":"Measured quality and speed of quantized LLM files (GGUF for llama.cpp, MLX for Apple Silicon), each compared with its own full-precision original.","source":"https://quantdiff.com/data/export/README.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"CC BY 4.0","verdict":"attribution","scope":"data","note":null,"evidence":{"quote":"License (data): **CC BY 4.0** - credit \"QuantDiff (quantdiff.com)\" with a link.","source":"https://quantdiff.com/data/export/README.md","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-30","method":"Date stated on the page","url":"https://quantdiff.com/data/export/README.md","quote":"data as of 2026-10-30","quote_verified":true}],"last_updated":"2026-10-30","status":"active","last_checked":"2026-10-31"},{"slug":"rewardbench","name":"RewardBench / RewardBench 2","operator":"Allen Institute for AI (Ai2)","url":"https://huggingface.co/spaces/allenai/reward-bench","page":"https://benchdata.branolabs.com/l/rewardbench","category":"Reward models","test_runner":"submissions","summary":"Evaluation of reward models. Results are Hugging Face datasets.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/allenai/reward-bench-2-results","note":"RewardBench 2 results.","evidence":null},{"kind":"HF dataset","url":"https://huggingface.co/datasets/allenai/reward-bench-results","note":"RewardBench v1 results.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"unclear","scope":"repo","note":"Apache-2.0 covers the code repo; the results datasets carry no licence field.","evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/allenai/reward-bench/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2025-12-11","method":"Hugging Face lastModified of dataset allenai/reward-bench-2-results","url":"https://huggingface.co/datasets/allenai/reward-bench-2-results","quote":null}],"last_updated":"2025-12-11","status":"stale","last_checked":"2026-10-31"},{"slug":"scale-seal","name":"Scale SEAL Leaderboards","operator":"Scale AI","url":"https://labs.scale.com/leaderboard","page":"https://benchdata.branolabs.com/l/scale-seal","category":"General / multi-benchmark","test_runner":"own_runs","summary":"Expert-curated evaluations, mixing private datasets with open ones.","raw_data":null,"raw_data_note":"Private datasets; no download.","licence":{"label":"No licence found","verdict":"no_licence","scope":"site","note":"No data licence found on the leaderboard page.","evidence":null,"more":null},"submissions":{"status":"restricted","note":null,"evidence":{"quote":"If you'd like to add your model to this leaderboard or a future version, please contact","source":"https://labs.scale.com/leaderboard","quote_verified":true},"more":null},"freshness":[],"last_updated":null,"status":"unknown","last_checked":"2026-10-31"},{"slug":"simplebench","name":"SimpleBench","operator":"SimpleBench Team","url":"https://simple-bench.com/","page":"https://benchdata.branolabs.com/l/simplebench","category":"Reasoning","test_runner":"own_runs","summary":"Multiple-choice trick and common-sense questions with a human baseline.","raw_data":null,"raw_data_note":"A public sample dataset and code exist; full results are not published as a file.","licence":{"label":"MIT (code)","verdict":"unclear","scope":"repo","note":"GitHub reports MIT for the code repo.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2024-12-20","method":"Last commit to the GitHub repo","url":"https://github.com/simple-bench/SimpleBench/commits","quote":null}],"last_updated":"2024-12-20","status":"stale","last_checked":"2026-10-31"},{"slug":"swe-bench","name":"SWE-bench","operator":"SWE-bench team (Princeton / Stanford)","url":"https://www.swebench.com/","page":"https://benchdata.branolabs.com/l/swe-bench","category":"Coding agents","test_runner":"submissions","summary":"Resolving real GitHub issues. Official submissions are kept in the SWE-bench/experiments repository.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/SWE-bench/experiments","note":"Submission entries (not full artifacts).","evidence":{"quote":"All official submissions to the SWE-bench leaderboard are maintained at SWE-bench/experiments","source":"https://www.swebench.com/submit.html","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"No licence on the experiments repo","verdict":"no_licence","scope":"repo","note":"GitHub reports no licence for SWE-bench/experiments. The benchmark code (SWE-bench/SWE-bench) is MIT.","evidence":null,"more":null},"submissions":{"status":"restricted","note":null,"evidence":{"quote":"SWE-bench Verified and Multilingual now only accepts submissions from academic teams and research institutions with open source methods and peer-reviewed publications.","source":"https://raw.githubusercontent.com/SWE-bench/experiments/main/README.md","quote_verified":true},"more":{"quote":"Anybody is still welcome to submit to the SWE-bench Multimodal leaderboard.","source":"https://raw.githubusercontent.com/SWE-bench/experiments/main/README.md","quote_verified":true}},"freshness":[{"date":"2026-09-01","method":"Last commit touching evaluation/verified","url":"https://github.com/SWE-bench/experiments/commits/HEAD/evaluation/verified","quote":null}],"last_updated":"2026-09-01","status":"active","last_checked":"2026-10-31"},{"slug":"terminal-bench","name":"Terminal-Bench","operator":"Stanford / Laude Institute (Harbor)","url":"https://www.tbench.ai/leaderboard","page":"https://benchdata.branolabs.com/l/terminal-bench","category":"Coding agents","test_runner":"submissions","summary":"Agents completing tasks in a terminal. Terminal-Bench 2.0 leaderboard entries are a Hugging Face dataset.","raw_data":[{"kind":"HF dataset","url":"https://huggingface.co/datasets/harborframework/terminal-bench-2-leaderboard","note":"Terminal-Bench 2.0 leaderboard submissions.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"attribution","scope":"dataset","note":null,"evidence":{"quote":"license: apache-2.0","source":"https://huggingface.co/datasets/harborframework/terminal-bench-2-leaderboard/raw/main/README.md","quote_verified":true},"more":null},"submissions":{"status":"closed","note":null,"evidence":{"quote":"SUBMISSIONS CLOSED - All PRs opened before May 14th have been reviewed and merged if valid.","source":"https://huggingface.co/datasets/harborframework/terminal-bench-2-leaderboard/raw/main/README.md","quote_verified":true},"more":{"quote":"We are working on a new submission process for the Terminal Bench 2.0 Leaderboard.","source":"https://huggingface.co/datasets/harborframework/terminal-bench-2-leaderboard/raw/main/README.md","quote_verified":true}},"freshness":[{"date":"2026-05-15","method":"Hugging Face lastModified of dataset harborframework/terminal-bench-2-leaderboard","url":"https://huggingface.co/datasets/harborframework/terminal-bench-2-leaderboard","quote":null}],"last_updated":"2026-05-15","status":"stale","last_checked":"2026-10-31"},{"slug":"ugi-leaderboard","name":"UGI Leaderboard","operator":"DontPlanToEnd","url":"https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard","page":"https://benchdata.branolabs.com/l/ugi-leaderboard","category":"Uncensored","test_runner":"own_runs","summary":"Uncensored General Intelligence: willingness and knowledge on sensitive topics, with private test questions.","raw_data":null,"raw_data_note":"Data is inside the Space app; no separate download found.","licence":{"label":"Apache-2.0 (Space)","verdict":"unclear","scope":"space","note":"The Space metadata is tagged apache-2.0, which covers the app code; no statement for the results.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":"The Space metadata is tagged submission:manual and test:private.","evidence":null,"more":null},"freshness":[{"date":"2026-10-17","method":"Hugging Face lastModified of Space DontPlanToEnd/UGI-Leaderboard","url":"https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard","quote":null}],"last_updated":"2026-10-17","status":"active","last_checked":"2026-10-31"},{"slug":"vals-ai","name":"Vals AI benchmarks","operator":"Vals AI","url":"https://www.vals.ai/benchmarks","page":"https://benchdata.branolabs.com/l/vals-ai","category":"Industry / domain","test_runner":"own_runs","summary":"Domain benchmarks (legal, finance, healthcare, coding and more), several on private datasets, and Vals indexes.","raw_data":null,"raw_data_note":"No public download found.","licence":{"label":"No licence found","verdict":"no_licence","scope":"site","note":"We found no data licence; vals.ai/terms returned 404 on 2026-10-31.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-29","method":"Date stated on the page","url":"https://www.vals.ai/benchmarks","quote":"Proprietary Vals Index Updated: 10/29/2026","quote_verified":true}],"last_updated":"2026-10-29","status":"active","last_checked":"2026-10-31"},{"slug":"vectara-hallucination","name":"Vectara Hallucination Leaderboard","operator":"Vectara","url":"https://github.com/vectara/hallucination-leaderboard","page":"https://benchdata.branolabs.com/l/vectara-hallucination","category":"Hallucination","test_runner":"own_runs","summary":"Summarisation hallucination rates measured with Vectara's HHEM model, published in the repo README.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/vectara/hallucination-leaderboard","note":"Results tables in the README.","evidence":null}],"raw_data_note":null,"licence":{"label":"Apache-2.0","verdict":"attribution","scope":"repo","note":null,"evidence":{"quote":"Apache License Version 2.0, January 2004","source":"https://raw.githubusercontent.com/vectara/hallucination-leaderboard/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2026-10-23","method":"Last commit to the GitHub repo","url":"https://github.com/vectara/hallucination-leaderboard/commits","quote":null}],"last_updated":"2026-10-23","status":"active","last_checked":"2026-10-31"},{"slug":"wildbench","name":"WildBench","operator":"Allen Institute for AI (Ai2)","url":"https://huggingface.co/spaces/allenai/WildBench","page":"https://benchdata.branolabs.com/l/wildbench","category":"General chat","test_runner":"own_runs","summary":"Challenging tasks from real user conversations, judged by an LLM.","raw_data":null,"raw_data_note":"Tasks are on Hugging Face (allenai/WildBench); no result file confirmed.","licence":{"label":"Apache-2.0 (code)","verdict":"unclear","scope":"repo","note":"GitHub reports Apache-2.0 for the code repo.","evidence":null,"more":null},"submissions":{"status":"not_stated","note":null,"evidence":null,"more":null},"freshness":[{"date":"2024-08-01","method":"Last commit to the GitHub repo","url":"https://github.com/allenai/WildBench/commits","quote":null}],"last_updated":"2024-08-01","status":"stale","last_checked":"2026-10-31"},{"slug":"tau-bench","name":"τ-bench (tau2-bench)","operator":"Sierra","url":"https://taubench.com/","page":"https://benchdata.branolabs.com/l/tau-bench","category":"Agents / tool use","test_runner":"submissions","summary":"Customer-service agent simulations (airline, retail, telecom, banking, voice). Leaderboard data is in the tau2-bench repo.","raw_data":[{"kind":"GitHub repo","url":"https://github.com/sierra-research/tau2-bench","note":"Submissions arrive as pull requests; the leaderboard site lives in web/.","evidence":{"quote":"upload the trajectories to external storage, and open a pull request","source":"https://raw.githubusercontent.com/sierra-research/tau2-bench/main/docs/leaderboard-submission.md","quote_verified":true}}],"raw_data_note":null,"licence":{"label":"MIT (repo)","verdict":"attribution","scope":"repo","note":null,"evidence":{"quote":"MIT License","source":"https://raw.githubusercontent.com/sierra-research/tau2-bench/main/LICENSE","quote_verified":true},"more":null},"submissions":{"status":"open","note":null,"evidence":{"quote":"1.x submissions are still accepted.","source":"https://raw.githubusercontent.com/sierra-research/tau2-bench/main/docs/leaderboard-submission.md","quote_verified":true},"more":null},"freshness":[{"date":"2026-10-30","method":"Last commit touching web","url":"https://github.com/sierra-research/tau2-bench/commits/HEAD/web","quote":null}],"last_updated":"2026-10-30","status":"active","last_checked":"2026-10-31"}],"excluded":[{"name":"WebArena","url":"https://webarena.dev/","reason":"The leaderboard is linked from the README, but we could not confirm where it lives or how it is updated."},{"name":"OpenAI simple-evals","url":"https://github.com/openai/simple-evals","reason":"An evaluation library with reference numbers, not a maintained leaderboard."},{"name":"Vellum LLM Leaderboard","url":"https://www.vellum.ai/llm-leaderboard","reason":"A marketing page; the terms we found cover Vellum's product, not the leaderboard, and no sources or data are published."},{"name":"PublicAI Index","url":"https://publicai.io/model-index","reason":"A meta-index that rescales other boards; we found no data licence statement on the pages we checked."},{"name":"LLMBenchmark.ai","url":"https://llmbenchmark.ai/methodology","reason":"A meta-index built on other leaderboards; no data licence found."},{"name":"peerbench DACH","url":"https://dach.peerbench.ai/","reason":"A German-language aggregate ranking; operator and data licence could not be confirmed from its pages."},{"name":"OpenCompass academic leaderboard","url":"https://rank.opencompass.org.cn/","reason":"Not fetched successfully; the OpenVLM leaderboard from the same team is listed."}]}