Dataset
Berkeley-Function-Calling-Leaderboard
by Gorilla LLM (UC Berkeley) gorilla-llm/Berkeley-Function-Calling-Leaderboard
The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools).
Dataset Card
By Gorilla LLM (UC Berkeley), published under apache-2.0, revision 61fc0608cfd8.
The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools). We built this dataset from our learnings to be representative of most users' function calling use-cases, for example, in agents, as a part of enterprise workflows, etc. To this end, our evaluation dataset spans diverse categories, and across multiple languages.
Checkout the Leaderboard at gorilla.cs.berkeley.edu/leaderboard.html and our release blogs:
BFCL V1: Our initial BFCL release
BFCL V2: Our second release, employing enterprise and OSS-contributed live data
BFCL V3: Introduces multi-turn and multi-step function calling scenarios
Latest Version Release Date: 09/22/2024
Original Release Date: 02/26/2024
Prepare Evaluation Dataset
To use the BFCL dataset, please follow the instructions detailed in the README here.
Details
- Repository
- gorilla-llm/Berkeley-Function-Calling-Leaderboard
- Publisher
- Gorilla LLM (UC Berkeley)
- Task category
- Not stated by the source
- Tags
- Not stated by the source
- Size category
- Not stated by the source
- Languages
- en
- Revision
- 61fc0608cfd831fcfbbaa676ebdfef0ed963eeda
- Last updated
- 2026-04-29
SAVRN's Notes on Berkeley-Function-Calling-Leaderboard
This repo is the evaluation data behind the live function-calling leaderboard from Gorilla LLM at UC Berkeley: 11.7 MB across 52 files, 143.5k monthly downloads, no rows published because the unit is function-calling test cases. The card traces three releases, V1 from 02/26/2024 through V2's enterprise and open-source-contributed live data to V3's multi-turn, multi-step scenarios, with the latest release dated 09/22/2024.
Apache 2.0 makes commercial use straightforward, and the size makes it trivial to mirror at 11.7 MB total. The intended role is grading agentic tool use, not training: if you tune a model on these cases, your BFCL number stops meaning anything. Pin the release version when comparing scores, since V1, V2 and V3 measure different things and the suite is still being extended.
Files
52 files, 11.7 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| BFCL_v3_chatable.json | Data | 25.0 KB | — |
| BFCL_v3_exec_multiple.json | Data | 82.1 KB | — |
| BFCL_v3_exec_parallel.json | Data | 56.3 KB | — |
| BFCL_v3_exec_parallel_multiple.json | Data | 73.9 KB | — |
| BFCL_v3_exec_simple.json | Data | 88.0 KB | — |
| BFCL_v3_irrelevance.json | Data | 160.1 KB | — |
| BFCL_v3_java.json | Data | 85.9 KB | — |
| BFCL_v3_javascript.json | Data | 46.7 KB | — |
| BFCL_v3_live_irrelevance.json | Data | 2.2 MB | — |
| BFCL_v3_live_multiple.json | Data | 3.9 MB | — |
| BFCL_v3_live_parallel.json | Data | 15.6 KB | — |
| BFCL_v3_live_parallel_multiple.json | Data | 82.3 KB | — |
| BFCL_v3_live_relevance.json | Data | 54.1 KB | — |
| BFCL_v3_live_simple.json | Data | 260.1 KB | — |
| BFCL_v3_multi_turn_base.json | Data | 391.1 KB | — |
| BFCL_v3_multi_turn_composite.json | Data | 404.5 KB | — |
| BFCL_v3_multi_turn_long_context.json | Data | 392.7 KB | — |
| BFCL_v3_multi_turn_miss_func.json | Data | 401.8 KB | — |
| BFCL_v3_multi_turn_miss_param.json | Data | 403.2 KB | — |
| BFCL_v3_multiple.json | Data | 316.6 KB | — |
| BFCL_v3_parallel.json | Data | 172.1 KB | — |
| BFCL_v3_parallel_multiple.json | Data | 347.1 KB | — |
| BFCL_v3_rest.json | Data | 208.2 KB | — |
| BFCL_v3_simple.json | Data | 280.5 KB | — |
| BFCL_v3_sql.json | Data | 152.6 KB | — |
| multi_turn_func_doc/gorilla_file_system.json | Data | 14.0 KB | — |
| multi_turn_func_doc/math_api.json | Data | 9.2 KB | — |
| multi_turn_func_doc/message_api.json | Data | 6.1 KB | — |
| multi_turn_func_doc/posting_api.json | Data | 10.6 KB | — |
| multi_turn_func_doc/ticket_api.json | Data | 6.6 KB | — |
| multi_turn_func_doc/trading_bot.json | Data | 14.1 KB | — |
| multi_turn_func_doc/travel_booking.json | Data | 15.9 KB | — |
| multi_turn_func_doc/vehicle_control.json | Data | 17.5 KB | — |
| possible_answer/BFCL_v3_java.json | Data | 15.7 KB | — |
| possible_answer/BFCL_v3_javascript.json | Data | 7.5 KB | — |
| possible_answer/BFCL_v3_live_multiple.json | Data | 199.7 KB | — |
| possible_answer/BFCL_v3_live_parallel.json | Data | 5.0 KB | — |
| possible_answer/BFCL_v3_live_parallel_multiple.json | Data | 7.4 KB | — |
| possible_answer/BFCL_v3_live_simple.json | Data | 47.0 KB | — |
| possible_answer/BFCL_v3_multi_turn_base.json | Data | 80.4 KB | — |
| possible_answer/BFCL_v3_multi_turn_composite.json | Data | 115.9 KB | — |
| possible_answer/BFCL_v3_multi_turn_long_context.json | Data | 116.0 KB | — |
| possible_answer/BFCL_v3_multi_turn_miss_func.json | Data | 82.1 KB | — |
| possible_answer/BFCL_v3_multi_turn_miss_param.json | Data | 82.4 KB | — |
| possible_answer/BFCL_v3_multiple.json | Data | 32.3 KB | — |
| possible_answer/BFCL_v3_parallel.json | Data | 65.9 KB | — |
| possible_answer/BFCL_v3_parallel_multiple.json | Data | 74.6 KB | — |
| possible_answer/BFCL_v3_simple.json | Data | 60.8 KB | — |
| possible_answer/BFCL_v3_sql.json | Data | 20.5 KB | — |
| README.md | Documentation | 15.3 KB | — |
| eval.yaml | Other | 736 B | — |
| .gitattributes | Repository | 2.3 KB | — |
License and Download
- License
- apache-2.0
- Access
- No access gate
Released by Gorilla LLM (UC Berkeley) through its official repository on Hugging Face. Read the license.