| --- |
| title: README |
| emoji: 🌍 |
| colorFrom: purple |
| colorTo: red |
| sdk: static |
| pinned: false |
| --- |
| |
| MTEB is a Python framework for evaluating embeddings and retrieval systems for both text and image. |
| MTEB covers more than 1000 languages and diverse tasks, from classics like classification and clustering to use-case specialized tasks such as legal, code, or healthcare retrieval. |
|
|
| You can get started using [`mteb`](https://github.com/embeddings-benchmark/mteb). |
|
|
| | Overview | | |
| |--------------------------------|--------------------------------------------------------------------------------------| |
| | 📈 [Leaderboard] | The interactive leaderboard of the benchmark | |
| | **Get Started**. | | |
| | 🏃 [Get Started] | Overview of how to use mteb | |
| | 🤖 [Defining Models] | How to use existing model and define custom ones | |
| | 📋 [Selecting tasks] | How to select tasks, benchmarks, splits etc. | |
| | 🏭 [Running Evaluation] | How to run the evaluations, including cache management, speeding up evaluations etc. | |
| | 📊 [Loading Results] | How to load and work with existing model results | |
| | **Overview**. | | |
| | 📋 [Tasks] | Overview of available tasks | |
| | 📐 [Benchmarks] | Overview of available benchmarks | |
| | 🤖 [Models] | Overview of available Models | |
| | **Contributing** | | |
| | 🤖 [Adding a model] | How to submit a model to MTEB and to the leaderboard | |
| | 👩💻 [Adding a dataset] | How to add a new task/dataset to MTEB | |
| | 👩💻 [Adding a benchmark] | How to add a new benchmark to MTEB and to the leaderboard | |
| | 🤝 [Contributing] | How to contribute to MTEB and set it up for development | |
|
|
| [Get Started]: https://embeddings-benchmark.github.io/mteb/ |
| [Defining Models]: https://embeddings-benchmark.github.io/mteb/get_started/usage/defining_the_model/ |
| [Selecting tasks]: https://embeddings-benchmark.github.io/mteb/get_started/usage/selecting_tasks/ |
| [Running Evaluation]: https://embeddings-benchmark.github.io/mteb/get_started/usage/running_the_evaluation/ |
| [Loading Results]: https://embeddings-benchmark.github.io/mteb/get_started/usage/loading_results/ |
| [Tasks]: https://embeddings-benchmark.github.io/mteb/overview/available_tasks/retrieval/ |
| [Benchmarks]: https://embeddings-benchmark.github.io/mteb/overview/available_benchmarks/ |
| [Models]: https://embeddings-benchmark.github.io/mteb/overview/available_models/text/ |
| [Contributing]: https://embeddings-benchmark.github.io/mteb/CONTRIBUTING/ |
| [Adding a model]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_model/ |
| [Adding a dataset]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_dataset/ |
| [Adding a benchmark]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_benchmark/ |
| [Leaderboard]: https://huggingface.co/spaces/mteb/leaderboard |
|
|
