2.10.6 (2026-03-05)

Fix

• fix: fleurs loading (#4197)

• fix fleurs

• fix dataset transform signature (009209a)

Unknown

• Start video (#4148)

• start video integration

• start video integration

• upd task structure

• upd batched input

• upd video input type

• combine video and audio to dict

• use only one video per time

• remove __main__

• remove PostProcessingCollator (34d060c)

• model: Add Perplexity pplx-embed-v1 models (0.6B and 4B) (#4189)

• model: Add Perplexity pplx-embed-v1 models (0.6B and 4B)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

• Update mteb/models/model_implementations/perplexity_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/perplexity_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Add training datasets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (1c90bfd)

2.10.5 (2026-03-03)

Fix

• fix: vn models (#4190)

fix vn models (a80fae9)

Unknown

• model: LateOn-Code models definition (#4175)

• First draft of LateOn code models definition

• Fix reference for LateOn-Code

• Fix reference LateOn code edge pretrain

• Add memory_usage_mb (and embed_dim)

• fix lint

• Add training datasets

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (4345e63)

• model: Vietnamese model for VN-MTEB (#4187)

• [ADD] Vietnamese model for VN-MTEB

• [ADD] Vietnamese model for VN-MTEB (rename variable) (3ab29f4)

2.10.4 (2026-03-02)

Fix

• fix: remove select column from dataloader (#4185)

• remove select column

• fix sts (7477902)

2.10.3 (2026-02-28)

Fix

• fix: Add repr to benchmarks to avoid excessive prints (#4180)

• fix: don't specify keyerror when it is a keyerror

• fix: Add repr for benchmarks

It now looks like:

Benchmark(name='BEIR', desciption='BEIR is a heterogeneous benchmark containing diver..., tasks=[...] (#15), ...)

• format (8b6cd0c)

2.10.2 (2026-02-27)

Fix

• fix: add hit_rate to retrieval metrics and remove cv_recall (#4142)

• add hit_rate

• upd lotte metric

• fix tests

• fix name

• remove cv_recall

• upd lotte comment (3efc32f)

2.10.1 (2026-02-27)

Fix

• fix: move mteb(multilingual, v2) datasets to the mteb org (#4172)

move datasets to mteb (55d6103)

• fix: prompt type for non-retrieval types (#4174)

• fix prompt names

• fix pyproject

• rollback pyproject.toml (e453c15)

Unknown

• Update zeroshot classification template path for pyproject.toml (#4171)

• update path for zeroshot classification templates

• fix path for zeroshot classification template (4a305d8)

2.10.0 (2026-02-25)

Feature

• feat: save and load experiments results (#4071)

• start experiments

• sanitize experiment name

• sanitize inputs

• add methods to load results

• fix to python test

• fix loading with model meta

• fix test path

• apply suggestions

• don't serilize classes

• Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• add different loading strategies

• align implementation

• align implementation

• upd docstring

• update enum

• don't change logic for modelmeta

• fix typing

• try to run with -n auto

• always work with metadata copy

• simplify test

• fix corner cases and add doc

• add one more test

• fix when model loading from meta with experiment

• remove additional typos from pyppproject

• rename to experiment_kwargs

• fixes after review

• rename baseline everywhere

• add model_name_with_experiment

• Update mteb/models/model_implementations/random_baseline.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• change examples

---

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (ed4f476)

2.9.1 (2026-02-25)

Fix

• fix: Update _calculate_memory_usage_mb to optionally fetch_from_hf' (#4156)

• Update _calculate_memory_usage_mb to optionally fetch_from_hf

• Apply changes at other places (8f8d8bc)

Unknown

• Change leaderboard healthcheck schedule to every 12 hours (#4165) (469611c)

• fix eval from existing (#4166)

fix eval (5e57dd2)

2.9.0 (2026-02-24)

Ci

• ci: Add to notify us when the leaderboard is down (#4162)

leaderboard health check (c9c623e)

Feature

• feat: Integrate eval results (#4114)

• start integration of eval results

• fix models

• Update mteb/abstasks/_eval/eval_model.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• working version

• add method to model meta for uploading

• fix tests

• test file dump in tests

• add what's new

• fix dumping

• rename bm25

• upd what's new

• fix notes https://github.com/embeddings-benchmark/mteb/issues/4150

• fix test

• raname baseline models

• rename models in tests

• fix naming

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (d892fd7)

Fix

• fix: Deprecate GritLM Wrapper and use Sentence Transformers (#4085)

• Depreceate GritLM Wrapper and use Sentence Transformers

• Add include_prompt in InstructSentenceTransformerModel

• Update Wrapper to InstructSentenceTransformerModel for remaining models

• apply_instruction_to_passages=False in loader_kwargs

• update docstring

• Add depreciation warning

• Added warning to both class and function (338af33)

• fix: languages filtering scores (#4145)

fix languages filtering scores (3a2416d)

2.8.10 (2026-02-24)

Documentation

• docs: add cache/search backends to doc (#4143)

• add cache backend to doc

• Update docs/advanced_usage/cache_embeddings.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• remove duplicates

---

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> (25c9a2b)

Fix

• fix: Refactor pylate models such that it does not follow the Encoder interface (#4151)

• make multivec not comatible with most tasks

• fix

• remove numpy (b0e5f5a)

• fix: pylate reranking (#4152)

fix pylate reranking (0b11c69)

• fix: Update bright task dataset path (#4144) (23856d1)

2.8.9 (2026-02-24)

Ci

• ci: raise leaderboard docker workflow timeout to 12min (#4153)

• feat: enhance leaderboard Docker timeout debugging

• Add explicit log flushing after each Step 1/7 through 7/7 completion for better Docker/CI visibility into initialization bottlenecks
• Add log flushing for cache loading strategies to identify which cache approach is being used (existing cache vs download vs git clone)
• Increase Docker workflow timeout from 9 to 12 minutes (720s)
• Enhance progress detection to include "Prerun complete" logs
• Show last 3 progress lines instead of 2 for better context

This enables clear identification of which initialization phase causes timeouts, with granular timing data for each step.

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Revert "feat: enhance leaderboard Docker timeout debugging"

This reverts commit cba77a5e9781d05efa949371f2d48d21931d03be.

• feat: increase Docker workflow timeout from 9 to 12 minutes

• Update timeout-minutes from 9 to 12 in leaderboard Docker workflow
• Update script timeout check from 480s to 720s (8min to 12min)
• Allow more time for container initialization in CI environment

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

---

Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> (8edfd07)

Fix

• fix: Metadata for nvidia colembed models (#4157)

• fix metadata

• fix metadata (017024e)

Unknown

• Fix Error in Performance Per Model Size Plot in MIEB (#4149)

• Fix Error in Performance Plot in MIEB

• added as customizable column (71fca40)

2.8.8 (2026-02-23)

Fix

• fix: num_proc for reuploading (#4146)

fix num_proc (007ab9e)

2.8.7 (2026-02-23)

Fix

• fix: lotte wrong main metric (#4128)

• fix lotte

• fix function name (c997546)

• fix: pylate model prompts (#4120)

• fix pylate model prompts

• fix log message

• simplify a bit

• fix imoprt

• Update mteb/models/model_implementations/pylate_models.py

Co-authored-by: Antoine Chaffin <38869395+NohTow@users.noreply.github.com>

---

Co-authored-by: Antoine Chaffin <38869395+NohTow@users.noreply.github.com> (dc3aa46)

2.8.6 (2026-02-23)

Documentation

• docs: fix incorrect example in docs for MAEB (#4113)

docs: fix incorrect example in docs (8e04559)

Fix

• fix: leaderboard missing results (#4140)

fix leaderboard build (e6d8bb2)

2.8.5 (2026-02-23)

Fix

• fix: add performance over time (#4134)

• add performance over time

• change default metric (b2cb4c4)

Unknown

• dataset: Added Ukr Toxicity classification dataset (#4108)

• Added Ukr Toxicity classification dataset

• Update mteb/tasks/classification/ukr/ukr_toxicity_classification.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• format citiatons and lint

• add descriptive statistics

• fix citations

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (7a18af4)

• Add msmarco-minilm-v2 series (#4132)

• Add msmarco-minilm-v2 series

• fix metadata "n_embedding_param" (7f1fa2b)

2.8.4 (2026-02-22)

Fix

• fix: Change transformers version to 5.0.0 for nemotron-colembed-v2 models and update uv.lock (#4125)

• change transformers version to 5.0.0 for nemotron-colembed and update lockfile

• update model revisions (fc929e0)

Unknown

• Fill up n_embedding_parameters value in ModelMeta (#4050)

• Fill up n_embedding_parameters value in ModelMeta

• fix naming

• add more results and fix tests

• Remmove private models from _HISTORIC_MODELS

• fix tests

• Address comments

• Add parameter for vlm2vec models

• Add fetch_from_hf parameter

• apply suggestions

• apply suggestions

• fx typechecking

• correctly apply suggestion

• make lint

• fix test

• fix documentation

• Reuse Loaded json (72bde5c)

• add method for creating collection for benchmark (#4115)

• add method for pushing

• simplify

• add comment to description

• simplify imports

• customize collection name (5f12691)

2.8.3 (2026-02-20)

Fix

• fix: Pair classification convert to dict (#4119)

fix pair classification (88fb54e)

Unknown

• updating to correct revision for miniac-embed (#4116)

• Add ManiacLabs/miniac-embed to MTEB

Adds ManiacLabs/miniac-embed. Sentence Transformers–compatible; 1024-d output, cosine similarity.

New file: mteb/models/model_implementations/maniac_labs.py

---

Checklist

• [ ] I have filled out the ModelMeta object to the extent possible
• [ ] I have ensured that my model can be loaded using
  • [ ] mteb.get_model(model_name, revision) and
  • [ ] mteb.get_model_meta(model_name, revision)
• [ ] I have tested the implementation works on a representative set of tasks.
• [ ] The model is public, i.e., is available either as an API or the weights are publicly available to download

Co-authored-by: Cursor <cursoragent@cursor.com>

• add HuggingFace reference link and model revision

• Add reference URL: https://huggingface.co/ManiacLabs/miniac-embed
• Add revision: 0fe5413163ce75cf13e6351b39a8b6f321b64e79 (latest commit)

Addresses reviewer feedback from @ayush1298

Co-authored-by: Cursor <cursoragent@cursor.com>

• training_datasets: use dataset names only (drop author/org prefix)

Co-authored-by: Cursor <cursoragent@cursor.com>

• remove citation (leave blank)

Co-authored-by: Cursor <cursoragent@cursor.com>

• remove query prompt from loader_kwargs

Co-authored-by: Cursor <cursoragent@cursor.com>

• update revision and release_date for ManiacLabs/miniac-embed

Co-authored-by: Cursor <cursoragent@cursor.com>

---

Co-authored-by: JD Pruett <85522589+jdpruett44@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> (3a7566f)

2.8.2 (2026-02-19)

Fix

• fix: refactor from_* function in model metas (#4101)

• fix: refactor from_* function in model metas

• minor fixes based on review

• fixes from review

• format

• reworked computation of embedding size

• minor fixes

• fixed failed tests and typecheck

• Update mteb/models/model_meta.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (c916c0b)

Unknown

• add: jina-embeddings-v5-text family (#4102)

• add: jina-v5

• fix: linting error

• fix: missing values

• fix: remove useless code

• add: citation

• Update mteb/models/model_implementations/jina_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• fix: use different mapping strategy

• fix implementation

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (f607bae)

• model: Add the model ManiacLabs/miniac-embed (#4096)

• Add ManiacLabs/miniac-embed to MTEB

Adds ManiacLabs/miniac-embed. Sentence Transformers–compatible; 1024-d output, cosine similarity.

New file: mteb/models/model_implementations/maniac_labs.py

---

Checklist

• [ ] I have filled out the ModelMeta object to the extent possible
• [ ] I have ensured that my model can be loaded using
  • [ ] mteb.get_model(model_name, revision) and
  • [ ] mteb.get_model_meta(model_name, revision)
• [ ] I have tested the implementation works on a representative set of tasks.
• [ ] The model is public, i.e., is available either as an API or the weights are publicly available to download

Co-authored-by: Cursor <cursoragent@cursor.com>

• add HuggingFace reference link and model revision

• Add reference URL: https://huggingface.co/ManiacLabs/miniac-embed
• Add revision: 0fe5413163ce75cf13e6351b39a8b6f321b64e79 (latest commit)

Addresses reviewer feedback from @ayush1298

Co-authored-by: Cursor <cursoragent@cursor.com>

• training_datasets: use dataset names only (drop author/org prefix)

Co-authored-by: Cursor <cursoragent@cursor.com>

• remove citation (leave blank)

Co-authored-by: Cursor <cursoragent@cursor.com>

• remove query prompt from loader_kwargs

Co-authored-by: Cursor <cursoragent@cursor.com>

---

Co-authored-by: JD Pruett <85522589+jdpruett44@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> (e64ed80)

• model: Add NIFE models (#4058)

• model: Add NIFE models

closes #3586

@stephantul can I ask you to review the metadata

ran it using:

import mteb

model = mteb.get_model("stephantulkens/NIFE-mxbai-embed-large-v1")

# dummy small tasks
task1 = mteb.get_task("LccSentimentClassification")
task2 = mteb.get_task("TwitterHjerneRetrieval")

results = mteb.evaluate(model, [task1, task2], encode_kwargs={"device": "cpu"}) # to prevent MPS error

which uses the encode function for the classification task Is that intended?

• updated based on review

• updated to new models and metadata

• fixes from review

• fixed revision (7f24538)

2.8.1 (2026-02-17)

Fix

• fix: Remove duplicate citations and add test to prevent it going forward (#4032)

• test: add test to detect duplicate citations

• quality

• move changes to task file

• fix falsepositives

• update citations

• add models and benchmarks

• search close titles

• fix maeb

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (377f1b4)

Unknown

• benchmark: add 6 new VisRAG retrieval tasks and corresponding stats (#4059)

• dataset: add 6 new VisRAG retrieval tasks and corresponding stats

• Introduced VisRAGRetArxivQA, VisRAGRetChartQA, VisRAGRetInfoVQA, VisRAGRetMPDocVQA, VisRAGRetPlotQA, and VisRAGRetSlideVQA classes for various retrieval tasks.
• Added JSON files containing descriptive statistics for each task, including sample counts, image dimensions, and query statistics.
• Updated the retrieval module's init.py to include the new tasks in the module exports.

• fix a linter error

• dataset: introduce VisRAG Retrieval Benchmark

• fix: metadata update of VisRag

• Update benchmark metadata

• Update VisRAG datasets metadata, including one-line description and the domains

• Update slideVQA domain

• Add Aliases for VisRAG

• Fix bibtex format

• Update dataset metadata to point to the mteb versions

• Remove redundant data loading

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (7a9e653)

• fix: Remove MAEB+ and MAEB(extended) from leaderboard and add "beta" to all MAEB (#4103)

• fix: Remove MAEB+ and MAEB(extended)

related to #3470

We could consider keeping the two benchmarks (would still need to be beta as paper is review) so they could change.

• fix: Remove MAEB+ and MAEB(extended) from leaderboard and make add "beta" to all MAEB

related to #3470

• Remove MAEB+ and MAEB(extended) from leaderboard
• Added the beta tag to denote that these might change

Currently implemented it as keeping the two temporary benchmarks. We could consider removing them as well (I am unsure how much of a burden it is for us to maintain them), but I would probably not add them to the leaderboard.

All of these changes should be backward compatible

• docs: Added whatsnew

• implement fixes

• update description to explain beta status (77ac52b)

2.7.30 (2026-02-12)

Fix

• fix: correct reference link for MIRACLVisionRetrieval task (#4092) (d3e9b06)

2.7.29 (2026-02-12)

Documentation

• docs: Improved adding a benchmark docs (#4087)

• docs: Improved adding a benchmark docs

expanded the explanation of provide more information about the process.

• Update docs/contributing/adding_a_benchmark.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• fix

• fix

• minor heading change

• Apply suggestions from code review

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (8149d4a)

Fix

• fix: constrain the transformers library version for jina-clip (#4061)

• fix: constrain the transformers library version for jina-clip to avoid compatibility issue

• add require package

• ad to conflicts

• try to run again

• upd lock

• tmp

• try

• fix: pylate dependency on outdated version of transformers

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (eee82cf)

Unknown

• dataset: Add MTEB(spa) Spanish language benchmark (#4053)

• dataset: Add MTEB(spa) Spanish language benchmark

Define MTEB(spa, v1) benchmark grouping 23 existing Spanish tasks across 6 task types: Classification (8), Clustering (3), PairClassification (2), Reranking (1), Retrieval (5), and STS (4).

• fix: Replace MIRACLRetrieval with HardNegatives.v2 per review

• fix: Remove tasks with known issues, add contact, reduce to 16 tasks

• Apply suggestion from @KennethEnevoldsen

---

Co-authored-by: Clemente <clemente@Clementes-MacBook-Pro.local> Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (2507bec)

• Move MIEB datasets to mteb HuggingFace org (#4070)

• Move 15 MIEB datasets to mteb HuggingFace org

Update dataset paths and revisions for tasks that now use datasets forked to the mteb org:

• MMSoc_HatefulMemes, MMSoc_Memotion (Ahren09 -> mteb)
• blink-it2i, blink-it2i-multi, blink-it2t, blink-it2t-multi (JamieSJS -> mteb)
• gld-v2-i2t, imagecode, imagecode-multi (JamieSJS -> mteb)
• imagenet-10, imagenet-dog-15, met (JamieSJS -> mteb)
• r-oxford-easy-multi, r-oxford-medium-multi, r-oxford-hard-multi (JamieSJS -> mteb)

Part of #4049

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• transfer mrbench

• Move 8 isaacchung MIEB datasets to mteb HuggingFace org

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Move 12 MIEB datasets to mteb HuggingFace org

Datasets moved:

• dpdl-benchmark/sun397
• ethz/food101
• floschne/xflickrco
• floschne/xm3600
• flwrlabs/ucf101
• nyu-visionx/CV-Bench
• tanganke/dtd
• tanganke/stl10
• timm/eurosat-rgb
• timm/resisc45
• uoft-cs/cifar10
• uoft-cs/cifar100

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Move 15 MIEB datasets to mteb HuggingFace org

Datasets moved:

• JamieSJS/r-paris-easy-multi
• JamieSJS/r-paris-medium-multi
• JamieSJS/r-paris-hard-multi
• JamieSJS/rp2k
• JamieSJS/sketchy
• JamieSJS/stanford-online-products
• JamieSJS/vizwiz
• JamieSJS/vqa-2
• Pixel-Linguist/rendered-sts12
• Pixel-Linguist/rendered-sts13
• Pixel-Linguist/rendered-sts14
• Pixel-Linguist/rendered-sts15
• Pixel-Linguist/rendered-sts16
• ylecun/mnist
• zh-plus/tiny-imagenet

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Move 7 clip-benchmark datasets to mteb HuggingFace org

Datasets moved:

• clip-benchmark/wds_country211
• clip-benchmark/wds_fer2013
• clip-benchmark/wds_gtsrb
• clip-benchmark/wds_renderedsst2
• clip-benchmark/wds_vtab-clevr_closest_object_distance
• clip-benchmark/wds_vtab-clevr_count_all
• clip-benchmark/wds_vtab-pcam

Note: wds_imagenet1k failed due to storage limits.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Move 14 MIEB datasets to mteb HuggingFace org

Datasets migrated:

• clip-benchmark/wds_imagenet1k → mteb/wds_imagenet1k
• m-a-p/SciMMIR → mteb/SciMMIR
• yjkimstats/SUGARCREPE_fmt → mteb/SUGARCREPE_fmt
• nelorth/oxford-flowers → mteb/oxford-flowers
• vidore/arxivqa_test_subsampled_beir → mteb/arxivqa_test_subsampled_beir
• vidore/docvqa_test_subsampled_beir → mteb/docvqa_test_subsampled_beir
• vidore/infovqa_test_subsampled_beir → mteb/infovqa_test_subsampled_beir
• vidore/shiftproject_test_beir → mteb/shiftproject_test_beir
• vidore/syntheticDocQA_artificial_intelligence_test_beir → mteb/syntheticDocQA_artificial_intelligence_test_beir
• vidore/syntheticDocQA_energy_test_beir → mteb/syntheticDocQA_energy_test_beir
• vidore/syntheticDocQA_government_reports_test_beir → mteb/syntheticDocQA_government_reports_test_beir
• vidore/syntheticDocQA_healthcare_industry_test_beir → mteb/syntheticDocQA_healthcare_industry_test_beir
• vidore/tabfquad_test_subsampled_beir → mteb/tabfquad_test_subsampled_beir
• vidore/tatdqa_test_beir → mteb/tatdqa_test_beir

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• update rest

---

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (2ef04f8)

• model: add voyage-4-nano (#4086)

• model: add voyage-4-nano model implementation

• Apply suggestion from @Samoed

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (2ce07c4)

2.7.28 (2026-02-11)

Fix

• fix: Remove task performance by type tab when there is only one type (#4067)

• Remove task performance by type Tab when the Radar plot can't be generated

• Apply suggestion (9f95b58)

Unknown

• Correct Embedding Dimension for paraphrase-multilingual-MiniLM-L12-v2 (#4089) (334d690)

2.7.27 (2026-02-11)

Documentation

• docs: Outline for adding a task documentation (#4082)

• docs: Outline for adding a task documentation

This is a suggested structure, PR is just to get feedback before I finish it up.

fixes #4077

• upd docs

• install dependencies in ci

• add example with retrieval

• filled out the missing segments

• lint and format

• Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• fix numerating and indent

• add missing imports

• fix links

• add full example for retrieval dataset

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> (42b8058)

• docs: Improve docstring for some of the main abstasks (#4083)

• docs: fix AbsTaskClassification docstring formatting and improve docstrings for some of the main tasks

• format (50bd0fa)

Fix

• fix: Add performance per language tab to more benchmarks (#4066)

Add Performance per language Tab to more benchmarks (4ca1922)

Unknown

• dataset: add 'law-ir_ko' dataset for IR task (#4052)

• law_ir_ko

• Update mteb/tasks/retrieval/kor/law_ir_ko.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• law_ir_ko info revision

• description

• metadata-info rev

• metadata-info rev

• Update mteb/tasks/retrieval/kor/law_ir_ko.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• statistics(), reference

• format citation

• author & howpublished rev

• make lint

• description rev

• Update mteb/tasks/retrieval/kor/law_ir_ko.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update mteb/tasks/retrieval/kor/law_ir_ko.py

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (1cdc662)

• model: Add ModelMeta for geoffsee/auto-g-embed-st (#4074)

Add ModelMeta for geoffsee/auto-g-embed-st (81540a2)

• Add MetaCLIP 2 model integration (#4065)

• Add MetaCLIP 2 model integration

Add support for facebook/metaclip-2-mt5-worldwide-b32, a multilingual vision-language model using mT5 tokenizer for worldwide language support.

• 254M parameters, 512 embedding dimension
• Supports 99 languages (XLMR language set)
• Handles MetaCLIP 2's BaseModelOutputWithPooling return format

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Add n_embedding_parameters for MetaCLIP 2 model

Set n_embedding_parameters to 128,057,344 (mT5 vocab size 250,112 × embed_dim 512) to fix test_n_embedding_parameters test failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Add training metadata for MetaCLIP 2 model

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Apply suggestion from @Samoed

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (4280dd9)

2.7.26 (2026-02-07)

Fix

• fix: filter corrupted image in Birdsnap (#4068)

• fix: filter corrupted image in Birdsnap and drop unused splits in zero-shot tasks

• Filter out corrupted/truncated image at index 3854 in Birdsnap train split
• Add dataset_transform to AbsTaskZeroShotClassification to keep only eval splits (zero-shot tasks don't need train splits)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• fix: handle BaseModelOutputWithPooling in CLIP model wrapper

In transformers 5.x, get_text_features and get_image_features return BaseModelOutputWithPooling instead of a tensor directly. Extract the pooler_output when needed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• fix: add None check for dataset in zeroshot classification transform

Fixes mypy type errors where self.dataset could be None when accessing .keys() and deleting splits in dataset_transform method.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> (6c00506)

Unknown

• Backfill missing metadata for historic datasets (#4063)

• Backfill missing metadata for historic datasets

Fill in missing TaskMetadata fields for ~90 historic datasets as described in issue #2502. This includes:

• Classification tasks (Polish, Chinese)
• Clustering tasks (German, French, Spanish, Swedish, Chinese, Multilingual)
• Pair classification tasks (Polish, Chinese)
• Reranking tasks (English, French, Chinese)
• Retrieval tasks (German, English, Japanese, Korean, Polish, Spanish, Chinese, Multilingual)
• STS tasks (German, English, French, Korean, Spanish, Chinese)

Fields filled include: date, domains, task_subtypes, license, annotations_creators, dialect, sample_creation, and bibtex_citation.

The _HISTORIC_DATASETS list is reduced from ~90 entries to just 4 aggregate tasks whose metadata computation has a separate issue (the compute* methods return None for single-valued fields).

Closes #2502

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Fix type annotation in _compute_license method

Add StrURL to the return type and set type annotation to match the license field type (Licenses | StrURL | None).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> (b5fb471)

2.7.25 (2026-02-07)

Fix

• fix: mscoco (#4062)

• fix mscoco

• fix jina clip (c2d1bfe)

Unknown

• fix Remove the hardcoded batch_size=1 when generating text and image embeddings for Nemotron-Colembed-v2 models (#4054)

remove hardcoded batch_size 1 (1682b2f)

• Update nemotron v2 citation (#4051)

update nemotron v2 citation (0be1df3)

2.7.24 (2026-02-05)

Fix

• fix: leaderboard errors (#3969)

• fix leaderboard

• fix leaderboard errors

• simplify

• upd description (fd37337)

Unknown

• fix docs deploy command (#4044) (acb3d8c)

• fix docs links (#4043) (5d3b1a8)

2.7.23 (2026-02-04)

Fix

• fix: Fill in embedding and total parameters in ModelMeta (#4031)

• Filling Embedding/Total Parameters in ModelMeta

• Add parameter for other models

• Add parameters for more models

• Added exact value for n_parameters

• Fix tests

• set n_embedding_parameters to None

• Add results of some more models

• Add tests

• Add _HISTORIC_MODELS list in test

• Update tests/test_models/test_model_meta.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• fix tests

• correct tests

• fix _HISTORIC_MODELS list

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (bc6e6cb)

Unknown

• dataset: Add ERESS reranking task (#3991)

• dataset: Add ERESS reranking task

• Add ERESSReranking task for e-commerce product relevance reranking
• Dataset: thebajajra/eress with ~72k query-product pairs
• Supports graded relevance (0-100 integer scale)
• Main metric: nDCG@5
• Add E-commerce domain and Product Reranking subtypes to TaskMetadata
• Include descriptive statistics

• fix: align dataset_transform signature with base class

• fix: dataset reuploaded, custom transformation removed

• fix: rev updated with title + text combination

• description moved away from docstring

• Update mteb/tasks/reranking/eng/ecommerce_product_relevance_reranking.py

---

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (fe67f8e)

2.7.22 (2026-02-03)

Documentation

• docs: Added changelog (#3741)

• docs: Added changelog

• Clean up docs to prepare for adding the changelog. By adding missing links and removing references to documentation that does not exist
• Added whats new section
• Added changes from 2.0 upwards. I might be missing some

I think going forward we can just update this as well go.

• minor fix

• added autogenerated changelog

• rename

• add autogenerated workflows

• updates

• update

• update (2082d3e)

Fix

• fix: backfilling historic tasks (#4034)

• fix: backfilling historic tasks

• [x] Backfilled task metadata
• [x] extended test to ensure that backfilled tasks are removed from the historic list

addresses #2502

• back citation, date and task subtypes where only those are missing

• Update mteb/tasks/pair_classification/pol/polish_pc.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• add famteb citation

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (e542519)

2.7.21 (2026-02-03)

Fix

• fix: Add Optional dependencies for NemotronColEmbed models as extras (#4036)

• fix: Move nemotron-colembed models to separate module

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: Add optional extra section for nemotron-colembed-vl-v2

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: add requires_package to init method of Llama nemotron class

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: consolidate and rename nemotron colembed classes

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: Update transformers version check to reference extra name

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: revert change to llama-embed-nemotron extra

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: restore transformers_version_constraint in ModelMeta definitions

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: set citation to None for llama-nemotron-colembed-vl-3b-v2

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: revert unrelated citation change

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: change version typo for 4b model

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: cleanup now unused transformers_version_constraint kwarg

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: revert change to revision and transformers version

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• fix: rename extra for llama nemotron models to llama-nemotron-colembed-vl

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>

• Update nemotron extras with accelerate for clarify

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (2cc8dc2)

2.7.20 (2026-02-03)

Fix

• fix: avoid supplying trust_remote_code twice to `OpenSearch-AI/Ops-Co… (#4042)

fix: avoid supplying trust_remote_code twice to OpenSearch-AI/Ops-Colqwen3-4B

related to: #4040 (9aa5ae5)

2.7.19 (2026-02-03)

Fix

• fix: change np.bool to np.bool_ (#4041)

change np.bool (53de8c1)

Unknown

• model: add the model of boom (#4022)

• add the model of boom

• add the model of boom

• rename the file from boom_models.py to ict_time_and_querit_models.py, match MTEB task names, and remove the #

• rename the file from boom_models.py to ict_time_and_querit_models.py, match MTEB task names, and remove the #

• lint

• dirctly use the InstructSentenceTransformerModel class

• Remove unused boom_4b_v1_loader function

Removed commented-out loader function for BOOM_4B_v1 model.

• Removed the commented code

• lint again

• Removed the commented code

• lint again

• Remove the description comment at the beginning

• Remove the description comment at the beginning and update the n_parameters

• Add adapted_from field to model metadata

• add the description of some instruction templates

• reformatted the ict_time_and_querit_models.py

• update the model revision

---

Co-authored-by: zhanghengran <zhanghengran@baidu.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (62d354a)

2.7.18 (2026-02-02)

Fix

• fix: set num proc to None by default (#4038)

set num proc to none by default (b3a51c6)

2.7.17 (2026-02-02)

Fix

• fix: Improve array typing by also specifying dtype (#4018)

• Correct array typing

• Added arrat typing in docs

• fix typechecking

• Address comments

• fix typechecking

• simplify typechecking

• fix typecheck

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (3a40adc)

• fix: Update ViDoRe3 to rank based on the mean (#4033)

• Update ViDoRe3 benchmark and leaderboard

• revert uv.lock (b616950)

2.7.16 (2026-02-01)

Fix

• fix: remove undocumented models or models with no model card (#4030)

• fix: don't fill-missing metedata by default

This include two changes:

1. rename compute_missing to fill_missing, it seems like this change was only applied to get_model_meta, but not throughout the stack. This is technically breaking so I added a deprecation.
2. passed fill_missing to from_hub, I believe this was unintentionally missed

fixes #4027

• set fill_missing=False for from_{model}

• propegate deprecation

• add framework default

• fix: remove undocumented models or models with no model card

Also removed scripts/generate_metadata.py

fixes #3746

• compute_missing > compute_metadata

• clean up tests to match new functionality

• format

• lint (dbc7334)

2.7.15 (2026-01-31)

Ci

• ci: Pre-commit should not be forced (#4024)

We want to support people using pre-commit, but we don't want to force it. (b45ac3a)

Fix

• fix: don't fill-missing metedata by default (#4029)

• fix: don't fill-missing metedata by default

This include two changes:

1. rename compute_missing to fill_missing, it seems like this change was only applied to get_model_meta, but not throughout the stack. This is technically breaking so I added a deprecation.
2. passed fill_missing to from_hub, I believe this was unintentionally missed

fixes #4027

• set fill_missing=False for from_{model}

• propegate deprecation

• add framework default (d06184f)

2.7.14 (2026-01-30)

Fix

• fix: Make mteb.get_model compatible with CrossEncoders (#3988)

• Made mteg.get_model compatible with CrossEncoders and SparseEncoders

• update loader for sparseEncoder

• fix import

• Simplify structure

• Add model_type to sparseEncoder models

• remove detection logic of sparsencoder

• Add tests and documentation

• simplified tests

• updated docs

• fix docs

• fix

• fix grammar

• Update docs/usage/defining_the_model.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• Update docs/advanced_usage/two_stage_reranking.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• Update docs/index.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• address comments

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> (8cacef7)

Unknown

• Add to_disk and from_disk to ModelResults (#3973)

• Add to_disk and from_disk to ModelResults

• fix signature

• change import to high-level

• Changed model_validate to model_validate_json

• fix tests

• remove duplicate path (4a33277)

• Fix support for datsets 4.5 with pandas 3 (#3983)

• fix test

• fix: sanitize type for label during array conversion

• lint

• revert typo fix

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (8876169)

• rename bm25s to baseline/bm25s (#4007)

• rename bm25s to baseline/bm25s

• Update mteb/models/get_model_meta.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

• remove logger message

• rename Human to baseline/Human

---

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> (3bab8d9)

2.7.13 (2026-01-30)

Fix

• fix: simplify dependencies (#4017) (7f07463)

Unknown

• Build image on leaderboard refresh (#4015)

build image on leaderboard refresh (b9ff905)

• model: added Querit/Querit (#3996)

• querit_models_add

• Querit_Models_Change

• Update

• format revise

• add future

• format revise

• format revise

• last format revison

• last last revise

• last last last revison

• revise

• revise

• change the instruction

• last revison

• revise

• revise

• revise

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (5ca3387)

• Adding nvidia/nemotron-colembed models (#3941)

• Adding nvidia/nemotron-colembed models

• add colembed 4b, 8b model meta

• fix colembed-3b-v2 model name

• update revision for colembed 3b

• update revisions

• Update mteb/models/model_implementations/nvidia_llama_nemoretriever_colemb.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• lint

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (47d59ad)

• model: added nomic-ai/nomic-embed-code (#4006)

• Add model metadata for nomic-embed-code

Added new model metadata for 'nomic-embed-code'

• fix nomic_embed_code

• lint

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (88e72c6)

• model: Adding Ops-Colqwen3 models (#3987)

• Create ops_colqwen3_models.py

• Refactor OpsColQwen3 model and processor classes

• Update model revision in ops_colqwen3_models.py

• Remove calculate_probs method and fix model name

Removed the calculate_probs method and updated model name.

• format

• fix ds name

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (7c9bab2)

2.7.12 (2026-01-25)

Fix

• fix: all dataset transform (#4002)

fix dataset transform (78687f7)

• fix: dataset_transform signature in PubChemWikiPairClassification (#4001)

fix: add num_proc arg to PubChemWikiPairClassification dataset_transform (4faaf36)

2.7.11 (2026-01-25)

Fix

• fix: BedrockModel initialization arguments (#3999)

fix: add model_name arg to BedrockModel init to prevent multiple values for model_id (0b9de9b)

2.7.10 (2026-01-25)

Fix

• fix: NomicWrapper get_prompt_name call (#3995)

fix(models): correct get_prompt_name call in NomicWrapper (74dc105)

• fix: BAAI/bge-small-en model revision (#3993)

fix(models): update invalid bge-small-en revision (8c6a4d6)

2.7.9 (2026-01-24)

Fix

• fix: add kwargs to pub chem load data (#3990)

add kwargs to pub chem load data (aeb22cd)

2.7.8 (2026-01-20)

Fix

• fix: Filled active_parameter_overiride for GritLM/GritLM-8x7B nomic-ai/nomic-embed-text-v2-moe (#3967)

• Filled active_parameter_overiride for ritLM/GritLM-8x7B and nomic-ai/nomic-embed-text-v2-moe

• add correct parameters for nomic-ai/nomic-embed-text-v2-moe (dbd4287)

2.7.7 (2026-01-20)

Fix

• fix: leaderboard Nan handling (#3965)

• fix leaderboard

• fix loading aggregated tasks

• Update mteb/results/task_result.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (a369c26)

• fix: Add fill_missing parameter in get_model_meta (#3801)

• Add compute missing parameter in get_model_meta

• fix logs

• fix

• fix from comments

• apply suggestion

• fix method

• add test and fix logic

• address comments

• rename compute_missing to fill_missing

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (90536d4)

• fix: Ensure that retrieval tasks only evaluate on specified subsets instead of all (#3946)

• fix dataset loading

• update logging

• add test (8186392)

Unknown

• Merge branch 'main' of https://github.com/embeddings-benchmark/mteb (523b5bc)

• model: Adding voyage-4-large (2048d) model configs (#3970)

• Adding voyage-4-large (2048d) model configs

• Adding voyage-4-large 2048d model configs

• Adding voyage-4-large 2048d model configs (961a43b)

2.7.6 (2026-01-20)

Fix

• fix: saving aggregated tasks (#3915)

fix saving (ced5f71)

2.7.5 (2026-01-19)

Fix

• fix: use num_proc for dataset processing (#3832)

• add typehint for encode kwargs

• remove num_proc

• start adding num_proc

• remove all num proc

• fix import

• add num proc to transform

• add to push to hub

• use num proc in vidore v2

• move num proc to evaluate

• pass num proc everywhere

• fix tests

• fix pylate

• fix image text pair

• fix num workers

• add kwargs to load_data (daf2b6f)

2.7.4 (2026-01-19)

Fix

• fix: Update metadata to include active number of parameter to ModelMeta (#3837)

• Add active parameter column on LB

• update ModelMeta with parameters

• update ModelMeta of models

• Delete parameter_update_results.csv

• fix test

• fix tests

• delete script

• rename for consistency

• convert active_parameter to property

• rename and fix property

• update embedding parameters for model2vec models

• remove duplicate loading of models

• fix

• lintter

• fix

• remove separate method for embedding parameter calculation

• fix embedding calculation to pass typecheck

• lintter

• fix checking

• rename active parameters

• upd docstring

• fix tests

• remove n_active_parameters_override from ModelMeta of all models

• lintter

• rename file instead of merging main

• fix tests

• correct tests

• Delete model total and active parameters - model_parameters.csv

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (a45359e)

Refactor

• refactor: split BRIGHT benchmark into individual subset tasks (#3285)

• refactor: split BRIGHT benchmark into individual subset tasks

• readd bright

• readd bright subset tasks

• feat: add descriptive stats for BRIGHT subsets retrieval tasks

• feat: add top_ranked for excluded_ids handling

• change main score to recall@1 for long version

• improve BRIGHT task descriptions

• add prompts to BRIGHT retrieval tasks

• refactor: BRIGHT(v1.1)

• calculate descriptive stats for BRIGHTLongRetrieval

• update prompts

• normalize names in prompts

• don't filter tasks

• remove filter_queries_without_positives and update revision

• don't create top ranked if not necessary

• get back naucs

• fix instructions

• add warning

• fix import

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (2c9b9e9)

2.7.3 (2026-01-19)

Fix

• fix: temporarily remove private column from RTEB (#3932)

• fix: temporarily remove private column from RTEB

Link is still missing the note as I am waiting for @isaac-chung and @Samoed to confirm the write-up.

fixes #3902

• added issue link

• fix remove mean (Task)

• lint

• merge in fixes to remove_private (#3940)

fix: exclude private tasks from Borda rank calculation in RTEB

Co-authored-by: bflhc <kunka.xgw@gmail.com>

---

Co-authored-by: bflhc <kunka.xgw@gmail.com> (b968433)

• fix: correct inverted unload_data condition in evaluate (#3929)

Add tests verifying preloaded data is preserved.

Co-authored-by: Daniel Svonava <daniel@superlinked.com> (1c5d9c6)

Refactor

• refactor: Activate TC (#3800)

• activate tc

• activate TC

• small import fix

• fix imports

• fix imports

• fix pil import

• fix benchmark result validation

• full benchmark fix

• update

• fix unpack imports

• upd vllm type (16e0211)

Unknown

• dedup colnomic_7b and fix loader (#3957)

• dedup colnomic_7b and fix loader

• remove flash_attention_2 (8c96b79)

• fix colSmol-256M revision (#3956) (b3c2718)

• fix typo (#3954) (d7acd57)

2.7.2 (2026-01-15)

Documentation

• docs: fix vllm broken link (#3936)

fix vllm link (d045d53)

Fix

• fix: expose ResultCache directly as mteb.ResultCache (#3912)

• fix: expose ResultCache directly as mteb.ResultCache

fixes #3910

• docs: Update docs usage of ResultCache (3103f97)

• fix: computation of results with missing scores (#3874)

• fix computation of results with missing scores

• fix test

• change 0 to nan

• change 0 to nan

• remove fill_missing_scores (d60e916)

Unknown

• model: add pixie_models (#3938)

• model: add pixie_models

• Apply lint formatting (8b54f0e)

• model: mixedbread-ai/mxbai-edge-colbert-v0-32m and mixedbread-ai/mxbai-edge-colbert-v0-17m (#3931)

• Add model: mixedbread-ai/mxbai-edge-colbert-v0-32m and mixedbread-ai/mxbai-edge-colbert-v0-17m

• Lintter

• Add quotes

• Update dataset name

• Apply suggestions from code review

• Update mixedbread_ai_models.py

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (5de2194)

2.7.1 (2026-01-14)

Fix

• fix: Minor logging fixes by activate LOG rule (#3820)

activate logger rule (65313c9)

Unknown

• model: Adding voyage-4 model (#3927)

• Adding voyage-4 model

• Adding voyage-4 model configs (b80da30)

• Update references and citations for ViDoRe V3 benchmark (#3930)

• fix: Update references and citations for ViDoRe V3 benchmark

• foramat citation

• format again

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (e7d077e)

• model: add nemotron rerank (#3750)

• add nemotron rerank

• move to nvidia models

• removed extra params

• Apply suggestions from code review

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• remove or

• add docstring

• Update mteb/models/model_implementations/nvidia_models.py

Co-authored-by: Yauhen Babakhin <ybabakhin@nvidia.com>

• update

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com> Co-authored-by: Yauhen Babakhin <ybabakhin@nvidia.com> (330a601)

• dataset: Add EuroPIRQRetrieval dataset (#3924)

• dataset: Add EuroPIRQRetrieval dataset

• Removed unnecessary load dataset functions (7966e06)

• dataset: add ChemRxivRetrieval task to ChemTEB benchmark (#3923)

• dataset: add ChemRxivRetrieval task to ChemTEB benchmark

• fix: add descriptive statistics

• feat: add ChemTEB v1.1 with ChemRxivRetrieval task

• fix: chemteb v1.1 alias (86359fd)

2.7.0 (2026-01-13)

Documentation

• docs: Resolve problems with missing documentation links (#3834)

• resolve problems with missing documentation links

• split into files (0d277cd)

Feature

• feat: Add vLLM support (#3794)

• init

• init

Signed-off-by: wang.yuqi <noooop@126.com>

• ruff

Signed-off-by: wang.yuqi <noooop@126.com>

• • vllm_loader

Signed-off-by: wang.yuqi <noooop@126.com>

• • TYPE_CHECKING

Signed-off-by: wang.yuqi <noooop@126.com>

• Make vLLM exit properly.

Signed-off-by: wang.yuqi <noooop@126.com>

• rename

Signed-off-by: wang.yuqi <noooop@126.com>

• support rerank

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• refine

Signed-off-by: wang.yuqi <noooop@126.com>

• refine

Signed-off-by: wang.yuqi <noooop@126.com>

• Update mteb/models/vllm_wrapper.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• refine

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • benchmark

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• • more benchmark

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• refine docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• refine docs

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• fix typing

• move type ignore

• doc upd

• add test

• Update Makefile

• add support for prompts

• add support for prompts

• • demo

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• make mypy happy

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

• fix typehints

• update pyproject

• update pyproject

• update pyproject

• The pooling + dp fails to run.

• fix uv lock

• fix docs

• simplify conflicts

• upd lock

• upd lock

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Apply suggestions from code review

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Update docs/advanced_usage/vllm_wrapper.md

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Apply suggestion from @Samoed

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• update

Signed-off-by: wang.yuqi <noooop@126.com>

• update

Signed-off-by: wang.yuqi <noooop@126.com>

---

Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (4d568a8)

Unknown

• model: Update the nemo retriever reversions to avoid error when loading the model (#3925)

• Update the nemo retriever versions to fix the crash issue with visual_config

• Update mteb/models/model_implementations/nvidia_llama_nemoretriever_colemb.py

• Update mteb/models/model_implementations/nvidia_llama_nemoretriever_colemb.py

---

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (dcd31fa)

• model: Adding voyage-4-large, voyage-4 and voyage-4-lite (#3885)

• Adding voyage-4-large and voyage-4-lite

• Adding voyage-4-large and voyage-4-lite

• Adding voyage-4

• Reverting voyage-4 (as the tokenizer is not yet available publicly)

• added superseeded_by

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com> (64ce6ba)

2.6.9 (2026-01-12)

Fix

• fix: model load test (#3914)

• fix model load test

• trigger on dependencies change (227ce95)

Unknown

• Artifacts for llama-embed-nemotron-8b model (#3919)

add artifacts for llama-embed-nemotron-8b model (c38ed02)

2.6.8 (2026-01-11)

Fix

• fix: KoVidore2EnergyRetrieval revision fix (#3913) (0221b94)

Unknown

• add model: bflhc/Octen-Embedding-0.6B (#3906) (c5c481f)

• model: mixedbread-ai/mxbai-rerank-large-v1 (#3905)

• Add model: mixedbread-ai/mxbai-rerank-large-v1

• apply suggestions

• Added xsmall and base version of reranker models

• lintter (cb29623)

• Add typehint for encode kwargs (#3831)

• add typehint for encode kwargs

• remove num_proc

• remove all num proc

• fix import

• fix docstrings (a417426)

• add dataset: KoViDoRe(v2) (#3876)

• add dataset: KoViDoRe v2

• fix citation format

• add direct loading

• lint format

• delete benchmark language view

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (24f3f85)

2.6.7 (2026-01-08)

Fix

• fix: nv embed version (#3715)

• fix nv embed wrapper

• try to fix

• fix sbert version (d3020f3)

Unknown

• Don't sync make lint (#3841)

• don't sync make lint

• don't sync make typecheck

• upd ci

• upd ci

• upd ci

• upd ci

• upd ci

• swap (cb26458)

• model: add missing sentence transformers and jina models (#3808)

• add sentence transformers models

• add jina v2

• fix modalities (27619d1)

2.6.6 (2026-01-07)

Fix

• fix: Simplify conflicts (#3875)

• simplify conflicts

• add lock

• remove torch (1bc3fba)

Test

• test: Add HF Space Dockerfile using pre-built leaderboard image (#3838)

• Add HF Space Dockerfile using pre-built leaderboard image

Adds a lightweight Dockerfile for HuggingFace Space deployment that uses the pre-built ghcr.io/embeddings-benchmark/mteb/leaderboard image as base. Also adds a workflow to test the Dockerfile.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

• Delete .github/workflows/hf_space_docker.yml

• test: Add CI workflow for HF Space Dockerfile validation

---

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: MTEB Agent <agent@example.com> (b905e27)

Unknown

• model: Update bflhc/Octen-Embedding-8B revision (#3869)

update bflhc/Octen-Embedding-8B revision (1e78793)

• dataset: Vietnamese VN-MTEB TVPLRetrieval, NanoClimateFEVER-VN, NanoFEVER-VN, NanoDBPedia-VN, NanoNQ-VN, NanoHotpotQA-VN, NanoMSMARCO-VN (#3810)

• [ADD] Vietnamese VN-MTEB TVPLRetrieval, NanoClimateFEVER-VN, NanoFEVER-VN, NanoDBPedia-VN, NanoNQ-VN, NanoHotpotQA-VN, NanoMSMARCO-VN

• [UPDATE] descriptive stats

• [UPDATE] bibtext

• [UPDATE] dataset path

• [UPDATE] nano db pedia retrieval

• [UPDATE] size dataset from 1M corpus to 100k

• [ADD] add note about what's different in nano version

• [ADD] TVPLRetrieval description (adb5b42)

2.6.5 (2026-01-05)

Documentation

• docs: Fix docs build strict mode errors (#3809)

• fix: resolve mkdocs strict mode errors

• fix: remove duplicate line in installation.md

• build: add --strict flag to mkdocs build

• fix: resolve invalid BibTeX keys in task citations

• feat: filter BibTeX warnings in strict docs build

• Update docs/usage/defining_the_model.md

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• fix: dynamic mkdocs path discovery for CI

• fix: improve docs build script with clear warning counts

• fix: resolve 6 real docs build warnings

• fix: remove broken PylateSearchEncoder reference

• Remove unused build scripts

• docs: wrap multimodal example as code snippet

• fix: export SklearnModelProtocol for docs API

• docs: add API reference for SklearnModelProtocol

• fix: remove SklearnModelProtocol export to avoid circular import

• feat: add SklearnModelProtocol docs with lazy import

• fix: convert Sphinx cross-references to MkDocs syntax for proper linking

Convert :class: Sphinx syntax to [Text][module.path] MkDocs syntax to ensure cross-references are properly clickable in the generated documentation.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• chore: remove SklearnModelProtocol docs to avoid circular import

• chore: remove SklearnModelProtocol export from _evaluators

• style: fix indentation in _evaluators/init.py

• fix: enable mkdocs build --strict without warnings

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: rename evaluator to multilabel_classifier to avoid type conflict

• fix: use evaluator_model instead of evaluator to avoid type conflict

• Change parameter name from multilabel_classifier to evaluator_model
• Maintains SklearnModelProtocol type hint as requested in PR review
• Resolves mypy type error by using parent class's evaluator_model field
• Keeps explicit protocol reference in docstring for clarity

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

---

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> (3723d27)

Fix

• fix: Extend framework annotations for ModelMeta (#3819)

• Update framework and filter based on them

• update ModelMeta of models

• update ModelMeta of models

• update ModelMeta of models

• update ModelMeta of models

• update ModelMeta of models

• update ModelMeta of models

• update ModelMeta

• add csv

• update ModelMeta

• added framework to ModelMeta

• update ModelMeta of models

• update ModelMeta of models

• update framework in ModelMeta of models

• update framework

• update framework

• update framework in ModelMeta

• fix tests

• Add models

• fix tests

• add tags extraction in from_hub()

• fix typecheck

• apply suggestions

• apply suggestions

• keep only static method

• delete csv and script (d033c24)

Unknown

• fix dataset generation tags (#3835) (bf2627a)

• model: Add SauerkrautLM-ColPali visual document retrieval models (#3804)

• model: Add SauerkrautLM-ColPali visual document retrieval models

Add inference code and requirements for SauerkrautLM-ColPali visual document retrieval models.

These are multi-vector embedding models based on the ColPali architecture:

• ColQwen3 (Qwen3-VL backbone): 1.7B Turbo, 2B, 4B, 8B variants
• ColLFM2 (LFM2-VL backbone): 450M variant
• ColMinistral3 (Ministral3 backbone): 3B variant

All models produce 128-dimensional embeddings per text/image token and use MaxSim (late interaction) for retrieval scoring.

Model checkpoints:

• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColQwen3-1.7b-Turbo-v0.1
• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColQwen3-2b-v0.1
• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColQwen3-4b-v0.1
• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColQwen3-8b-v0.1
• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColLFM2-450M-v0.1
• https://huggingface.co/VAGOsolutions/SauerkrautLM-ColMinistral3-3b-v0.1

• fix: Address review comments

• Remove loader functions, use classes directly in ModelMeta
• Remove unused get_fused_embeddings method
• Move model.to(device) and model.eval() to base class init
• Pass torch_dtype directly to ColMinistral3.from_pretrained

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• Update pyproject.toml

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• fix: Update release_date to 2025-12-20

• fix: address review comments - remove partial, add adapted_from and training_datasets

• Update mteb/models/model_implementations/slm_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

• fix: import COLPALI_CITATION from colpali_models and add model_type

• add training datasets

• fix: remove section headers and use PyPI package instead of Git URL

• fix: resolve merge conflicts and remove section headers

• fix: use COLPALI_TRAINING_DATA for training_datasets

• fix: use exact n_parameters and memory_usage_mb values from HuggingFace

• don't build 3.14

• lint

---

Co-authored-by: David Golchinfar <d.golchin@web.de> Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (44e9b20)

2.6.4 (2026-01-03)

Fix

• fix: Add leaderboard docker workflow (#3828)

• Add GitHub workflow to test leaderboard Dockerfile

• Add .github/workflows/leaderboard_docker.yml workflow that:
  • Builds the Docker image
  • Tests container startup with 6-minute timeout
  • Monitors for container exit codes and failures
  • Shows progress updates during long initialization
  • Validates leaderboard dependencies are available
• Include Dockerfile for leaderboard containerization
• Fix missing typer dependency in leaderboard extras to resolve ModuleNotFoundError
• Update uv.lock with typer dependency

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Optimize leaderboard Docker workflow with smart exit conditions

• Add intelligent startup progress detection instead of blind 5.5min wait
• Monitor key milestones: app startup, Step 1/7 completion
• Only exit early on actual completion signals (Gradio server ready, full initialization)
• Hard timeout failure at 5.5min regardless of progress
• Improved logging with 30s progress updates
• Tested with act: reduces wait time from 5.5min to ~2.5min when appropriate

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Remove redundant Test container dependencies step

• Dependencies already verified during Docker build process
• Runtime verification handled by smart exit conditions
• Eliminates environment-specific import failures in act testing
• Streamlines workflow to focus on essential container startup validation

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Fix Docker image availability and add GHCR caching

• Add load: true to ensure built image is available for docker run
• Add GHCR login and push for enhanced caching across workflow runs
• Include commit-specific and latest tags for better cache utilization

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Fix Docker Hub push error by separating GHCR push from local testing

• Only push to GHCR to avoid Docker Hub authentication issues
• Pull GHCR image and tag locally for testing
• Maintains same local tag name for test step compatibility

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Add free disk space step to prevent storage errors

• Remove unused software installations (~10GB)
• Clean Docker cache before build operations
• Prevents "no space left on device" errors during image pull

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• Fix duplicate detection messages and improve server response validation

• Add INIT_COMPLETE_DETECTED state tracking to prevent repeated "initialization complete" messages
• Remove Step 1/7 detection logic that was causing duplicate output
• Add server response test after initialization complete detection
• Clean up progress status display and exit conditions

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: remove redundant typer dependency from leaderboard extras

• fix: only push Docker images from main branch

• fix: use COPY instead of git clone in Dockerfile

---

Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> (c7c04e5)

2.6.3 (2026-01-03)

Fix

• fix: Allow passing device to model (#3812)

• Allow passing device to model

• revert incorrect modification and fix typeerror

• add device to get_model and address comments

• Correct CDEWrapper (17ef363)

2.6.2 (2026-01-02)

Ci

• ci: Switch CI to use uv (#3702)

• use uv to all make commands

• read the docs a bit more...

• try out system flag

• fix: remove redundant pip install uv commands from Makefile

Removes duplicate uv installations that were conflicting with the properly configured uv from astral-sh/setup-uv GitHub Action. The GitHub Action already installs and configures uv correctly, so the Makefile pip installs were overwriting this configuration and causing "No system Python installation found" errors.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: remove --system flag from uv pip install commands

The astral-sh/setup-uv GitHub Action configures uv to manage its own Python installations, not to use system Python. The --system flag was causing "No system Python installation found" errors because uv expects to use its managed Python environment.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: migrate Makefile to use correct uv workflow

• Replace 'uv pip install' with 'uv sync' for dependency management
• Add proper --extra flags for all optional dependencies
• Use 'uv run' for all Python command executions
• Follow official uv GitHub Actions best practices

This aligns with uv's recommended project workflow and should resolve the CI environment issues we were experiencing.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: update all GitHub Actions workflows to remove UV_SYSTEM_PYTHON

• Remove UV_SYSTEM_PYTHON: 1 from all workflow files
• Fix documentation.yml to use 'uv sync --group docs' instead of 'uv pip install'
• Fix leaderboard_build.yml to use 'uv sync --extra leaderboard --group dev'
• Ensures consistent uv workflow across all CI jobs

Updated workflows:

• lint.yml
• documentation.yml
• model_loading.yml
• dataset_loading.yml
• leaderboard_build.yml

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• fix: lint workflow to use correct dependency group

• Change from 'make install' to 'uv sync --group lint' since pre-commit is in the lint group
• Add explicit pre-commit install step
• Use 'uv run' for lint commands (ruff, typos) to ensure proper environment
• Fixes "pre-commit: No such file or directory" error in lint workflow

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

• remove 3.14

• try out 3.14 again with python_full_version

• specify torch version for pylate dep

• try to skip colpali

• try split torch

• Add --no-sync flag and group/extra flags to uv run commands

Address review comments from PR #3702:

1. Add --no-sync to all uv run commands in Makefile for:

   • Faster execution (avoids re-syncing on each command)
   • pip compatibility (users can remove 'uv run' prefix)

2. Add appropriate group/extra flags to uv run commands:

   • test commands: --group test
   • docs commands: --group docs
   • typecheck: --group typing
   • leaderboard: --extra leaderboard

3. Update CI workflows to use --no-sync and appropriate groups:

   • lint.yml: Add --no-sync --group lint to all uv run commands
   • documentation.yml: Add uv run --no-sync --group docs to mkdocs gh-deploy

These changes improve performance while maintaining compatibility for contributors who prefer using pip directly.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

• try removing install block

• add back install block

• remove install block in doc CI without --no-sync

• add uv lock file

• replace install-for-test with just install

• install pre-commit with uv

• fix doc workflow

• address review comments

• remove no-sync from run-leaderboard make command

• remove --no-sync from selected make commands

• update typechecking

• fix type checking

• sync to install

• fix tests

• test pre-commit setup

• remove test file

• fix: separate install and install-for-tests with uv commands

• fix: add leaderboard extra to typecheck command for gradio imports

• fix: add faiss-cpu extra to test targets

• fix: update CI workflows for uv dependency management

• docs: update all documentation for uv migration

• Add uv installation options alongside pip in README.md
• Update installation.md with comprehensive migration guide for contributors
• Add uv context to CONTRIBUTING.md for development setup
• Update all usage docs to include uv alternatives for extras:
  • openai, leaderboard, image, xet, faiss-cpu dependencies
• Fix incorrect extra name: faiss -> faiss-cpu in retrieval_backend.md
• Ensure consistent dual-option approach (pip/uv) throughout documentation

This provides users and contributors with modern, fast uv tooling while maintaining backward compatibility with existing pip workflows.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

---

Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (75743f1)

Fix

• fix: handle git lfs content for cached zip file (#3827) (043ea38)