Instructions to use TomoroAI/tomoro-colqwen3-embed-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TomoroAI/tomoro-colqwen3-embed-8b with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("TomoroAI/tomoro-colqwen3-embed-8b", trust_remote_code=True, device_map="auto") - sentence-transformers
How to use TomoroAI/tomoro-colqwen3-embed-8b with sentence-transformers:
from sentence_transformers import MultiVectorEncoder model = MultiVectorEncoder("TomoroAI/tomoro-colqwen3-embed-8b", trust_remote_code=True) queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - ColPali
How to use TomoroAI/tomoro-colqwen3-embed-8b with ColPali:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Fix transformers 5.x loading and add Sentence Transformers support
Hello!
The MultiVectorEncoder class ships in the next Sentence Transformers release, planned for around the 18th, so for now the install below pulls from source. I would love to feature this model in that release's blog post and documentation, especially once it loads without the revision pin (that is, once this PR is merged).
Heads up, this PR was AI-generated and human-reviewed. Here's a summary of the changes as reported by my agent:
Pull Request overview
- Fix five small
transformers5.x incompatibilities that stop the model from loading. - Add a Sentence Transformers loading path (multi-vector, ColBERT-style late interaction) via
MultiVectorEncoder.
Details
This model shares the five transformers 5.x compatibility fixes from the tomoro-colqwen3-embed-4b PR (the modeling files are byte-identical between the two repositories, so the patch is identical): a sub_configs dataclass-field clash, sub-config init order, and three tie_weights / mm_token_type_ids signature mismatches in configuration_colqwen3.py and modeling_colqwen3.py. Without them, AutoModel.from_pretrained(..., trust_remote_code=True) fails on transformers 5.x. The weights are untouched and existing AutoModel plus AutoProcessor usage is unchanged apart from now working on current transformers.
The model also loads directly into Sentence Transformers as a Transformer(retrieval) -> MultiVectorMask pipeline (its own forward already handles the projection, normalization and masking), so it needs only the standard ST config files and no custom module. I verified it against AutoModel plus AutoProcessor (in float16, since it does not fit in 24 GB at float32): MaxSim scores agree to 3.3e-3, query embeddings to 1.9e-3, document embeddings are bit-identical, and each query retrieves its own page.
pip install "sentence-transformers[image] @ git+https://github.com/huggingface/sentence-transformers.git"
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("TomoroAI/tomoro-colqwen3-embed-8b", revision="refs/pr/1", trust_remote_code=True)
queries = [
"What is the variable represented on the y-axis of the graph?",
"Total outlay is maximum in which year?",
]
documents = [
f"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc{i}.jpg"
for i in range(1, 5)
]
query_embeddings = model.encode_query(queries, convert_to_tensor=True)
document_embeddings = model.encode_document(documents, convert_to_tensor=True)
print(tuple(query_embeddings[0].shape), tuple(document_embeddings[0].shape))
# (23, 320) (1251, 320)
print(model.similarity(query_embeddings, document_embeddings))
# tensor([[12.4336, 8.5479, 6.7305, 5.1968],
# [ 4.5449, 11.1719, 4.8320, 4.5195]])
- Tom Aarsen
Hi Tom, it looks good to me, thanks for fixing this.
@hxssgaa I'm glad you like it! I've also opened a PR against the 4B variant: https://huggingface.co/TomoroAI/tomoro-colqwen3-embed-4b/discussions/4
I'll also update my Sentence Transformers MultiVectorEncoder blogpost draft to report that this model can load directly from the official model repository 🤗 It should release in ~4 days!
- Tom Aarsen