silv4souz4/embeddinggemma-2-coreai-w8a16

Model Card for EmbeddingGemma 2 — Apple Core AI, int8 weights / fp16 activations (W8A16)

google/embeddinggemma-2, the multimodal embedding model from Google DeepMind (text, images, video and audio in one 768-dimensional space), converted to Apple Core AI (.aimodel) with 8-bit integer weights in blocks of 32 and fp16 activations, for on-device embeddings on Apple silicon. Tested on macOS 27.2 (Apple M5, 24 GB) with the Core AI Python runtime; not tested on iOS.

Read the limits first. Text up to 2,048 tokens (the original supports 8,192). Images at the default 280 soft tokens only (the original allows 70–1,120); video at 140 tokens per frame; audio up to 30 s. Inputs that mix several modalities in one sequence are supported by the host design but were not validated; only pure text, image, video and audio were. The original card warns against float16; these graphs run fp16 activations on purpose (norms scaled by the row maximum) and were checked on a small set only — see Evaluation.

Model Details

Model Description

• Original model developed by: Google DeepMind (google/embeddinggemma-2).
• Conversion by: silv4souz4 (independent; not affiliated with or endorsed by Google or Apple).
• Model type: multimodal embedding model; 24-layer bidirectional text encoder (130 M transformer + 140 M token embedder), 16-layer vision encoder (170 M), 12-layer audio encoder (300 M).
• Relation to base: quantized (int8 weights); same math, re-authored for Core AI with static shapes; no retraining or fine-tuning.
• Language: multilingual (100+ languages), inherited from the original.
• License: Apache License 2.0, inherited from google/embeddinggemma-2.

Assets (0.89 GB)

File	Content
embeddinggemma2_text_w8a16.aimodel (214 MB)	text encoder, three functions: embeddinggemma2_texto_l{128,512,2048}_w8a16_g32 (x = inputs_embeds [1,L,512], valid [1,L] → emb [1,768], mean over the valid tokens, L2-normalised)
embeddinggemma2_vision_w8a16.aimodel (163 MB)	vision encoder + projection, two functions: embeddinggemma2_visao_p2520_w8a16_g32 (image: 2,520 patches → 280 soft tokens) and ..._p1260_... (video frame: 1,260 patches → 140 tokens)
embeddinggemma2_audio_w8a16.aimodel (314 MB)	audio encoder + projection, two functions: embeddinggemma2_audio_t{1000,3000}_w8a16_g32 (feat [1,T,128] log-mel, fmask [1,T] → soft [1,T/4,512])
embeddinggemma2_embed_tokens_int8.npz (135 MB)	token embedding table 262,144×512, int8 per row + fp16 scale (the host does the gather and the √512 scaling)
embeddinggemma2_vision_pos_table.npz (31 MB)	2-D patch position table 2×10,240×768 in fp16 (exact)
embeddinggemma2_audio_mel.npz	HTK mel filter bank [257,128] and Hann window [320] of the feature extractor
host/embeddinggemma2_coreai_host.py	self-contained reference host (numpy, Pillow, tokenizers + Core AI)
tokenizer.json, tokenizer_config.json, chat_template.jinja, preprocessor_config.json, processor_config.json	unchanged from google/embeddinggemma-2

(The function names inside the assets keep the Portuguese part names texto/visao used during conversion.) All functions of an asset share the int8 weights.

What the host does and the graphs do not: tokenization and task prefixes (task: search result | query: , title: none | text: , ...); token-embedding gather; aspect-ratio-preserving resize (bicubic) to at most 2,520 (image) or 1,260 (video frame) 16×16 patches, positions (x, y; −1 = padding), the 2-D position embedding, the axial RoPE tables and the 3×3 pooling matrix; log-mel features (128 mel, 20 ms window, 10 ms hop, ≤ 30 s); choice of the smallest static lane; padding; replacing the image/video/audio placeholder tokens by the soft tokens; Matryoshka truncation (128/256/512/768) followed by re-normalisation. Sequences: <bos> <|image> <|image|>×n <image|> <eos> (image), <bos> + per frame <|image> <|video|>×n <image|> + <eos> (video), <bos> <|audio> <|audio|>×n <audio|> <eos> (audio). The final embedding is the mean over all tokens of the sequence, computed inside the text graph.

Model Sources

• Original model: https://huggingface.co/google/embeddinggemma-2

Uses

Direct Use

Local semantic search, retrieval, classification, clustering and similarity over text, images, video frames and audio on Apple silicon through the Core AI runtime.

Out-of-Scope Use

• Text longer than 2,048 tokens; image budgets other than 280 tokens; audio longer than 30 s.
• Mixed-modality single inputs, unless validated by the user.
• The uses prohibited by the Gemma Prohibited Use Policy referenced by the original.
• High-stakes decisions without human review: the limitations of the original apply.

Bias, Risks, and Limitations

• Inherits the limitations, biases and risks of google/embeddinggemma-2 (see its model card); quality is not equal across languages.
• Quality was compared with the official transformers implementation on a small set (11 texts, 7 images, 2 videos, 5 audio clips), not on retrieval benchmarks such as MTEB.
• The original recommends bfloat16/float32 because its activation range exceeds float16. Here the layer outputs of the test set stayed below 1,200 in magnitude (maximum 1,126.5 in the vision encoder, far from the fp16 limit of 65,504) and no NaN occurred, but internal products were not instrumented; unusual inputs could behave differently.
• Latency was not measured on a quiet machine, so no speed figure is claimed.

How to Get Started with the Model

hf download silv4souz4/embeddinggemma-2-coreai-w8a16 --local-dir eg2

pip install numpy tokenizers pillow   # plus the Apple Core AI Python runtime (coreai-core)
python eg2/host/embeddinggemma2_coreai_host.py eg2 text "What causes the northern lights?" query
python eg2/host/embeddinggemma2_coreai_host.py eg2 text "The northern lights are caused by charged particles from the sun." document 256
python eg2/host/embeddinggemma2_coreai_host.py eg2 image photo.png
python eg2/host/embeddinggemma2_coreai_host.py eg2 audio speech.wav        # 16 kHz mono 16-bit WAV

Use the same prefix family on both sides of a comparison (query/document for retrieval; the same one for symmetric tasks) and the same dimension; the host re-normalises truncated vectors.

Ahead-of-time compilation (Apple M5, optional, faster cold load):

mkdir -p eg2/aot_h17g && for f in eg2/embeddinggemma2_*_w8a16.aimodel; do coreai-build compile "$f" --platform macOS --architecture h17g --output eg2/aot_h17g; done

(iPhone 15: --platform iOS --architecture h15p, untested.) The host uses aot_h17g/*.h17g.aimodelc automatically when present.

Evaluation

Reference: the official transformers 5.20 implementation (EmbeddingGemma2Model) in fp32, one item at a time. The floor is the same model run in bfloat16 (the checkpoint's own precision) against fp32. Cosine of the final 768-d embedding, mean over the cases (W8A16 here = Core AI runtime, default; the compiled h17g assets and the preferred-ANE mode gave identical numbers):

Modality	Cases	cos(W8A16, fp32)	floor: cos(bf16, fp32)	Largest pairwise-similarity difference (W8A16 / floor)
Text	11 (English, Portuguese, Japanese, Arabic, code, a 1,993-token document)	0.999915 (min 0.999747)	0.999964	0.0015 / 0.0015
Image	7 (1024², 832², landscape, portrait, 200×150)	0.999874	0.999895	0.0019 / 0.0019
Video	2 (6 frames 640×800; 3 frames 500×300)	0.999806 / 0.999902	0.999778 / 0.999915	—
Audio	5 (speech EN/PT, synthetic, 25 s, 30 s)	0.999777	0.999808	0.0035 / 0.0024

Soft tokens of the vision and audio encoders against fp32: 28.8–32.7 dB (image), 29.3–29.5 dB (video), 25.9–37.0 dB (audio). The re-authored fp32 graphs match the official model at 100–121 dB per layer. The reference host was also checked end to end with its own preprocessing: the tokenizer gives ids identical to the official processor on all 11 texts; its Pillow bicubic resize and patchify give identical positions and patches within 0.004 (one 8-bit level; 65–71 dB) of the official torchvision processor, and the resulting embeddings match fp32 at cos 0.9997–0.9999 (images), 0.9997 (the 6-frame video) and 0.9998–0.9999 (audio, text), the same level as above. Memory: with the three compiled assets loaded, 0.84 GB footprint (1.32 GB after calling every function), measured with footprint -p.

Summary

On this test set the W8A16 conversion is within the floor set by the checkpoint's own bf16 precision for text, image and video, and about 1.5× the floor in pairwise similarity for audio, at 0.86 GB (versus 1.49 GB in bf16).

Environmental Impact

No training. Conversion and evaluation ran locally on one Apple M5 laptop; negligible.

Technical Specifications

Compute Infrastructure

• Hardware: Apple M5 (24 GB unified memory).
• Software: macOS 27.2, Python 3.12, PyTorch 2.13, coreai-torch 0.4.3, coreai-opt 0.3.0, coreai-core 1.0.0b3, coreai-build 3605.5.4; reference with transformers 5.20.0 and PyTorch 2.14.1.

Citation

If you use this model, please cite and credit the original release, EmbeddingGemma 2 by Google DeepMind (https://huggingface.co/google/embeddinggemma-2); the original card does not publish a BibTeX entry.

License and Attribution

• License: Apache License 2.0 — full text in LICENSE; the license of google/embeddinggemma-2.
• Original model: © Google DeepMind (google/embeddinggemma-2).
• Modifications (Apache 2.0 §4(b)): re-authoring of the text, vision and audio encoders for Core AI (static shapes, host-side position/pooling/RoPE tables, banded local attention in the audio encoder), int8 weight quantization, split into three assets; tokenizer, chat template and processor configs unchanged; no retraining. See NOTICE.
• "Apple" and "Core AI" are trademarks of Apple Inc.; "Gemma" is a trademark of Google; their use here is descriptive only.

Model Card Authors

silv4souz4

Model Card Contact

Please open a discussion in the Community tab of this repository.