Static embeddings for Ukrainian retrieval: 32.8 MB, about 0.1 ms per query on a CPU

Encode Ukrainian text

from model2vec import StaticModel

model = StaticModel.from_pretrained("SyntaxSpirits/ukr-static-m3-64k")
embeddings = model.encode(["Київ — столиця України.", "Як оформити закордонний паспорт?"])

With sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("SyntaxSpirits/ukr-static-m3-64k").float()
embeddings = model.encode(["Київ — столиця України."])

Keep the .float(): the weights are stored in FP16, and without it sentence-transformers returns NaN for inputs that contain no known tokens, such as an empty string.

Embeddings are L2-normalised, so a dot product is the cosine similarity. Inputs that leave no known tokens, such as an empty string or "!!!", get a zero vector. Both loaders cut texts at 512 tokens, and Model2Vec by default also cuts at 3,584 characters before tokenizing. To embed whole texts with Model2Vec, pass max_length=None to model.encode; otherwise split long documents into chunks.

Retrieval quality

nDCG@10 for question → passage retrieval.

• Belebele ukr_Cyrl: 900 questions, 488 passages. WebFAQ ukr test: 6,851 questions, 68,501 answers. UA-SQuAD val: 5,726 answerable questions, 1,160 paragraphs.
• Differences from the strongest other static model, with 95% bootstrap intervals: Belebele +0.124 [+0.100, +0.147] (POTION), WebFAQ +0.082 [+0.074, +0.091] (static-similarity-mrl), UA-SQuAD +0.071 [+0.060, +0.081] (POTION).
• "+ BM25" is reciprocal-rank fusion (k = 60, top 100 from each system, no tuning). BM25 is Okapi with default parameters on lowercased word tokens.
• Belebele and WebFAQ were used during development, to choose the vocabulary and to decide to train. UA-SQuAD was run with a protocol fixed beforehand.
• An earlier build had a tokenizer bug and was retrained before release. All three sets were evaluated with both builds, and no score differs by more than 0.0015.

The full evaluation notes, including overlap checks against the training text, are on the model card.

When BM25 is the better choice

On Belebele the model alone scores 0.776 against BM25's 0.693, and fusing the two does not improve on it (0.770). On WebFAQ and UA-SQuAD it is the other way round: BM25 alone scores higher (0.490 and 0.761, against 0.434 and 0.684), and fusing the two adds only +0.014 and +0.005 nDCG@10 over BM25 alone. UA-SQuAD is a Ukrainian translation of the SQuAD 2.0 dev set, so its questions share many words with their passages, which favours BM25.

For keyword-style questions, use the model alongside BM25, not instead of it. A minimal hybrid search with rank-bm25-rs and the same reciprocal-rank fusion as in the table, k = 60 over the top 100 of each ranking (pip install model2vec rank-bm25-rs):

import re

import numpy as np
from model2vec import StaticModel
from rank_bm25_rs import BM25Okapi

docs = [
    "Київ — столиця і найбільше місто України.",
    "Закордонний паспорт можна оформити в ЦНАП або в Державній міграційній службі.",
    "Дніпро — найдовша річка, що протікає Україною.",
]
query = "Де зробити закордонний паспорт?"


def words(text):
    return re.findall(r"\w+", text.lower())


def rrf(*scores, k=60, top=100):
    fused = np.zeros(len(docs))
    for s in scores:
        for rank, i in enumerate(np.argsort(-s, kind="stable")[:top], start=1):
            fused[i] += 1 / (k + rank)
    return np.argsort(-fused, kind="stable")


model = StaticModel.from_pretrained("SyntaxSpirits/ukr-static-m3-64k")
dense = model.encode(docs) @ model.encode([query])[0]
sparse = BM25Okapi([words(d) for d in docs]).get_scores(words(query))

docs[rrf(dense, sparse)[0]]
# 'Закордонний паспорт можна оформити в ЦНАП або в Державній міграційній службі.'

Size, speed and memory

At the same FP16 precision, POTION's weights would be 256 MB, 7.8 times this model's. Encoding the 900 Belebele questions one at a time after warm-up, on an Apple M1 Pro CPU:

About 150 MiB of the peak memory is Python and the datasets library.

How it was built

1. Vocabulary. The 32,000 most frequent lowercased Ukrainian words in 60 M characters of Ukrainian Wikipedia and FineWeb-2, plus the 32,000 BGE-M3 subword tokens most frequent in the same text, so that unseen word forms still get vectors. The tokenizer prefers whole words over subword splits.
2. Distillation. Each token was embedded by BGE-M3, reduced to 256 dimensions with PCA and weighted by corpus frequency.
3. Tokenlearn. The model was then trained with Tokenlearn to reproduce BGE-M3's mean token embedding on 119,430 chunks from the same sources. Chunks that share an 8-word sequence with Belebele or WebFAQ were removed.

Limitations

• Input is lowercased, word order is ignored, and long texts are truncated.
• It is far behind its teacher: 0.776 against BGE-M3's 0.937 on Belebele.
• The vocabulary was built for Ukrainian. Other languages, including Russian and English, are covered only by subwords seen in Ukrainian text and were not evaluated.
• Only retrieval was evaluated; similarity, clustering and classification were not.

How to cite

Oleksandr Kholodniak (ORCID 0009-0000-1012-7361).

@misc{kholodniak2026ukrstatic,
  author       = {Kholodniak, Oleksandr},
  title        = {ukr-static-m3-64k: a static embedding model for Ukrainian retrieval},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/SyntaxSpirits/ukr-static-m3-64k}}
}