We rewrite popular pure-Python libraries in Rust and test every output against the original, down to the last bit of every float. Change one import: your scores, languages and rankings stay exactly the same.
rouge-score → rouge-score-rs
ROUGE for summarization and LLM evaluation
30–80×
faster on one core, up to 353× with score_batch
The implementation behind Hugging Face Evaluate, with the same tokenizer, Porter stemmer and rougeLsum. Unicode tokenizers for other languages.
A faster library that gives slightly different numbers changes every result built on it. So each port is compared with the original without any tolerance, and a single differing float fails the tests.
Gradient reach in recurrent networks: code and training runs
How dropout, zoneout and gate initialisation change how far gradients reach back in time in LSTM and GRU networks, and whether that reach, measured early in training, predicts which runs learn long-range dependencies. Trained with rust-lstm.
Comparing IRT models and simple baselines for budget-constrained LLM evaluation
Item response models against simple estimators on public model × item correctness matrices: how well each estimates item difficulty, a new model's capability and its full-benchmark accuracy when only 0.5–80% of the items are run.
QoS-aware orchestration of local and cloud large language models in distributed information systems
Choosing per request between a small on-device model, a larger LAN server and a cloud API: latency model, routing policies, a netem testbed and the measurement data.
SyntaxSpirits is maintained by Oleksandr Kholodniak, a software engineer and PhD researcher at the National Aerospace University “Kharkiv Aviation Institute” in Ukraine. The libraries come out of production Ruby work and research on evaluating language models.
Is there a slow Python library you would like ported, or a result that differs? Open an issue on the library's GitHub page, or write to hello@syntaxspirits.dev.