How Verbora uses Rust
Verbora is fast for an unglamorous reason: it does not allocate much. The algorithms are the standard ones — Levenshtein is Levenshtein, Jaro–Winkler is Jaro–Winkler. What differs is the data that flows through them, and how often it has to be copied.
This section covers the techniques that produce that, where each one surfaces in the API, and what it means for the code you write.
The techniques, and where to find them
| Technique | Where it shows up in the API | Page |
|---|---|---|
| Borrowing | Fourteen tokenizers yield &str slices of your input | Zero-copy |
Cow | Four of six normalizers; all 17 ja::converters; Stemmer::stem; three tokenizers | Zero-copy |
| Lazy iterators | tokens(), ngrams_iter(), iter_keys_with_prefix() | Iterator vs _into |
| Caller-owned buffers | tokenize_into(), pluralize_into(), stem_into() | Buffer reuse |
| Choosing the smallest working set | Levenshtein's bit-vector / row / matrix modes | Cache locality |
| Struct-of-arrays | The Levenshtein search matrix | Cache locality |
| Flat arenas | Trie's Vec<Node> addressed by u32 | Cache locality |
| Inline small collections | SmallVec children per trie node | Cache locality |
| Stack buffers for small inputs | Jaro–Winkler's match flags | Allocation |
| Cheaper hash keys | Dice hashes (u16, u16) instead of a String per bigram | Allocation |
| Exact fast paths | ASCII &[u8] vs Vec<u16> promotion in distance and phonetics | Zero-copy |
| Monomorphised predicates | CharClass as a zero-sized type, so each tokenizer's scan inlines | Cache locality |
Read these in order
1. Ergonomics vs throughput →When to reach for a performance-oriented API and — more often — when not to.2. Iterator vs reusable buffer →Two shapes people assume are alternatives. They solve different problems.3. Buffer reuse →What
clear() does and does not free, the append-vs-clear conventions, and how to size a buffer up front.4. Zero-copy and Cow →Borrowed tokens, Cow-returning normalizers, and the ASCII fast paths that keep UTF-16 exactness free.5. Allocation behaviour →A per-API reference: what allocates, how much, and how often.6. Batch vs streaming →Bounded memory and early output against preallocation and shared setup.7. Parallelism →The thirteen opt-in par_* APIs, and how to parallelise everything else yourself.8. Cache locality and data layout →Working sets, arenas, struct-of-arrays — where the big wins actually came from.How to read the numbers here
Timings are measured; allocation counts are not. Published timings come from the benchmark pages, and today they cover
verbora-distance. Criterion benchmarks for tokenizers, phonetics, n-grams, normalizers, inflectors and the trie exist in-tree (crates/*/benches/) but their tables are not published yet, so this section says "fewer allocations" rather than quoting a speed figure for those subsystems. Where a page describes allocation behaviour it describes what the code does, read from the source — allocation counting and peak-RSS instrumentation are not in the repository.