Parallelism

Fourteen crates ship an optional, feature-gated par_* batch API. For everything else, Verbora's types are built so that parallelising at your own call site is a two-line change.

The built-in APIs

Enable the parallel feature on the crate you need it from:

[dependencies]
verbora-tokenizers = { version = "0.2", features = ["parallel"] }

parallel is never on by default. It pulls in rayon as an optional dependency and adds one or more par_* functions, each a thin fan-out over the crate's existing sequential primitive:

Pending re-measurement. The per-word costs on this page are 0.1.0 measurements. They are retained because the argument they support — that a per-word task is far below rayon's dispatch overhead — is structural and holds across an order of magnitude, but the numbers themselves predate 0.2.0 and no figure here should be quoted as current.

Crate	API	Granularity
verbora-spellcheck	Spellcheck::par_corrections_batch	per word
verbora-wordnet	WordNet::par_lookup_batch	per word
verbora-tagger	BrillTagger::par_tag_batch	per document
verbora-distance	par_levenshtein_batch and siblings, one per metric	per pair
verbora-tokenizers	par_tokenize_batch	per document
verbora-normalizers	par_remove_diacritics_batch	per document
verbora-sentiment	SentimentAnalyzer::par_get_sentiment_batch	per document
verbora-analyzers	par_analyze_batch	per sentence
verbora-transliterators	par_transliterate_ja_batch	per document
verbora-stemmers	TokenizeAndStem::par_tokenize_and_stem_batch	per document, not per word — per-word cost is as low as ~26 ns, far below task-dispatch overhead
verbora-phonetics	par_encode_batch / par_encode_double_batch	chunked (par_chunks) — same overhead problem at ~42–183 ns/word
verbora-classifiers	par_classify_batch on Classifier<E>	per document — MaxEntClassifier is excluded; it is already Send + Sync, but no par_* API ships here without sequential-vs-parallel benchmark evidence, and there is none yet for this model
verbora-tfidf	TfIdf::par_add_documents	per document, split phase — see below
verbora-language	par_detect_batch	per text; generic over any LanguageDetector + Sync

Two guarantees hold for all of them: output is identical to the sequential call (each is a fan-out over the primitive you would have called in a loop, not a second implementation), and nothing runs in parallel unless you ask — the feature is opt-in and the functions are called by name.

verbora-tfidf is the one split case.add_document takes &mut self and mutates shared corpus state — the term table, the document-frequency table and the document list — so par_add_documents runs the stateless phase in parallel (Analyzer::terms: tokenizing, case folding and stop-word filtering, all pure functions of the text and the analyzer) and replays the resulting term lists through the sequential counting primitive, in the original order. The result is identical to the sequential loop — same positions, same counts, same term-id assignment order, same serialized bytes — and the sequential phase is a real, un-parallelised fraction of the total, which bounds what fan-out can buy. It also costs one String per term, which add_document does not: its terms are borrowed from the text.

Where there is deliberately no par_* API

• verbora-trie — a contains is one hash of the folded bytes and a short probe, at or below task-dispatch overhead; construction is inherently sequential against one shared arena (insert takes &mut self).
• verbora-inflectors — ~360 ns/word, the same overhead problem.
• verbora-util — its graph algorithms operate on one shared graph per call, not independent items; there is no batch shape to parallelize.

verbora-ngrams has not been quantified for Rayon — read its absence as "not yet evaluated" rather than "evaluated and rejected".

For anything else, parallelising at your call site is easy because Verbora's types are already built for it: tokenizers, phonetic encoders, inflectors and distance functions are stateless values (most are zero-sized types), the distance and normalizer entry points are free functions, Trie is Send + Sync, and nothing has interior mutability on a hot path.

The exception you must know about

One list in the workspace is process-global and mutable.verbora_core's global stop-word list (add_global_stopword, remove_global_stopword, reset_global_stopwords, is_global_stopword) is stored behind an RwLock plus an AtomicBool, so it is not a memory-safety hazard — no data race, no undefined behaviour, whatever you do concurrently. The hazard is correctness: a thread calling add_global_stopword while others are reading gives those readers a nondeterministic mix of the old and new value, with no error to tell you. Its one consumer inside the workspace is verbora-stemmers' English and Lancaster stop-word helpers.

Nothing else reads it. phoneticize_tokens takes an explicit &StopWords and has no global-reading variant; a TfIdf owns its own Analyzer — its tokenizer, its case folding and its stop-word list — so two corpora in one program cannot silently change each other's answers. Prefer the owned or argument form everywhere; the global list exists for programs that genuinely want one process-wide setting.

Doing it yourself

For anything without a built-in API, add rayon to your Cargo.toml:

[dependencies]
rayon = "1"

Independent documents

use rayon::prelude::*;
use verbora_tokenizers::{BorrowingTokenizer, WordTokenizer};

let counts: Vec<usize> = corpus
    .par_iter()
    .map(|doc| WordTokenizer.tokens(doc).count())   // zero-sized: free
    .collect();

With a per-thread buffer

map_init gives each worker its own scratch, which is how you combine buffer reuse with parallelism — a &mut Vec cannot be shared:

use rayon::prelude::*;
use verbora_tokenizers::{BorrowingTokenizer, WordTokenizer};

let counts: Vec<usize> = corpus
    .par_iter()
    .map_init(Vec::new, |buf, doc| {
        buf.clear();
        WordTokenizer.tokenize_borrowed_into(doc, buf);
        buf.len()
    })
    .collect();

A shared read-only index

use verbora_trie::Trie;
use rayon::prelude::*;
use std::sync::Arc;

let trie = Arc::new(build_trie());          // built once, sequentially

let hits: Vec<bool> = queries
    .par_iter()
    .map(|q| trie.contains(q))              // &self — no locking needed
    .collect();

Trie construction cannot be parallelised — insert takes &mut self. Build on one thread, then share. If the index never changes afterwards, share the FrozenTrie that freeze() returns instead: it is Send + Sync too, and its keys_slice hands each worker a borrowed &[String] rather than a fresh Vec.

Chunking to control granularity

Per-item tasks that take a few hundred nanoseconds are dominated by scheduling overhead. Give each task more work:

use rayon::prelude::*;

let total: usize = corpus
    .par_chunks(1024)                       // one task per 1024 documents
    .map(|chunk| chunk.iter().map(|d| tokenizer.tokens(d).count()).sum::<usize>())
    .sum();

When parallelism actually helps

Parallel does not automatically mean faster.

The two crossover tables this section carried are withdrawn.verbora-spellcheck's correction contract and verbora-tfidf's ingestion path both changed underneath the figures that described them, so neither set is current, and both crates now say so in their own documentation rather than carrying a number forward. Which batch size each one crosses over at must come from a re-run, not from the direction of the change. spellcheck_par_corrections_batch_d2 in benches/spellcheck.rs and parallel_batch in benches/tfidf.rs compare the sequential loop against each parallel method across several batch sizes, and are what the next campaign should answer this with.

What can be said without a measurement is the shape of each trade, which is a property of the code rather than of a run:

• Spellcheck is the favourable shape. Each correction is independent work of its own, with no shared mutable state, so par_corrections_batch is a plain fan-out and the only question is whether the batch is big enough to amortize fork-join.
• TF-IDF is Amdahl-limited by construction. par_add_documents parallelises the analyzer and replays the counting sequentially, in order, so the sequential fraction is real and bounded below by the corpus update itself. Fork-join has a fixed cost on top, so a handful of short documents will be slower this way; the crate's own documentation states the crossover is unmeasured for the current ingestion path.

Four checks before you parallelise anything yourself:

Compare the per-item cost to the scheduling cost. A rayon task costs on the order of a microsecond to schedule. Verbora's fastest stemmer processes a word in ~26 ns — three orders of magnitude below that floor.

Check whether you are memory-bound. Tokenization is a linear scan that allocates little. Sixteen cores scanning sixteen documents can saturate memory bandwidth long before they saturate the ALUs. Distance calculations on longer inputs, which do real arithmetic per cell, scale better.

Check whether the work is already small. hamming/4 is 6.6 ns † — pending re-measurement, but orders of magnitude below a task's scheduling cost either way. No amount of threading makes an operation that small faster; you would be measuring the scheduler.

Check what else is running. In a web server every request already occupies a thread. Adding intra-request parallelism there usually reduces total throughput by oversubscribing the CPU, even when it improves one request's latency.

If the operation has a built-in par_* API, enable the parallel feature and call it — each one's doc comment states what it costs and, where a crossover has been measured, where it sits. Otherwise, leave it sequential unless total CPU time in the stage is measured in seconds, the items are independent, and you are not already running one request per thread. Then par_chunks with a chunk size that makes each task ≥ ~100 µs, and measure.

What to measure

1. Wall-clock, not CPU time. Parallel code that halves latency while quadrupling CPU time is a bad trade in a shared environment.
2. Scaling curve, not a single point. Run at 1, 2, 4, 8, 16 threads. A flat curve past 4 means you are bound by something other than the CPU.
3. The sequential version, optimised first. A tokenize_into loop that removed ten million allocations may make the parallel version unnecessary. It is cheaper, and it composes.

Related

• Batch vs streaming — you need a batch before you can split it.
• Buffer reuse — map_init is how the two combine.
• Massive parallel corpora — a complete worked example.
• Benchmarks — how to reproduce every number on this page.