14× faster embeddings: how we rebuilt the ONNX path in Manticore

Manticore Search 27.1.5 ships a new ONNX Runtime backend for auto-embeddings that delivers ~14× faster throughput than the previous SentenceTransformers/Candle path on CPU. The old path was stuck at 5–11 docs/sec regardless of concurrency or batch size; the new one reaches 70–233 docs/sec. Key engineering decisions: sharing a single ORT session across concurrent callers (safe on Linux/macOS per ORT's C API docs), processing one document per inference call instead of batching (padding overhead made batching slower with variable-length inputs), and disabling intra-op thread spinning to free CPU for the rest of the system. For maximum bulk ingest throughput, the recommended pattern is large batches (32–128 docs) from a single client thread, since ORT already parallelises internally. GPU support and Windows perf parity are planned for future releases.