Cleanup on60 runs
Core ML Parakeet + local cleanup
- Micro WER
- 3.46%
- Deletion rate
- 0.41%
- Median processing
- 1,124 ms
- p95 processing
- 2,807 ms
- Maximum processing
- 2,958 ms
State of Private Dictation · 2026
Measure what dictation leaves out.
Accuracy, deletions, latency, and observable network behaviour across on-device speech tools. We are publishing the method, the limits, and the sanitized data beside the numbers.
01 · Pilot results
The first numbers, in full view.
With local cleanup enabled, the pilot recorded a 1,124 ms median processing time and 3.46% word error rate. The same 20 clean synthetic fixtures were decoded three times per lane on one M1 Max MacBook Pro using Yaps 2.3.2129. Lower error and latency values are better within this exact setup. These are Yaps configurations, not competing products.
- Fixtures
- 20
- Runs per lane
- 60
- Words, with repeats
- 1,473
- Speech origin
- Synthetic
- Test device
- 1
- Error aggregation
- Micro
Cleanup off60 runs
Core ML Parakeet, cleanup off
- Micro WER
- 3.67%
- Deletion rate
- 0.20%
- Median processing
- 250 ms
- p95 processing
- 722 ms
- Maximum processing
- 1,863 ms
Cleanup off60 runs
Whisper Base, cleanup off
- Micro WER
- 6.52%
- Deletion rate
- 0.41%
- Median processing
- 844 ms
- p95 processing
- 1,669 ms
- Maximum processing
- 2,189 ms
Processing time includes recognition, pipeline finalization and local cleanup when enabled. It excludes initial runtime construction, fixture loading, live recording, shortcut dispatch and cursor insertion. All lanes score final evaluator output; cleanup-off lanes can still apply vocabulary and formatting changes.
Latency uses 60 equally weighted decode samples per lane and R-7 linear quantiles. The 20 fixtures contain 491 reference-word positions per pass, counted three times in the 1,473-word denominator. Repeats are not independent utterances. Quality-threshold failures remain included: 3 with Core ML cleanup on, 3 with Core ML cleanup off and 6 with Whisper.
Interpretation corrected 6 September 2026. Original numbers are unchanged. Read the evidence audit, external references, remaining limitations and reproduction instructions.
A matched cleanup-off observation.
In these runs, Whisper Base with cleanup off had 2.85 percentage points higher WER, 0.20 points higher deletion rate, a 595 ms higher median, and a 947 ms higher p95 than Core ML Parakeet with cleanup off.
A general product or privacy claim.
The pilot excludes human speech, accents, multilingual speech, live microphone capture, controlled thermal order, network observation, additional devices, and third-party products.
02 · Product layer
A model is not a dictation experience.
Why Yaps belongs in the report
Downloading model weights is the easy part. A useful dictation system must capture audio reliably, choose the right runtime, prepare speech for that model, detect incomplete output, preserve the user’s intent, and place the result wherever they are working.
Yaps turns capable local speech models into a single private tool people can use across their everyday apps, without assembling model servers, scripts, audio pipelines, or export steps themselves.
Capture
A shortcut or mobile keyboard starts a private recording without moving the user into a separate transcription tool.
Prepare
Yaps applies speech detection, silence handling, and device-shaped audio preparation before recognition.
Route
The app selects an installed local engine for the device, language, quality preference, and available hardware.
Recognize
Parakeet, Cohere, Whisper, or a specialist route converts the prepared audio into candidate text.
Guard
Completeness checks, rescue paths, vocabulary handling, and local cleanup protect the delivered result.
Deliver
The finished text is committed at the cursor or into the mobile keyboard workflow, ready to use immediately.
This product context explains why the report distinguishes cleanup settings and the measured output stage. It is not evidence that Yaps is best, and it does not turn product positioning into a benchmark result.
03 · Wider evidence
The research programme is already larger than one pilot.
These companion studies used different devices, corpora, metrics, and test protocols. Their numbers explain current product decisions, but they must not be combined into a single leaderboard.
Apple Silicon · multilingualRelated study
Cohere changes the routing conversation.
A broader M1 Max bake-off used 144 synthetic clips across 24 languages and 65 human FLEURS clips across 13 languages. On the matched 20-clip human subset shared by Cohere and every model, Cohere recorded 4.2% audited WER versus 12.0% for MLX Parakeet.
- Cohere WER
- 4.2%
- MLX Parakeet
- 12.0%
- Matched human clips
- 20
Windows 11 · CohereRelated study
Higher multilingual reliability, with a latency cost.
On a Ryzen 7 5700U Windows laptop, Cohere and ONNX Parakeet ran the same 70 human FLEURS clips across 14 languages. Cohere returned 70 non-empty transcripts; Parakeet returned 61. Cohere won 12 of 14 automated language comparisons, but averaged 3.654 seconds per clip versus 1.394 seconds for Parakeet.
- Cohere non-empty
- 70/70
- Parakeet non-empty
- 61/70
- Cohere mean
- 3.654 s
Android · Arabic previewRelated study
Device speed can pass while corpus coverage still fails.
An Arabic FastConformer pack ran on a Pixel 10 Pro. Excluding one invalid reference pair, 39 FLEURS clips produced 9.8% WER and 4.1% CER, with a 624 ms median and 984 ms p95. A broader clean and noisy MASC slice on Mac CPU later scored 22.5% WER, so the route remains Preview rather than becoming a general Arabic claim.
- Valid clips
- 39
- Median decode
- 624 ms
- p95 decode
- 984 ms
04 · Platform coverage
Show the gaps instead of hiding them.
macOS · Apple Silicon
Published pilotCurrent evidenceThe current 20-fixture pilot, plus a separate 144-synthetic and 65-human multilingual model bake-off.
Next evidence gateAdd spontaneous speech, more Mac generations, controlled thermal order, and live microphone capture.
Windows · x64
Current evidenceA separate 70-clip, 14-language human FLEURS validation compared Cohere and ONNX Parakeet on Windows 11.
Next evidence gatePublish sanitized rows under the frozen protocol and add lower-memory and additional GPU classes.
Android · ARM64
Current evidenceA Pixel 10 Pro evaluation measured an Arabic FastConformer pack and exercised the real keyboard path.
Next evidence gateAdd spontaneous dialect speech, more phone tiers, English route parity, battery, and sustained-use measurements.
Linux · x64
Research gapCurrent evidenceYaps is available on Linux, but this report does not contain a report-grade native Linux accuracy or latency run.
Next evidence gateRun native Ubuntu fixture replay, installed-package dictation, CPU and Vulkan paths, X11, and Wayland.
iOS · ARM64
Research gapCurrent evidenceYaps is available on iOS, but this report does not yet include an iPhone device benchmark.
Next evidence gateMeasure the real keyboard extension, model loading, memory pressure, energy use, and end-to-end commit latency.
05 · Research question
A fluent transcript can still be wrong.
Accuracy
How many reference words are substituted, inserted, or deleted after deterministic normalization?
Silent deletion
Does a transcript lose whole words or phrases while still sounding fluent enough to escape a quick review?
Responsiveness
How long does processing take across a distribution, not just in the single fastest run?
Network behaviour
What connections and payloads are observed during a defined workflow, with the limits of packet observation stated plainly?
06 · Publication design
Evidence you can follow back to its source.
The pilot distinguishes semantic cleanup settings, scores final evaluator output, reports substitutions, insertions, and deletions independently, and keeps private speech content outside the public release. Comparable systems receive the same audio bytes where their interfaces permit it; incompatible capture paths belong in clearly separated strata.
01Frozen public or consented corpus
02Declared system, model, and device
03Repeated runs with failures retained
04Word-level S / I / D alignment
05Content-free public rows and aggregates
07 · Read the record
Inspect the method and every public row.
ProtocolMethodologyResearch phases, corpus rules, primary metrics, publication gates, and known limitations.EvidencePilot resultsThree Yaps-only pilot lanes on one synthetic corpus and one device. No competitor ranking.ReproduceDownload dataSanitized rows, aggregate metrics, manifests, checksums, and the public release index.PortableLocalized Markdown reportA complete text edition containing the pilot, companion evidence, platform gaps, methodology summary, and source files.