Yappy LLM backends

An LLM does exactly one job in dictation: tidying the raw Whisper transcript before Yappy types it. Which one runs that step is configured in Settings → Engine, or as llm.provider in ~/.yappy/config.json.

The options

Provider	What it is	Typical latency	Key needed?
`hosted`	The Yappy plan's hosted inference	under 1s	No — vended by the plan
`ollama`	Fully local open-source models (default `qwen2.5:3b`)	1–3s	No
`custom`	Any OpenAI-compatible endpoint — LM Studio, vLLM, llama.cpp	varies	No
`claudeCode`	Your Claude subscription, via the `claude` CLI	4–6s	No
`grokCode`	Your xAI subscription, via the `grok` CLI	~2s	No
`anthropic`	Claude via the API	1–2s	Yes
`openai`	OpenAI via the API	1–2s	Yes

Choosing one

Latency is the whole argument. Cleanup runs between you releasing the key and the text appearing, so you feel every millisecond of it. Sub-second is the difference between dictation that feels instant and dictation you wait on. That is why the hosted plan is the paid default rather than a bigger model.

If you want to spend nothing: claudeCode runs on a Claude subscription you probably already have, at 4–6 seconds per dictation. Slower, free, no key.

If the data cannot leave the building: ollama. Legal, healthcare, finance, anything under NDA — most commercial dictation tools are a non-starter there because they ship your audio, and sometimes your screen, to their servers. With a local model, dictation, cleanup, and screen context all happen on hardware you own. That is an architecture, not a marketing claim.

If you want nothing at all in the path: turn AI Cleanup off. Yappy pastes the raw transcript instantly.

Fully local in two commands

brew install ollama && ollama pull qwen2.5:3b

Then in ~/.yappy/config.json:

{ "llm": { "provider": "ollama", "model": "qwen2.5:3b" } }

Plane mode is yap mode.

Cleanup is not Agent Mode

llm.provider controls dictation cleanup on the Right ⌥ key only. **Agent Mode (Right ⌘) always runs the claude CLI** and needs it installed regardless of this setting. Setting the cleanup provider to ollama does not make Agent Mode local.

Custom endpoints

custom accepts any OpenAI-compatible /v1/chat/completions server:

{
  "llm": {
    "provider": "custom",
    "baseURL": "http://127.0.0.1:1234/v1",
    "model": "your-model-name"
  }
}

If the endpoint stops answering, dictation cleanup stops with it — Yappy falls back to inserting the raw transcript rather than blocking on a dead server.