# Yappy LLM backends

An LLM does exactly one job in dictation: tidying the raw Whisper transcript
before Yappy types it. Which one runs that step is configured in
**Settings → Engine**, or as `llm.provider` in `~/.yappy/config.json`.

## The options

| Provider | What it is | Typical latency | Key needed? |
|---|---|---|---|
| `hosted` | The Yappy plan's hosted inference | under 1s | No — vended by the plan |
| `ollama` | Fully local open-source models (default `qwen2.5:3b`) | 1–3s | No |
| `custom` | Any OpenAI-compatible endpoint — LM Studio, vLLM, llama.cpp | varies | No |
| `claudeCode` | Your Claude subscription, via the `claude` CLI | 4–6s | No |
| `grokCode` | Your xAI subscription, via the `grok` CLI | ~2s | No |
| `anthropic` | Claude via the API | 1–2s | Yes |
| `openai` | OpenAI via the API | 1–2s | Yes |

## Choosing one

**Latency is the whole argument.** Cleanup runs between you releasing the key and
the text appearing, so you feel every millisecond of it. Sub-second is the
difference between dictation that feels instant and dictation you wait on. That
is why the hosted plan is the paid default rather than a bigger model.

**If you want to spend nothing:** `claudeCode` runs on a Claude subscription you
probably already have, at 4–6 seconds per dictation. Slower, free, no key.

**If the data cannot leave the building:** `ollama`. Legal, healthcare, finance,
anything under NDA — most commercial dictation tools are a non-starter there
because they ship your audio, and sometimes your screen, to their servers. With a
local model, dictation, cleanup, and screen context all happen on hardware you
own. That is an architecture, not a marketing claim.

**If you want nothing at all in the path:** turn AI Cleanup off. Yappy pastes the
raw transcript instantly.

## Fully local in two commands

```bash
brew install ollama && ollama pull qwen2.5:3b
```

Then in `~/.yappy/config.json`:

```json
{ "llm": { "provider": "ollama", "model": "qwen2.5:3b" } }
```

Plane mode is yap mode.

## Cleanup is not Agent Mode

`llm.provider` controls dictation cleanup on the Right ⌥ key only. **Agent Mode
(Right ⌘) always runs the `claude` CLI** and needs it installed regardless of
this setting. Setting the cleanup provider to `ollama` does not make Agent Mode
local.

## Custom endpoints

`custom` accepts any OpenAI-compatible `/v1/chat/completions` server:

```json
{
  "llm": {
    "provider": "custom",
    "baseURL": "http://127.0.0.1:1234/v1",
    "model": "your-model-name"
  }
}
```

If the endpoint stops answering, dictation cleanup stops with it — Yappy falls
back to inserting the raw transcript rather than blocking on a dead server.

---

Canonical: https://yappy.biz/docs/backends/ · Last updated: 2026-08-25