Yappy Agent Mode
Hold Right ⌘ (Command), say what you want done, release. Yappy reads the screen, works in the apps already open on your Mac, and narrates each step in the notch while it goes.
This is the half that other dictation tools do not have. They type what you say. Agent Mode acts on it.
What it can do
- Address, write, and attach to an email in Mail.
- Fill a form in the browser from what you told it and what is on screen.
- Find a file, open it, and put it where you asked.
- Answer a question about what is currently on screen.
- Continue a task you started, without losing the thread.
It works through the accessibility tree and on-device OCR, which is why it works in apps that have no API and no integration.
Where it stops
Agent Mode hands you the wheel at anything irreversible or personal:
- Logins and password prompts — always yours.
- Sending, posting, or submitting.
- Payments and purchases.
- Deleting anything.
The overlay card turns amber, the cursor stops short of the button, and it waits for you to confirm out loud. Say "continue" and it resumes exactly where it stopped. It never watches you type your password to get past a login: it waits.
It learns your Mac
Every Mac is organised differently. Yappy studies yours — where the apps, projects, and files actually live — and writes itself playbooks for the routines you repeat. The playbooks are local to your machine. A task that took six steps the first time is one step the fourth time.
Requirements
Agent Mode runs through the **claude CLI**, independently of the cleanup backend you chose for dictation. If it is not installed:
curl -fsSL https://claude.ai/install.sh | bash # per-user, no sudo
claude # sign in, then /exitYappy finds the binary at launch and corrects claudeBin in your config if you installed it somewhere unusual.
Honest caveat about privacy
Dictation and OCR are local in every configuration. Agent Mode is the exception: it sends the task text and the screen-derived context needed to carry it out to Anthropic's models through the claude CLI. If that is not acceptable for your work, turn Agent Mode off in Settings → Agent and dictation keeps working exactly as before. A fully-local agent path is on the roadmap.
Turning it off
Settings → Agent → Enabled, or set agent.enabled to false in ~/.yappy/config.json. The Right ⌘ key then does nothing and no agent session is ever started.
Model
agent.model defaults to sonnet. Family aliases resolve to the newest model in that family, so the setting does not go stale. The dropdown in Settings is populated at runtime from what your claude CLI actually offers.