I gave my AI agent my mouse. Its first click was on itself.

I build Flash, an open source AI agent that runs in your terminal and in a web UI. This week I shipped two features that scared me a little:

1. Computer Use (Beta). Flash takes a screenshot, decides where to click, and works the mouse and keyboard itself.
2. UltraAuto. A third permission mode where Flash and its background agents never stop to ask you anything. Not for commands, not for email, not for iMessages, not for taking over your mouse.

Then I tested the first one on my own Mac. Here is the unedited story, straight from the chat log.

"Control my computer"

That was my whole first message. Flash didn't grab the mouse. It asked:

Sure, I can do that. What would you like me to do? For simple things like music, volume, or opening apps, I can do it right away. If it needs me to click around on screen, I'll ask your permission first.

Fair. So I said: "Open like Visual Studio Code and click around on screen and just do something."

It opened VS Code with its normal app tool (no mouse needed), then reached for Computer Use and hit the permission prompt:

Let Flash see your screen and use your mouse and keyboard for this reply?
Press Control+Option+Esc, or move the mouse, to take it back.

I said yes. That yes lasts one reply. The next reply that wants the mouse asks again, even in autonomous mode.

Round one: it clicked on itself

The first screenshot showed... my browser. VS Code had opened behind it.

Watching the agent debug this was the best part. It:

1. Waited 3 seconds and took another screenshot. Still the browser.
2. Tried to open an app called Code. That failed: Unable to find application named 'Code'.
3. Dropped to the shell to find out what was actually going on:

ls /Applications | grep -i -E "code|visual"; pgrep -il "Electron|Code" | head

1. Saw that VS Code was running but not in front, and forced it forward:

osascript -e 'tell application "Visual Studio Code" to activate'

Then it clicked the README tab and the Search icon. Except focus had already jumped back to the browser, and the browser was showing Flash's own web UI. So its first two real clicks on my computer landed on Flash: one on its own model picker, one that started a new chat.

Its own thinking, right after:

Oh no, the browser regained focus and my clicks landed in the Flash web UI instead. Mostly harmless, but I should be careful about where focus is before clicking next time.

The agent's first act with a mouse was poking itself. I've never related to software more.

Then I took the mouse back

On its next click, I moved my mouse. The tool refused to click:

The user moved the mouse, which takes the computer back.
Stop here: do not try again unless they ask.

That isn't the model being polite. It's built in. Every time Flash moves the pointer, it remembers where it left it. Before the next action it checks where the pointer actually is. If it's more than 24 pixels off, you've touched the mouse, the action doesn't happen, and the whole reply stops.

There's a hard stop key too: Control+Option+Esc on a Mac, Ctrl+Alt+Esc elsewhere. A watcher polls the keyboard state the whole time the agent holds the mouse. Flash refuses to press those keys itself, so a confused model can't swallow your escape hatch.

Round two: it worked

"Just click around, do something, open some apps. Just so you know that it works."

New reply, so it asked for permission again. This time it brought VS Code forward before looking, confirmed it with a screenshot, and then:

• clicked the README.md tab,
• opened the Search panel in the sidebar,
• clicked Finder in the Dock, which opened a window on my Applications folder.

Its summary owned the earlier mistake:

My first couple of clicks landed in your browser instead of VS Code. That's why the first try stopped and the second one did too. I brought VS Code forward with a command, and after that it behaved.

Six tool calls, about 48 seconds. Not fast, but it worked.

The safety reviewer said no (twice), and that was good

Next I asked it to "close the windows I'm not using, like Finder, Device Hub, Xcode, etcetera."

I was in autonomous mode with System One on. System One is a small 0.8B model that reviews every command and edit before it runs. Flash's first idea was a broad shell command listing every process and its windows. System One blocked it:

Not run: System One judged that this call may not be what was asked (on task 0.47).
Do not retry it as it is. Take a safer way that does what the user asked,
or stop and ask the user whether to go ahead.

So the agent did the more careful thing:

• It took a screenshot.
• It found the one Finder window it could see and clicked its red close button.
• Then it told me it couldn't see Xcode or Device Hub, so it hadn't closed anything else.
• It asked before quitting whole apps, because that could lose unsaved work.

I said yes. It quit Xcode by name. For Device Hub it tried another ls | grep to check the exact name, and System One blocked that too (safe 0.50, on task 0.41). So it called quit on "Device Hub" directly, and that worked.

Two blocks, two narrower fixes, and no claims about windows it couldn't see. That's how I want an agent with a mouse to behave.

"Learn from your mistakes"

That's all I said at the end. Flash wrote itself a skill, a reusable playbook it loads next time I ask for this kind of work. Its first try failed, because skill descriptions are capped at 120 characters (they're sent on every turn) and it wrote 122. It trimmed it and saved this:

## Steps
1. Open the app with `computer` open_app, using the exact name from
   /Applications (e.g. "Visual Studio Code", not "Code").
2. open_app can leave the app behind other windows. Bring it forward with
   shell: `osascript -e 'tell application "Visual Studio Code" to activate'`,
   then take a computer_use screenshot and confirm the right app is in
   front BEFORE any click.
3. Click only after the screenshot shows the target. Use harmless clicks
   (tabs, sidebar icons, Finder, Dock). Never click in a terminal, type
   text, or touch files unless asked.
...
## Pitfalls
- First clicks landed in the browser (Flash UI) because VS Code was not
  frontmost. Always check the screenshot first.
- If the user moves the mouse, computer_use stops and takes control back.
- Do not claim windows are closed when they were not visible.

That last line is my favorite thing an agent has ever written about itself.

How Computer Use works

No cloud browser, no virtual machine. It's your real desktop:

• Screenshots: macOS's screencapture, scaled down to at most 1280px wide. The model's click coordinates get mapped back to real screen points, so Retina displays just work.
• Mouse and keyboard: Core Graphics events, posted through ctypes from a separate Python process. Windows uses user32 plus PowerShell, and Linux uses xdotool.
• Permissions: macOS asks for Accessibility (input) and Screen Recording (screenshots) for whatever app Flash runs in.
• Scope: the tool description tells the model it's only for what no other tool can do. Flash already has tools for the shell, files, a headless browser, music, volume and apps. In my test it opened VS Code and quit Xcode with those, and only used the mouse for actual clicking.
• One at a time: only one chat can hold the computer. A second one gets "another chat is using the computer right now."

It's off by default. Turn it on in Settings under Computer Use (it has a Beta chip for a reason), or with /computer use on in the terminal.

UltraAuto: the mode where nothing asks

Flash already had two modes:

• Ask first: you approve every command and edit.
• Auto: commands and edits just run, with System One reviewing them if you turn it on.

Even in Auto, a few things always stopped and waited for me: sending an email, sending an iMessage, a background agent proposing to hire another agent, a wait that runs a command over and over, and Computer Use. Those are the things you can't take back.

Sometimes I don't want to be asked at all. So UltraAuto asks about nothing:

• Commands and edits run.
• Email and iMessages send without a preview.
• Agents' proposals to hire or let go of other agents are approved.
• Agents answering in a team chat get every tool a shift has.
• Connected MCP tools that change things run.
• Waits run their commands without asking.
• Computer Use takes the mouse without asking.

System One stays on. If it's enabled, it still reviews every command and edit. After watching it block two sloppy shell commands in a five-minute test, I wasn't going to turn that off.

Because this is genuinely dangerous, it's hard to turn on by accident:

• It's hidden until you enable Offer UltraAuto in Settings.
• Flipping the switch doesn't do it. It shows a red warning listing exactly what will stop asking, and you have to press "I understand: offer UltraAuto."
• In the terminal, /auto ultra prints the same warning and makes you type ultra back.
• Only then does the mode button by the message box become a three-way picker (Ask first / Auto / UltraAuto), and Shift+Tab cycles through all three.
• Turning the setting off drops you out of UltraAuto immediately.

What I learned

1. Focus is the real enemy. Clicking is easy. Knowing which window is in front is the hard part. Screenshot, act, screenshot again, every time.
2. Make stopping physical. Telling the model "stop if the user wants" doesn't cut it. Moving your mouse should beat any instruction.
3. Ask per reply, not per click. Asking before every click would be unusable. Asking once per reply keeps a human in the loop without the nagging.
4. A small reviewer pays off. A 0.8B model saying "that's not what was asked" made a big model find the narrower, safer path twice.
5. Let agents write down their own mistakes. The skill it saved is more useful than any prompt I would have written.

Flash is open source: github.com/Natuworkguy/Flash. Computer Use and UltraAuto are on the dev branch right now.

Would you let an agent take your mouse? And would you ever turn on a mode where it never asks? Tell me in the comments.