記事一覧に戻る
What Claude Computer Use Changes — When AI Starts Operating Your Screen

What Claude Computer Use Changes — When AI Starts Operating Your Screen

ZenChAIne·
AI AgentClaudeAnthropicComputer Use

Introduction

On March 24, 2026, Anthropic officially added Computer Use to Claude. An AI that sees your desktop, clicks your mouse, types on your keyboard, and operates your apps — what once felt like science fiction is now reality for Pro and Max plan subscribers on macOS.

This is not just "a useful new feature." It marks the evolution of AI agents from "writing code" to "using computers." This article examines how Computer Use works and what it means for developers and organizations.

Key Takeaways

  • Claude Computer Use autonomously operates apps, browsers, and files on macOS
  • Dispatch lets you assign tasks from your phone and have Claude complete them on your desktop
  • Claude automatically selects the best tool: connectors first, then browser, then desktop app control
  • Security is controlled through per-app permissions and a configurable blocklist

What Can Computer Use Do — and What Can't It?

Computer Use is an autonomous desktop control feature available in both Cowork (Claude's desktop app) and Claude Code.

What It Can Do

  • Launch and operate apps: Open files, enter data into spreadsheets, create presentations
  • Control browsers: Fill web forms, collect data, handle login flows
  • Manage files: Organize folders, rename files, aggregate data
  • Cross-app workflows: Check Slack messages → pull files from Google Drive → send a report via email — all in one flow

Current Limitations

  • macOS only for now: Windows x64 support is coming soon
  • Financial apps are blocked: Trading platforms and crypto wallets are restricted by default
  • Not perfect: Anthropic openly states that "Claude can make mistakes." Critical operations need human verification
  • Computer must be awake: The desktop app must be running; sleep mode won't work

Why the "Connectors First" Architecture Matters

The most important design decision in Computer Use is its tool selection priority:

1. Connectors (API integrations) → Fastest and most accurate
2. Browser control              → Fallback when no connector exists
3. Desktop app control          → Last resort when browser won't work

According to Anthropic's documentation, when Claude receives a task, it first checks for available connectors (Slack, Google Calendar, Gmail, Google Drive API integrations). If a connector exists, it uses that. If not, it falls back to controlling the browser's web version.

This is a smart design. API integrations are dramatically faster and less error-prone than screen navigation. Computer Use is positioned as a last resort when connectors aren't available — not as a universal desktop automation tool.

Thinking of Computer Use as a "universal desktop automation tool" is a misconception. The actual design philosophy is: use API connectors whenever possible, and only fall back to screen control when no other option exists.

What New Work Patterns Does Dispatch Enable?

Dispatch is the feature that lets you trigger Computer Use remotely. Send instructions from the Claude app on your phone, and Claude executes the work on your macOS desktop at home or the office.

Practical Use Cases

  • While commuting: "Summarize yesterday's Slack channel discussion and create a briefing in Google Docs"
  • During a meeting: "Pull Q1 sales data from last week's spreadsheet and create a report with charts"
  • On the weekend: "Organize my /Downloads folder — move images to /Photos and PDFs to /Documents"

Tasks that previously required sitting at your computer can now be completed with a phone instruction. This isn't just convenience — it has the potential to fundamentally decouple work from location.

What It Means for Developers

From a developer's perspective, Dispatch feels like "triggering a CI/CD pipeline from your phone." The fundamental difference is that the target isn't a CI pipeline — it's your entire desktop environment.

Combined with Claude Code, a fully remote development flow becomes technically possible: instruct implementation from your phone → Claude writes code on your desktop → runs tests → creates a PR.

How Is Security Designed?

Letting AI operate your desktop — the security concerns are obvious. Anthropic addresses them with these mechanisms:

1. Per-app permission system

Claude asks for permission before accessing each new application. Once approved, subsequent access is automatic, but the first time always requires explicit consent.

2. Blocklist

Financial trading platforms and cryptocurrency wallets are blocked by default. Users can also configure their own custom blocklist.

3. Screen visibility

During Computer Use execution, everything Claude does is displayed on screen in real time. There's no "operating invisibly in the background."

The security design philosophy is "restricted by default, explicitly permitted by the user" — an opt-in approach. This shares the same design principle as the "delegation with oversight" concept we discussed in cmux-delegate.

How Does This Relate to Claude Code?

Computer Use is available not just in Cowork but also in Claude Code.

Claude Code was originally a terminal-based development agent, but with Computer Use, it can go beyond "writing code" to autonomously handle tasks like "checking browser previews," "running GUI app tests," and "taking screenshots for reports."

This means the parts of development workflows that required "manually operating a screen" can now be automated. E2E test execution, design verification, post-deployment checks — tasks that previously required a human to open a browser and visually confirm results could be handled by Claude Code.

FAQ

Q. Is Computer Use free?

A. No. It's currently available only to Claude Pro ($20/month) and Claude Max subscribers. Free plans don't have access.

Q. Does it work on Windows?

A. macOS only for now. Anthropic has announced plans for Windows x64 support in the future.

Q. What happens if Claude makes a wrong operation?

A. Claude asks for app access permission before operating. All actions are displayed on screen in real time, so you can stop it immediately if something looks wrong. However, Anthropic acknowledges that mistakes are possible and recommends human verification for critical operations.

Q. Is Computer Use in Claude Code the same as in Cowork?

A. The underlying technology is the same, but the use cases differ. Cowork targets general desktop tasks (email, file management, document creation), while Claude Code is optimized for development workflows (test execution, preview verification, GUI debugging).

Summary

Claude Computer Use extends AI agent capabilities from "text generation" and "code writing" to "computer operation." Remote task assignment via Dispatch, the connectors-first architecture, and per-app permission security — these design choices represent a careful, rational approach to moving AI from "tool" toward "autonomous worker."

That said, this is still a research preview with acknowledged potential for errors. The right approach is "Trust, but verify."

At ZenChAIne, we continue to explore and validate development workflow automation using Claude Code's Computer Use capabilities.

References

🇯🇵 日本語で読む