# GitHub Copilot can now operate desktop apps, not just code

GitHub Copilot’s CLI and desktop app can now control GUI applications on macOS and Windows in public preview, extending coding-agent workflows into software that has no API, CLI or MCP integration.

The useful boundary change is that Copilot can now cross from code and terminals into ordinary desktop interfaces, with per-app approval and organisation-level controls.

- Status: Active
- Published: 2026-10-04T09:23:38+13:00
- Updated: 2026-10-04T09:23:38+13:00
- Categories: Artificial Intelligence, Web Development, AI Agents, AI Coding, Developer Tools
- Tags: AI agents, computer use, Developer tools, GitHub, GitHub Copilot
- Canonical HTML: https://beyondthe.news/dossiers/github-copilot-computer-use-desktop-apps

## What changed

On October 1, 2026, GitHub put computer use into public preview in GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows. Once enabled, Copilot can read accessible application content and visual context, click controls, enter and edit text, press keys, scroll, drag and navigate workflows across desktop applications. GitHub explicitly positions this for legacy and GUI-only software that lacks an API, command-line interface or MCP integration.

## Why it matters

Coding agents have mostly been strongest where software exposes files, shells, APIs or structured tool interfaces. Computer use gives Copilot a fallback for the opposite case: applications whose only practical automation surface is their GUI. That can connect development work to browsers, presentations, internal tools and older desktop software without writing a bespoke integration first. The trade-off is a larger action and security surface: GitHub requires approval before Copilot controls an app, macOS requires Accessibility and Screen Recording permissions, and organisations can disable the feature. It remains a public preview, so builders should treat it as supervised automation rather than a stable unattended execution layer.

## Copilot gets a GUI fallback when no tool interface exists

Computer use can inspect accessible content and visual context, then click, type, scroll, drag and move through application workflows. GitHub's examples include summarising browser notifications, editing presentation content and moving information through multi-application workflows.

## The permission boundary is explicit

Copilot asks for approval before controlling an application. Users can review or reset applications they have chosen to always allow. On macOS the feature requires Accessibility and Screen Recording permissions, and organisation-managed settings can disable computer use.

## This broadens what a coding agent can finish end to end

A repository task often ends at a boundary where a developer must use a GUI-only admin tool, browser workflow or legacy application. Computer use lets the same Copilot session attempt that final step instead of requiring an API, CLI or MCP server to exist first.

## Key details

- Computer use entered public preview on October 1, 2026.
- It is available in GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows.
- Copilot can read accessible and visual context and perform clicks, typing, scrolling, dragging and keyboard actions.
- Users must approve app control; always-allowed apps can be reviewed or reset.
- Organisation-managed settings can disable the feature.
- In Copilot CLI the feature is enabled with /computer on.

## Builder takeaways

- Use computer use where an important workflow has no API, CLI or MCP integration rather than building brittle glue immediately.
- Keep approval enabled for consequential applications and avoid treating preview GUI automation as unattended infrastructure.
- Audit macOS Accessibility and Screen Recording permissions as part of the agent's effective authority.
- Test workflows against application UI changes because GUI automation has a different failure mode from structured tool calls.

## What to watch

- Whether computer use graduates from public preview and becomes available in additional Copilot clients.
- Whether GitHub adds finer-grained organisation policies, app allowlists or action restrictions.
- Reliability measurements for long multi-application workflows and recovery after UI changes.
- Whether computer-use actions become auditable in enterprise activity or agent logs.

## Uncertainties

- GitHub has not published reliability or task-success benchmarks for the preview.
- The changelog does not define a complete application compatibility matrix.
- Public-preview behaviour, permissions and supported clients may change.

## Sources

- [GitHub Copilot can now interact with desktop apps with computer use](https://github.blog/changelog/2026-10-01-github-copilot-can-now-interact-with-desktop-apps/) — GitHub · primary · 2026-10-01T00:00:00+13:00. Primary release announcement, capabilities, permissions and client availability.