Vercel agent-desktop: A command-line tool that enables AI agents to truly "operate computers"Vercel agent-desktop:让 AI Agent 真正「操作电脑」的命令行工具
No screenshots, no pixel matching: It directly reads the accessibility tree of the operating system, transforming desktop applications into JSON structures with ref numbers. AI clicks, inputs, and verifies everything with precision down to the control level. Written in Rust, it works across three platforms – macOS, Windows, and Linux – and topped GitHub's growth chart the week it was released. It marks the beginning of the era of "agents controlling real computers".
不截图、不像素匹配:它直接读取操作系统的无障碍树,把桌面应用变成带 ref 编号的 JSON 结构,AI 点击、输入、验证全部精确到控件级别。Rust 编写,macOS / Windows / Linux 三平台通吃,发布当周登顶 GitHub 增速榜,是「Agent 操控真实计算机」元年的开山之作。

How is it different from the screenshot solution?它和截图方案差在哪
Mainstream computer use solutions rely on screenshots + pixel coordinates: the model looks at the image and guesses the position of the button. If it clicks the wrong one, it has to start over again. This is both slow and unreliable. agent-desktop takes a different approach – it directly reads the system's Accessibility Tree and returns a JSON where each interactive element has a reference in the format @s8f3k2p9:e8.
The AI's operation loop becomes: snapshot obtains the structure → click/type is performed on the ref → snapshot again to verify the result. Each command outputs a JSON envelope, and no visual model is required throughout the process, making it both fast and accurate. Note that it does not have any built-in models itself, and the loop logic is driven by your Agent.
主流的 Computer Use 方案靠截图 + 像素坐标:模型看图猜按钮位置,点错了再来一遍,又慢又不稳定。agent-desktop 换了条路——直接读系统无障碍树(Accessibility Tree),返回的 JSON 里每个可交互元素都带一个形如 @s8f3k2p9:e8 的 ref。
AI 的操作循环变成:snapshot 拿到结构 → 对 ref 执行 click / type → 再 snapshot 验证结果。每条命令输出一个 JSON 信封,全程无需视觉模型参与,又快又准。注意它本体没有内置任何模型,循环逻辑由你的 Agent 驱动。
Quick Start快速上手
The simplest installation is npm install -g agent-desktop (the package contains x64 and ARM64 native binaries). After installation, run agent-desktop status to check platform and permission support. macOS users can use agent-desktop permissions --guide to complete accessibility authorisation step by step.
The three core tools: agent-desktop snapshot --app Finder -i List interactive elements → agent-desktop click @s8f3k2p9:e8 Click on the specified element → Then snapshot to verify. The clipboard, screenshots, and command output can all be handled with a single command.
最简安装是 npm install -g agent-desktop(包内含 x64 与 ARM64 原生二进制)。装好先跑 agent-desktop status 检查平台与权限支持,macOS 用户用 agent-desktop permissions --guide 一步步完成无障碍授权。
核心三板斧:agent-desktop snapshot --app Finder -i 列出交互元素 → agent-desktop click @s8f3k2p9:e8 点击指定元素 → 再 snapshot 验证。剪贴板、屏幕截图、命令输出同样是一条命令的事。
This site provides native Windows packages本站提供 Windows 原生包
Windows users who do not want to mess with the npm environment can directly download the v1.3.0 x86_64 native executable file packaged by this site. Extract it to get agent-desktop.exe, and there is no need to install Node.js. macOS/Linux users are recommended to use npm or Homebrew (brew install agent-desktop).
Security Reminder: Snapshots, clipboard readings, and screenshots may contain private desktop content. Do not post them to public logs or upload them to services.
不想折腾 npm 环境的 Windows 用户,直接下载本站打包的 v1.3.0 x86_64 原生可执行文件,解压即得 agent-desktop.exe,无需安装 Node.js。macOS / Linux 用户建议走 npm 或 Homebrew(brew install agent-desktop)。
安全提醒:快照、剪贴板读取和截图可能包含隐私桌面内容,别把它们传到公开日志或上传服务。
Applicable Scenario适用场景
Give Agents like Claude Code/Codex "hands": automate desktop software testing, perform batch operations on GUI applications, and automate file management processes. Together with the FFI header file from the official documentation, you can also embed desktop control capabilities into your own programme.
给 Claude Code / Codex 等 Agent 挂上「手」:自动化桌面软件测试、批量操作 GUI 应用、文件管理流程自动化。配合官网文档的 FFI 头文件,还能把桌面操控能力嵌进你自己的程序。
⬇ Download · 点击下载:agent-desktop v1.3.0 Windows x64 原生可执行文件
Apache-2.0 许可 · 解压即得 agent-desktop.exe · 也可 npm install -g agent-desktop
阅读与点赞数据保存在你的浏览器本地,欢迎留下你的想法。
今日正能量学一点,用一点;今天种下的种子,会长成明天的能力。去免费下载专区 →广告
评论 文明发言,让讨论更有价值