Melaya Computer Control · computer use agent05 / 05

The AI agent thatcontrols your computer.

Excel, Outlook, your ERP, any app with a window, on Windows, macOS and Linux. It reads the real window, plans many steps in one call and checks every click.

Open the app← Back to home
Let Claude Code, Codex or Cursor drive your desktop apps too.Set up in 1 min
Reads real windowsthe accessibility tree first, OCR when an app draws its own pixels
No false successevery action is checked against the window before the next one
Per-app permissionsview, click or full, with sensitive apps off by default
System One fast modeone model call plans many steps, Melaya runs and checks them

Not a screenshot guesser.An agent that reads your apps.

A computer use agent is an AI that operates a computer the way you do. Plenty of real work still lives in desktop apps with no API: the ERP from 2009, the spreadsheet everyone shares, the inbox rules nobody remembers. Computer Control lets a Melaya agent use them. It reads the window as named buttons, fields and tables, acts on exact elements, and checks what changed after every step. Think of it as a careful colleague at your keyboard who never touches your passwords.

Three steps. No terminal.

Install it, connect once, then hand it work.

  1. Install the app

    One click in the Microsoft Store. Free, signed by Microsoft.

  2. Choose what agents may touch

    Pick a level for each app. Try it here.

    • Excel
    • Outlook
    • Password managerAlways off
  3. Give it a task

    Plain words in, checked steps out. The important ones wait for you.

    Copy this week's invoices from Outlook into Invoices.xlsx and draft a summary for Sam.

    1. Outlook: 12 invoices read
    2. Invoices.xlsx: 12 rows filled and checked
    3. Summary for Sam draftedApprove send
Desktop app setup guide

A screenshot agent asks the AI 20 times. Melaya asks once.

Same job, same 20 cells, same clock. One reads pixels and asks before every click. Melaya reads the window once, plans everything, then checks each step itself.

14.3× faster in this illustration: a 20-cell form at about 4 seconds per AI call. Melaya asks the AI again only when a step does not do what the plan said.

  • Reads, not screenshots Windows hands Melaya each window as named buttons, fields and cells, so nothing is guessed from pixels.
  • One plan, many steps The AI writes the whole sequence in one call: up to 500 steps in fast mode, 5,000 in a batch.
  • Checked as it goes Every step is verified in the app. On a surprise, Melaya stops and asks the AI again from that step.
How fast mode works

Fast, batch, plans, Laya.

Other agents ask a big AI before every single click. Melaya asks once, then clicks, checks and keeps going on its own.

Computer Control · batch mode · one stored plan

An agent painted the Mona Lisa in Microsoft Paint.

call
1
brush strokes
17,386
minutes, start to finish
8:32

The agent planned every stroke as code, saved it as a plan, then started it with one call. Melaya moved the brush, switched colours and checked its own work. This is the real canvas.

  1. 01Fast modePlain steps. Each one found, done, checked.melaya_computer_fast
  2. 02Batch modeExact steps. Thousands in one go.melaya_computer_batch
  3. 03PlansWrite the job once. Start up to 5,000 steps in one call.melaya_computer_plan
  4. 04LayaPicks the right button when the words are fuzzy.
2.8 s35 steps typed, desktop app6.1 s108 sevent page, Brave13.2 s107 sTikTok search, Android

Desktop automation AI for every app with a window

The same moves work in Excel, Outlook, Teams, File Explorer, Finder, SAP GUI, a CRM or an in-house tool. If you can do it with a mouse and a keyboard, an agent can do it under your rules.

screen · ocr

Reads the real window

UI Automation on Windows, Accessibility on macOS, AT-SPI2 on Linux: named elements with stable references. OCR and marked screenshots fill in for apps that draw their own UI.

click · type · press

Clicks, types, shortcuts

Menus, combo boxes, checkboxes, drag and drop, Ctrl+S or Cmd+Shift+4. Values can be set directly instead of typed.

file_dialog

Open and Save dialogs

Fills file dialogs from file handles, never raw paths. A file it saves comes back to the run as a handle you control.

read_table · set_value

Tables and spreadsheets

Reads grids as rows in Excel, ERPs and CRMs, so an agent can loop over every open invoice instead of squinting at pixels.

delivery: background

Background first

Where the OS allows, actions go through the accessibility layer: no cursor grab, no window jumping to the front. When it needs the foreground, it says so.

expect · observe

Checked, every time

After each write, Melaya looks at what changed. Nothing changed? The step reports no effect instead of a false success.

handoff

Hands back for logins

Sign-in, 2FA or a CAPTCHA? The agent pauses, you do that step yourself, press Continue, and it picks up from a fresh read.

keep_awake

Ready for scheduled runs

Keeps the computer awake during a scheduled run, and says plainly when the screen is locked instead of clicking at nothing.

playbooks

App playbooks

Built-in notes for 24 common desktop apps, from Excel, Outlook and Teams to Finder, with the keyboard routes that work best.

connectors

Connectors first

When Slack, Notion or Teams is connected, the agent uses the connector instead of clicking through the app.

Melaya vs OpenClaw vs Hermes Agent vs Claude Cowork

Four ways to let an AI agent control a computer. All four are serious tools. Here is how they differ, side by side.

What mattersMelayaOpenClawHermes AgentClaude Cowork
How it sees the screenAccessibility tree first (UI Automation, macOS Accessibility, AT-SPI2), OCR and marked screenshots as fallback, one reference spaceScreenshots plus accessibility tree with element refs, Peekaboo on macOS by defaultSet-of-marks screenshots, vision or accessibility-tree modesScreenshots and coordinates, no accessibility tree given to the model, images downscaled to about 1372x887
Model calls and token useOne model call plans many steps (up to 100), Melaya runs and checks themUsers report 80 to 150k input tokens per taskNot documentedReviewers found it slow and token-hungry (a 5-hour Pro allowance used in 30 minutes, PCWorld)
Verified writes, no false successEvery write checked against the window, "no effect" instead of a false success, admin windows reported up frontNot documentedDocumented: on elevated Windows windows, capture returns 0 elements and click reports success while doing nothingNot documented
Background control on WindowsBackground first through accessibility patterns, says when it had to use the foregroundWindows and Linux driver marked experimental, primary display onlyBackground-first by designWorks on the visible screen on Windows (background windows on macOS 15+)
Linux and headless VMsLinux on X11 and Wayland, plus headless servers and VMs with npx @melaya/runner --computerWindows and Linux driver marked experimentalNot documentedNot documented
Per-app permission levelsView, click or full control per app, sensitive apps off by defaultNo per-action confirmation, safety through durable arming and tool policyApproval modes, including a bounded modeStrong per-app approval tiers
Stop and safetyStop button and shortcut, pauses when you touch the mouse or keyboard, never types into password fieldsCVE-2026-25253 (one-click RCE) and malicious skills found in its skill hubRelies on private macOS APIs for background input, which can break with OS updatesEsc stops it from anywhere, one session at a time (machine-wide lock)
Phone, browser and computer in oneYes: the same agents drive your Android phone, your browser and your computerNot documentedNot documentedNot documented
Scheduled runs on your computerScheduled pipelines run on your own computer, kept awake for the runCron and heartbeat schedulingNot documentedThe machine must stay awake for local apps
Any model, MCP accessClaude, GPT, Gemini, Qwen or local models, and reachable from Claude Code, Codex and Cursor over MCPNot documentedNot documentedRuns on Claude, a very capable model

Competitor details per public documentation, October 2026. "Not documented" means we found no statement in their public docs, not that the feature is missing.

Melaya vs OpenClaw

OpenClaw is open source, with a huge skill ecosystem and cron and heartbeat scheduling. Per its public docs, its Windows and Linux driver is experimental and primary-display only, and users report 80 to 150k input tokens per task. Melaya runs on Windows, macOS, Linux and headless VMs, plans many steps per model call, and checks every write.

OpenClaw alternative: full comparison

Melaya vs Hermes Agent

Hermes Agent from Nous Research is background-first, with a learning loop that writes skills and approval modes including a bounded one. Its own docs note that on elevated Windows windows, a click can report success while doing nothing, and that background input on macOS relies on private APIs. Melaya detects admin windows up front and reports no effect instead.

Hermes Agent alternative: full comparison

Melaya vs Claude Cowork

Claude Cowork brings a very capable model, strong per-app approval tiers and Esc to stop from anywhere. It works from screenshots and coordinates, uses the visible screen on Windows and runs one session at a time. Melaya reads the accessibility tree, works in the background where the OS allows, plans many steps per call, and can run Claude or any other model.

Claude Cowork alternative: full comparison

Your computer, your rules

Agents only see and touch the apps you switch on, at the level you pick. These rules hold on every run, from every surface.

Per-app levels: View reads only, Click adds clicks and scrolling, Full allows typing and files. Approvals still apply on Full.
Password managers, banking and trading apps, terminals, system settings and Melaya itself start switched off.
Never types into password fields, or anything that looks like one: one-time codes, card numbers, API keys.
Asks before it sends, deletes, buys or installs in Safe mode, in the desktop app and on the web.
A global Stop shortcut and a Stop button, plus Stop from the web. Touch the mouse or keyboard yourself and the run pauses.
A live timeline of every action and what changed. The on-screen driving indicator never shows up in what the agent sees.
Admin windows are detected up front and reported, never silently clicked.
Safety and permissions in the docs
Allowed appsOffice PC
Excel

Spreadsheets

ViewClickFull
Outlook

Email and calendar

ViewClickFull
File Explorer

Files and folders

ViewClickFull
SAP GUI

ERP

ViewClickFull
1Password

Password manager

Off · sensitive
Terminal

Command line

Off · sensitive
policy

Checked twice: the server applies your levels, then the app on your computer checks again before it acts.

A computer use agent for Linux, headless servers and VMs

Linux is a first-class desktop here, not an experiment. The desktop app reads Linux apps through AT-SPI2 on X11 and Wayland. No screen at all? Run the Melaya runner on a headless server or VM and it becomes a computer your agents can use, with the same permissions, approvals and timeline.

Runner CLI and VM guide
npx @melaya/runner --computer
  • Desktop app for Linux as AppImage, .deb or .rpm
  • AT-SPI2 accessibility tree on X11 and Wayland
  • Headless servers and VMs: npx @melaya/runner --computer
  • Multi-monitor and per-monitor DPI on every OS
  • Scheduled pipelines run on that machine, kept awake for the run

Ask in chat. Schedule it. Call it from your own AI.

One set of computer tools, with the same app levels, approvals and Stop everywhere you use them.

Assistant

Type the job in plain words and watch it happen, with approvals inline.

Pipelines

Scheduled runs on your own computer, from the tray, while you sleep or sit in meetings.

Claude CodeCodexCursorClaude

Your own AI, over MCP

Claude Code, Codex, Cursor or Claude get computer tools with your permissions and approvals.

Works next to Browser Control and Device Control: read the web, update the desktop app, confirm on the phone, all in one run.

Work that lives in desktop apps

Five jobs teams hand to Computer Control first.

Excel automation with an AI agent

Clean a monthly export, fill formulas down, pivot by region and save a CSV where finance expects it. The agent reads cells as rows, not pixels, so it works through the whole sheet.

Excel

Automate Outlook with an AI agent

Sort the inbox by sender and topic, file invoices, flag what needs you and draft replies for your approval. Sending always asks first in Safe mode.

Outlook

ERP export without an API

Pull the month-end ledger out of a desktop ERP, reshape it in Excel and save it on the shared drive. Schedule it and it runs every month on your own computer.

Desktop ERP · Excel

CRM data entry

Copy leads from a spreadsheet into the desktop CRM field by field, and check that every record saved before moving on.

Excel · Desktop CRM

QA testing for desktop apps

Click through your own app after each build, compare what appeared with what should, and report the exact step that broke. Run it on a Linux VM in CI.

Your app · Windows, macOS, Linux

Questions people actually ask

Short version: the agent works in the apps you allow, checks every step, and stops when you say stop.

What is a computer use agent?

An AI agent that operates a computer through its screen, mouse and keyboard, the way a person does, so it can work in apps that have no API. Melaya's computer use agent reads the app's accessibility tree first, acts on exact elements and checks each result.

Which systems does it work on?

Windows 10 and 11, macOS 13 or later, and Linux desktops on X11 and Wayland. Headless Linux machines and VMs work through the Melaya runner, started with npx @melaya/runner --computer.

Why is System One faster than screenshot agents?

Screenshot agents make a model call and take a screenshot for every click. System One reads the window once, plans many steps in one call, and Melaya grounds and checks each step on the accessibility tree. Far fewer model calls and images per task.

Is Melaya an OpenClaw alternative?

Yes. OpenClaw is open source with a large skill ecosystem. Per its public docs (October 2026), its Windows and Linux driver is experimental and primary-display only. Melaya runs on Windows, macOS, Linux and headless VMs, with per-app levels, approvals and verified writes.

How is Melaya different from Claude Cowork?

Claude Cowork has a very capable model and strong per-app approval tiers. Per Anthropic's docs (October 2026), it works from screenshots and coordinates, without an accessibility tree. Melaya reads the accessibility tree, plans many steps per call, and can use Claude or any other model.

How does Melaya compare with Hermes Agent?

Hermes Agent is background-first and has a learning loop that writes skills. Its docs (October 2026) note that on elevated Windows windows, a click can report success while doing nothing. Melaya detects admin windows up front and reports no effect instead of a false success.

Can it see my passwords or my banking app?

Password managers, banking and trading apps, terminals and system settings are off by default, and the agent never types into password fields. You decide which apps it may use and how far it may go: view, click or full.

Which AI models can drive it?

Any model you use in Melaya: Claude, GPT, Gemini, Qwen, local models through Ollama or LM Studio, or your own Claude Code or Codex subscription. Fast mode keeps model calls low, so smaller models do well too.

Does what is on my screen leave my computer?

The app and every action run on your computer. What the agent reads from the apps you allowed goes to the model you picked. Pick a local model and it stays on your machine.

How is it different from Browser Control and Device Control?

Browser Control drives web pages, Device Control drives an Android phone, Computer Control drives desktop apps. Same agent, same permissions and approvals, and one run can use all three.

Full computer use comparisons:OpenClaw alternativeHermes Agent alternativeClaude Cowork alternative

Put an agent at your desk

Install the desktop app, connect this computer, switch on the apps you want help with, and hand over the first boring job.

Runs on your computer. Apps stay off until you allow them. Stop is one shortcut away.
// Cookies
Melaya uses a small set of first-party cookies that are strictly necessary to authenticate you, maintain your session, and protect the platform from abuse. We do not use advertising cookies, cross-site trackers, or third-party analytics by default. The full cookie list is in our Privacy Policy.