Android accessibility service AI agent: a UI automation tutorial without root
Step-by-step tutorial: set up an Android accessibility service AI agent for UI automation without root, from permission and pairing to approved actions.
An Android accessibility service AI agent automates the UI without root. You grant one accessibility permission, the agent reads the screen's accessibility tree, taps, types or swipes on real elements, then checks the new screen before the next step. The tutorial below sets it up in seven steps with Melaya Device Control, an app allowlist, and approval before anything publishes. Secure or custom-drawn screens stay off-limits.
New to the topic? Start with what an Android AI agent can and cannot do on your phone.
What accessibility control can do
With the user's explicit permission, an accessibility service can observe parts of the current UI and perform navigation or gesture actions. That enables an agent to interact with apps through the visible interface without root access.
Tutorial: set up an accessibility AI agent in 7 steps
This walkthrough uses Melaya Device Control on a real Android phone running Android 8 or newer. No root, and no API from the apps the agent operates.
- 1. Install the Melaya app on the Android phone you want the agent to operate.
- 2. Pair it: scan the QR code shown under Device Control on your desktop, or tap Pair this phone inside the app.
- 3. Turn on control: allow the restricted setting, then enable the Melaya accessibility service. This is the permission that lets the agent read the screen and act.
- 4. In the Apps panel, switch on only the apps the agent may use. Every other app stays blocked on the phone, whatever the prompt says.
- 5. Pick the model: a cloud provider key, your Claude Code or Codex subscription, or a local model through Ollama or LM Studio.
- 6. Run a device agent. It reads the accessibility tree, targets real on-screen elements instead of pixel coordinates, performs one action, then checks the result.
- 7. Approve or stop. Posts and comments wait for your tap on the phone, and you can end the run from Device Control at any time.
What it cannot guarantee
Custom canvases, games, secure windows, media surfaces, operating-system restrictions, and app changes can limit what is observable or actionable. Accessibility metadata may be incomplete or misleading. The workflow must be able to stop when the state is uncertain.
- No promise of universal app compatibility
- No bypass of secure screens or platform controls
- No self-granting permissions
- No hidden assumption that coordinates remain stable
Use the permission responsibly
Explain why the permission is needed, show when control is active, let the user revoke it, and keep the agent inside a declared app and action scope. Accessibility exists to help users; automation must not abuse it to conceal behavior or defeat consent.
Frequently asked questions
Can an AI agent control my Android phone without root?
Yes. With an accessibility service you grant, an AI agent can read the on-screen UI and perform taps, swipes, and text entry across apps without root or a jailbreak. Some protected or custom-rendered screens stay off-limits by design.
How do I automate Android with accessibility services?
Enable the accessibility permission for the agent, scope it to an app allowlist, and let it observe the UI and perform gestures. Keep actions bounded, require approval for consequential writes, and let it stop when a screen is unreadable.
Is accessibility automation safe?
It can be, with explicit consent, a clear active-control indicator, an app and action allowlist, human approval on protected writes, and easy revocation. Accessibility exists to help users, so automation must never hide behavior or defeat consent.
Does accessibility automation require root?
No. Android accessibility services operate through user-granted platform permissions rather than root access.
Can it access secure screens?
Not reliably. Apps and Android can protect sensitive surfaces, and a safe agent should treat inaccessible state as a stop condition.
Is accessibility automation the same as Appium?
No. Appium is primarily a testing automation stack. An accessibility-based agent can support interactive real-device workflows, but the tools have different control, test, and deployment models.
