A developer recently released Omi, a phone agent app, on Reddit, claiming it can operate any Android app the way a human would, without requiring an API or connector from the target app. The developer also shared engineering lessons from Omi's on-device inference: an early version that used a small model as a "judge" to assess information importance generated many false positives, while switching to an EmbeddingGemma classifier achieved zero false positives.
Technical approach: reading screen elements, not screenshots
According to the developer, Omi reads screen elements through the Android accessibility API rather than relying on screenshots. A cloud model decides the next tap or input, looping until the task is complete. The developer says reading elements is faster and cheaper than screenshots, and locating buttons no longer relies on guesswork.
In terms of architecture, the decision model is currently deployed in the cloud, while memory, search and chat functions run on an on-device 2B model. The developer plans to migrate the whole system on-device in the future.
Current status: stable on single apps, limited across apps and in money-related scenarios
The developer says Omi is fairly stable on single-app tasks, but cross-app tasks — such as finding information in one app and acting in another — still produce errors. In addition, banking apps refuse to run when they detect the accessibility service, so money-related scenarios are naturally excluded.
On-device inference lessons: LLM judge misfires, classifier hits zero false positives
In a separate post, the developer described engineering lessons from building Omi, a fully offline Android app. The app can read messages, emails, documents and photos, and remember important items such as bills and deadlines. The first version used a small model to judge the importance of each item one by one, producing 9–30 false-positive cards, and was slow and power-hungry.
The developer then switched to EmbeddingGemma 2 to convert each message, photo and document into a vector in one pass, and used a self-trained small classification head to judge importance. According to the developer, 293 test lines produced zero false positives, the cost is extremely low and it can run in the background all day, and the same set of vectors also supports semantic search.
A fine-tuned Qwen3.5 2B only handles answering questions and routine organizing while charging, and all chat stays on the device.
The developer also pointed to the bottleneck in on-device inference: on mid-range phones, the slow part is not generation but reading the prompt. Keeping prompts short and reusing the KV cache is more effective than switching to a faster model.
What to watch
It remains to be seen whether Omi's cloud-based decision model and its plan to migrate on-device can solve cross-app task errors, and where the accessibility API approach hits its limits in sensitive apps such as banking. The developer's on-device inference experience also offers a reference for engineering phone agents.
