If voice typing isn't enough, Coral AI can literally take control of the mouse pointer via PyAutoGUI bindings.
Core Architecture
How It Works Under The Hood
The Mouse Coordinate Emulation module is built on a highly optimized C++ and Python bridge. By bypassing standard Windows UI restrictions, Coral AI directly interfaces with system memory, native Win32 APIs, and DOM structures to achieve near-zero latency execution.
Visual Ray-Casting
Calculates coordinate grids based on screen vision to click exactly where needed.
Drag & Drop Execution
Simulates mouse-down and mouse-up events to move files between visible folders.
Smooth Scrolling
Command it to 'Scroll down slowly' to read an article hands-free.
Macro Chaining
Click multiple buttons across a UI sequentially to automate repetitive tasks.
This module does not operate in isolation. It is dynamically invoked by the Coral PlannerAgent via JSON-RPC, allowing it to be chained endlessly with vision and memory modules.