Advanced desktop automation with pixel-perfect mouse control, keyboard input, screen capture, window management, and clipboard operations. Built on PyAutoGUI.
by
matagul
Desktop Control provides comprehensive desktop automation capabilities, allowing OpenClaw to interact with any application on your computer.
It offers pixel-perfect mouse control, fast keyboard input, screen capture with image recognition, window management, and clipboard operations with built-in safety features.
The skill uses PyAutoGUI as the core engine and exposes a DesktopController class for automation.
Mouse moves, clicks, drags, scrolling, and keyboard events are routed through DesktopController.
Screenshots, pixel checks, and optional OpenCV image recognition help locate UI targets.
Failsafe corners, pause controls, and optional approval mode help prevent unsafe actions.
Absolute and relative positioning, smooth movement, left/right/middle/double click, drag-and-drop, and scrolling.
Fast text typing with configurable WPM, hotkeys, special keys, and hold/release control.
Full-screen and region screenshots, pixel color detection, image recognition, and multi-monitor support.
List windows, activate by title, read window info, and minimize/maximize control.
Copy text to the system clipboard and read text back from it.
Failsafe corner abort, emergency pause, bounds checking, and action logging.
Automatically fill out web forms or desktop application fields with typed input.
Control any desktop application through mouse and keyboard simulation.
Capture screenshots, detect specific UI elements, and respond to visual changes.
Drag and drop files between folders and multi-select with Ctrl+click.
Automate manual GUI testing workflows across desktop applications.
npx clawhub@latest install desktop-controlRun in your terminal, or click the Install button at the top of this page for one-click setup.
PyAutoGUI, Pillow, and PyGetWindow are required. OpenCV (opencv-python) is optional for image recognition.
Yes. It supports macOS, Linux, and Windows, though some actions may need elevated privileges.
No. A graphical desktop environment is required.
Yes. It includes a failsafe corner, pause controls, optional approval mode, and logging.
Install OpenCV (opencv-python). It enables image matching in screen operations.