Local Computer Use Skill
This skill provides local Mac desktop control - screenshots, mouse, and keyboard automation.
When to Use
- •Taking screenshots of the local Mac screen
- •Clicking at coordinates on the local screen
- •Typing text locally
- •Pressing keyboard shortcuts
- •Moving the mouse cursor
- •Scrolling
- •Checking accessibility permissions
Available Tools
- •
local_screenshot- Capture the local Mac screen - •
local_computer_action- Perform mouse/keyboard actions:- •
left_click- Left click at coordinates - •
right_click- Right click at coordinates - •
double_click- Double click - •
triple_click- Triple click - •
mouse_move- Move cursor to position - •
left_click_drag- Click and drag - •
type- Type text - •
key- Press keyboard key with modifiers - •
scroll- Scroll up/down/left/right - •
cursor_position- Get current cursor position
- •
- •
local_get_display_info- Get display resolution, scale factor - •
local_check_permissions- Check accessibility permissions status
Requirements
The local computer use feature requires Accessibility permissions in macOS: System Settings > Privacy & Security > Accessibility
Example Usage
code
User: Take a screenshot of my screen Assistant: [Uses local_screenshot] User: Click on the Dock at position 500, 900 Assistant: [Uses local_computer_action with action="left_click", x=500, y=900] User: Type "Hello World" in the current app Assistant: [Uses local_computer_action with action="type", text="Hello World"] User: Press Cmd+C to copy Assistant: [Uses local_computer_action with action="key", key="c", modifiers=["cmd"]]