win-ui-auto

Windows UI automation using .NET UI Automation and Win32 APIs. The Windows counterpart to Peekaboo (macOS).

No external dependencies — uses built-in PowerShell, .NET Framework, and Win32 APIs.

Capabilities

Feature	Peekaboo (macOS)	win-ui-auto (Windows)
See UI elements	`peekaboo see`	UI Automation tree walk
Click elements	`peekaboo click`	SetCursorPos + mouse_event
Type text	`peekaboo type`	SendKeys
Hotkeys	`peekaboo hotkey`	SendKeys with modifiers
List windows	`peekaboo list windows`	EnumWindows / UI Automation
Focus window	`peekaboo window focus`	SetForegroundWindow
Move/resize	`peekaboo window set-bounds`	MoveWindow
Launch app	`peekaboo app launch`	Start-Process
Screenshot	`peekaboo image`	win-screenshot skill

Recommended Workflow

code

1. List windows     → find target window
2. See elements     → enumerate UI tree, get element positions
3. Screenshot       → capture window (use win-screenshot skill)
4. Analyze image    → use OpenClaw image tool to understand layout
5. Click / Type     → interact with specific elements

List Windows

List all visible windows with position and size:

powershell

powershell.exe -NoProfile -Command "
Add-Type -AssemblyName UIAutomationClient
Add-Type -AssemblyName UIAutomationTypes
$root = [System.Windows.Automation.AutomationElement]::RootElement
$wc = [System.Windows.Automation.PropertyCondition]::new(
    [System.Windows.Automation.AutomationElement]::ControlTypeProperty,
    [System.Windows.Automation.ControlType]::Window)
$wins = $root.FindAll([System.Windows.Automation.TreeScope]::Children, $wc)
foreach ($w in $wins) {
    $n = $w.Current.Name; $c = $w.Current.ClassName; $r = $w.Current.BoundingRectangle
    if ($n.Length -gt 0 -and -not [System.Double]::IsInfinity($r.Width)) {
        Write-Host ('{0} | class={1} | {2},{3} {4}x{5}' -f $n,$c,[math]::Round($r.X),[math]::Round($r.Y),[math]::Round($r.Width),[math]::Round($r.Height))
    }
}
"

See UI Elements

Inspect the UI Automation tree of a specific window. Returns control type, name, automation ID, and bounding rectangle for each element.

powershell

powershell.exe -NoProfile -Command "
Add-Type -AssemblyName UIAutomationClient
Add-Type -AssemblyName UIAutomationTypes
$root = [System.Windows.Automation.AutomationElement]::RootElement
$wc = [System.Windows.Automation.PropertyCondition]::new(
    [System.Windows.Automation.AutomationElement]::NameProperty, 'WINDOW_TITLE')
$win = $root.FindFirst([System.Windows.Automation.TreeScope]::Children, $wc)
if (-not $win) { Write-Host 'Window not found'; exit 1 }
$all = [System.Windows.Automation.Condition]::TrueCondition
$els = $win.FindAll([System.Windows.Automation.TreeScope]::Descendants, $all)
$i = 0
foreach ($el in $els) {
    $ct = $el.Current.ControlType.ProgrammaticName -replace 'ControlType\.',''
    $nm = $el.Current.Name
    $aid = $el.Current.AutomationId
    $r = $el.Current.BoundingRectangle
    if (-not [System.Double]::IsInfinity($r.Width) -and $r.Width -gt 0) {
        $cx = [math]::Round($r.X + $r.Width/2); $cy = [math]::Round($r.Y + $r.Height/2)
        Write-Host ('E{0} {1} | name={2} | id={3} | center={4},{5}' -f $i,$ct,$nm,$aid,$cx,$cy)
    }
    $i++
    if ($i -ge 100) { Write-Host '... (truncated at 100 elements)'; break }
}
"

Replace WINDOW_TITLE with the exact title from the window list. Each element is prefixed with an ID like E0, E1, etc. The center coordinates can be used for clicking.

Filter by Control Type

Show only buttons:

powershell

powershell.exe -NoProfile -Command "
Add-Type -AssemblyName UIAutomationClient
Add-Type -AssemblyName UIAutomationTypes
$root = [System.Windows.Automation.AutomationElement]::RootElement
$wc = [System.Windows.Automation.PropertyCondition]::new(
    [System.Windows.Automation.AutomationElement]::NameProperty, 'WINDOW_TITLE')
$win = $root.FindFirst([System.Windows.Automation.TreeScope]::Children, $wc)
if (-not $win) { Write-Host 'Window not found'; exit 1 }
$bc = [System.Windows.Automation.PropertyCondition]::new(
    [System.Windows.Automation.AutomationElement]::ControlTypeProperty,
    [System.Windows.Automation.ControlType]::Button)
$btns = $win.FindAll([System.Windows.Automation.TreeScope]::Descendants, $bc)
foreach ($b in $btns) {
    $r = $b.Current.BoundingRectangle
    if (-not [System.Double]::IsInfinity($r.Width)) {
        $cx = [math]::Round($r.X + $r.Width/2); $cy = [math]::Round($r.Y + $r.Height/2)
        Write-Host ('Button: {0} | id={1} | center={2},{3}' -f $b.Current.Name,$b.Current.AutomationId,$cx,$cy)
    }
}
"

Other control types: Edit, Text, CheckBox, RadioButton, ComboBox, List, ListItem, Menu, MenuItem, Tab, TabItem, Tree, TreeItem, Hyperlink, Image, Slider, ProgressBar.

Click

Click at specific screen coordinates:

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class Mouse {
    [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y);
    [DllImport(\"user32.dll\")] public static extern void mouse_event(uint f, int dx, int dy, int d, IntPtr e);
    public const uint DOWN = 0x02, UP = 0x04;
    public static void Click(int x, int y) {
        SetCursorPos(x, y);
        System.Threading.Thread.Sleep(50);
        mouse_event(DOWN, 0, 0, 0, IntPtr.Zero);
        mouse_event(UP, 0, 0, 0, IntPtr.Zero);
    }
}
'@
[Mouse]::Click(X, Y)
Write-Host 'Clicked at X,Y'
"

Replace X and Y with target coordinates (e.g., from the center output of the See command).

Double Click

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class Mouse {
    [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y);
    [DllImport(\"user32.dll\")] public static extern void mouse_event(uint f, int dx, int dy, int d, IntPtr e);
    public const uint DOWN = 0x02, UP = 0x04;
    public static void DoubleClick(int x, int y) {
        SetCursorPos(x, y);
        System.Threading.Thread.Sleep(50);
        mouse_event(DOWN, 0, 0, 0, IntPtr.Zero); mouse_event(UP, 0, 0, 0, IntPtr.Zero);
        System.Threading.Thread.Sleep(80);
        mouse_event(DOWN, 0, 0, 0, IntPtr.Zero); mouse_event(UP, 0, 0, 0, IntPtr.Zero);
    }
}
'@
[Mouse]::DoubleClick(X, Y)
Write-Host 'Double-clicked at X,Y'
"

Right Click

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class Mouse {
    [DllImport(\"user32.dll\")] public static extern bool SetCursorPos(int X, int Y);
    [DllImport(\"user32.dll\")] public static extern void mouse_event(uint f, int dx, int dy, int d, IntPtr e);
    public const uint RDOWN = 0x08, RUP = 0x10;
    public static void RightClick(int x, int y) {
        SetCursorPos(x, y);
        System.Threading.Thread.Sleep(50);
        mouse_event(RDOWN, 0, 0, 0, IntPtr.Zero);
        mouse_event(RUP, 0, 0, 0, IntPtr.Zero);
    }
}
'@
[Mouse]::RightClick(X, Y)
Write-Host 'Right-clicked at X,Y'
"

Type Text

Send keystrokes to the focused window:

powershell

powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('YOUR_TEXT_HERE')
"

Special characters in SendKeys:

•{ENTER} — Enter key
•{TAB} — Tab key
•{BACKSPACE} or {BS} — Backspace
•{DELETE} or {DEL} — Delete
•{ESCAPE} or {ESC} — Escape
•{UP}, {DOWN}, {LEFT}, {RIGHT} — Arrow keys
•{HOME}, {END}, {PGUP}, {PGDN} — Navigation
•{F1} through {F12} — Function keys
•+ — Shift, ^ — Ctrl, % — Alt (modifiers)

Type multi-line text

powershell

powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('Line 1{ENTER}Line 2{ENTER}Line 3')
"

Hotkeys

Send keyboard shortcuts:

powershell

# Ctrl+S (Save)
powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('^s')
"

# Ctrl+C (Copy)
powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('^c')
"

# Ctrl+V (Paste)
powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('^v')
"

# Alt+F4 (Close)
powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('%{F4}')
"

# Ctrl+Shift+T (Reopen tab)
powershell.exe -NoProfile -Command "
Add-Type -AssemblyName System.Windows.Forms
[System.Windows.Forms.SendKeys]::SendWait('^+t')
"

# Win+D (Show desktop)
powershell.exe -NoProfile -Command "
\$shell = New-Object -ComObject Shell.Application
\$shell.ToggleDesktop()
"

Focus Window

Bring a window to the foreground by title:

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class WinFocus {
    [DllImport(\"user32.dll\")] public static extern IntPtr FindWindow(string cls, string title);
    [DllImport(\"user32.dll\")] public static extern bool SetForegroundWindow(IntPtr hWnd);
    [DllImport(\"user32.dll\")] public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
}
'@
$h = [WinFocus]::FindWindow($null, 'WINDOW_TITLE')
if ($h -eq [IntPtr]::Zero) { Write-Host 'Window not found'; exit 1 }
[WinFocus]::ShowWindow($h, 9)  # SW_RESTORE
[WinFocus]::SetForegroundWindow($h)
Write-Host 'Focused: WINDOW_TITLE'
"

Move / Resize Window

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class WinMove {
    [DllImport(\"user32.dll\")] public static extern IntPtr FindWindow(string cls, string title);
    [DllImport(\"user32.dll\")] public static extern bool MoveWindow(IntPtr hWnd, int X, int Y, int W, int H, bool repaint);
}
'@
$h = [WinMove]::FindWindow($null, 'WINDOW_TITLE')
if ($h -eq [IntPtr]::Zero) { Write-Host 'Window not found'; exit 1 }
[WinMove]::MoveWindow($h, X, Y, WIDTH, HEIGHT, $true)
Write-Host 'Moved window to X,Y WIDTHxHEIGHT'
"

Minimize / Maximize / Close Window

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class WinCtl {
    [DllImport(\"user32.dll\")] public static extern IntPtr FindWindow(string cls, string title);
    [DllImport(\"user32.dll\")] public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
    [DllImport(\"user32.dll\")] public static extern bool PostMessage(IntPtr hWnd, uint Msg, IntPtr wParam, IntPtr lParam);
}
'@
$h = [WinCtl]::FindWindow($null, 'WINDOW_TITLE')
if ($h -eq [IntPtr]::Zero) { Write-Host 'Window not found'; exit 1 }

# Minimize: ShowWindow($h, 6)
# Maximize: ShowWindow($h, 3)
# Restore:  ShowWindow($h, 9)
# Close:    PostMessage($h, 0x0010, [IntPtr]::Zero, [IntPtr]::Zero)

[WinCtl]::ShowWindow($h, ACTION)
Write-Host 'Done'
"

Action codes: 6 = minimize, 3 = maximize, 9 = restore. For close, use PostMessage with 0x0010 (WM_CLOSE).

Launch Application

powershell

# By name
powershell.exe -NoProfile -Command "Start-Process 'notepad'"

# By path
powershell.exe -NoProfile -Command "Start-Process 'C:\Program Files\MyApp\app.exe'"

# With arguments
powershell.exe -NoProfile -Command "Start-Process 'code' -ArgumentList 'C:\projects\myrepo'"

# Open URL in default browser
powershell.exe -NoProfile -Command "Start-Process 'https://example.com'"

# Open file with default app
powershell.exe -NoProfile -Command "Start-Process 'C:\Users\me\doc.pdf'"

List Running Processes

powershell

powershell.exe -NoProfile -Command "
Get-Process | Where-Object { \$_.MainWindowTitle.Length -gt 0 } |
    Select-Object Id, ProcessName, MainWindowTitle | Format-Table -AutoSize
"

Quit Application

powershell

# Graceful close by process name
powershell.exe -NoProfile -Command "
Get-Process -Name 'notepad' -ErrorAction SilentlyContinue | ForEach-Object { $_.CloseMainWindow() }
"

# Force kill
powershell.exe -NoProfile -Command "Stop-Process -Name 'notepad' -Force"

Mouse Scroll

powershell

powershell.exe -NoProfile -Command "
Add-Type @'
using System;
using System.Runtime.InteropServices;
public class Scroll {
    [DllImport(\"user32.dll\")] public static extern void mouse_event(uint f, int dx, int dy, int d, IntPtr e);
    public const uint WHEEL = 0x0800;
    public static void Up(int clicks) { mouse_event(WHEEL, 0, 0, clicks * 120, IntPtr.Zero); }
    public static void Down(int clicks) { mouse_event(WHEEL, 0, 0, -clicks * 120, IntPtr.Zero); }
}
'@
# Scroll down 3 clicks
[Scroll]::Down(3)
"

Full Workflow Example

code

# 1. List windows
→ Found: "Notepad - myfile.txt" at 100,100 800x600

# 2. Focus window
→ SetForegroundWindow("Notepad - myfile.txt")

# 3. See elements
→ E5 Edit | name= | id=15 | center=500,400
→ E8 MenuItem | name=File | id= | center=120,125

# 4. Click on Edit area
→ Click(500, 400)

# 5. Type text
→ SendKeys("Hello from PCClaw!")

# 6. Save with Ctrl+S
→ SendKeys("^s")

# 7. Screenshot to verify
→ (use win-screenshot skill)

Notes

•Works on Windows 10/11 with PowerShell 5.1+.
•No external dependencies — uses .NET Framework UI Automation and Win32 API.
•UI Automation requires the target app to expose an accessibility tree (most modern apps do).
•SendKeys requires the target window to be focused. Always focus first, then type.
•SetCursorPos + mouse_event uses absolute screen coordinates.
•Some apps with elevated privileges (admin) may not respond to input from a non-elevated process.
•For complex automation flows: see → screenshot → analyze with image tool → click/type.
•Combine with win-screenshot for visual verification and win-clipboard for data transfer.
•Use the element center coordinates from the See command as click targets.
•Truncate element lists at a reasonable limit (50-100) for large windows.