Remote Agent HQ Startup & Debug

Troubleshooting guide for starting Agent HQ services on remote machines when they fail or crash.

Trigger Keywords

Use this skill when you see:

•"AMS not responding", "SSE connection lost"
•"port 8041", "port 8037"
•"environment variable", "Missing/invalid AMS port", "Missing/invalid port"
•"vite config", "failed to load config"
•"agent-hq debug", "Agent HQ died"
•"AMS_TMUX_PORT", "AGENT_HQ_UI_PORT", "AGENT_MGMT_PORT"

Quick Reference: Remote Ports

Service	Port	Purpose
AMS (Agent Management Server)	8041	`/api/agent-top/stream` SSE endpoint
Agent HQ UI	8037	Web UI for viewing agents
Claude Shim	8787	Claude vscode-shim server
Codex Shim	9288	Codex vscode-shim server

Problem: AMS Not Responding (SSE Connection Lost)

Symptoms

•UI shows "SSE connection lost"
•"Connecting to SSE..." stuck forever
•/api/agent-top/stream returns connection reset

Diagnosis

bash

# Check if AMS is running on remote
ssh -p <PORT> <HOST> "lsof -nP -iTCP:8041 -sTCP:LISTEN"

Fix: Start AMS

bash

# Start AMS in tmux (requires PATH and env vars)
ssh -p <PORT> <HOST> "export PATH=\"/opt/homebrew/bin:\$PATH\" && \
  cd ~/AgenticProjects/agent-box-v1/services/ams-tmux && \
  AMS_TMUX_PORT=8041 ./launchers/run_ams_tmux_server.sh"

If tmux: command not found:

bash

ssh -p <PORT> <HOST> "export PATH=\"/opt/homebrew/bin:\$PATH\" && \
  AMS_TMUX_PORT=8041 \
  ~/AgenticProjects/agent-box-v1/services/ams-tmux/launchers/run_ams_tmux_server.sh"

Problem: Agent HQ UI Not Responding (port 8037)

Symptoms

•http://localhost:28037 shows "This site can't be reached"
•"ERR_CONNECTION_RESET"

Diagnosis

bash

# Check if UI is running on remote
ssh -p <PORT> <HOST> "lsof -nP -iTCP:8037 -sTCP:LISTEN"

Fix: Start Agent HQ UI

⚠️ CRITICAL: Must source .env file first!

The vite config requires these environment variables:

•AMS_TMUX_PORT (or AGENT_MGMT_PORT)
•AGENT_HQ_UI_PORT (or VITE_PORT)

bash

# Start in tmux with proper env (CORRECT WAY)
ssh -p <PORT> <HOST> "export PATH=\"/opt/homebrew/bin:\$PATH\" && \
  tmux new-session -d -s agent-hq-ui-8037 \
  -c ~/AgenticProjects/agent-box-v1/apps/agent-hq-ui \
  'bash -lc \"cd ~/AgenticProjects/agent-box-v1 && set -a && source .env && set +a && \
  cd apps/agent-hq-ui && npm run dev:web -- --port 8037 --host 127.0.0.1; exec bash\"'"

Common Error:

code

Error: Missing/invalid AMS port: set AMS_TMUX_PORT (preferred) or AGENT_MGMT_PORT
Error: Missing/invalid port: set AGENT_HQ_UI_PORT (preferred) or VITE_PORT

Cause: .env file not sourced before starting vite.

Fix: Use set -a && source .env && set +a pattern shown above.

Problem: SSH Tunnel Not Forwarding

Symptoms

•Local port not listening
•"Connection refused" when accessing localhost:28xxx

Diagnosis

bash

# Check local tunnel ports
lsof -nP -iTCP:28787 -sTCP:LISTEN
lsof -nP -iTCP:28041 -sTCP:LISTEN

# Check tunnel tmux session
tmux list-sessions | grep tunnel
tmux capture-pane -t ssh-tunnel-to-m2 -p -S -30

Fix: Restart Tunnel

bash

cd ~/swe/vscode-shims && python launchers/launch_ssh_tunnel_to_m2_tmux.py --verbose \
  --map 28787:8787,29288:9288,28037:8037,28041:8041

Environment Variables Reference

From ~/AgenticProjects/agent-box-v1/.env:

Variable	Value	Used By
`AMS_TMUX_PORT`	8041	AMS server, vite config
`AGENT_HQ_UI_PORT`	8037	Agent HQ UI vite config
`AGENT_MGMT_PORT`	(fallback for AMS_TMUX_PORT)	Legacy
`VITE_PORT`	(fallback for AGENT_HQ_UI_PORT)	Legacy

Complete Startup Sequence (Remote m2)

bash

# 1. Start AMS (port 8041)
ssh -p 11111 m2@113.161.41.101 "export PATH=\"/opt/homebrew/bin:\$PATH\" && \
  cd ~/AgenticProjects/agent-box-v1/services/ams-tmux && \
  AMS_TMUX_PORT=8041 ./launchers/run_ams_tmux_server.sh"

# 2. Start Agent HQ UI (port 8037)
ssh -p 11111 m2@113.161.41.101 "export PATH=\"/opt/homebrew/bin:\$PATH\" && \
  tmux new-session -d -s agent-hq-ui-8037 \
  -c ~/AgenticProjects/agent-box-v1/apps/agent-hq-ui \
  'bash -lc \"cd ~/AgenticProjects/agent-box-v1 && set -a && source .env && set +a && \
  cd apps/agent-hq-ui && npm run dev:web -- --port 8037 --host 127.0.0.1; exec bash\"'"

# 3. Start SSH tunnel (on LOCAL machine)
cd ~/swe/vscode-shims && python launchers/launch_ssh_tunnel_to_m2_tmux.py --verbose \
  --map 28787:8787,29288:9288,28037:8037,28041:8041

# 4. Verify
curl -sS http://127.0.0.1:28041/api/agent-top/stream | head -5  # AMS
curl -sS http://127.0.0.1:28037/ | head -5  # Agent HQ UI

Tmux Session Names

Session	Service
`ams_tmux`	AMS server (port 8041)
`agent-hq-ui-8037`	Agent HQ UI (port 8037)
`ssh-tunnel-to-m2`	SSH tunnel (local)

Related Skills

•agentic-ecosystem-remote-deployment - Full deployment guide
•config-transparency-centralized - .env philosophy