RAM Consumption Verification

[Created by Claude: ab7e3fa0-6c2e-49c0-a55f-21779f70b64a 2026-01-25]

Purpose

Verify that a process's self-reported RAM consumption is accurate by:

•Measuring total system RAM with htop
•Killing the target process(es)
•Measuring total system RAM again
•Comparing the difference

Prerequisites

•htop installed
•tmux installed
•Agent must know their suffix(5) (last 5 chars of Agent ID)

⚠️ CRITICAL: Always Clean Up

You MUST clean up the tmux session after the test completes.

The tmux session is ONLY for running htop temporarily. Failure to clean up leaves orphaned sessions.

Step-by-Step Procedure

Step 1: Create tmux session with htop

Use your suffix(5) to name the session:

bash

tmux new-session -d -s ram-test-{suffix5} && \
tmux send-keys -t ram-test-{suffix5} 'htop' Enter

Example for agent with suffix 0b64a:

bash

tmux new-session -d -s ram-test-0b64a && \
tmux send-keys -t ram-test-0b64a 'htop' Enter

Step 2: Wait and capture BEFORE measurement

bash

sleep 2 && tmux capture-pane -t ram-test-{suffix5} -p | grep "^  Mem"

Record this value (e.g., 53.8G/128G)

Step 3: Kill target process(es)

Use pkill -9 for forceful termination:

bash

pkill -9 -f "{process-pattern}"

Examples:

bash

# Kill ingestors
pkill -9 -f "ingest-to-centralized-db"

# Kill a specific python script
pkill -9 -f "my_script.py"

# Kill by PID
kill -9 {pid1} {pid2}

Step 4: Wait and capture AFTER measurement

bash

sleep 1 && tmux capture-pane -t ram-test-{suffix5} -p | grep "^  Mem"

Record this value (e.g., 43.2G/128G)

Step 5: Verify processes are dead

bash

pgrep -f "{process-pattern}" || echo "Processes confirmed dead"

Step 6: Calculate and report

code

RAM Before: 53.8 GB
RAM After:  43.2 GB
Difference: 10.6 GB (this is the actual RAM used by killed processes)

Step 7: ⚠️ CLEANUP (MANDATORY)

Always run this step, even if earlier steps failed:

bash

tmux kill-session -t ram-test-{suffix5}

Complete Example Script

For an agent with suffix e274f verifying ingestor RAM:

bash

# Step 1: Create session
tmux new-session -d -s ram-test-e274f && \
tmux send-keys -t ram-test-e274f 'htop' Enter

# Step 2: Measure BEFORE
sleep 2 && tmux capture-pane -t ram-test-e274f -p | grep "^  Mem"
# Output: Mem[||||||||||||||||||||||||||||||||||||53.8G/128G]

# Step 3: Kill processes
pkill -9 -f "ingest-to-centralized-db"

# Step 4: Measure AFTER
sleep 1 && tmux capture-pane -t ram-test-e274f -p | grep "^  Mem"
# Output: Mem[||||||||||||||||||||||||||||||||||||43.2G/128G]

# Step 5: Verify dead
pgrep -f "ingest-to-centralized-db" || echo "Ingestors confirmed dead"

# Step 6: Report
echo "RAM dropped from 53.8GB to 43.2GB = 10.6GB used by ingestors"

# Step 7: CLEANUP
tmux kill-session -t ram-test-e274f

Interpreting Results

Scenario	RAM Change	Interpretation
Large drop (GBs)	-10 GB	Process was using significant RAM
Small drop (MBs)	-50 MB	Process was using minimal RAM
No change / slight increase	±0.5 GB	Process RAM was negligible; fluctuation is noise

Why htop Instead of ps/top?

•ps aux RSS values can be misleading (shared memory, copy-on-write)
•htop shows real-time system-wide memory that the kernel reports
•Measuring before/after gives ground truth of actual memory freed

Troubleshooting

htop not showing in capture

bash

# Increase sleep time
sleep 3 && tmux capture-pane -t ram-test-{suffix5} -p | grep "^  Mem"

Session already exists

bash

# Kill existing and recreate
tmux kill-session -t ram-test-{suffix5} 2>/dev/null
tmux new-session -d -s ram-test-{suffix5}

Forgot to cleanup

bash

# List all ram-test sessions
tmux ls | grep ram-test

# Kill all ram-test sessions
tmux ls | grep ram-test | cut -d: -f1 | xargs -I{} tmux kill-session -t {}