What I Do

Provide comprehensive guidance for administering RabbitMQ 4.1.4 in the crypto-scout ecosystem, including Streams, AMQP, topology configuration, and operational tasks.

Core Concepts

RabbitMQ Streams

High-throughput, append-only log messaging with:

•Non-destructive reads: Multiple consumers at different offsets
•Offset tracking: Per-consumer position management
•Retention policies: Time and size-based retention
•Protocol: Binary protocol on port 5552

AMQP 0.9.1

Traditional message queuing with:

•Queue-based: FIFO message delivery
•Acknowledgments: Explicit delivery confirmation
•Routing: Exchange-based message routing
•Protocol: Binary protocol on port 5672

Management UI

Web-based administration interface:

•Port: 15672 (localhost only in production)
•Features: Monitoring, configuration, user management
•API: RESTful API for automation

Topology Reference

Exchanges

Exchange	Type	Purpose
`crypto-scout-exchange`	direct	Main message routing
`dlx-exchange`	direct	Dead letter handling

Streams

Stream	Retention	Max Size	Segment Size	Purpose
`bybit-stream`	1 day	2GB	100MB	Bybit market data
`crypto-scout-stream`	1 day	2GB	100MB	CMC/parser data

Stream retention policy (stream-retention):

•Pattern: .*-stream$
•max-length-bytes: 2000000000 (2GB)
•max-age: 1D
•stream-max-segment-size-bytes: 100000000 (100MB)

Queues

Queue	Type	Arguments	Purpose
`collector-queue`	Classic	lazy, TTL 6h, max 2500, reject-publish, DLX	Control messages
`chatbot-queue`	Classic	lazy, TTL 6h, max 2500, reject-publish, DLX	Notifications
`dlx-queue`	Classic	lazy, max 10k, TTL 7d	Dead letters

Queue arguments:

•
collector-queue / chatbot-queue:
- •x-max-length: 2500
- •x-message-ttl: 21600000 (6 hours)
- •x-queue-mode: lazy
- •x-overflow: reject-publish
- •x-dead-letter-exchange: dlx-exchange
- •x-dead-letter-routing-key: dlx
•
dlx-queue:
- •x-queue-mode: lazy
- •max-length: 10000
- •max-age: 7D
- •overflow: reject-publish

Bindings

Source	Routing Key	Destination
crypto-scout-exchange	bybit	bybit-stream
crypto-scout-exchange	crypto-scout	crypto-scout-stream
crypto-scout-exchange	collector	collector-queue
crypto-scout-exchange	chatbot	chatbot-queue
dlx-exchange	dlx	dlx-queue

Configuration Management

definitions.json

Declarative topology configuration loaded at startup via:

ini

load_definitions = /etc/rabbitmq/definitions.json

Structure:

json

{
  "vhosts": [{"name": "/"}],
  "exchanges": [...],
  "queues": [...],
  "bindings": [...],
  "policies": [...]
}

rabbitmq.conf

Key settings:

ini

# Stream configuration
stream.listeners.tcp.1 = 0.0.0.0:5552
stream.advertised_host = crypto_scout_mq
stream.advertised_port = 5552

# Definitions
load_definitions = /etc/rabbitmq/definitions.json

# Resource limits
disk_free_limit.absolute = 2GB
vm_memory_high_watermark.relative = 0.6

# Management
management.tcp.ip = 0.0.0.0
management.rates_mode = basic

# Cluster (single node)
cluster_formation.peer_discovery_backend = classic_config
cluster_formation.classic_config.nodes.1 = rabbit@crypto_scout_mq

Environment Variables

bash

RABBITMQ_ERLANG_COOKIE=secret_cookie

Stored in secret/rabbitmq.env with 600 permissions.

CLI Commands

Node Operations

bash

# Check status
rabbitmq-diagnostics -q ping
rabbitmq-diagnostics -q status

# Start/stop (inside container)
rabbitmqctl stop
rabbitmqctl start_app

User Management

bash

# List users
rabbitmqctl list_users

# Add user using helper script
./script/rmq_user.sh -u username -p 'password' -t administrator

# Add user manually
rabbitmqctl add_user username 'password'
rabbitmqctl set_user_tags username administrator
rabbitmqctl set_permissions -p / username ".*" ".*" ".*"

# Change password
rabbitmqctl change_password username 'new_password'

# Delete user
rabbitmqctl delete_user username

Queue Operations

bash

# List queues
rabbitmqctl list_queues name messages consumers

# List queues with memory
rabbitmqctl list_queues name memory messages

# Purge queue
rabbitmqctl purge_queue queue_name

# Delete queue
rabbitmqctl delete_queue queue_name

Stream Operations

bash

# List streams
rabbitmqctl list_streams name retention_policy

# Stream consumer tracking
rabbitmqctl list_stream_consumers stream_name

# Stream publisher info
rabbitmqctl list_stream_publishers stream_name

Monitoring Commands

Health Checks

bash

# Using helper script
./script/rmq_compose.sh status

# Basic health
rabbitmq-diagnostics -q ping

# Listeners
rabbitmq-diagnostics -q listeners

# Alarms
rabbitmq-diagnostics -q alarms

# Memory
rabbitmq-diagnostics -q memory

# Overview
rabbitmq-diagnostics -q overview

Connection Monitoring

bash

# List connections
rabbitmqctl list_connections peer_host peer_port state user

# List channels
rabbitmqctl list_channels connection peer_pid user

# List consumers
rabbitmqctl list_consumers

Security Best Practices

Network Security

bash

# Verify port exposure (container network only for AMQP/Streams)
podman inspect crypto-scout-mq | grep -A 5 "PortBindings"

# Management UI localhost only
management.tcp.ip = 127.0.0.1

Access Control

bash

# Principle of least privilege
rabbitmqctl set_permissions -p / user "^bybit-.*" "^bybit-.*" "^bybit-.*"

# Remove default user
rabbitmqctl delete_user guest

Secret Management

bash

# Secure Erlang cookie
cd crypto-scout-mq
COOKIE=$(openssl rand -base64 48 | tr -dc 'A-Za-z0-9' | head -c 48)
printf "RABBITMQ_ERLANG_COOKIE=%s\n" "$COOKIE" > secret/rabbitmq.env
chmod 600 secret/rabbitmq.env

Troubleshooting

Connection Refused

bash

# Check if running
podman ps | grep crypto-scout-mq

# Check logs
podman logs crypto-scout-mq

# Verify ports
podman exec crypto-scout-mq rabbitmq-diagnostics -q listeners

Authentication Failed

bash

# Check user exists
rabbitmqctl list_users | grep username

# Reset password
rabbitmqctl change_password username 'new_password'

# Check permissions
rabbitmqctl list_permissions -p /

High Memory Usage

bash

# Memory breakdown
rabbitmq-diagnostics -q memory

# Top queues by memory
rabbitmqctl list_queues name memory | sort -k2 -n | tail

# Connections
rabbitmqctl list_connections name peer_host memory_reduction

Streams Not Working

bash

# Check plugin
rabbitmq-plugins list | grep stream

# Verify stream listeners
rabbitmq-diagnostics -q listeners | grep 5552

# Check stream existence
rabbitmqctl list_streams

Helper Scripts Reference

Script	Purpose	Usage
`./script/network.sh`	Create network	`./script/network.sh`
`./script/rmq_compose.sh`	Manage service	`./script/rmq_compose.sh up -d`
`./script/rmq_user.sh`	User management	`./script/rmq_user.sh -u admin -p 'pass' -t administrator`

When to Use Me

Use this skill when:

•Configuring RabbitMQ topology
•Managing users and permissions
•Monitoring service health
•Troubleshooting connectivity
•Understanding Streams vs AMQP
•Performing operational tasks
•Setting up security policies