OpenTelemetry Demo Architecture

Quick reference for the OpenTelemetry Demo microservices system. Focus on service dependencies, critical paths, and common failure patterns.

For detailed observability queries: See the Observability Query Guides section below for comprehensive metrics, traces, and logs references.

Service Dependency Matrix

Critical Service Paths

User Request Flow:

Internet → frontend-proxy:ENVOY_PORT → frontend → [cart, product-catalog, recommendation, checkout]

Checkout/Order Flow:

checkout → cart (gRPC)
        → currency (gRPC)
        → payment (gRPC)
        → product-catalog (gRPC)
        → shipping (HTTP) → quote (HTTP)
        → email (HTTP)
        → kafka → [accounting, fraud-detection]

Telemetry Flow:

All services → otel-collector:4317(gRPC)/4318(HTTP) → [jaeger, prometheus, tempo, opensearch]
                                                     → grafana (visualization)

Observability Stack Details

Metrics Pipeline

Services → otel-collector (OTLP) → prometheus (scrape/remote-write) → grafana

Traces Pipeline

Services → otel-collector (OTLP) → [jaeger, tempo] → grafana

Logs Pipeline

Services → docker json-file → alloy → loki → grafana

Log Configuration

All services use json-file driver:

• •Max size: 5M
• •Max files: 2
• •Auto-rotation

Observability Query Guides

Detailed reference guides for querying observability data:

Metrics Guide

See metrics-guide.md for:

• •Complete Prometheus metrics catalog - All 260+ available metrics organized by category
• •Service-specific metrics - Application metrics for each demo service
• •HTTP/RPC metrics - Request rates, latencies, error rates
• •Runtime metrics - Go, .NET, JVM, Node.js, Python runtime instrumentation
• •Container metrics - CPU, memory, network, disk I/O
• •Feature flag metrics - Flag evaluation and impression tracking
• •OTEL Collector metrics - Pipeline health and throughput
• •Common label patterns - Service identification, filtering, aggregation
• •Query patterns - Request rates, error rates, percentiles, top-N

When to use: Building dashboards, investigating performance issues, analyzing resource utilization, monitoring service health.

Traces Guide

See traces-guide.md for:

• •TraceQL syntax and patterns - Complete query language reference
• •Resource attributes - Service, host, process, runtime identification (27 attributes)
• •Span attributes - HTTP, gRPC, database, application-specific (150+ attributes)
• •Event attributes - Exceptions, feature flags, business events
• •Intrinsic attributes - Duration, status, kind, instrumentation
• •Service-specific attributes - Cart, product, payment, shipping, ad service patterns
• •Common query patterns - Error investigation, performance analysis, feature flag impact

When to use: Debugging distributed transactions, investigating latency issues, understanding service dependencies, tracing business transactions.

Logs Guide

See logs-guide.md for:

• •LogQL syntax and patterns - Complete query language reference
• •Available labels - service_name, container, project (5 labels)
• •Service identification - All 16 application and infrastructure services
• •Common query patterns - Errors, HTTP requests, performance, business events
• •Log parsing - JSON parsing, pattern extraction, label filtering
• •Aggregations - Count, rate, percentiles, error percentages
• •Multi-service analysis - Cross-service errors, communication tracing

When to use: Investigating errors, debugging application logic, analyzing request patterns, root cause analysis.