Java Structured Logging (JSON/logfmt + Correlation + Safety)

Scope

In scope

• •Structured log format: JSON (preferred) or logfmt.
• •Field schema: consistent keys across services.
• •Correlation: traceId, spanId, requestId, correlationId.
• •Log levels policy, sampling, and noise control.
• •Redaction rules for secrets/PII.
• •Minimal integration patterns for HTTP entrypoints and background jobs.

Out of scope

• •Choosing a vendor-specific logging pipeline (ELK/Datadog/etc).
• •Full SIEM design.

Goals

• •Make logs queryable and joinable with metrics/traces.
• •Make incident response faster by standardizing fields and policies.
• •Prevent accidental leakage of secrets/PII.

Format and schema

Preferred format: JSON

Every log record should be a single JSON object with stable keys.

Recommended baseline fields

• •timestamp (ISO-8601)
• •level (TRACE/DEBUG/INFO/WARN/ERROR)
• •logger (class/category)
• •thread
• •message
• •service.name
• •service.version
• •deployment.environment (dev/staging/prod)
• •traceId (if tracing enabled)
• •spanId (if tracing enabled)
• •requestId (if available)
• •correlationId (if you use a custom correlation id)
• •http.method, http.path, http.status (for request logs)
• •event.name (semantic event label; stable)
• •error.type, error.message (safe), error.stack (internal-only, never to clients)

Field stability rules

• •Do not rename fields without migration plan.
• •Prefer adding new fields rather than changing meaning.
• •Avoid high-cardinality fields unless explicitly needed:
  • •Never log raw user identifiers; hash or redact.
  • •Never log full request/response bodies by default.

Correlation strategy

What identifiers to use

1. •Prefer traceId/spanId (from tracing context).
2. •Also include a requestId (gateway id or generated at ingress).
3. •For async flows, propagate correlation through message headers:
   • •include traceparent (W3C) and optional correlationId.

How to implement (Java pattern)

• •Use MDC (Mapped Diagnostic Context) to attach correlation fields per thread/request.
• •At request entry:
  • •extract incoming traceparent / requestId if present
  • •otherwise generate requestId
  • •set MDC keys: traceId, spanId, requestId
• •Ensure MDC cleanup at request end.

Async and thread hops

• •MDC is thread-local by default. For executors:
  • •wrap Runnable/Callable to copy MDC context
  • •or use framework/otel context propagation when available.

Logging policy (levels)

Default guidance

• •INFO: business milestones (state transitions), start/stop, essential events.
• •WARN: recoverable anomalies, retries, degraded mode.
• •ERROR: failures requiring attention, request failed.
• •DEBUG/TRACE: development-only, guarded by config; never enable globally in prod.

Anti-pattern: "log everything at INFO"

This creates noise, cost, and hides signals.

Security and privacy

Redaction rules (must-have)

• •Redact credentials: Authorization headers, tokens, API keys.
• •Redact secrets: passwords, private keys, session cookies.
• •Redact or hash PII: email, phone, address, national id.
• •Never log full JWTs; at most log token fingerprint (hash).

Safe exception logging

• •Log error class and sanitized message.
• •Store full stack traces only in internal logs with restricted access, or sample them.

Sampling and rate limiting

• •For high-volume endpoints:
  • •sample INFO request logs (e.g., 1%)
  • •always keep WARN/ERROR
• •Rate limit repetitive error logs to avoid log storms.

Standard log events (recommended)

• •event.name=request.received
• •event.name=request.completed
• •event.name=dependency.call
• •event.name=dependency.failed
• •event.name=job.started
• •event.name=job.completed
• •event.name=security.auth.failed (without leaking details)

Outputs / artifacts

• •docs/logging.md (schema + policies)
• •logging/log-schema.json (optional JSON schema)
• •logging/redaction-rules.md
• •Code changes:
  • •ingress filter/middleware for MDC
  • •JSON encoder configuration
  • •unit tests for redaction and MDC cleanup

Definition of Done (DoD)

• • Logs are structured (JSON/logfmt) and consistent.
• • Correlation identifiers present for requests and background jobs.
• • Redaction rules applied and tested.
• • Sampling/rate-limiting strategy documented.
• • No secret/PII leakage found in test logs.

Guardrails (What NOT to do)

• •Never log raw secrets or tokens.
• •Never dump full request/response by default.
• •Avoid unbounded string concatenation in hot paths (use lazy logging).
• •Avoid logging in tight loops without rate limiting.

Common failure modes & fixes

• •Symptom: cannot correlate logs with traces -> Fix: include traceId and propagate context; add MDC.
• •Symptom: log storms during incidents -> Fix: add sampling + rate limiting; reduce noisy INFO logs.
• •Symptom: leaked secrets -> Fix: redaction filters; code review checklist; scanners.

References (see references/)

• •references/owasp-logging-rules.md
• •references/log-schema-fields.md
• •references/mdc-propagation.md