Observability and Resilience¶
Observability lets operators infer system state from outputs. Resilience defines how the system behaves when dependencies slow down, fail, or recover.
Signals¶
- logs record discrete events with structured context;
- metrics aggregate numerical behavior over time;
- traces connect work across component and service boundaries.
Spring Boot Actuator integrates with Micrometer for structured logs, metrics, and traces through observations and instrumentation. Use low-cardinality metric tags; an unbounded identifier such as a user or request ID creates a time series per value and can overwhelm storage.
Service indicators¶
Begin with user-visible behavior: request rate, error rate, latency distribution, and saturation. Percentiles reveal tails that averages hide. Separate readiness from liveness: a temporarily unavailable dependency may affect whether traffic should be sent without implying that the process must be restarted.
Resilience mechanisms¶
| Mechanism | Purpose | Main risk |
|---|---|---|
| Timeout/deadline | Bound waiting | Too short causes false failure |
| Retry | Reattempt transient work | Amplifies load and duplicates effects |
| Exponential backoff + jitter | Spread retries | Increases completion latency |
| Circuit breaker | Stop calls likely to fail | Stale or premature open state |
| Bulkhead | Isolate resource exhaustion | Underutilized reserved capacity |
| Rate limit | Protect capacity/fairness | Rejects legitimate bursts |
Retries, backoff, and deadlines require an idempotency analysis. Multiplying retries across layers can create a retry storm. The separate guide to rate limits, circuit breakers, and bulkheads explains their state and interactions. A circuit breaker is not a substitute for timeouts, and a fallback must be semantically honest rather than returning plausible incorrect data.
Operational safety¶
Do not expose sensitive actuator endpoints publicly. Redact credentials and personal data from logs and attributes. Trace context must be propagated through supported mechanisms, especially across asynchronous boundaries.
See the official observability and metrics documentation.