Monitoring & Logging
Document: 29-monitoring-logging.md
Version: 1.0.0
Purpose
This document defines the monitoring, logging, and observability standards used across all Rundown projects.
Monitoring ensures that applications remain healthy, available, and performant after deployment. Logging provides the information required to understand application behaviour, diagnose issues, and improve long-term reliability.
Together, they provide operational visibility into every production system.
Observability Philosophy
Applications should report their own health.
Operational issues should be detected before they significantly affect users.
Monitoring is not about collecting data—it is about providing actionable insight.
Responsibilities
The monitoring layer owns:
-
Application health
-
Uptime monitoring
-
Error reporting
-
Log collection
-
Performance metrics
-
Resource utilization
-
Alerting
-
Operational visibility
Non-Responsibilities
Monitoring should not own:
-
Business logic
-
Incident resolution
-
Infrastructure provisioning
-
Deployment decisions
Monitoring identifies problems.
Engineering teams resolve them.
Monitoring Categories
Every production application should monitor:
Availability
Examples:
-
Uptime
-
HTTP availability
-
Health endpoints
-
SSL certificate validity
Performance
Examples:
-
Response times
-
Core Web Vitals
-
API latency
-
Database query performance
-
Page load times
Infrastructure
Examples:
-
CPU utilization
-
Memory usage
-
Disk usage
-
Container health
-
Network availability
Application
Examples:
-
Application errors
-
Failed requests
-
Authentication failures
-
Background job status
Logging Principles
Logs should be:
-
Structured
-
Consistent
-
Searchable
-
Actionable
-
Relevant
Logs should explain what happened without exposing sensitive information.
Log Levels
Standard log levels:
DEBUG
INFO
WARN
ERROR
FATAL
Production environments should minimize verbose debugging logs.
Error Tracking
Production applications should capture:
-
Unhandled exceptions
-
Failed API requests
-
Application crashes
-
Background job failures
Errors should include sufficient context to reproduce the issue.
Health Checks
Critical services should expose health endpoints.
Typical checks include:
-
Application status
-
Database connectivity
-
Storage availability
-
External service connectivity
Health checks support automated recovery and deployment validation.
Alerting
Alerts should be:
-
Actionable
-
Prioritized
-
Timely
Typical alert conditions include:
-
Service unavailable
-
High error rates
-
Resource exhaustion
-
Failed deployments
-
Backup failures
Avoid excessive alerting that leads to notification fatigue.
Dashboards
Operational dashboards should present:
-
Application status
-
Performance metrics
-
Infrastructure health
-
Deployment history
-
Error trends
Dashboards should enable rapid understanding of system health.
Log Retention
Log retention policies should balance:
-
Operational requirements
-
Storage costs
-
Compliance obligations
Logs should be retained only as long as necessary.
Security
Logs should never expose:
-
Passwords
-
API keys
-
Authentication tokens
-
Personal information
-
Database credentials
Sensitive data should be masked or omitted.
Monitoring Tools
The specific monitoring platform may vary by project.
Preferred characteristics include:
-
Real-time monitoring
-
Alerting
-
Historical metrics
-
Searchable logs
-
Dashboard support
Tool selection should align with project complexity and operational requirements.
Engineering Decision
Proactive Monitoring
Status: Accepted
Decision
Every production Rundown application must include monitoring and logging sufficient to detect operational issues before they become client-reported incidents.
Reasoning
Proactive monitoring:
-
Reduces downtime.
-
Improves reliability.
-
Simplifies troubleshooting.
-
Supports continuous improvement.
-
Improves client confidence.
Monitoring Checklist
Before production launch, verify:
-
Uptime monitoring configured.
-
Health endpoints available.
-
Error logging enabled.
-
Alerts configured.
-
Infrastructure metrics available.
-
Sensitive information excluded from logs.
-
Operational dashboard reviewed.
Summary
Monitoring and logging provide continuous visibility into the health of every Rundown application.
By combining proactive monitoring, structured logging, and actionable alerting, Rundown ensures that production systems remain reliable, maintainable, and capable of supporting long-term operational excellence.
Related Documents
Related ADRs
- ADR-003: Documentation Information Architecture