Logging & Monitoring Policy
1. Purpose
Provide timely detection of anomalous events and support forensic investigations.
2. Log Sources
Source | Collected Data | Retention | Location |
EC2 & VPC Flow Logs | Instance activity, network flows | 365 days | Datadog Log Management |
Application | Structured JSON logs | 90 days hot, 365 days cold | Datadog Log Management |
3. Collection & Storage
- All logs routed to Amazon CloudWatch Logs → Kinesis Firehose → S3.
- Logs are immutable and access controlled via IAM.
- All logs are sent directly to Datadog Log Management, where retention policies enforce 90-day hot storage and 12-month cold archive in S3 (Glacier tier).
4. Alerting Rules
-
5 failed logins in 5 min per user → Better Stack Minor.
- Suspicious GeoIP login → Slack alert.
- DB queries without prepared statements flagged.
5. Reviews
Daily triage queue processed by SRE; weekly summary sent to Security.
6. Retention & Disposal
After retention period, logs are purged via lifecycle policies; cryptographic hash chain verifies no tampering.
7. Advanced Detection
Amazon Lookout for Metrics performs continuous anomaly detection on key application and infrastructure KPIs (request rate, error rate, latency). Outliers beyond 3 σ trigger Better Stack P2 incidents and auto-create Jira security tickets.
Version 1.0