WAF-COST-070 – Observability & Logging Cost Tiers

Pillar: Cost | Severity: Medium | Category: Logging Monitoring | Automatable: High

Description

Logging and observability resources MUST be organized in explicit retention tiers: Hot (7-30 days), Warm (30-90 days), Cold (90-365 days), Archive (>365 days). CloudWatch Log Groups MUST NOT have retention_in_days = 0 (infinite). Operational log groups MUST have retention_in_days ⇐ 365. DEBUG-level logging in production is prohibited without Architecture Board justification. High-volume trace sampling MUST be configured for services with >1000 req/s.

Rationale

Risk	Description
Risk	Security incidents go undetected without centralised, tamper-proof audit logs.
Risk	Forensic investigations are delayed or inconclusive when logs are missing.

Risk

Description

Risk

Security incidents go undetected without centralised, tamper-proof audit logs.

Risk

Forensic investigations are delayed or inconclusive when logs are missing.

Requirements

Logging and observability resources MUST be organized in explicit retention tiers: Hot (7-30 days), Warm (30-90 days), Cold (90-365 days), Archive (>365 days).
CloudWatch Log Groups MUST NOT have retention_in_days = 0 (infinite).
Operational log groups MUST have retention_in_days ⇐ 365.
High-volume trace sampling MUST be configured for services with >1000 req/s.

Implementation Guidance

Document the requirement in a version-controlled policy or standard.
Assign ownership and establish a review cadence (e.g., quarterly).
Implement the required configuration as code in Terraform or equivalent IaC.

Maturity Levels

Level	Name	Criteria
L1	Ad hoc – control not consistently applied	* Manual evidence exists for some workloads, but coverage is unknown.
L2	Standardised – policy defined and communicated	* A version-controlled policy, ADR, or standard describes the requirement.
L3	Integrated – automated validation in CI/CD	* Terraform checks run automatically on pull requests and report results.
L4	Auditable – metrics and exceptions tracked	* Control coverage, failures, and approved exceptions are reported quarterly.
L5	Optimised – continuous improvement based on data	* Control effectiveness is measured and iteratively improved.

Level

Name

Criteria

Ad hoc – control not consistently applied

* Manual evidence exists for some workloads, but coverage is unknown.

Standardised – policy defined and communicated

* A version-controlled policy, ADR, or standard describes the requirement.

Integrated – automated validation in CI/CD

* Terraform checks run automatically on pull requests and report results.

Auditable – metrics and exceptions tracked

* Control coverage, failures, and approved exceptions are reported quarterly.

Optimised – continuous improvement based on data

* Control effectiveness is measured and iteratively improved.

Checks

waf-cost-070.tf.aws.cloudwatch-retention-not-infinite

Engine: terraform | Provider: aws | Automated: Yes

CloudWatch Log Groups must not use infinite retention (retention_in_days = 0)

aws_cloudwatch_log_group resources must have retention_in_days set to a value > 0. The value 0 means infinite retention. Omitting retention_in_days also results in infinite retention. Both configurations constitute Architectural Cost Debt.

Scope: resource — aws_cloudwatch_log_group

Assertions: * retention_in_days attribute exists — retention_in_days must be explicitly set. Missing value defaults to infinite retention. * retention_in_days greater than or equal 1 — retention_in_days must be >= 1. Value 0 means infinite retention (cost debt).

Remediation: Set retention_in_days on all aws_cloudwatch_log_group resources. Recommended values by tier: operational=30, access=90, security-audit=365. Compliance-driven long retention: use S3 export + lifecycle instead of CloudWatch.

Compliant

resource "aws_cloudwatch_log_group" "app_logs" {
  name              = "/app/production/payment-service"
  retention_in_days = 30
  tags = merge(module.mandatory_tags.tags, {
    log-tier = "hot"
    log-type = "operational"
  })
}

Non-Compliant

resource "aws_cloudwatch_log_group" "app_logs" {
  name = "/app/production/payment-service"
  # retention_in_days omitted = infinite retention = violation
}

waf-cost-070.tf.aws.cloudwatch-operational-retention-max-365

Engine: terraform | Provider: aws | Automated: Yes

Operational CloudWatch Log Groups should have retention ⇐ 365 days

CloudWatch Log Groups tagged as operational (log-type = operational or log-tier = hot/warm) should have retention_in_days ⇐ 365. Long-term compliance logs should be exported to S3 with lifecycle policies, not stored in CloudWatch indefinitely.

Scope: resource — aws_cloudwatch_log_group

Assertions: * retention_in_days less than or equal 365 — Operational log groups should have retention ⇐ 365 days. Use S3 export for long-term retention.

Remediation: For operational logs needing > 365 days retention: configure CloudWatch Logs subscription to export to S3, then apply S3 lifecycle policies for long-term archive. This is significantly cheaper than retaining in CloudWatch.

Compliant

resource "aws_cloudwatch_log_group" "audit_logs" {
  name              = "/app/production/audit"
  retention_in_days = 90  # Within CloudWatch; exported to S3 via subscription for archival
  tags = merge(module.mandatory_tags.tags, {
    log-tier = "warm"
    log-type = "audit"
  })
}
# S3 export subscription configured separately with lifecycle to Glacier

Non-Compliant

resource "aws_cloudwatch_log_group" "everything" {
  name              = "/app/production/all"
  retention_in_days = 2555  # 7 years in CloudWatch = very expensive
  # Should be: 90d in CloudWatch, then S3 export with Glacier lifecycle
}

waf-cost-070.tf.azurerm.log-analytics-retention-explicit

Engine: terraform | Provider: azurerm | Automated: Yes

Azure Log Analytics workspaces must have explicit retention days configured

Every azurerm_log_analytics_workspace resource must define retention_in_days with a finite value. Omitting the value or setting it to 0 results in indefinite retention and unbounded observability cost.

Scope: resource — azurerm_log_analytics_workspace

Assertions: * retention_in_days attribute exists — Azure Log Analytics workspace must define retention_in_days. * retention_in_days greater than or equal 1 — retention_in_days must be >= 1. Use Azure Storage archive for long-term retention.

Remediation: Set retention_in_days on every azurerm_log_analytics_workspace. Use tiered retention: operational logs 30 days, audit logs 365 days. Archive older data to Azure Storage via diagnostic settings and lifecycle policies.

Compliant

resource "azurerm_log_analytics_workspace" "ops" {
  name                = "ops-workspace"
  location            = var.location
  resource_group_name = azurerm_resource_group.main.name
  sku                 = "PerGB2018"
  retention_in_days   = 30
  tags = {
    log-tier = "hot"
  }
}

Non-Compliant

resource "azurerm_log_analytics_workspace" "ops" {
  name                = "ops-workspace"
  location            = var.location
  resource_group_name = azurerm_resource_group.main.name
  # retention_in_days omitted = default indefinite retention
}

waf-cost-070.tf.google.logging-bucket-retention

Engine: terraform | Provider: google | Automated: Yes

GCP logging buckets must have explicit retention days configured

Every google_logging_project_sink destination bucket or google_storage_bucket used for centralized logs must have a lifecycle_rule or retention_policy enforcing finite retention. Indefinite log storage is classified as Architectural Cost Debt.

Scope: resource — google_storage_bucket

Assertions: * lifecycle_rule attribute exists — GCP log bucket must define lifecycle rules with finite retention.

Remediation: Define lifecycle_rules on log destination google_storage_bucket resources to transition or delete old logs. Set retention_days on google_logging_project_bucket_config resources.

Compliant

resource "google_storage_bucket" "logs" {
  name          = "${var.project_id}-logs"
  location      = var.region
  storage_class = "STANDARD"

  lifecycle_rule {
    action {
      type = "Delete"
    }
    condition {
      age = 90
    }
  }
}

Non-Compliant

resource "google_storage_bucket" "logs" {
  name     = "${var.project_id}-logs"
  location = var.region
  # No lifecycle rules – logs accumulate indefinitely
}

waf-cost-070.tf.stackit.log-retention-explicit

Engine: terraform | Provider: stackit | Automated: Yes

STACKIT log instances must have an explicit retention period

STACKIT log instances must have an explicit retention period. This check validates the Terraform configuration uses the expected resource and attribute.

Scope: resource — stackit_logme_instance, stackit_argus_instance

Assertions: * retention_days attribute exists — STACKIT log/observability instance must define retention days.

Remediation: Review the Terraform configuration for the listed resource types and ensure the required attribute or tag is present. See the control narrative for provider-specific guidance.

Compliant

# Example resource meets the requirement.
resource "stackit_logme_instance" "example" {
  # required attribute/tag/label present
}

Non-Compliant

# Example resource does not meet the requirement.
resource "stackit_logme_instance" "example" {
  # Missing required attribute/tag/label
}

waf-cost-070.tf.ovh.log-retention-explicit

Engine: terraform | Provider: ovh | Automated: Yes

OVHcloud log destinations must have an explicit retention period

OVHcloud log destinations must have an explicit retention period. This check validates the Terraform configuration uses the expected resource and attribute.

Scope: resource — ovh_cloud_project_database_postgresql, ovh_cloud_project_kube

Assertions: * retention attribute exists — OVHcloud service must define log retention.

Remediation: Review the Terraform configuration for the listed resource types and ensure the required attribute or tag is present. See the control narrative for provider-specific guidance.

Compliant

# Example resource meets the requirement.
resource "ovh_cloud_project_database_postgresql" "example" {
  # required attribute/tag/label present
}

Non-Compliant

# Example resource does not meet the requirement.
resource "ovh_cloud_project_database_postgresql" "example" {
  # Missing required attribute/tag/label
}

waf-cost-070.tf.hcloud.log-retention-explicit

Engine: terraform | Provider: hcloud | Automated: Yes

Hetzner Cloud log destinations must have an explicit retention period

Hetzner Cloud log destinations must have an explicit retention period. This check validates the Terraform configuration uses the expected resource and attribute.

Scope: resource — hcloud_server

Assertions: * labels.log-retention-days key exists — Hetzner Cloud server must declare a log-retention-days label.

Remediation: Review the Terraform configuration for the listed resource types and ensure the required attribute or tag is present. See the control narrative for provider-specific guidance.

Compliant

# Example resource meets the requirement.
resource "hcloud_server" "example" {
  # required attribute/tag/label present
}

Non-Compliant

# Example resource does not meet the requirement.
resource "hcloud_server" "example" {
  # Missing required attribute/tag/label
}

Evidence

Kind Required Description

Kind	Required	Description
`iac`	Required	Terraform configuration implementing the control.
`governance`	Required	Policy, ADR, or standard documenting the requirement.
`config`	Optional	Cloud console or API output confirming runtime state.

iac

Required

Terraform configuration implementing the control.

governance

Required

Policy, ADR, or standard documenting the requirement.

config

Optional

Cloud console or API output confirming runtime state.

Related Controls

Framework	Controls
GDPR	* Art. 5(1)(e) – Storage limitation (logs containing personal data must not be kept indefinitely)
FinOps Foundation	* Optimize Phase – Observability Cost Management

Framework

Controls

GDPR

* Art. 5(1)(e) – Storage limitation (logs containing personal data must not be kept indefinitely)

FinOps Foundation

* Optimize Phase – Observability Cost Management