WAF-AGN-070 – Agent Observability

Pillar: Agentic | Severity: Medium | Category: Logging Monitoring | Automatable: High

Description

Agent activities must be visible and measurable. Telemetry MUST be collected for all agents including tokens, latency, GPU utilization, and decision metrics. Metrics MUST be ingested into a monitoring system. Distributed tracing MUST link agent operations end-to-end.

Rationale

Risk	Description
Risk	Invisible agent activity delays detection of misuse, drift, or attacks.
Risk	Without metrics, agent behavior cannot be governed or continuously improved.

Risk

Description

Risk

Invisible agent activity delays detection of misuse, drift, or attacks.

Risk

Without metrics, agent behavior cannot be governed or continuously improved.

Requirements

Agent activities must be visible and measurable.
Telemetry MUST be collected for all agents including tokens, latency, GPU utilization, and decision metrics.
Metrics MUST be ingested into a monitoring system.
Distributed tracing MUST link agent operations end-to-end.

Implementation Guidance

Document the requirement in a version-controlled policy or standard.
Assign ownership and establish a review cadence (e.g., quarterly).
Implement the required configuration as code in Terraform or equivalent IaC.

Maturity Levels

Level	Name	Criteria
L1	Ad hoc – control not consistently applied	* Manual evidence exists for some workloads, but coverage is unknown.
L2	Standardised – policy defined and communicated	* A version-controlled policy, ADR, or standard describes the requirement.
L3	Integrated – automated validation in CI/CD	* Terraform checks run automatically on pull requests and report results.
L4	Auditable – metrics and exceptions tracked	* Control coverage, failures, and approved exceptions are reported quarterly.
L5	Optimised – continuous improvement based on data	* Control effectiveness is measured and iteratively improved.

Level

Name

Criteria

Ad hoc – control not consistently applied

* Manual evidence exists for some workloads, but coverage is unknown.

Standardised – policy defined and communicated

* A version-controlled policy, ADR, or standard describes the requirement.

Integrated – automated validation in CI/CD

* Terraform checks run automatically on pull requests and report results.

Auditable – metrics and exceptions tracked

* Control coverage, failures, and approved exceptions are reported quarterly.

Optimised – continuous improvement based on data

* Control effectiveness is measured and iteratively improved.

Checks

waf-agn-070.tf.aws.metrics-configured

Engine: terraform | Provider: aws | Automated: Yes

Agent metrics must be configured (tokens, latency, GPU)

Agent telemetry must include token counts, inference latency, GPU utilization, and error rates. Metrics must be exported to CloudWatch or a third-party monitoring system.

Scope: resource — aws_cloudwatch_metric_alarm, aws_bedrockagent

Assertions: * alarm_name not empty — Metrics alarms must be configured for agent monitoring.

Remediation: Configure Bedrock metrics export to CloudWatch. Use CloudWatch Insights to query agent metrics. Set up dashboards for token usage, latency, and error rates. Configure alarms for anomalies.

Compliant

resource "aws_cloudwatch_metric_alarm" "agent_latency" {
  alarm_name          = "agent-latency-alarm"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  metric_name         = "Latency"
  namespace           = "AWS/Bedrock"
  period              = 60
  statistic           = "Average"
  threshold           = 5000
  unit                = "Milliseconds"
  alarm_description   = "Agent latency exceeds 5 seconds"
}

resource "aws_cloudwatch_metric_alarm" "agent_tokens" {
  alarm_name          = "agent-token-alarm"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "TokenCount"
  namespace           = "AWS/Bedrock"
  period              = 300
  statistic           = "Sum"
  threshold           = 100000
  unit                = "Count"
  alarm_description   = "Agent token usage exceeds limit"
}

Non-Compliant

# No metrics configured
# WAF-AGN-070 Violation
# No visibility into agent performance

waf-agn-070.tf.aws.distributed-tracing-enabled

Engine: terraform | Provider: aws | Automated: Yes

Distributed tracing must be enabled for agent operations

Agent operations must be traced using AWS X-Ray. Traces must include agent ID, operation type, latency, input/output size, and decision path. X-Ray must link to CloudWatch Logs for full context.

Scope: resource — aws_lambda_function, aws_bedrockagent

Assertions: * environment.variables.AWS_XRAY_TRACING_NAME not empty — Distributed tracing (X-Ray) must be enabled.

Remediation: Enable X-Ray tracing for Bedrock invocations. Use AWS SDK with X-Ray integration. Instrument agent code with X-Ray SDK for custom segments. Configure trace sampling.

Compliant

resource "aws_lambda_function" "agent_traced" {
  function_name = "agent-traced"
  environment {
    variables = {
      AWS_XRAY_TRACING_NAME = "agent-service"
      AWS_XRAY_DAEMON_ADDRESS = "127.0.0.1:2000"
    }
  }
}

resource "aws_cloudwatch_log_group" "agent_trace_logs" {
  name = "/aws/lambda/agent-traced"
}

Non-Compliant

resource "aws_lambda_function" "agent_untraced" {
  function_name = "agent-untraced"
  # No X-Ray tracing
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.aws.log-integration-configured

Engine: terraform | Provider: aws | Automated: Yes

Agent logs must be integrated with monitoring system

Agent logs must be integrated with a centralized monitoring system. Logs must include agent ID, session ID, timestamp, operation type, input/output size, and result status.

Scope: resource — aws_cloudwatch_log_group, aws_cloudwatch_log_metric_filter

Assertions: * name not empty — Log integration with monitoring must be configured.

Remediation: Configure CloudWatch Logs Insights queries for agent log analysis. Set up dashboards showing agent health, error rates, and performance trends. Configure alarms for error patterns.

Compliant

resource "aws_cloudwatch_log_group" "agent_logs" {
  name = "/aws/bedrock/agent-logs"

  retention_in_days = 90
}

resource "aws_cloudwatch_log_metric_filter" "agent_errors" {
  name = "agent-error-metric"

  pattern = "{ $.level = \"error\" || $.error != null }"

  log_group_name = aws_cloudwatch_log_group.agent_logs.name

  metric_transformations {
    metric_name      = "AgentErrors"
    metric_namespace = "AgentMonitoring"
    metric_value     = "1"
  }
}

Non-Compliant

resource "aws_cloudwatch_log_group" "agent_logs" {
  name = "/aws/bedrock/agent-logs"
  # No metric filter configured
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.azurerm.app-insights-agent

Engine: terraform | Provider: azurerm | Automated: Yes

Azure Application Insights must monitor agent operations

Azure AI agent components must be monitored via Application Insights to collect latency, token usage, error rates, and distributed traces.

Scope: resource — azurerm_application_insights, azurerm_linux_function_app

Assertions: * workspace_id not empty — Application Insights must be linked to a Log Analytics workspace. * app_settings.APPINSIGHTS_INSTRUMENTATIONKEY attribute exists — Agent host must reference Application Insights instrumentation key.

Remediation: Deploy azurerm_application_insights and connect it to agent hosting services (App Service, Container Apps, Functions).

Compliant

resource "azurerm_application_insights" "agent_ai" {
  name                = "agent-insights"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  workspace_id        = azurerm_log_analytics_workspace.ai_logs.id
  application_type    = "web"
}

resource "azurerm_linux_function_app" "agent_host" {
  name                = "agent-function"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  service_plan_id     = azurerm_service_plan.agent_plan.id

  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.agent_ai.instrumentation_key
  }
}

Non-Compliant

resource "azurerm_linux_function_app" "agent_host" {
  name = "agent-function"
  # No Application Insights integration
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.google.monitoring-agent

Engine: terraform | Provider: google | Automated: Yes

GCP Cloud Monitoring must collect agent telemetry

GCP Vertex AI / Cloud Run agent services must have monitoring dashboards and alerting policies for latency, errors, and token usage.

Scope: resource — google_monitoring_alert_policy

Assertions: * conditions attribute exists — Cloud Monitoring alert policy must define conditions for agent metrics. * notification_channels attribute exists — Alert policy must route to notification channels.

Remediation: Configure google_monitoring_alert_policy resources for agent services and google_monitoring_dashboard for agent metrics.

Compliant

resource "google_monitoring_alert_policy" "agent_latency" {
  display_name = "Agent high latency"
  combiner     = "OR"

  conditions {
    display_name = "Latency > 5s"
    condition_threshold {
      filter          = "resource.type="cloud_run_revision" AND metric.type="run.googleapis.com/request_latencies""
      duration        = "60s"
      comparison      = "COMPARISON_GT"
      threshold_value = 5000
    }
  }

  notification_channels = [google_monitoring_notification_channel.team.id]
}

Non-Compliant

resource "google_cloud_run_service" "agent" {
  name     = "agent-service"
  location = "europe-west3"
  # No Cloud Monitoring alert policies configured
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.stackit.agentic-observability

Engine: terraform | Provider: stackit | Automated: Yes

Agent activities must be visible and measurable for stackit

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on stackit_project. This validates the control is tracked as code.

Scope: resource — stackit_project

Assertions: * labels key exists — stackit_project is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the stackit_project resource to show the stackit implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "stackit_project" "example" {
  labels = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "stackit_project" "example" {
  # Missing agentic-observability labels
}

waf-agn-070.tf.ovh.agentic-observability

Engine: terraform | Provider: ovh | Automated: Yes

Agent activities must be visible and measurable for ovh

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on ovh_cloud_project. This validates the control is tracked as code.

Scope: resource — ovh_cloud_project

Assertions: * tags key exists — ovh_cloud_project is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the ovh_cloud_project resource to show the ovh implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "ovh_cloud_project" "example" {
  tags = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "ovh_cloud_project" "example" {
  # Missing agentic-observability tags
}

waf-agn-070.tf.hcloud.agentic-observability

Engine: terraform | Provider: hcloud | Automated: Yes

Agent activities must be visible and measurable for hcloud

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on hcloud_server. This validates the control is tracked as code.

Scope: resource — hcloud_server

Assertions: * labels key exists — hcloud_server is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the hcloud_server resource to show the hcloud implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "hcloud_server" "example" {
  labels = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "hcloud_server" "example" {
  # Missing agentic-observability labels
}

Evidence

Kind Required Description

Kind	Required	Description
`iac`	Required	Terraform configuration implementing the control.
`governance`	Required	Policy, ADR, or standard documenting the requirement.
`config`	Optional	Cloud console or API output confirming runtime state.

iac

Required

Terraform configuration implementing the control.

governance

Required

Policy, ADR, or standard documenting the requirement.

config

Optional

Cloud console or API output confirming runtime state.

Related Controls

Framework	Controls
ISO 27001:2022	* A.8.1 – Information security roles and responsibilities * A.8.2 – Privileged access rights
ISO 27017	* CLD.9.5 – Segregation in virtual computing environments
CIS Controls v8	* CIS 13 – Content Filtering
SOC 2 Type II	* CC4.1 – Monitoring activities * CC7.1 – Infrastructure and software monitoring
NIST SP 800-53	* AU-2 – Event logging * AU-3 – Event content * SI-4 – Information system monitoring
NIST CSF 2.0	* DE.CM – Continuous monitoring * DE.DP – Data protection
GDPR	* Art. 32 – Security of processing * Art. 5(1)(f) – Integrity and confidentiality
PCI DSS v4.0	* Req 10 – Monitor and detect threats
CSA STAR	* CCM Logging and Monitoring
BSI C5:2020	* OPS-01 – Operational monitoring
TISAX	* Information security – Monitoring and logging
ANSSI SecNumCloud	* Domain – Logging and monitoring
BIO	* BIO – Logboekhouding en monitoring
ENS High	* op.exp.6 – Gestión de cambios
UK NCSC CAF	* A4 – Policy and assurance
FedRAMP	* AU-2, AU-3, SI-4 (Moderate baseline)
CMMC 2.0	* AU.L2-3.8.1 – Flow audit logs to central log management
IT-Grundschutz	* ORP.1 – Organisation
IRAP	* ISM – Logging and monitoring
CCCS PBMM	* AU-2, AU-3 – Logging and audit
MAS TRM	* Ch.9 – Change management
ISMAP	* Monitoring and logging controls
FISC	* Technical measures – Logging and monitoring

Framework

Controls

ISO 27001:2022

* A.8.1 – Information security roles and responsibilities * A.8.2 – Privileged access rights

ISO 27017

* CLD.9.5 – Segregation in virtual computing environments

CIS Controls v8

* CIS 13 – Content Filtering

SOC 2 Type II

* CC4.1 – Monitoring activities * CC7.1 – Infrastructure and software monitoring

NIST SP 800-53

* AU-2 – Event logging * AU-3 – Event content * SI-4 – Information system monitoring

NIST CSF 2.0

* DE.CM – Continuous monitoring * DE.DP – Data protection

GDPR

* Art. 32 – Security of processing * Art. 5(1)(f) – Integrity and confidentiality

PCI DSS v4.0

* Req 10 – Monitor and detect threats

CSA STAR

* CCM Logging and Monitoring

BSI C5:2020

* OPS-01 – Operational monitoring

TISAX

* Information security – Monitoring and logging

ANSSI SecNumCloud

* Domain – Logging and monitoring

BIO

* BIO – Logboekhouding en monitoring

ENS High

* op.exp.6 – Gestión de cambios

UK NCSC CAF

* A4 – Policy and assurance

FedRAMP

* AU-2, AU-3, SI-4 (Moderate baseline)

CMMC 2.0

* AU.L2-3.8.1 – Flow audit logs to central log management

IT-Grundschutz

* ORP.1 – Organisation

IRAP

* ISM – Logging and monitoring

CCCS PBMM

* AU-2, AU-3 – Logging and audit

MAS TRM

* Ch.9 – Change management

ISMAP

* Monitoring and logging controls

FISC

* Technical measures – Logging and monitoring