WAF-AGN-090 β Cost & Performance Monitoring
Description
Agent operations must be economic and performant. Token usage and compute costs MUST be monitored. Budget alerts MUST be configured to prevent cost overruns. Model selection MUST be optimized for cost-performance tradeoffs. Efficient caching and context management MUST be implemented.
Rationale
| Risk | Description |
|---|---|
Risk |
Unbounded agent compute consumption leads to exploding costs and increased carbon footprint. |
Risk |
Inefficient agent architectures miss latency requirements and waste resources. |
Requirements
-
Agent operations must be economic and performant.
-
Token usage and compute costs MUST be monitored.
-
Budget alerts MUST be configured to prevent cost overruns.
-
Model selection MUST be optimized for cost-performance tradeoffs.
-
Efficient caching and context management MUST be implemented.
Implementation Guidance
-
Document the requirement in a version-controlled policy or standard.
-
Assign ownership and establish a review cadence (e.g., quarterly).
-
Implement the required configuration as code in Terraform or equivalent IaC.
Maturity Levels
| Level | Name | Criteria |
|---|---|---|
L1 |
Ad hoc β control not consistently applied |
* Manual evidence exists for some workloads, but coverage is unknown. |
L2 |
Standardised β policy defined and communicated |
* A version-controlled policy, ADR, or standard describes the requirement. |
L3 |
Integrated β automated validation in CI/CD |
* Terraform checks run automatically on pull requests and report results. |
L4 |
Auditable β metrics and exceptions tracked |
* Control coverage, failures, and approved exceptions are reported quarterly. |
L5 |
Optimised β continuous improvement based on data |
* Control effectiveness is measured and iteratively improved. |
Checks
waf-agn-090.tf.aws.token-monitoring-configured
Engine: terraform | Provider: aws | Automated: Yes
Agent token usage must be monitored
Token usage (input, output, total) for all agents must be tracked and monitored. Usage must be exported to CloudWatch with daily aggregation. Anomalies in token usage must trigger alerts.
Scope: resource β aws_cloudwatch_metric_alarm, aws_cloudwatch_dashboard
Assertions:
* alarm_name not empty β Token usage monitoring must be configured.
Remediation: Configure CloudWatch Metrics for Bedrock token usage. Create dashboards showing token consumption by agent. Set alarms for unusual token spikes that may indicate runaway prompts or model issues.
resource "aws_cloudwatch_metric_alarm" "token_usage" {
alarm_name = "agent-token-spike"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 1
metric_name = "InputTokenCount"
namespace = "AWS/Bedrock"
period = 300
statistic = "Sum"
threshold = 50000
unit = "Count"
alarm_description = "Agent token usage anomaly detected"
}
resource "aws_cloudwatch_dashboard" "agent_dashboard" {
dashboard_name = "agent-metrics"
dashboard_body = jsonencode({
widgets = [{
type = "metric"
x = 0
y = 0
width = 12
height = 6
properties = {
metrics = [
["AWS/Bedrock", "InputTokenCount", "AgentName", "customer-agent"]
]
period = 300
stat = "Sum"
region = "us-east-1"
}
}]
})
}
# No token monitoring configured
# WAF-AGN-090 Violation
# Cannot track or control costs
waf-agn-090.tf.aws.budget-alerts-configured
Engine: terraform | Provider: aws | Automated: Yes
Budget alerts must be configured for agent costs
Cost budgets must be set for agent operations with alerts at 50%, 75%, 90%, and 100% of budget. Alerts must be sent to designated recipients before thresholds are reached.
Scope: resource β aws_budgets_budget
Assertions:
* budget_name not empty β Budget alerts must be configured for agent costs.
Remediation: Configure AWS Cost Explorer budgets for Bedrock spending. Set up budget alerts via SNS topics. Define budget periods (monthly, quarterly) and threshold percentages.
resource "aws_budgets_budget" "agent_budget" {
budget_name = "agent-monthly-budget"
budget_type = "COST"
limit_amount = "500"
limit_unit = "USD"
time_unit = "MONTHLY"
cost_types {
include_tax = true
include_subscription = true
include_recurring = true
include_other_subscription = true
include_discounts = true
use_amortized = false
use_blended = false
}
time_period_start = "2026-01-01_00:00"
time_period_end = "2026-12-31_23:59"
notification {
notification_type = "ACTUAL"
comparison_operator = "GREATER_THAN"
threshold = 50
threshold_type = "PERCENTAGE"
notification_state = "ALARM"
subscriber_email_addresses = ["team-ops@example.com"]
}
notification {
notification_type = "ACTUAL"
comparison_operator = "GREATER_THAN"
threshold = 75
threshold_type = "PERCENTAGE"
notification_state = "ALARM"
subscriber_email_addresses = ["team-ops@example.com", "finance@example.com"]
}
notification {
notification_type = "ACTUAL"
comparison_operator = "GREATER_THAN"
threshold = 90
threshold_type = "PERCENTAGE"
notification_state = "ALARM"
subscriber_email_addresses = ["team-ops@example.com", "finance@example.com"]
}
notification {
notification_type = "ACTUAL"
comparison_operator = "GREATER_THAN"
threshold = 100
threshold_type = "PERCENTAGE"
notification_state = "ALARM"
subscriber_email_addresses = ["team-ops@example.com", "finance@example.com", "management@example.com"]
}
}
# No budget alerts configured
# WAF-AGN-090 Violation
# Cost overruns not detected
waf-agn-090.tf.aws.model-optimization-configured
Engine: terraform | Provider: aws | Automated: Yes
Model selection must be optimized for cost-performance
Agents must use the most cost-effective model that meets quality requirements. Model selection should consider input/output length, latency requirements, and cost per token. Optimization should be reviewed quarterly.
Scope: resource β aws_bedrock_foundation_model
Assertions:
* model_id not empty β Model selection must be documented and optimized.
Remediation: Document model selection criteria for each agent use case. Compare Claude Instant vs Claude Sonnet vs Claude Opus based on quality requirements. Use OpenAI GPT-3.5 Turbo vs GPT-4 based on task complexity. Implement model fallback for cost management.
# Documented model selection criteria
# - Simple queries: Claude Instant (cheapest)
# - Moderate complexity: Claude Sonnet (balanced)
# - High complexity: Claude Opus (best quality)
# Review model performance quarterly
resource "aws_bedrock_foundation_model" "model_selection" {
model_id = "anthropic.claude-3-haiku-20240307-v1:0" # Cost-optimized choice
# Model selection documented in architecture decision record
}
# Using expensive models for simple tasks
# WAF-AGN-090 Violation
# Cost-performance not optimized
waf-agn-090.tf.aws.caching-configured
Engine: terraform | Provider: aws | Automated: Yes
Caching must be implemented to reduce redundant API calls
Caching must be implemented for repeated queries, common prompts, and similar inputs. Cache keys must include input hash and model identifier. Cache TTL must be configured based on data freshness requirements.
Scope: resource β aws_dynamodb_table, aws_lambda_function
Assertions:
* name not empty β Caching must be configured to reduce redundant API calls.
Remediation: Implement caching layer (ElastiCache, DynamoDB TTL) for agent responses. Cache based on input hash to handle similar queries efficiently. Configure cache invalidation based on data changes. Monitor cache hit rates.
resource "aws_dynamodb_table" "agent_cache" {
name = "agent-response-cache"
billing_mode = "PAY_PER_REQUEST"
hash_key = "cache_key"
attribute {
name = "cache_key"
type = "S"
}
ttl {
attribute_name = "expires_at"
enabled = true
}
# Cache keys based on input hash
# TTL based on data freshness needs
}
resource "aws_lambda_function" "cache_middleware" {
function_name = "agent-cache-middleware"
# Middleware checks cache before API call
}
# No caching implemented
# WAF-AGN-090 Violation
# All queries hit API directly
waf-agn-090.tf.azurerm.budget-alert-agent
Engine: terraform | Provider: azurerm | Automated: Yes
Azure Cost Management budget alerts must be configured for AI agent spending
Azure Cost Management budgets must monitor AI service spending (OpenAI, Cognitive Services) with alerts at 75%, 90%, and 100% thresholds.
Scope: resource β azurerm_consumption_budget_subscription, azurerm_consumption_budget_resource_group
Assertions:
* notification attribute exists β Budget must have notification thresholds configured.
* amount greater than 0 β Budget must define a positive spending limit.
Remediation: Configure azurerm_consumption_budget_subscription or azurerm_consumption_budget_resource_group with filters for AI service resource groups and alert email contacts.
resource "azurerm_consumption_budget_subscription" "agent_budget" {
name = "agent-spend-budget"
subscription_id = data.azurerm_subscription.current.id
amount = 1000
time_grain = "Monthly"
time_period {
start_date = "2026-01-01T00:00:00Z"
end_date = "2026-12-31T23:59:59Z"
}
notification {
enabled = true
threshold = 75
operator = "EqualTo"
contact_emails = ["team-ops@example.com"]
}
notification {
enabled = true
threshold = 90
operator = "EqualTo"
contact_emails = ["team-ops@example.com", "finance@example.com"]
}
}
resource "azurerm_cognitive_account" "ai" {
name = "ai-services"
# No spending budget or alert configured
# WAF-AGN-090 Violation
}
waf-agn-090.tf.google.budget-alert-agent
Engine: terraform | Provider: google | Automated: Yes
GCP budgets must be configured for agent cost control
GCP budgets must monitor Vertex AI, Cloud Run, and Cloud Logging spending for agent operations with threshold alerts.
Scope: resource β google_billing_budget
Assertions:
* threshold_rules attribute exists β Billing budget must define threshold alert rules.
* amount attribute exists β Billing budget must define a spending amount.
Remediation: Configure google_billing_budget with filters for Vertex AI / Cloud Run services and alert pubsub/email channels.
resource "google_billing_budget" "agent_budget" {
billing_account = data.google_billing_account.account.id
display_name = "Agent Operations Budget"
amount {
specified_amount {
currency_code = "EUR"
units = "1000"
}
}
threshold_rules {
threshold_percent = 75
}
threshold_rules {
threshold_percent = 90
}
all_updates_rule {
monitoring_notification_channels = [google_monitoring_notification_channel.agent_budget.id]
}
}
resource "google_cloud_run_service" "agent" {
name = "agent-service"
# No billing budget configured
# WAF-AGN-090 Violation
}
waf-agn-090.tf.stackit.agentic-efficiency-reviewed
Engine: terraform | Provider: stackit | Automated: Yes
Agent operations must be economic and performant for stackit
Agent operations must be economic and performant must be documented in IaC via a agentic-efficiency-reviewed tag/label on stackit_project. This validates the control is tracked as code.
Scope: resource β stackit_project
Assertions:
* labels key exists β stackit_project is missing the agentic-efficiency-reviewed tag/label.
Remediation: Add a agentic-efficiency-reviewed tag/label to the stackit_project resource to show the stackit implementation of WAF-AGN-090 is tracked in IaC.
resource "stackit_project" "example" {
labels = {
agentic-efficiency-reviewed = "true"
}
}
resource "stackit_project" "example" {
# Missing agentic-efficiency-reviewed labels
}
waf-agn-090.tf.ovh.agentic-efficiency-reviewed
Engine: terraform | Provider: ovh | Automated: Yes
Agent operations must be economic and performant for ovh
Agent operations must be economic and performant must be documented in IaC via a agentic-efficiency-reviewed tag/label on ovh_cloud_project. This validates the control is tracked as code.
Scope: resource β ovh_cloud_project
Assertions:
* tags key exists β ovh_cloud_project is missing the agentic-efficiency-reviewed tag/label.
Remediation: Add a agentic-efficiency-reviewed tag/label to the ovh_cloud_project resource to show the ovh implementation of WAF-AGN-090 is tracked in IaC.
resource "ovh_cloud_project" "example" {
tags = {
agentic-efficiency-reviewed = "true"
}
}
resource "ovh_cloud_project" "example" {
# Missing agentic-efficiency-reviewed tags
}
waf-agn-090.tf.hcloud.agentic-efficiency-reviewed
Engine: terraform | Provider: hcloud | Automated: Yes
Agent operations must be economic and performant for hcloud
Agent operations must be economic and performant must be documented in IaC via a agentic-efficiency-reviewed tag/label on hcloud_server. This validates the control is tracked as code.
Scope: resource β hcloud_server
Assertions:
* labels key exists β hcloud_server is missing the agentic-efficiency-reviewed tag/label.
Remediation: Add a agentic-efficiency-reviewed tag/label to the hcloud_server resource to show the hcloud implementation of WAF-AGN-090 is tracked in IaC.
resource "hcloud_server" "example" {
labels = {
agentic-efficiency-reviewed = "true"
}
}
resource "hcloud_server" "example" {
# Missing agentic-efficiency-reviewed labels
}
Evidence
| Kind | Required | Description |
|---|---|---|
|
Required |
Terraform configuration implementing the control. |
|
Required |
Policy, ADR, or standard documenting the requirement. |
|
Optional |
Cloud console or API output confirming runtime state. |
Related Controls
| Framework | Controls |
|---|---|
ISO 27001:2022 |
* A.8.2 β Privileged access rights * A.12.1 β Control of operational software |
ISO 27017 |
* CLD.9.5 β Segregation in virtual computing environments |
CIS Controls v8 |
* CIS 10 β Data Accuracy |
SOC 2 Type II |
* CC4.1 β Monitoring activities * CC7.1 β Infrastructure and software monitoring |
NIST SP 800-53 |
* SC-5 β Resource availability * SC-39 β Process isolation |
NIST CSF 2.0 |
* DE.CM β Continuous monitoring * DE.AE β Anomalies and events |
GDPR |
* Art. 32 β Security of processing * Art. 5(1)(f) β Integrity and confidentiality |
PCI DSS v4.0 |
* Req 10 β Monitor and detect threats |
CSA STAR |
* CCM Logging and Monitoring |
BSI C5:2020 |
* OPS-01 β Operational monitoring |
TISAX |
* Information security β Monitoring and logging |
ANSSI SecNumCloud |
* Domain β Logging and monitoring |
BIO |
* BIO β Logboekhouding en monitoring |
ENS High |
* op.exp.6 β GestiΓ³n de cambios |
UK NCSC CAF |
* A4 β Policy and assurance |
FedRAMP |
* SC-5, SC-39 (Moderate baseline) |
CMMC 2.0 |
* SC.L2-3.8.1 β Limit information system use |
IT-Grundschutz |
* ORP.1 β Organisation |
IRAP |
* ISM β Logging and monitoring |
CCCS PBMM |
* SC-5 β Resource availability |
MAS TRM |
* Ch.9 β Change management |
ISMAP |
* Monitoring and logging controls |
FISC |
* Technical measures β Logging and monitoring |