Introduction to SQS Monitoring
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that decouples and scales microservices, distributed systems, and serverless applications. However, simply creating a queue and sending messages to it is not enough — you need comprehensive monitoring to ensure your messaging infrastructure remains healthy, performant, and cost-effective. Monitoring SQS involves tracking key metrics, configuring alarms to alert on anomalous conditions, and building dashboards that provide at-a-glance visibility into queue behavior.
This tutorial covers everything you need to know about monitoring SQS queues: the built-in CloudWatch metrics available, how to interpret them, how to set up alarms for critical scenarios, and how to build informative dashboards. You'll also learn best practices drawn from real-world production experience.
Why SQS Monitoring Matters
Without proper monitoring, SQS issues can cascade silently through your architecture. A backed-up queue might mean orders are not being processed, customers are not receiving notifications, or critical data pipelines are stalled. Monitoring helps you:
- Detect consumer failures early — when the
ApproximateAgeOfOldestMessagespikes, messages are sitting idle, indicating a consumer problem. - Prevent message loss — Dead-letter queue (DLQ) monitoring ensures poison messages don't vanish unnoticed.
- Optimize throughput — By tracking
NumberOfMessagesSentandNumberOfMessagesReceived, you can right-size consumer capacity. - Control costs — Empty queue polling generates charges; monitoring
NumberOfEmptyReceiveshelps you tune polling strategies. - Meet SLA commitments — Alarms on processing latency let you uphold delivery guarantees.
Key SQS Metrics in CloudWatch
Every SQS queue automatically publishes metrics to CloudWatch at one-minute intervals (if the queue has activity). These metrics fall into several logical categories. Understanding each one is essential before building alarms and dashboards.
Visibility and Processing Metrics
- ApproximateAgeOfOldestMessage — The age (in seconds) of the oldest message in the queue that has not been deleted. This is arguably the most important metric; a rising value indicates consumers are falling behind or have failed.
- ApproximateNumberOfMessagesVisible — The number of messages available for delivery. Useful for understanding queue depth and backlog.
- ApproximateNumberOfMessagesDelayed — Messages that are delayed and not yet available for processing.
- ApproximateNumberOfMessagesNotVisible — Messages in flight (received by consumers but not yet deleted or returned). High in-flight counts can indicate slow processing.
Throughput Metrics
- NumberOfMessagesSent — Total messages pushed to the queue (including those sent to a DLQ).
- NumberOfMessagesReceived — Total messages received by consumers.
- NumberOfMessagesDeleted — Messages successfully processed and removed from the queue.
- NumberOfEmptyReceives — The number of
ReceiveMessageAPI calls that returned zero messages. High values indicate inefficient polling and wasted money.
Error Metrics
- NumberOfMessagesFailed — Applicable only to FIFO queues; counts messages that failed to be processed.
- SentMessageSize / ReceivedMessageSize — Average size of messages, useful for tracking payload trends and approaching the 256 KB limit.
Setting Up CloudWatch Alarms
CloudWatch alarms watch a single metric over a defined time window and trigger an action — typically an SNS notification — when a threshold is breached. Below are the most critical alarms you should configure for any production SQS queue.
Alarm 1: Oldest Message Age Exceeds Threshold
This alarm detects when messages are not being processed in a timely manner. A common threshold is 300 seconds (5 minutes), but you should tune this based on your application's SLA.
# AWS CLI: Create alarm for ApproximateAgeOfOldestMessage
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-oldest-message-age-critical" \
--alarm-description "Alert when oldest message exceeds 5 minutes" \
--namespace AWS/SQS \
--metric-name ApproximateAgeOfOldestMessage \
--dimensions Name=QueueName,Value=MyQueue \
--statistic Maximum \
--period 60 \
--evaluation-periods 5 \
--threshold 300 \
--comparison-operator GreaterThanThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts \
--treat-missing-data ignore
The key parameters explained:
- period: 60 — Evaluates the metric every 60 seconds.
- evaluation-periods: 5 — Requires 5 consecutive data points above threshold before triggering, preventing false positives from transient spikes.
- statistic: Maximum — Uses the max age across the evaluation window, ensuring you catch the worst-case message.
Alarm 2: High Visible Message Count (Backlog)
Detects when the queue is filling up faster than consumers can drain it, indicating a throughput mismatch.
# AWS CLI: Alarm for queue depth
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-high-visible-messages" \
--alarm-description "Queue backlog exceeds 1000 messages" \
--namespace AWS/SQS \
--metric-name ApproximateNumberOfMessagesVisible \
--dimensions Name=QueueName,Value=MyQueue \
--statistic Average \
--period 300 \
--evaluation-periods 3 \
--threshold 1000 \
--comparison-operator GreaterThanThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts
Alarm 3: Dead-Letter Queue Receiving Messages
When messages land in a DLQ, it means they've exceeded the max receive count and something is fundamentally broken. Even a single message in a DLQ should trigger an alert.
# AWS CLI: Alarm for DLQ activity
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-dlq-message-detected" \
--alarm-description "Messages have been moved to the dead-letter queue" \
--namespace AWS/SQS \
--metric-name ApproximateNumberOfMessagesVisible \
--dimensions Name=QueueName,Value=MyDLQ \
--statistic Sum \
--period 60 \
--evaluation-periods 1 \
--threshold 1 \
--comparison-operator GreaterThanOrEqualToThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-critical
Alarm 4: High Empty Receive Rate
Excessive empty receives mean you're polling a queue that has no messages, wasting both money and CPU cycles on consumers. This is especially relevant for long-polling consumers that still get empty responses during idle periods.
# AWS CLI: Alarm for inefficient polling
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-high-empty-receives" \
--alarm-description "Empty receive rate exceeds 10 per minute" \
--namespace AWS/SQS \
--metric-name NumberOfEmptyReceives \
--dimensions Name=QueueName,Value=MyQueue \
--statistic Sum \
--period 60 \
--evaluation-periods 5 \
--threshold 10 \
--comparison-operator GreaterThanThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts
Using CloudFormation to Deploy Alarms
For infrastructure-as-code teams, defining alarms alongside the queue keeps configuration synchronized and auditable.
# CloudFormation snippet: SQS queue + alarm
Resources:
ProcessingQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: order-processing-queue
VisibilityTimeout: 300
RedrivePolicy:
deadLetterTargetArn: !GetAtt DeadLetterQueue.Arn
maxReceiveCount: 3
DeadLetterQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: order-processing-dlq
OldestMessageAgeAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub "${ProcessingQueue.QueueName}-oldest-message-age"
AlarmDescription: "Oldest message in queue exceeds 10 minutes"
Namespace: AWS/SQS
MetricName: ApproximateAgeOfOldestMessage
Dimensions:
- Name: QueueName
Value: !GetAtt ProcessingQueue.QueueName
Statistic: Maximum
Period: 60
EvaluationPeriods: 5
Threshold: 600
ComparisonOperator: GreaterThanThreshold
AlarmActions:
- !Ref AlertTopicArn
TreatMissingData: ignore
AlertTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: sqs-alerts
Subscription:
- Endpoint: ops@example.com
Protocol: email
Terraform Equivalent
For teams using Terraform, here's the same alarm configuration:
# Terraform: SQS queue with CloudWatch alarm
resource "aws_sqs_queue" "processing" {
name = "order-processing-queue"
visibility_timeout_seconds = 300
redrive_policy = jsonencode({
deadLetterTargetArn = aws_sqs_queue.dlq.arn
maxReceiveCount = 3
})
}
resource "aws_sqs_queue" "dlq" {
name = "order-processing-dlq"
}
resource "aws_sns_topic" "alerts" {
name = "sqs-alerts"
}
resource "aws_sns_topic_subscription" "email" {
topic_arn = aws_sns_topic.alerts.arn
protocol = "email"
endpoint = "ops@example.com"
}
resource "aws_cloudwatch_metric_alarm" "oldest_message_age" {
alarm_name = "order-processing-queue-oldest-message-age"
alarm_description = "Oldest message exceeds 10 minutes"
namespace = "AWS/SQS"
metric_name = "ApproximateAgeOfOldestMessage"
dimensions = {
QueueName = aws_sqs_queue.processing.name
}
statistic = "Maximum"
period = 60
evaluation_periods = 5
threshold = 600
comparison_operator = "GreaterThanThreshold"
alarm_actions = [aws_sns_topic.alerts.arn]
treat_missing_data = "ignore"
}
Building CloudWatch Dashboards
While alarms notify you of problems, dashboards give you continuous visibility into queue health. A well-designed dashboard lets you spot trends, investigate anomalies, and verify that systems are operating normally without digging through raw metrics.
Creating a Dashboard via the Console
In the AWS Console, navigate to CloudWatch → Dashboards → Create dashboard. Give it a name like "SQS-Overview". Add widgets for each queue you want to monitor. A typical layout includes:
- A line graph showing ApproximateNumberOfMessagesVisible (queue depth over time)
- A line graph showing NumberOfMessagesSent, NumberOfMessagesReceived, and NumberOfMessagesDeleted overlaid (throughput health)
- A single-value widget showing the current ApproximateAgeOfOldestMessage
- A line graph of NumberOfEmptyReceives (polling efficiency)
Creating a Dashboard Programmatically
You can define dashboards as JSON and deploy them via CLI or Infrastructure as Code. Here's a complete dashboard JSON that monitors two queues — a primary queue and its DLQ:
# dashboard.json - Complete SQS monitoring dashboard
{
"widgets": [
{
"type": "metric",
"x": 0,
"y": 0,
"width": 12,
"height": 6,
"properties": {
"metrics": [
["AWS/SQS", "ApproximateNumberOfMessagesVisible", "QueueName", "order-processing-queue", { "stat": "Average", "label": "Visible Messages" }],
["AWS/SQS", "ApproximateNumberOfMessagesNotVisible", "QueueName", "order-processing-queue", { "stat": "Average", "label": "In-Flight Messages" }]
],
"view": "timeSeries",
"stacked": false,
"region": "us-east-1",
"title": "Queue Depth - Order Processing",
"period": 60
}
},
{
"type": "metric",
"x": 12,
"y": 0,
"width": 12,
"height": 6,
"properties": {
"metrics": [
["AWS/SQS", "NumberOfMessagesSent", "QueueName", "order-processing-queue", { "stat": "Sum", "label": "Sent" }],
["AWS/SQS", "NumberOfMessagesReceived", "QueueName", "order-processing-queue", { "stat": "Sum", "label": "Received" }],
["AWS/SQS", "NumberOfMessagesDeleted", "QueueName", "order-processing-queue", { "stat": "Sum", "label": "Deleted" }]
],
"view": "timeSeries",
"stacked": false,
"region": "us-east-1",
"title": "Throughput - Order Processing",
"period": 300
}
},
{
"type": "metric",
"x": 0,
"y": 6,
"width": 6,
"height": 6,
"properties": {
"metrics": [
["AWS/SQS", "ApproximateAgeOfOldestMessage", "QueueName", "order-processing-queue", { "stat": "Maximum", "label": "Oldest Message Age (seconds)" }]
],
"view": "timeSeries",
"stacked": false,
"region": "us-east-1",
"title": "Message Age - Order Processing",
"period": 60
}
},
{
"type": "metric",
"x": 6,
"y": 6,
"width": 6,
"height": 6,
"properties": {
"metrics": [
["AWS/SQS", "NumberOfEmptyReceives", "QueueName", "order-processing-queue", { "stat": "Sum", "label": "Empty Receives" }]
],
"view": "timeSeries",
"stacked": false,
"region": "us-east-1",
"title": "Empty Receives - Order Processing",
"period": 300
}
},
{
"type": "metric",
"x": 12,
"y": 6,
"width": 12,
"height": 6,
"properties": {
"metrics": [
["AWS/SQS", "ApproximateNumberOfMessagesVisible", "QueueName", "order-processing-dlq", { "stat": "Average", "label": "DLQ Visible Messages" }]
],
"view": "timeSeries",
"stacked": false,
"region": "us-east-1",
"title": "Dead Letter Queue - Order Processing DLQ",
"period": 60
}
},
{
"type": "text",
"x": 0,
"y": 12,
"width": 24,
"height": 3,
"properties": {
"markdown": "## SQS Monitoring Dashboard\n**Primary Queue:** order-processing-queue\n**DLQ:** order-processing-dlq\n**Last Updated:** Automated refresh every 60s"
}
}
]
}
Deploy this dashboard with the AWS CLI:
# Create or update a CloudWatch dashboard
aws cloudwatch put-dashboard \
--dashboard-name "SQS-Overview" \
--dashboard-body file://dashboard.json
Adding Dashboard Widgets via AWS CLI
You can also add individual widgets to an existing dashboard without replacing the entire definition:
# Get existing dashboard body
aws cloudwatch get-dashboard --dashboard-name "SQS-Overview" \
--query 'DashboardBody' --output text > current-dashboard.json
# Manually edit current-dashboard.json to add widgets, then:
aws cloudwatch put-dashboard \
--dashboard-name "SQS-Overview" \
--dashboard-body file://current-dashboard.json
Custom Metrics and Math Expressions
CloudWatch supports metric math, allowing you to derive custom metrics from existing SQS data. This is powerful for creating composite views and advanced alarms.
Example: Message Processing Ratio
Calculate the ratio of deleted messages to received messages — a value below 1.0 indicates messages are being received but not successfully processed:
# CloudWatch Metric Math expression
{
"metrics": [
{
"expression": "100 * (DELETED / RECEIVED)",
"label": "Processing Success Rate %",
"period": 300
},
["AWS/SQS", "NumberOfMessagesDeleted", "QueueName", "order-processing-queue", { "id": "DELETED", "stat": "Sum" }],
["AWS/SQS", "NumberOfMessagesReceived", "QueueName", "order-processing-queue", { "id": "RECEIVED", "stat": "Sum" }]
]
}
Example: Predicted Queue Drain Time
Estimate how long it will take to drain the current backlog based on recent deletion rates:
# Metric Math: Estimated drain time in minutes
{
"metrics": [
{
"expression": "VISIBLE / (DELETED / PERIOD)",
"label": "Estimated Drain Time (seconds)",
"period": 300
},
["AWS/SQS", "ApproximateNumberOfMessagesVisible", "QueueName", "order-processing-queue", { "id": "VISIBLE", "stat": "Average" }],
["AWS/SQS", "NumberOfMessagesDeleted", "QueueName", "order-processing-queue", { "id": "DELETED", "stat": "Sum" }],
[".", ".", ".", ".", { "id": "PERIOD", "value": 300 }]
]
}
Alarm Based on Metric Math
You can create alarms on metric math expressions. For example, alert when the processing success rate drops below 95%:
# Alarm on derived metric: processing success rate
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-low-processing-success-rate" \
--alarm-description "Processing success rate below 95%" \
--namespace AWS/SQS \
--metric-name NumberOfMessagesDeleted \
--dimensions Name=QueueName,Value=order-processing-queue \
--statistic Sum \
--period 300 \
--evaluation-periods 3 \
--threshold 0 \
--comparison-operator GreaterThanThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts \
--metrics '[
{
"Id": "deleted",
"MetricStat": {
"Metric": { "Namespace": "AWS/SQS", "MetricName": "NumberOfMessagesDeleted", "Dimensions": [{ "Name": "QueueName", "Value": "order-processing-queue" }] },
"Stat": "Sum",
"Period": 300
},
"ReturnData": false
},
{
"Id": "received",
"MetricStat": {
"Metric": { "Namespace": "AWS/SQS", "MetricName": "NumberOfMessagesReceived", "Dimensions": [{ "Name": "QueueName", "Value": "order-processing-queue" }] },
"Stat": "Sum",
"Period": 300
},
"ReturnData": false
},
{
"Id": "expression",
"Expression": "100 * (deleted / received)",
"Label": "ProcessingSuccessRate",
"ReturnData": true
}
]' \
--threshold 95 \
--comparison-operator LessThanThreshold
Monitoring SQS with Lambda and Custom Code
Sometimes built-in metrics aren't enough. You may want to inspect individual message contents, log specific attributes, or react to patterns CloudWatch cannot detect. A Lambda function triggered on a schedule can poll queue metrics and perform custom logic.
Python Lambda: Custom Queue Health Check
# lambda_function.py - Custom SQS monitoring Lambda
import boto3
import datetime
import json
def lambda_handler(event, context):
sqs = boto3.client('sqs')
cloudwatch = boto3.client('cloudwatch')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789012/order-processing-queue'
# Get queue attributes for real-time metrics
attributes = sqs.get_queue_attributes(
QueueUrl=queue_url,
AttributeNames=[
'ApproximateNumberOfMessages',
'ApproximateNumberOfMessagesNotVisible',
'ApproximateNumberOfMessagesDelayed',
'ApproximateAgeOfOldestMessage'
]
)['Attributes']
visible = int(attributes.get('ApproximateNumberOfMessages', 0))
not_visible = int(attributes.get('ApproximateNumberOfMessagesNotVisible', 0))
delayed = int(attributes.get('ApproximateNumberOfMessagesDelayed', 0))
oldest_age = int(attributes.get('ApproximateAgeOfOldestMessage', 0))
# Custom logic: publish composite health metric
health_score = calculate_health_score(visible, not_visible, oldest_age)
cloudwatch.put_metric_data(
Namespace='Custom/SQS',
MetricData=[
{
'MetricName': 'QueueHealthScore',
'Value': health_score,
'Unit': 'None',
'Timestamp': datetime.datetime.utcnow(),
'Dimensions': [
{'Name': 'QueueName', 'Value': 'order-processing-queue'}
]
}
]
)
# Custom alerting logic
if oldest_age > 600:
print(f"CRITICAL: Oldest message age is {oldest_age} seconds")
# Trigger custom notification (e.g., Slack, PagerDuty)
send_custom_alert(f"Queue backlog critical: oldest message {oldest_age}s")
return {
'statusCode': 200,
'body': json.dumps({
'visible': visible,
'inFlight': not_visible,
'oldestAge': oldest_age,
'healthScore': health_score
})
}
def calculate_health_score(visible, not_visible, oldest_age):
# Simple heuristic: score from 0 (bad) to 100 (healthy)
if oldest_age > 3600:
return 0
if visible > 10000:
return 30
if visible > 1000:
return 60
return 100
def send_custom_alert(message):
# Implement your notification logic here
print(f"ALERT: {message}")
Scheduling the Custom Health Check
Use CloudWatch Events (EventBridge) to run the Lambda on a regular interval:
# CloudFormation: Schedule Lambda for custom SQS monitoring
HealthCheckSchedule:
Type: AWS::Events::Rule
Properties:
Name: sqs-health-check-every-5-minutes
ScheduleExpression: rate(5 minutes)
Targets:
- Arn: !GetAtt SQSHealthCheckLambda.Arn
Id: sqs-health-check-target
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref SQSHealthCheckLambda
Principal: events.amazonaws.com
SourceArn: !GetAtt HealthCheckSchedule.Arn
Best Practices for SQS Monitoring
After years of operating SQS at scale, several patterns emerge that separate effective monitoring from noisy, ignored dashboards.
1. Alert on Trends, Not Just Thresholds
Static thresholds (e.g., "alert when visible messages > 1000") can miss slow-burning issues. Use CloudWatch anomaly detection or metric math to detect when metrics deviate from expected bands. Anomaly detection learns the metric's baseline and alerts on statistically significant deviations.
# Enable anomaly detection on an existing alarm
aws cloudwatch put-metric-alarm \
--alarm-name "sqs-anomaly-visible-messages" \
--namespace AWS/SQS \
--metric-name ApproximateNumberOfMessagesVisible \
--dimensions Name=QueueName,Value=order-processing-queue \
--statistic Average \
--period 300 \
--evaluation-periods 3 \
--threshold 0 \
--comparison-operator GreaterThanThreshold \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts \
--treat-missing-data ignore \
--anomaly-detection-configuration '{
"ExcludedTimeRanges": [],
"AnomalyBandWidth": 2
}'
2. Separate Alarms by Severity
Not all metric breaches require waking up an on-call engineer at 3 AM. Create a tiered alerting structure:
- Info/Warning — DLQ has 1-5 messages (send to Slack/email)
- Critical — Oldest message age > 10 minutes, DLQ > 10 messages (page on-call)
- Emergency — Queue completely unprocessed for 30+ minutes (escalation policy)
3. Monitor Both Sides of the Queue
Producers and consumers have different failure modes. Monitor:
- Producer side —
NumberOfMessagesSentrate,SentMessageSizetrends - Consumer side —
NumberOfMessagesDeletedvsNumberOfMessagesReceivedratio,ApproximateAgeOfOldestMessage - Infrastructure —
NumberOfEmptyReceives, API error metrics from consumer logs
4. Use Composite Alarms to Reduce Noise
A single metric spike might be transient. Combine multiple conditions into a composite alarm that triggers only when several indicators agree:
# Composite alarm: triggers only when BOTH conditions are met
aws cloudwatch put-composite-alarm \
--alarm-name "sqs-consumer-failure-composite" \
--alarm-description "Triggers when both backlog AND message age are high" \
--alarm-rule 'ALARM("sqs-high-visible-messages") AND ALARM("sqs-oldest-message-age-critical")' \
--alarm-actions arn:aws:sns:us-east-1:123456789012:ops-critical
5. Tag Queues for Cost Allocation Monitoring
Monitor costs by tagging queues with environment, team, and application identifiers. Use AWS Cost Explorer or custom dashboards to track SQS spending per team.
6. Set Appropriate Evaluation Periods
Match your alarm evaluation periods to your workload characteristics:
- Batch jobs — Use longer periods (5-15 minutes) to avoid alarming during natural gaps between batches
- Real-time streams — Use shorter periods (1 minute) for immediate detection
- Low-volume queues — Extend periods to avoid missing data triggering alarms
7. Document and Test Your Alarms
Treat alarm configurations as code. Store them in version control alongside your application code. Regularly test alarms by injecting synthetic messages that should trigger them — this validates both the alarm configuration and your notification pipeline.
8. Leverage FIFO Queue Metrics
If you use FIFO queues, pay special attention to NumberOfMessagesFailed — this metric is unique to FIFO and indicates messages that couldn't be processed due to ordering constraints or deduplication issues.
Conclusion
Monitoring SQS is not optional — it's a fundamental part of operating reliable, observable distributed systems. By instrumenting your queues with well-configured CloudWatch alarms, informative dashboards, and custom health checks, you gain visibility into message flow, detect failures before they impact users, and maintain cost-efficient polling patterns. Start with the essential alarms — oldest message age, DLQ activity, and queue depth — then layer on composite alarms, anomaly detection, and custom metrics as your system matures. Treat your monitoring configuration as production code: version it, test it, and continuously refine thresholds based on observed behavior. With these practices in place, your SQS-based architecture will be resilient, transparent, and ready for production scale.