|docs

For AI agents and assistants: visit https://docs.pagerduty.com/llms.txt for the index.

Workflow Actions

AWS: CloudWatch Query

Queries AWS CloudWatch logs with Logs Insights, CloudWatch metrics, and AWS X-Ray traces through an AWS integration connection.

View as Markdown

Availability

See the Incident Workflow Actions Overview table for the plans this workflow is available on.

Description

Query AWS CloudWatch logs, metrics, and X-Ray traces from one action. The Operation input selects what the action runs, and each operation takes one Search Query format.

This action replaces AWS: Get CloudWatch Logs by Query for new workflow steps. Existing steps that use the earlier action keep working.

Instructions

  1. If you have not done so, follow the instructions to Create an Incident Workflow.
  2. When the instructions prompt you to add actions, select this action.
  3. Enter the following Inputs and click Save.
  4. Continue following instructions to Publish the Workflow.
  5. When the action runs, you will see the Outputs listed below.

Integrating with AI Agents

PagerDuty’s AI Agents – such as Paige, PagerDuty's SRE Agent – can use this workflow action to search logs, metrics, and traces. For this action to be usable by AI Agents, select this option when configuring the Workflow Connection.

Required AWS Permissions

The IAM role on your AWS connection needs the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:GetMetricData",
                "cloudwatch:ListMetrics",
                "logs:DescribeLogGroups",
                "logs:StartQuery",
                "logs:GetQueryResults",
                "xray:GetTraceSegmentDestination",
                "xray:GetTraceSummaries",
                "xray:BatchGetTraces"
            ],
            "Resource": "*"
        }
    ]
}
  • logs:DescribeLogGroups is required only for Logs Insights queries that use SOURCE logGroups(...).
  • Search Traces and Get Traces call xray:GetTraceSegmentDestination first, and fail without it.
  • When X-Ray Transaction Search is on, logs:StartQuery and logs:GetQueryResults must also cover the aws/spans log group.

Update Existing Roles

A role created for AWS: Get CloudWatch Logs by Query has only the logs permissions. Add cloudwatch:GetMetricData and cloudwatch:ListMetrics before you use the metric operations, and the xray: permissions before you use the trace operations.

Inputs

Field References

Fields with the {+} icon accept Field References, which can be useful for referencing incident data or outputs created in prior workflow steps. To add Field References, click {+}, or enter {{, and select relevant fields. Refer to the Field References article for more information.

NameDescription
IntegrationSelect a Workflow Integration or click New AWS Connection to establish a new one.
OperationSelect what to run: Logs Insights, Logs Insights (Log Groups), Get Metric Data, Get Metric Statistics, List Metrics, Search Traces, or Get Traces.
Search QueryThe query for the selected operation. See Search Query Formats. Required for every operation except List Metrics and Search Traces.
RegionThe AWS region to query. If left blank, the action uses the default region of the connection, then us-east-1.
Start TimeThe start time for the query in ISO 8601 format. For example: 2023-01-01T00:00:00Z. Required for every operation except List Metrics and Get Traces.
End TimeThe end time for the query in ISO 8601 format. For example: 2023-01-01T01:00:00Z. Required for every operation except List Metrics and Get Traces. For Search Traces, the end time can be at most 24 hours after the start time.
Result LimitThe maximum number of log rows, metric series, metrics, or traces to return. Default is 100, maximum is 1000. For Logs Insights, a trailing | limit N in the query applies when it is smaller.
Next TokenMetric and trace operations only. To get the next page, enter the Next Token output of the previous run and keep every other input the same. Leave empty for the first page.

Search Query Formats

OperationSearch Query
Logs InsightsA Logs Insights query that starts with SOURCE logGroups(...). For example: SOURCE logGroups(namePrefix: ['/aws/lambda/checkout']) | fields @timestamp, @message | limit 20
Logs Insights (Log Groups)JSON with the query and the exact log group names. The query must not use SOURCE. For example: {"queryString": "fields @message | limit 20", "logGroupNames": ["/aws/lambda/checkout"]}
Get Metric DataA Metrics Insights SQL query that starts with SELECT. For example: SELECT AVG(CPUUtilization) FROM SCHEMA("AWS/EC2", InstanceId)
Get Metric StatisticsJSON for one metric. period is optional. For example: {"namespace": "AWS/EC2", "metricName": "CPUUtilization", "dimensions": {"InstanceId": "i-0abc123"}, "statistic": "Average", "period": 300}
List MetricsJSON where every field is optional, and a null dimension value matches any value. For example: {"namespace": "AWS/EC2", "metricName": "CPUUtilization", "dimensions": {"InstanceId": null}}
Search TracesAn X-Ray filter expression, or empty for every trace. For example: service("checkout") AND responsetime > 2
Get TracesA JSON array of one to five trace IDs. For example: ["1-5759e988-bd862e3fe1be46a994272793"]

Outputs

NameDescription
ResultsLog rows (one object per row, keyed by field name), metric series (id, label, statusCode, timestamps, values), metrics (namespace, metricName, dimensions), trace summaries, or full traces with their parsed segments.
Result CountThe number of rows, series, metrics, or traces returned.
Query UsedThe query that was sent to AWS. For Get Metric Statistics, a readable form of the request.
Time Range StartThe start time used for the query. Empty for List Metrics and Get Traces.
Time Range EndThe end time used for the query. Empty for List Metrics and Get Traces.
Next TokenMetric and trace operations only. Enter this as the Next Token input, with the same other inputs, to get the next page. Empty when there are no more results, and always empty for Logs Insights.
Search DurationThe duration of the query in milliseconds.
ResultValue that shows if the action was successful or not. Either "Success" or "Failed."
Result SummaryBrief description of what the action did. If the failure was caused by the inputs or the AWS permissions, it states the reason.
ErrorBrief description that is populated if the action failed unexpectedly, for example because of throttling or an AWS server error.

Tips

  • Logs pagination: Logs Insights does not return a Next Token. When the Result Summary says there may be more rows, narrow the time window, for example by setting End Time to the oldest @timestamp returned.
  • Transaction Search: When X-Ray Transaction Search is on, X-Ray only sees indexed spans, so an empty or short trace result is not conclusive. The Result Summary says so. All spans are in the aws/spans log group, which you can query with Logs Insights.
  • Trace annotations: Each annotation has stringValue, numberValue, and booleanValue, and only one is set. Use stringValue when it is not null.
  • Query syntax: See the AWS documentation for Logs Insights query syntax.