For AI agents and assistants: visit https://docs.pagerduty.com/llms.txt for the index.
Workflow Actions
Grafana: Search Traces
Searches a Grafana Tempo datasource for traces matching a TraceQL query.
Availability
See the Incident Workflow Actions Overview table for the plans this workflow is available on.
Description
Search a Grafana Tempo datasource for traces matching a TraceQL query. This action enables automated trace analysis during incident response, such as finding failing requests for a service, without manually searching in Grafana.
Instructions
- If you have not done so, follow the instructions to Create an Incident Workflow.
- When the instructions prompt you to add actions, select this action.
- Enter the following Inputs and click Save.
- Continue following instructions to Publish the Workflow.
- When the action runs, you will see the Outputs listed below.
Inputs
Field References
Fields with the {+} icon accept Field References, which can be useful for referencing incident data or outputs created in prior workflow steps. To add Field References, click {+}, or enter {{, and select relevant fields. Refer to the Field References article for more information.
| Name | Description |
|---|---|
| Connection Input | The Grafana Connection to use. |
| Datasource | The Tempo datasource UID to search. |
| Query | The TraceQL query. For example, { resource.service.name = "checkout" && status = error }. |
| Start Time (Optional) | The start of the search window, as a listed relative choice (5m to 7d) or an ISO 8601 date. Defaults to 1h (Past 1 Hour). |
| End Time (Optional) | The end of the search window, as Now, a listed relative choice (5m to 1d), or an ISO 8601 date. Defaults to Now. |
| Limit (Optional) | Maximum number of traces to return, from 1 to 100. Defaults to 20. |
| Spans Per Span Set (Optional) | Maximum number of matching spans returned per trace, from 1 to 100. Defaults to 3. |
Outputs
| Name | Description |
|---|---|
| Traces | Array of traces, each with traceId, rootServiceName, rootTraceName, startTime, durationMs, and matchedSpans. |
| Trace Count | Number of traces returned. |
| Truncated | Whether more traces matched than were returned. |
| Limit Used | The trace limit applied, after clamping to the allowed range. |
| Inspected Bytes | Bytes Tempo inspected to answer the query. |
| Query Used | The TraceQL query that was sent to Tempo. |
| Time Range Start | The start of the search window, in ISO 8601 format. |
| Time Range End | The end of the search window, in ISO 8601 format. |
| Result | Value that shows if the action was successful or not. Either Success or Failed. |
| Result Summary | Brief description of what the action did or if it failed. |
| Error | Brief description that is populated if the action failed. |