For AI agents and assistants: visit https://docs.pagerduty.com/llms.txt for the index.
AI & Automation
Grafana Self-Hosted
Connect a Grafana instance inside your network through the PD Automation Runner and use Grafana actions in Incident Workflows.
Early Access
This feature is currently in Early Access (EA). Click here to sign up for the Early Access program. This feature is only available for PD Reliability Platform accounts.
Use the Grafana Self-hosted connection for a Grafana instance that PagerDuty cannot reach directly, such as one inside a private network. PagerDuty sends requests through a PD Automation Runner running in your environment. For a Grafana instance reachable from the internet, use the Grafana connection instead (see Workflow Integrations).
Requirements
- A PD Automation Runner connection whose runner can reach your Grafana instance. To set one up, see PD Automation Runner. The runner can use any deployment method, and a single runner can serve Kubernetes, Prometheus, and Grafana connections.
- A Grafana service account token with the
datasources:readanddatasources:querypermissions. A service account with the Viewer role is sufficient.
Read-Only Access
PagerDuty only reads from Grafana and never modifies it. If your Grafana instance restricts data source access with custom roles, make sure the token's role includes both datasources:read and datasources:query.
Add a Grafana Self-Hosted Connection
- In PagerDuty, navigate to Integrations Automation Connectors.
- Select Grafana Self-hosted and click Add Connector.
- Enter the following fields:
| Field | Description |
|---|---|
| Connection Name | The name of the new connection. |
| Runner Connection | The PD Automation Runner connection that can reach this Grafana instance. |
| Grafana URL | The URL of your Grafana instance as reachable from the runner, for example https://grafana.example.internal. Do not include /api. |
| Service Account Token | The Grafana service account token. |
- Click Save.
How PagerDuty Validates the Connection
When you save the connection, the runner calls GET {Grafana URL}/api/org with the service account token. PagerDuty creates the connection only if Grafana returns your organization. Editing the runner, URL, or token runs the same check. Editing only the name does not.
Troubleshoot the Connection
| Error | What to Check |
|---|---|
Runner failed to reach Grafana: … | The runner cannot reach the URL. Check the host, port, and the runner's network access. |
Grafana rejected the credentials: … | The service account token is invalid, expired, or lacks the datasources:read and datasources:query permissions. |
The Grafana URL did not return a Grafana API response… | The URL responded, but it is not the Grafana API. For example, it is a login page, another service, or a URL that already includes /api. |
Grafana Workflow Actions
A Grafana Self-hosted connection works with the existing Grafana workflow actions: Grafana: Get Datasources, Grafana: Search Logs, Grafana: List Metrics, and Grafana: Query Metrics.
List Labels
Lists Prometheus label names, or the values for a specific label, from a Grafana datasource. You can scope results with a series selector and lookback window. Responders can discover which labels and values exist, such as namespaces, jobs, or pods, during an active incident without leaving the workflow, which makes it easier to build the next query.
To add the action to a workflow:
- If you have not done so, create an Incident Workflow.
- When the instructions prompt you to add actions, select Grafana: List Labels.
- Enter the inputs and click Save.
- Publish the workflow.
Field References
Fields with the icon accept Field References, which reference incident data or outputs from prior workflow steps. To add a Field Reference, click or enter {{, then select the field.
| Input | Description |
|---|---|
| Connection Input | The Grafana connection to use. |
| Datasource | The Prometheus datasource UID to discover labels from. |
| Label Name (Optional) | If set, lists the values for this label, for example namespace. If empty, lists all label names. |
| Match (Optional) | A PromQL series selector to scope results, for example {job="api"}. |
| Time Window (Optional) | The lookback window. Labels and values are scoped to series active within it. Defaults to 1h (Past 1 Hour). |
| Output | Description |
|---|---|
| Label Names | An array of label name strings. Populated when Label Name is empty. |
| Label Values | An array of value strings for the requested Label Name. Empty when listing label names. |
| Count | The number of items returned (label names or label values). |
| Mode | label_names or label_values. |
| Result | Whether the action succeeded: Success or Failed. |
| Result Summary | A brief description of what the action did, or why it failed. |
| Error | A brief description, populated if the action failed. |
List Names, Then Values
Leave Label Name empty to see which labels exist, then run the action again with a Label Name such as namespace to list its values. Use Match, for example {job="api"}, and Time Window to narrow results to a specific service and avoid stale values.