Grafana Explore Split View Workflow Prompt

You are a senior SRE who lives in Grafana Explore during incidents, correlating metrics, logs, and traces side by side.

I will provide:
- The symptom and the affected service
- The data sources available (Prometheus, Loki, Tempo)
- What I have found so far

Your job:

1. **Frame the Explore workflow**:
   - Open Explore, use the split button to get two panes
   - Left pane = metrics (Prometheus), right pane = logs (Loki)
   - Keep the same time range synced across panes
2. **Start from the metric**:
   - Query the RED/USE signal that alerted
   - Zoom to the spike, note the exact time window
3. **Pivot to logs**:
   - In the right pane query Loki with the same labels
   - Use the log volume histogram to spot the error burst
   - Filter with LogQL: `|= "error" | json | line_format`
4. **Pivot to traces**:
   - Use a derived field / data link from a log line's traceID
   - Or use exemplars from the metric to jump to Tempo
   - Open the trace in a third split
5. **Correlate labels**:
   - Confirm the same `pod`, `namespace`, `service` labels across sources
   - Use the query history and starred queries to reuse
6. **Capture evidence**:
   - Copy the shortened Explore URL (state is encoded)
   - Add to the incident timeline
7. **Hand off cleanly**:
   - Share the split-view URL so the next responder loads the same state

Mark DESTRUCTIVE: none in Explore itself (read-only), but flag if a suggested query is expensive on the data source.

---

Symptom and service: [DESCRIBE]
Data sources: [DESCRIBE]
Found so far: [DESCRIBE]

Why this prompt works

Explore is the fastest path from symptom to root cause because it needs no dashboard. But new responders under-use split view and lose time copy-pasting labels between tabs. This prompt encodes the metrics -> logs -> traces pivot and the label-correlation discipline that makes it fast.

How to use it

1. Open Explore and split into two panes.
2. Metric on the left, logs on the right, same time range.
3. Follow traceID via derived fields into Tempo.
4. Copy the short URL into the incident channel.

Useful commands

# Configure a Loki derived field so traceID becomes a Tempo link (provisioning)
# See Example config below. Verify it applied:
curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  http://grafana:3000/api/datasources/name/Loki | jq '.jsonData.derivedFields'

# Check exemplar support on the Prometheus data source
curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  http://grafana:3000/api/datasources/name/Prometheus | jq '.jsonData.exemplarTraceIdDestinations'

# Shorten current Explore state to a shareable link (done via UI "Share" button)
# grafana.ini toggle if short links are disabled:
#   [feature_toggles] enable = ...

Example config

# provisioning/datasources/loki.yaml — derived field links log traceID to Tempo
apiVersion: 1
datasources:
  - name: Loki
    type: loki
    uid: loki-uid
    jsonData:
      derivedFields:
        - name: TraceID
          matcherRegex: '"traceID":"(\w+)"'
          url: '$${__value.raw}'
          datasourceUid: tempo-uid
          urlDisplayLabel: 'View Trace'

Common findings this catches

• No trace link from logs → derived field regex not configured.
• Panes out of sync → time range not shared.
• Slow logs pane → LogQL range too wide.
• Wrong labels → service uses different label names in Loki vs Prometheus.
• Missing exemplars → metric not instrumented with them.
• Lost context on handoff → forgot to share the Explore URL.

When to escalate

• Data source overload from ad hoc queries — platform team rate limits.
• Missing instrumentation (no traceID in logs) — dev team change.
• Recurring incident pattern — promote the Explore queries into a dashboard.