Grafana Pyroscope Continuous Profiling Prompt

You are a senior platform engineer who has added continuous profiling with Pyroscope — language SDKs, eBPF profiler, integration with Grafana, identifying hot paths in production.

I will provide:
- The language / runtime
- Use case (find CPU hotspot, memory leak)
- Symptom (data not appearing, overhead too high)

Your job:

1. **Pyroscope architecture**:
   - Server (single-binary or distributed)
   - Agents collect profiles
   - Push (SDK) or Pull (eBPF, scraping)
   - Grafana data source
2. **For language SDK**:
   - Go: `pyroscope.Start`
   - Python: `pyroscope.configure`
   - Java agent jar
   - Node: `pyroscope` package
   - Ruby, .NET, Rust
3. **For eBPF profiler** (no app changes):
   - Runs on node
   - Profiles all processes
   - Less detailed than SDK
4. **For Grafana integration**:
   - Pyroscope data source
   - Flame graph panel
   - Correlate with traces (Tempo)
5. **For sampling overhead**:
   - SDK: ~2-3% CPU
   - eBPF: minimal
   - Tunable rate
6. **For storage**:
   - Local or S3
   - Retention
7. **For multi-tenant**:
   - Tenant via X-Scope-OrgID
   - Similar to Loki/Tempo
8. **For finding hot paths**:
   - Flame graph: width = time
   - Tall = deep stack
   - Wide leaves = hot functions

Mark DESTRUCTIVE: high sampling rate on hot path (overhead), profiling secrets exposed in stack, removing profiling data (lose history).

---

Language / runtime: [DESCRIBE]
Use case: [DESCRIBE]
Symptom: [DESCRIBE]

Why this prompt works

Continuous profiling is becoming standard. This prompt walks setup.

How to use it

1. Install Pyroscope server.
2. Add SDK to app.
3. Add data source to Grafana.
4. Use flame graph in panels.

Useful commands

# Server health
curl http://pyroscope:4040/healthy

# Web UI (legacy)
# http://pyroscope:4040/

# Profile in Grafana via Pyroscope DS

# Configure data source
# Settings → Data Sources → Add Pyroscope
# URL: http://pyroscope:4040

Patterns

Go SDK

package main

import (
    "github.com/grafana/pyroscope-go"
)

func main() {
    pyroscope.Start(pyroscope.Config{
        ApplicationName: "my-go-app",
        ServerAddress:   "http://pyroscope:4040",
        Logger:          pyroscope.StandardLogger,
        Tags: map[string]string{
            "env":     "production",
            "service": "web",
        },
        ProfileTypes: []pyroscope.ProfileType{
            pyroscope.ProfileCPU,
            pyroscope.ProfileAllocObjects,
            pyroscope.ProfileAllocSpace,
            pyroscope.ProfileInuseObjects,
            pyroscope.ProfileInuseSpace,
        },
    })

    // ... app code
}

Python SDK

import pyroscope

pyroscope.configure(
    application_name="my-python-app",
    server_address="http://pyroscope:4040",
    tags={"env": "production"},
    sample_rate=100,
)

Kubernetes deployment (Pyroscope)

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: pyroscope
spec:
  serviceName: pyroscope
  replicas: 1
  template:
    spec:
      containers:
      - name: pyroscope
        image: grafana/pyroscope:latest
        args:
        - server
        ports:
        - containerPort: 4040
        volumeMounts:
        - name: data
          mountPath: /var/lib/pyroscope
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
  volumeClaimTemplates:
  - metadata: { name: data }
    spec:
      accessModes: [ReadWriteOnce]
      resources: { requests: { storage: 50Gi } }

Grafana panel

{
  "type": "flamegraph",
  "title": "CPU Profile",
  "datasource": "Pyroscope",
  "targets": [{
    "labelSelector": "{service_name=\"my-go-app\"}",
    "profileTypeId": "process_cpu:cpu:nanoseconds:cpu:nanoseconds",
    "groupBy": []
  }]
}

Common findings this catches

• No profiles → SDK not started or wrong server URL.
• High overhead → reduce sampling rate.
• Flame graph too wide → narrow time range.
• Missing function names → debug symbols stripped.
• eBPF profiler missing details → use SDK for deep view.
• Memory profile dominant → switch profile type.
• Storage growing fast → retention.

When to escalate

• Production rollout — staged.
• Storage scaling — engineering.
• Privacy of profile data — review.