Grafana Elasticsearch Data Source Design Prompt

You are a senior observability engineer who has built Grafana dashboards on top of large Elasticsearch clusters.

I will provide:
- Index patterns and mappings
- Elasticsearch/OpenSearch version
- Dashboards or log views to build

Your job:

1. **Configure the datasource**: set the index pattern (with time pattern like `logs-*` or `[logstash-]YYYY.MM.DD`), the time field (`@timestamp`), and the ES version so Grafana picks the right query API.
2. **Bucket aggregations**: use Date Histogram on `@timestamp` with an auto interval, plus Terms buckets for split-by fields like `host` or `service`.
3. **Metric aggregations**: Count for log volume, Average/Percentiles for numeric fields, and Unique Count (cardinality) for distinct users.
4. **Lucene queries**: filter with `level:error AND service:checkout`; escape reserved characters and prefer `.keyword` sub-fields for exact Terms matches.
5. **Logs mode**: enable the Logs panel with a message field and level field so Grafana colorizes severities.
6. **Template variables**: use the datasource query type `{"find": "terms", "field": "service.keyword"}` for dynamic dropdowns.
7. **Performance**: cap Terms bucket size, avoid high-cardinality Terms on analyzed text fields, and use filters over wildcards.
8. **Provisioning**: express the datasource in YAML with `jsonData` (timeField, esVersion, maxConcurrentShardRequests).

Mark DESTRUCTIVE: pointing at a production index with expensive wildcard queries, changing the index pattern on a shared datasource, deleting a datasource in use.

---

Index patterns/mappings: [DESCRIBE]
ES/OpenSearch version: [DESCRIBE]
Dashboards/log views: [DESCRIBE]

Why this prompt works

Elasticsearch in Grafana is deceptively easy to misconfigure — the wrong time field yields empty panels, and Terms aggregations on analyzed text fields silently blow up cardinality. This prompt makes the model reason about mappings, keyword sub-fields, and aggregation shape before building panels, which is where most broken ES dashboards go wrong.

How to use it

1. Share the mapping so the assistant knows which fields are keyword vs text.
2. State the version — query APIs and supported metrics differ across ES/OpenSearch releases.
3. Describe the log schema (message field, level field) to enable Logs mode.
4. Ask for provisioning YAML for reproducibility.

Useful commands

# Inspect the index mapping to find keyword sub-fields
curl -s "http://es:9200/logs-*/_mapping?pretty"

# Check datasource health through Grafana
curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  http://localhost:3000/api/datasources/uid/es-logs/health

# Preview a Lucene query directly against ES
curl -s "http://es:9200/logs-*/_search?q=level:error&size=5&pretty"

Example config

# provisioning/datasources/elasticsearch.yaml
apiVersion: 1
datasources:
  - name: Elasticsearch Logs
    type: elasticsearch
    uid: es-logs
    access: proxy
    url: http://es:9200
    database: "logs-*"
    jsonData:
      timeField: "@timestamp"
      esVersion: "8.11.0"
      maxConcurrentShardRequests: 5
      logMessageField: message
      logLevelField: level
      interval: Daily

// Panel target: error volume by service over time
{
  "query": "level:error",
  "metrics": [{ "type": "count", "id": "1" }],
  "bucketAggs": [
    { "type": "terms", "field": "service.keyword",
      "settings": { "size": "10", "order": "desc", "orderBy": "1" }, "id": "3" },
    { "type": "date_histogram", "field": "@timestamp",
      "settings": { "interval": "auto" }, "id": "2" }
  ]
}

Common findings this catches

• Empty panels → wrong or missing timeField.
• Cardinality explosion → Terms on analyzed text instead of .keyword.
• Cluster overload → wildcard Lucene queries on production indices.
• Fragmented dropdowns → template variable not using keyword field.
• Unsupported aggregation → esVersion set wrong.
• Truncated results → Terms bucket size too small.
• No severity colors → logLevelField not configured for Logs mode.

When to escalate

• Cluster performance degradation from dashboard queries — involve the ES platform team.
• Index lifecycle or sharding redesign — data platform engineering.
• Cross-cluster search across regions — capacity and networking review.