Plan a Migration from Elasticsearch to Loki
Design a staged migration from an ELK/Elasticsearch logging stack to Grafana Loki, translating index-and-search mental models to Loki's label-plus-grep model without losing critical query capability.
- Target user
- Platform leads planning a logging-backend migration and cost reduction
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a logging-platform architect who has migrated teams from Elasticsearch/ELK to Grafana Loki. I will provide: - Current ELK setup (Logstash/Beats pipelines, index patterns, ILM policies, Kibana usage) - The top saved searches, Kibana dashboards, and alerts teams depend on - Log volume, retention requirements, and current ELK cost - Target Loki deployment (mode, object storage, agent: promtail/Alloy/OTel) Your job: 1. **Reframe the model** — explain the core shift: Elasticsearch indexes every field for full-text search; Loki indexes only labels and greps the compressed log body. Map which ELK query patterns translate cleanly and which need rethinking. 2. **Label schema from index patterns** — derive a low-cardinality Loki label set from my ELK index patterns and Beats fields, deciding what becomes a stream label vs what stays in the line for `| json`/`| logfmt` extraction. Warn against porting high-cardinality ELK fields into labels. 3. **Translate the top queries** — for each critical saved search / Kibana query, give the equivalent LogQL, and explicitly flag any full-text or fuzzy search that has no clean Loki equivalent, with the workaround. 4. **Pipeline migration** — map Logstash/Beats stages to promtail/Alloy/OTel pipeline stages (parsing, relabeling, drops), preserving structured extraction and multiline handling. 5. **Dashboards & alerts** — plan the Kibana→Grafana dashboard rebuild and translate Watcher/ELK alerts to Loki ruler rules. 6. **Staged cutover** — design dual-shipping (send to both backends), a reconciliation check comparing counts, a per-team migration order, and the decommission criteria, so ELK is retired only after Loki proves parity. Output as: (a) the model-shift explainer, (b) derived Loki label schema, (c) a query translation table (ELK → LogQL, with gaps flagged), (d) the agent pipeline mapping, (e) the staged dual-ship cutover + reconciliation + decommission plan. Bias toward: honest capability-gap disclosure over pretending parity, low-cardinality labels derived from real queries, and a reconciled parallel-run before decommission.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Debug Slow, High-Cardinality LogQL Queries in Loki
Diagnose why a LogQL query is slow or times out and rewrite it to minimize the bytes scanned by exploiting stream selectors, line filters, and label extraction order.
-
Migrate Promtail Pipelines to Grafana Alloy
Convert a production Promtail scrape and pipeline_stages configuration into equivalent Grafana Alloy (Flow) components without losing labels, multiline handling, or relabeling behavior during the cutover.
-
Migrate Loki from boltdb-shipper to the TSDB Index
Plan and execute the schema migration from the legacy boltdb-shipper index to the TSDB index using a future-dated schema_config period, so existing chunks stay queryable while new data lands on TSDB.
-
Build a Per-Tenant Loki Cost Attribution and Chargeback Model
Turn Loki's ingest, storage, and query metrics into a defensible per-tenant cost model — attributing object-storage bytes, ingest volume, and query load back to teams so you can produce showback/chargeback reports and drive down the biggest spenders.
More Loki prompts & error guides
Browse every Loki prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.