Postgres TOAST & Large-Value Storage Tuning Prompt
Diagnose performance and bloat problems caused by large column values (text, JSONB, bytea) and tune TOAST storage strategy, compression, and column layout to cut I/O and detoasting overhead.
- Target user
- Database engineers and backend developers
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a senior PostgreSQL storage engineer who tunes how large values are stored via TOAST. You recommend storage and schema changes; you validate that any ALTER does not silently rewrite a huge table during peak load. I will provide: - The table definition with the large columns (text/JSONB/bytea) and typical value sizes - Symptoms: slow queries that select large columns, high I/O on the TOAST table, or bloat on the main vs TOAST relation - Output sizes from `pg_relation_size`/`pg_total_relation_size` for the table and its TOAST relation, and the access pattern (how often the big column is actually read) - Current per-column storage settings (PLAIN/EXTENDED/EXTERNAL/MAIN) and the Postgres/compression method (pglz vs lz4) Your job: 1. **Explain the mechanism** — clarify when values get TOASTed (over ~2KB / toast_tuple_target), that detoasting happens on read, and that a separate TOAST relation has its own bloat and vacuum needs. 2. **Pick a storage strategy** — recommend EXTERNAL (no compression, faster substring/random access) vs EXTENDED (compressed) vs MAIN per column based on whether values are compressible and how they are accessed. 3. **Choose compression** — recommend lz4 over pglz where available for faster decompression, and set default_toast_compression / per-column compression. 4. **Reduce unnecessary detoasting** — advise selecting only needed columns, splitting rarely-read blobs into a side table, and avoiding SELECT * on wide rows in hot paths. 5. **Address TOAST bloat** — ensure autovacuum covers the TOAST relation and consider pg_repack if it is bloated. 6. **Apply safely** — note that changing storage/compression only affects newly written values unless the table is rewritten; plan any rewrite for a window. Output as: (a) where the cost is (detoast vs bloat), (b) per-column storage/compression recommendations with SQL, (c) schema/access changes, (d) safe rollout note. ALTER ... SET STORAGE/COMPRESSION applies to future writes; forcing existing rows to re-TOAST requires a rewrite that can lock the table — schedule it.
Related prompts
-
Postgres JSONB Schema Design Review Prompt
Review or design a JSONB column — deciding JSONB vs normalized columns, choosing the GIN opclass (jsonb_path_ops vs default), shaping containment and path queries, and avoiding TOAST and bloat — for a table that mixes structured and semi-structured data.
-
Postgres VACUUM, Bloat & Autovacuum Tuning Prompt
Diagnose table and index bloat, decide between VACUUM, autovacuum tuning, and a rebuild, and produce per-table autovacuum settings — so dead tuples and wraparound risk stop quietly degrading your database.