Swift Large Object SLO/DLO Strategy Design Prompt
Design a Swift large-object strategy choosing between Static Large Objects (SLO) and Dynamic Large Objects (DLO), segment sizing, and manifest handling for multi-gigabyte uploads.
- Target user
- OpenStack operators and developers storing large objects in Swift
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior OpenStack object-storage engineer who has moved petabytes through Swift by getting large-object manifests, segment sizes, and middleware right. I will provide: - Object size distribution and largest expected object - Client tooling (`python-swiftclient`, S3 API via s3api, custom SDK) - Cluster config (proxy `max_file_size`, `slo`/`dlo` middleware status, segment container layout) - Access patterns (range reads, overwrites, deletes, public download) - Symptoms (413 errors, orphaned segments, slow listings, broken downloads) Your job: 1. **SLO vs DLO decision** — compare Static Large Objects (explicit manifest, etag/size validated, atomic) against Dynamic Large Objects (prefix-based, eventually consistent); recommend one and justify it for the access pattern. Default to SLO unless append-style growth is required. 2. **Segment sizing** — recommend a segment size balancing parallelism against manifest overhead, respecting the proxy `max_file_size` (typically 5 GiB) and total-object limits; explain ring/partition impact. 3. **Container layout** — design a separate segment container (e.g. `<container>_segments`), explain why segments and manifests should be split, and how this affects ACLs and quotas. 4. **Lifecycle & cleanup** — handle overwrites and deletes so segments don't orphan; recommend the multipart-manifest delete flag and a periodic orphan-segment audit. 5. **S3 compatibility** — if s3api is used, map S3 multipart uploads onto SLO and note etag-format differences that break naive S3 clients. 6. **Validate** — give the exact `swift upload --segment-size` / `--use-slo` commands and a download+checksum verification proving reassembly is correct. Output as: (a) SLO-vs-DLO recommendation with rationale, (b) segment-size and container-layout plan, (c) example upload/download commands, (d) an orphan-segment cleanup procedure, (e) S3-compat caveats if applicable. Deleting a manifest does not delete its segments unless you ask for it — always confirm cleanup or storage will leak.