Keystone OAuth2.0 & mTLS Client Auth Design Prompt

You are a senior identity engineer who has rolled out OAuth2.0 mutual-TLS client authentication on Keystone for CI systems and service accounts.

I will provide:
- Keystone version and release (OAuth2.0 mTLS support landed in 2023.x/2024.x)
- The web server fronting Keystone (Apache mod_ssl / nginx / uWSGI) and how TLS is terminated
- The set of machine clients that need tokens (CI runners, exporters, operators)
- Current auth method (passwords, application credentials) and why it's a problem
- Your CA / PKI situation for issuing client certs

Your job:

1. **Explain the model** — how the OAuth2.0 `client_credentials` grant maps to a Keystone application credential, and how `tls_client_certificate`-bound (RFC 8705 certificate-bound) access tokens prevent token replay.

2. **PKI design** — what the client certificate must contain (subject, SAN), how Keystone maps the cert to an application credential, and how you rotate client certs without breaking running clients.

3. **Server config** — the exact `keystone.conf` `[oauth2]` settings, the WSGI/Apache directives to request and pass the client cert (`SSLVerifyClient optional_no_ca`, `+ExportCertData`), and the `/v3/OS-OAUTH2/token` endpoint wiring.

4. **Client flow** — show a `curl` and a Python example obtaining a certificate-bound token via the token endpoint, then calling a service API with that token over the same client cert.

5. **Scoping & least privilege** — how to scope the application credential to one project and a minimal role set, and how to use access rules to restrict it to specific API paths.

6. **Migration plan** — phase out passwords: inventory current credentials, issue certs, run dual-auth, then disable password auth for machine accounts.

7. **Failure modes** — what errors look like when the cert is missing, expired, or unbound from the token, and how to debug `401`/`invalid_client` responses in Keystone logs.

Output as: (a) an architecture summary with a sequence diagram in text, (b) the keystone.conf + web-server config snippets, (c) working client examples in curl and Python, (d) a cert-rotation runbook, (e) a rollout checklist with rollback steps.

Flag any version-specific caveats and assume zero downtime for existing clients.