🚨 Loki Error Guide: 'http: server gave HTTP response to HTTPS client' — Match the URL Scheme to the Endpoint

Overview

This error is a protocol-scheme mismatch. The client opened a TLS (https://) connection, but the Loki endpoint answered with plain HTTP, so the TLS handshake never happened and the client aborts:

http: server gave HTTP response to HTTPS client

The inverse appears when a plain-HTTP client hits a TLS port:

Client sent an HTTP request to an HTTPS server

Neither is a certificate problem — the two sides simply disagree on whether the connection is encrypted. It usually means the client url is set to https:// while Loki or its gateway serves plain HTTP on that port (commonly 3100), or TLS is terminated at a proxy the client is bypassing. The fix is to make the scheme match how the endpoint is actually served: either point the client at the plain-HTTP endpoint with http://, or enable TLS on the server and keep https://.

Symptoms

• Promtail/Alloy logs http: server gave HTTP response to HTTPS client and never delivers a batch.
• A curl http:// to the same host and port returns data, while curl https:// fails immediately.
• The error appeared right after someone changed a client url from http:// to https:// (or vice versa).
• The client is aimed straight at Loki on 3100 (plain) instead of the TLS-terminating gateway.
• Loki’s own server logs Client sent an HTTP request to an HTTPS server when a plain client hits a TLS port.

Common Root Causes

• url set to https:// but the endpoint serves plain HTTP — the most common case; the port answers HTTP with no TLS.
• TLS terminated at a proxy that the client bypasses — the gateway does TLS, but the client connects to Loki directly on the plain port.
• Missing http_tls_config on the server — nobody enabled TLS on Loki, yet clients assume https://.
• Wrong port — 3100 typically serves plain HTTP, while a separate TLS listener runs elsewhere; the scheme and port are mismatched.
• Scheme copied between environments — a https:// URL from a TLS-fronted prod pasted into a plain-HTTP dev cluster.

How to diagnose

1. Probe both schemes against the same endpoint to see which one the server actually speaks:

   curl -v http://loki-gateway:3100/ready    # works if plain HTTP
   curl -v https://loki-gateway:3100/ready   # "gave HTTP response to HTTPS client" if plain

2. Check whether the port negotiates TLS at all — a plain port will not complete a handshake:

   openssl s_client -connect loki-gateway:3100 </dev/null
   # plain HTTP: handshake fails / no peer certificate presented

3. Confirm what the client is configured to use and trace it to the real listener:

   kubectl get configmap promtail -o yaml | grep -A2 'url:'

4. Verify the server’s TLS configuration — is there an http_tls_config block, and on which port:

   # If this block is absent, Loki serves plain HTTP and clients must use http://
   server:
     http_listen_port: 3100

5. Check ulimits are not masking a flapping listener while you test, so a busy server does not look like a scheme error:

   ulimit -n   # ensure the listener is healthy, not fd-starved, during the probe

Fixes

Match the scheme to how the endpoint is served — if Loki answers plain HTTP on 3100, use http:// in the client. For Promtail:

clients:
  - url: http://loki-gateway:3100/loki/api/v1/push

For Alloy:

loki.write "default" {
  endpoint {
    url = "http://loki-gateway:3100/loki/api/v1/push"
  }
}

Point the client at the TLS-terminating gateway instead of Loki’s plain port, so https:// reaches something that actually serves TLS:

clients:
  # gateway terminates TLS on 443, then proxies to Loki on plain 3100
  - url: https://loki-gateway.example.com/loki/api/v1/push

Enable TLS on the Loki server if you want to keep https:// end to end — add an http_tls_config with a cert and key, then clients can connect over TLS directly:

server:
  http_listen_port: 3100
  http_tls_config:
    cert_file: /etc/loki/tls/server.pem
    key_file:  /etc/loki/tls/server.key

Fix the inverse mismatch — if the server now serves TLS and a client still uses http://, Loki logs Client sent an HTTP request to an HTTPS server; update the client scheme to https://:

clients:
  - url: https://loki-gateway:3100/loki/api/v1/push

Standardize scheme and port per environment so URLs are not copied across clusters with the wrong assumption — keep the endpoint in one templated variable:

# values.yaml: single source of truth for the endpoint
loki:
  pushUrl: http://loki-gateway:3100/loki/api/v1/push   # plain internal endpoint

What to watch out for

• This is not a certificate error, so adding a CA or setting insecure_skip_verify will not fix it — the mismatch is about the scheme, not trust.
• The default 3100 almost always means plain HTTP; assuming https:// on it is the single most common cause.
• If a gateway terminates TLS, clients must go through the gateway, not around it to Loki’s plain port, or they will hit this error.
• Enabling http_tls_config on the server flips every plain-HTTP client to the inverse error at once; roll the client scheme change out together with the server change.
• Health probes and scrape configs use the same scheme rules — update readiness checks and monitoring to match, or they will report the endpoint as down.

Related

• Loki Error Guide: ‘no org id’ — the next push-path check once the scheme is correct and the connection completes.
• Loki Error Guide: ‘failed to flush chunks’ — a write-path failure that can follow a misconfigured endpoint.
• Loki Error Guide: ‘context deadline exceeded’ — the timeout that can appear when a client keeps retrying against the wrong scheme or port.