#Example: HTTP Client Handle Compressed HTTP Responses

Go's http.Transport can transparently decompress gzip responses. When it does, resp.Body contains the decompressed data, not the bytes received from the server.

That affects response metadata, size limits, and connection reuse.

#Transparent gzip Decompression

The default Transport automatically requests gzip when:

• DisableCompression is false
• the request has no Accept-Encoding header
• the request has no Range header
• the method is not HEAD

It adds:

Accept-Encoding: gzip

If the server returns a gzip-encoded response, Transport decompresses it before the application reads resp.Body.

resp, err := client.Get("https://example.com")
if err != nil {
	return err
}
defer resp.Body.Close()

body, err := io.ReadAll(resp.Body)
if err != nil {
	return err
}

fmt.Println(len(body))

Transparent decompression also changes the response metadata:

resp.Uncompressed == true
resp.Header.Get("Content-Encoding") == ""
resp.Header.Get("Content-Length") == ""
resp.ContentLength == -1

resp.Uncompressed is the reliable signal that net/http performed the decompression. An empty Content-Encoding header is not: an ordinary uncompressed response has the same value.

If the application needs the compressed response instead, disable automatic compression handling:

transport := &http.Transport{
	DisableCompression: true,
}

client := &http.Client{
	Transport: transport,
}

DisableCompression controls the Transport's automatic gzip behavior. It is unrelated to connection reuse; DisableKeepAlives controls whether the Transport uses HTTP keep-alive connections.

#Application-Managed gzip

Set Accept-Encoding explicitly when the application needs to handle gzip itself.

ctx := context.Background()

req, err := http.NewRequestWithContext(
	ctx,
	http.MethodGet,
	"https://example.com",
	nil,
)
if err != nil {
	return err
}

req.Header.Set("Accept-Encoding", "gzip")

client := &http.Client{
	Timeout: 10 * time.Second,
}

resp, err := client.Do(req)
if err != nil {
	return err
}
defer resp.Body.Close()

gz, err := gzip.NewReader(resp.Body)
if err != nil {
	return fmt.Errorf("create gzip reader: %w", err)
}
defer gz.Close()

body, err := io.ReadAll(gz)
if err != nil {
	return fmt.Errorf("read gzip response: %w", err)
}

fmt.Println(len(body))

Because the application supplied Accept-Encoding, the Transport does not transparently decode the gzip response. gzip.Reader receives the compressed stream directly from resp.Body.

There are two distinct resources:

• gzip.Reader, which the application created
• resp.Body, which the HTTP client owns

gzip.Reader.Close does not close the underlying resp.Body. Both must be closed.

The defer resp.Body.Close() is registered immediately after Do succeeds, so it also covers errors from gzip.NewReader. There is no need to drain an invalid response merely because gzip initialization failed.

Reading a response to EOF and closing it has a different purpose: it can allow an HTTP/1.x keep-alive connection to be reused. Closing an unread body releases the response body, but the underlying connection may not be reusable.

#Limit the Decompressed Data

Content-Length describes the HTTP representation on the wire. It is not a safe limit for the amount of data produced by decompression.

A small gzip payload can expand into a very large response.

When Transport has already decompressed the response, put the limit directly around resp.Body:

const maxBodySize = 1 << 20 // 1 MiB

limited := io.LimitReader(resp.Body, maxBodySize+1)

body, err := io.ReadAll(limited)
if err != nil {
	return err
}

if int64(len(body)) > maxBodySize {
	return fmt.Errorf("response body exceeds %d bytes", maxBodySize)
}

Reading one byte beyond the limit distinguishes an acceptable response from an oversized one.

Do not use Content-Length for this check. After transparent decompression, resp.ContentLength is -1; even before decompression, the compressed length says nothing about the size of the decompressed data.

When gzip is handled by the application, put the limit around gzip.Reader:

limited := io.LimitReader(gz, maxBodySize+1)

body, err := io.ReadAll(limited)
if err != nil {
	return err
}

if int64(len(body)) > maxBodySize {
	return fmt.Errorf("response body exceeds %d bytes", maxBodySize)
}

The limit now applies to decompressed bytes.

If the limit is exceeded, stop reading and close resp.Body. Do not continue consuming an untrusted response simply to preserve an HTTP/1.x connection.

The trade-off is straightforward:

When a gzip.Reader is stopped before EOF, the complete gzip stream may not be consumed, so its final checksum may not be verified. That is acceptable when the response has already exceeded the application's size limit.

#Brotli and Zstandard

The standard net/http Transport provides transparent gzip handling. It does not provide equivalent automatic decompression for Brotli (br) or Zstandard (zstd).

If a client needs those encodings, decompression can be implemented in a custom http.RoundTripper.

The wrapper should:

1. call the underlying RoundTripper
2. inspect Content-Encoding
3. create the appropriate decompressor
4. replace resp.Body with an io.ReadCloser that owns both layers
5. update the response metadata to match the new body

The body wrapper should close the decompressor before closing the original response body:

type decompressedBody struct {
	reader io.Reader
	close  func() error
}

func (b *decompressedBody) Read(p []byte) (int, error) {
	return b.reader.Read(p)
}

func (b *decompressedBody) Close() error {
	return b.close()
}

For a real implementation, close should close the decompressor and then the original resp.Body, preserving the first error if both operations fail.

The response metadata must be updated consistently as well. Simply setting:

resp.Uncompressed = true

does not reproduce the standard library's transparent-decompression behavior.

If resp.Body now yields decompressed data, the compressed Content-Length must not remain as though it described the new body. Content-Encoding must likewise reflect what the application actually receives.

A reusable decompression layer should therefore define both its body ownership and its response metadata semantics explicitly.

#What to Keep in Mind

The important rules are small:

• resp.Uncompressed tells you that net/http performed transparent gzip decompression.
• Setting Accept-Encoding explicitly transfers decompression responsibility to the application.
• gzip.Reader and resp.Body have separate ownership and must both be closed.
• Size limits should apply to decompressed data when decompression can expand an untrusted response.
• Reading to EOF can preserve HTTP/1.x connection reuse; stopping early may not.
• A hard response-size limit should not be weakened just to preserve a keep-alive connection.