#Example: HTTP Client Read a Response Safely

Reading an HTTP response looks simple:

body, err := io.ReadAll(resp.Body)

The problem is not reading the bytes. The problem is deciding how many bytes to accept, how long to wait, what to do with errors, and who owns the response body.

This example focuses on safe response handling for production Go clients: bounded memory, status-aware handling, streaming large downloads, compressed responses, and correct body lifecycle management.

The patterns here work with Go 1.16 and later. The examples were verified with Go 1.27.1.

#Quick Example

For a small API response with a known practical size limit:

package main

import (
	"fmt"
	"io"
	"net/http"
)

const maxResponseSize = 1 << 20 // 1 MiB

func main() {
	client := &http.Client{}

	resp, err := client.Get("https://example.com/api/data")
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
		const maxErrorBody = 8 << 10 // 8 KiB

		body, err := io.ReadAll(io.LimitReader(resp.Body, maxErrorBody+1))
		if err != nil {
			panic(err)
		}

		truncated := int64(len(body)) > maxErrorBody
		if truncated {
			body = body[:maxErrorBody]
		}

		panic(fmt.Sprintf(
			"unexpected HTTP status %s: %s (truncated=%t)",
			resp.Status,
			string(body),
			truncated,
		))
	}

	if resp.ContentLength > maxResponseSize {
		panic(fmt.Sprintf(
			"response too large: %d bytes",
			resp.ContentLength,
		))
	}

	body, err := io.ReadAll(
		io.LimitReader(resp.Body, maxResponseSize+1),
	)
	if err != nil {
		panic(err)
	}

	if int64(len(body)) > maxResponseSize {
		panic(fmt.Sprintf(
			"response exceeds %d bytes",
			maxResponseSize,
		))
	}

	fmt.Println(string(body))
}

The important ordering is:

Do / Get
   │
   ├── error ───────────────→ return
   │
   └── response
        │
        ├── establish Body.Close() lifecycle
        │
        ├── check HTTP status
        │
        ├── reject obviously oversized ContentLength
        │
        ├── read through a hard size limit
        │
        └── return
              │
              └── Body.Close()

A successful http.Client.Do gives you a response body that must be closed. HTTP status is a separate concern from transport success: a 500 response is still a successfully received HTTP response.

The example uses panic only to keep the executable example short. Application and library code should normally return errors.

#resp.Body Is a Stream

resp.Body implements io.ReadCloser.

It is normally a one-shot stream:

body, err := io.ReadAll(resp.Body)

After that, the bytes have been consumed.

A second read does not return the same data:

body1, _ := io.ReadAll(resp.Body)
body2, _ := io.ReadAll(resp.Body)

// body2 is empty.

If multiple parts of your program need the response, read it once into a bounded buffer and pass the resulting bytes around.

Do not assume that resp.Body can be rewound.

#Always Close the Body

Once Do returns a non-nil response, establish the close lifecycle immediately:

resp, err := client.Do(req)
if err != nil {
	return err
}
defer resp.Body.Close()

This remains correct when later processing returns an error:

if resp.StatusCode != http.StatusOK {
	return fmt.Errorf("unexpected status: %s", resp.Status)
}

The deferred Close still runs.

Closing the body is a resource-lifecycle operation. It is separate from whether the response is a 2xx, whether the body is valid JSON, or whether the body passed your size check.

When practical, consuming the body to EOF can also allow the underlying connection to be reused. If you reject a response immediately because its declared size is too large, you may intentionally sacrifice connection reuse.

#io.ReadAll Is Fine When the Limit Is Real

This is reasonable:

body, err := io.ReadAll(
	io.LimitReader(resp.Body, maxResponseSize+1),
)

The important part is the limit.

This is dangerous for an untrusted or otherwise unbounded response:

body, err := io.ReadAll(resp.Body)
// No size limit on an untrusted response.

A server can return a much larger body than expected. io.ReadAll will keep growing the byte slice until EOF or an error.

The +1 is intentional:

maximum accepted size = 1 MiB

read up to 1 MiB + 1
              │
              ├── ≤ 1 MiB → accept
              │
              └── > 1 MiB → reject

If you read only maxSize bytes, you cannot distinguish:

exactly maxSize bytes

from:

more than maxSize bytes

Reading one extra byte makes the boundary observable.

#ContentLength Is an Early Check, Not a Size Limit

If the server declares a useful Content-Length:

if resp.ContentLength > maxResponseSize {
	return fmt.Errorf("response too large")
}

This can reject an obviously oversized response before reading it.

But it is not a replacement for a bounded reader.

ContentLength may be unknown:

resp.ContentLength == -1

Chunked responses, streaming responses, and other cases can have no known length.

Therefore the robust pattern is:

ContentLength
    │
    ├── known and too large → reject early
    │
    └── unknown or acceptable
             │
             ▼
       bounded reader
             │
             ▼
       enforce actual size

The actual stream still needs a limit.

#Do Not Treat ContentLength as the Decompressed Size

Compression makes this distinction particularly important.

With transparent gzip handling by net/http, the data flow is approximately:

compressed bytes on the wire
        │
        ▼
   HTTP transport
        │
        │ decompresses
        ▼
     resp.Body
        │
        │ decompressed bytes
        ▼
   your size limit

When the transport transparently decompresses gzip content, resp.Uncompressed is true and resp.ContentLength is set to -1.

That means an early ContentLength check cannot tell you how large the decompressed response will become.

The hard limit must therefore apply to the stream you actually consume.

For example:

body, err := io.ReadAll(
	io.LimitReader(resp.Body, maxResponseSize+1),
)

This limits the decompressed data exposed through resp.Body.

#Manual gzip handling

If you explicitly handle gzip yourself, there are two different sizes:

compressed response
       │
       ▼
compressed-size limit
       │
       ▼
gzip reader
       │
       ▼
decompressed-size limit

For example:

compressed := io.LimitReader(resp.Body, maxCompressedSize+1)

gz, err := gzip.NewReader(compressed)
if err != nil {
	return fmt.Errorf("create gzip reader: %w", err)
}
defer gz.Close()

body, err := io.ReadAll(
	io.LimitReader(gz, maxResponseSize+1),
)
if err != nil {
	return fmt.Errorf("read decompressed response: %w", err)
}

A limit on compressed bytes alone does not impose the same limit on decompressed bytes.

#Hard Limits and Memory Allocation

A bounded io.ReadAll prevents unbounded growth, but the peak allocation can still be larger than the logical response limit because slices grow geometrically.

For example:

body, err := io.ReadAll(
	io.LimitReader(resp.Body, maxResponseSize+1),
)

with a 1 MiB limit is usually perfectly reasonable for a small API response.

If the allowed size is hundreds of megabytes, however, io.ReadAll is usually the wrong design. Stream the data instead.

You may also see this optimization:

var buf bytes.Buffer
buf.Grow(int(maxResponseSize))

Pre-allocation can reduce reallocations, but it is only appropriate when the maximum size is small and practical to allocate up front.

Do not blindly preallocate hundreds of megabytes for untrusted or highly concurrent requests. For example, 100 concurrent requests with a 500 MiB preallocation can create enormous memory pressure even when the actual responses are tiny.

A size limit protects you from unbounded input. It does not automatically make an enormous preallocation safe.

#Size Does Not Bound Time

A response can be small but extremely slow.

For example:

server sends 100 bytes
      │
      │ one byte every few seconds
      ▼
request remains active for a long time

A size limit controls memory and input volume.

It does not impose a time limit.

Use a request context for request-specific cancellation:

ctx, cancel := context.WithTimeout(
	context.Background(),
	10*time.Second,
)
defer cancel()

req, err := http.NewRequestWithContext(
	ctx,
	http.MethodGet,
	"https://example.com/api/data",
	nil,
)
if err != nil {
	return err
}

resp, err := client.Do(req)
if err != nil {
	return err
}
defer resp.Body.Close()

A client-level timeout is another option:

client := &http.Client{
	Timeout: 10 * time.Second,
}

Client.Timeout covers the overall request lifecycle, including connection establishment, redirects, response headers, and body reading.

A request context is preferable when the timeout or cancellation belongs to one particular operation.

For more HTTP client patterns, see the HTTP examples on gobase.net.

#Streaming Large Downloads

Do not load a large file into memory just because it arrived through HTTP.

Stream it directly to disk.

A safe download also needs protection against oversized responses.

func download(
	resp *http.Response,
	path string,
	maxSize int64,
) (err error) {
	tmp, err := os.CreateTemp(filepath.Dir(path), ".download-*")
	if err != nil {
		return fmt.Errorf("create temporary file: %w", err)
	}

	tmpName := tmp.Name()

	defer func() {
		if closeErr := tmp.Close(); err == nil && closeErr != nil {
			err = fmt.Errorf("close temporary file: %w", closeErr)
		}
		if err != nil {
			_ = os.Remove(tmpName)
		}
	}()

	written, err := io.Copy(
		tmp,
		io.LimitReader(resp.Body, maxSize+1),
	)
	if err != nil {
		return fmt.Errorf("download response: %w", err)
	}

	if written > maxSize {
		return fmt.Errorf("download exceeds %d bytes", maxSize)
	}

	if err := tmp.Sync(); err != nil {
		return fmt.Errorf("sync temporary file: %w", err)
	}

	if err := tmp.Close(); err != nil {
		return fmt.Errorf("close temporary file: %w", err)
	}

	if err := os.Rename(tmpName, path); err != nil {
		return fmt.Errorf("rename temporary file: %w", err)
	}

	return nil
}

The important part is:

written, err := io.Copy(
	tmp,
	io.LimitReader(resp.Body, maxSize+1),
)

This means:

resp.Body
   │
   ▼
LimitReader(maxSize + 1)
   │
   ▼
io.Copy
   │
   ▼
temporary file

If the response ends normally before the limit, io.Copy returns successfully.

If more than maxSize bytes are available, the limited reader allows one extra byte through. written > maxSize then detects the oversized response.

This is also a useful distinction from io.ReadFull: io.CopyN and io.ReadFull have different EOF semantics. Here, io.Copy plus io.LimitReader makes the intended “copy until EOF, but never expose more than N bytes” behavior explicit.

#Why a temporary file?

This is unsafe for a production download:

file, err := os.Create(path)
if err != nil {
	return err
}

_, err = io.Copy(file, resp.Body)
if err != nil {
	return err
}

os.Create truncates an existing destination immediately.

If the network fails halfway through, the original file has already been destroyed.

The temporary-file pattern provides:

download
   │
   ▼
temporary file
   │
   ├── failure → delete temporary file
   │
   └── success
          │
          ├── Sync
          ├── Close
          └── Rename → final path

The temporary file should normally be created in the same directory as the destination so that the final rename has the appropriate filesystem semantics.

os.Rename replacement and atomicity details vary by platform and filesystem. If durability across sudden power loss matters, syncing the file is only part of the durability story; directory synchronization and filesystem-specific behavior may also matter.

#Bounded Draining

Sometimes you receive an HTTP error response and want to consume a small amount of the body before returning:

_, _ = io.CopyN(
	io.Discard,
	resp.Body,
	4<<10,
)

This can help connection reuse for small responses.

But it is not guaranteed to preserve reuse.

If the remaining response body is larger than the drain budget, the body will not reach EOF. The transport may then discard the connection.

So think of bounded draining as:

small remaining body
       │
       ▼
   drain to EOF
       │
       ▼
possible connection reuse

not:

drain 4 KiB
    ↓
connection definitely reusable

Also, if the response is intentionally rejected because its declared size is far too large, do not spend significant time reading it merely to preserve connection reuse.

#Error Bodies Need Their Own Limit

Error responses deserve a separate limit.

A 500 response might contain:

• stack traces
• internal paths
• database errors
• credentials accidentally included by an upstream service
• user-supplied data

Do not blindly log the entire response body.

A useful pattern is:

const maxErrorBody = 8 << 10

body, err := io.ReadAll(
	io.LimitReader(resp.Body, maxErrorBody+1),
)
if err != nil {
	return fmt.Errorf("read error response: %w", err)
}

truncated := int64(len(body)) > maxErrorBody
if truncated {
	body = body[:maxErrorBody]
}

return fmt.Errorf(
	"upstream returned %s: %s (truncated=%t)",
	resp.Status,
	string(body),
	truncated,
)

The truncated flag matters.

A truncated JSON response, for example, is not necessarily valid JSON and should not be presented as if it were the complete server response.

#Status and Body Are Separate

Transport success does not mean application success.

This:

resp, err := client.Do(req)

answers:

Did the HTTP exchange produce a response?

It does not answer:

Did the server accept my operation?

You normally need both:

if err != nil {
	return err
}

defer resp.Body.Close()

if resp.StatusCode < 200 || resp.StatusCode >= 300 {
	// Handle application-level failure.
}

Likewise, a 200 response does not guarantee valid JSON, valid UTF-8, a correct schema, or an acceptable payload size.

HTTP status, body size, body format, and application semantics are separate validation boundaries.

#JSON Responses

For small JSON responses, a bounded reader can be passed to json.Decoder:

decoder := json.NewDecoder(
	io.LimitReader(resp.Body, maxResponseSize+1),
)

var result Response
if err := decoder.Decode(&result); err != nil {
	return fmt.Errorf("decode response: %w", err)
}

But there is an important subtlety.

Decoder.Decode can stop after decoding the first complete JSON value. Therefore, decoding one value does not by itself prove that the entire response stayed within your intended size boundary.

If the contract requires the entire response to contain exactly one JSON value and remain within the size limit, you need to account for the remaining input as well.

For example:

decoder := json.NewDecoder(
	io.LimitReader(resp.Body, maxResponseSize+1),
)

var result Response
if err := decoder.Decode(&result); err != nil {
	return fmt.Errorf("decode response: %w", err)
}

var extra any
if err := decoder.Decode(&extra); !errors.Is(err, io.EOF) {
	if err == nil {
		return fmt.Errorf("response contains multiple JSON values")
	}
	return fmt.Errorf("check trailing JSON: %w", err)
}

For APIs where the complete bounded response must be retained anyway, another simple option is:

bounded body
     │
     ▼
ReadAll
     │
     ▼
json.Unmarshal

The trade-off is memory: retaining raw bytes while also holding the decoded object can increase peak memory substantially.

#resp.Body == nil

For responses returned by the standard HTTP client, Response.Body is normally non-nil.

A defensive helper may still protect itself against a manually constructed response:

if resp.Body == nil {
	return nil, errors.New("response body is nil")
}

This is mainly useful when testing or handling hand-built http.Response values.

Do not treat it as normal behavior from http.Client.Do.

#A Reusable Helper

If many callers need the same response-size policy, centralize it:

var ErrResponseTooLarge = errors.New("response too large")

func ReadResponseBody(
	resp *http.Response,
	maxSize int64,
) ([]byte, error) {
	if resp == nil {
		return nil, errors.New("nil response")
	}

	if resp.Body == nil {
		return nil, errors.New("nil response body")
	}

	if maxSize < 0 || maxSize == math.MaxInt64 {
		return nil, fmt.Errorf("invalid max response size: %d", maxSize)
	}

	defer resp.Body.Close()

	if resp.ContentLength > maxSize {
		return nil, ErrResponseTooLarge
	}

	body, err := io.ReadAll(
		io.LimitReader(resp.Body, maxSize+1),
	)
	if err != nil {
		return nil, fmt.Errorf("read response body: %w", err)
	}

	if int64(len(body)) > maxSize {
		return nil, ErrResponseTooLarge
	}

	return body, nil
}

The helper deliberately does not interpret the HTTP status code.

The caller decides whether the response is successful:

resp, err := client.Do(req)
if err != nil {
	return err
}

body, err := ReadResponseBody(resp, 1<<20)
if err != nil {
	return err
}

if resp.StatusCode < 200 || resp.StatusCode >= 300 {
	return fmt.Errorf("unexpected status: %s", resp.Status)
}

In real code, status handling should normally happen before or alongside body-policy decisions so that error responses can use a smaller error-body limit when appropriate.

The important contract is:

ReadResponseBody limits bytes. It does not define application-level success.

#Common Mistakes

#Reading an untrusted response without a limit

body, err := io.ReadAll(resp.Body)
// No size limit on an untrusted response.

Use a bounded reader.

#Checking only ContentLength

if resp.ContentLength > maxSize {
	return errTooLarge
}

ContentLength == -1 means the length is unknown.

Use a bounded reader as the actual enforcement mechanism.

#Assuming a size limit is a timeout

io.LimitReader(resp.Body, maxSize)

This limits bytes, not time.

Use request context or Client.Timeout for time bounds.

#Reading the body and then trying to decode it again

body, _ := io.ReadAll(resp.Body)
json.NewDecoder(resp.Body).Decode(&result)

The second operation sees an already-consumed stream.

Decode directly, or decode from the saved body.

#Logging the complete error body

return fmt.Errorf("upstream error: %s", body)

Error bodies may contain sensitive information and may be unexpectedly large.

Bound and sanitize them.

#Writing directly to the final download path

file, _ := os.Create(path)

A failed download can destroy the existing file.

Write to a temporary file and rename only after successful completion.

#Preallocating an enormous buffer

var buf bytes.Buffer
buf.Grow(int(maxSize))

Do not turn a logical size limit into a large upfront memory allocation, especially for untrusted and concurrent requests.

#Forgetting that compression changes what you measure

A compressed response can be small on the wire and much larger after decompression.

Apply the relevant limit to the stream whose size you actually need to control.

#Rule of Thumb

#Key Takeaways

• resp.Body is a one-shot stream and must be closed.
• HTTP transport success and HTTP application success are different checks.
• ContentLength is an early hint, not a complete size-enforcement mechanism.
• io.LimitReader(maxSize+1) lets you detect responses larger than the allowed limit.
• A size limit controls memory/input volume; it does not control how long a request can take.
• Transparent gzip means resp.Body can contain decompressed data even when the wire representation was much smaller.
• Large downloads should be streamed rather than buffered in memory.
• Download to a temporary file and publish it only after the transfer succeeds.
• Error bodies should have their own smaller limit and should not be logged blindly.
• Avoid large upfront bytes.Buffer.Grow allocations for untrusted or highly concurrent input.
• Keep size, time, status, parsing, and ownership as separate boundaries.