#Go I/O in Production: io, bufio, and the Standard Library I/O Stack

Go's I/O model is deliberately small.

At the center are two interfaces:

type Reader interface {
    Read(p []byte) (n int, err error)
}

type Writer interface {
    Write(p []byte) (n int, err error)
}

These two interfaces are enough to connect files, sockets, HTTP bodies, in-memory buffers, compression streams, encoders, and application-specific data sources without making application code depend on any particular implementation.

The important part is not memorizing io.Copy, io.ReadAll, bufio.Reader, or io.TeeReader.

It is understanding the contracts behind them:

• when data should remain a stream instead of becoming a []byte;
• when a read may be partial;
• when an exact byte count is required;
• where input boundaries should be enforced;
• when buffering actually helps;
• how multiple I/O components can be composed;
• how errors and resource lifetimes propagate;
• and when a high-level API can transparently reach a lower-level operating-system optimization.

Go 1.26 makes some of these details particularly interesting. io.ReadAll has received substantial allocation and performance improvements, while the existing io.Copy machinery continues to expose implementation-specific fast paths such as WriterTo, ReaderFrom, and OS-level zero-copy mechanisms.

The result is a small API surface that can still express surprisingly sophisticated production I/O pipelines.

#1. The Go I/O Stack

The io package does not open files, create TCP connections, or implement a network protocol.

Its primary job is to define common interfaces and operations that work on those interfaces.

Other standard-library packages provide concrete implementations.

The relationship is roughly:

                         io
                  Reader / Writer
                         │
          ┌──────────────┼──────────────┐
          │              │              │
         os             net           bytes
          │              │              │
         File           Conn          Buffer
          │              │              │
          └──────────────┼──────────────┘
                         │
                       bufio
                         │
                buffering / scanning
                         │
                         ▼
                    application

The key idea is simple:

io defines the common language. Other packages provide concrete sources, destinations, and transformations.

#2. io.Reader Is the Core Abstraction

The most important interface in Go I/O is probably the smallest:

type Reader interface {
    Read(p []byte) (n int, err error)
}

The caller provides the buffer.

The Reader fills as much of it as it currently can and reports how many bytes it actually produced.

For example:

buf := make([]byte, 32*1024)

n, err := r.Read(buf)

Only this part is newly read data:

buf[:n]

The fact that the buffer is 32 KiB does not mean Read will return 32 KiB.

It may return:

32 KiB
4 KiB
137 bytes

or another valid amount.

That is not an implementation accident. It is part of the Reader contract.

#3. A Read Does Not Mean "Fill This Buffer"

This is one of the most common mistakes in low-level Go I/O:

buf := make([]byte, 1024)

_, err := r.Read(buf)
if err != nil {
    return err
}

process(buf)

The Reader is allowed to return:

n = 137

In that case, only:

buf[:137]

contains the bytes returned by this call.

Correct low-level handling looks like:

n, err := r.Read(buf)

if n > 0 {
    process(buf[:n])
}

if err != nil {
    return err
}

There is another important detail:

n > 0
err != nil

is a valid result.

The caller must not discard the bytes merely because an error was also returned.

This is one reason hand-written Reader loops are easy to get subtly wrong.

#4. io.EOF Means End of Stream

io.EOF has a specific meaning:

The Reader has no more data to provide.

It is not a generic I/O failure.

A manual Reader loop therefore looks like:

buf := make([]byte, 32*1024)

for {
    n, err := r.Read(buf)

    if n > 0 {
        process(buf[:n])
    }

    if err == io.EOF {
        break
    }

    if err != nil {
        return err
    }
}

The ordering matters:

1. process returned bytes;
2. handle EOF;
3. handle other errors.

In application code, however, you should avoid writing this loop unless you actually need this level of control. For ordinary stream transfers, io.Copy is the better abstraction.

#5. io.Copy: More Than a Convenient Copy Loop

If the operation is simply:

Copy everything from this Reader to this Writer.

use:

_, err := io.Copy(dst, src)

For example:

src, err := os.Open("source.dat")
if err != nil {
    return err
}
defer src.Close()

dst, err := os.Create("target.dat")
if err != nil {
    return err
}
defer dst.Close()

_, err = io.Copy(dst, src)
return err

The obvious benefit is that io.Copy removes the repetitive Reader loop.

The more important benefit is that it preserves an optimization boundary.

io.Copy first checks whether the source implements io.WriterTo. If not, it checks whether the destination implements io.ReaderFrom. Only then does it fall back to the generic buffered copy path.

Conceptually:

io.Copy
   │
   ├── src implements WriterTo?
   │        └── use src.WriteTo
   │
   ├── dst implements ReaderFrom?
   │        └── use dst.ReadFrom
   │
   └── generic copy loop

This is important because those optional interfaces can take the operation far below the generic Read/Write level.

#6. io.Copy Can Reach the Kernel

This is one of the most useful details to understand if you care about production performance.

Consider:

io.Copy(conn, file)

where:

src = *os.File
dst = *net.TCPConn

At the API level, this still looks like:

Reader → Writer

But the actual execution path can be very different.

On supported systems, the concrete implementations can provide specialized ReadFrom or WriteTo methods. On Linux, Go's networking code can use kernel facilities such as sendfile and splice to reduce copying through user space.

For example, Go's Linux networking implementation tries splice first, then sendfile, and finally falls back to a generic implementation when the specialized path cannot handle the operation.

The underlying sendfile implementation transfers data from a file descriptor to another file descriptor, while splice uses a kernel pipe buffer to minimize copies to and from user space.

So the important lesson is not:

"io.Copy is a zero-copy API."

That would be too strong.

The accurate statement is:

io.Copy provides an abstraction boundary through which concrete implementations can select specialized, OS-level data-transfer mechanisms when the source, destination, and platform permit them.

A generic io.Reader/io.Writer pair may still use the ordinary buffered path.

This is precisely why application code should normally prefer io.Copy over manually rebuilding the copy loop: the higher-level code does not need to know which optimized path is available.

#7. Zero-Copy Does Not Mean "No Bytes Are Ever Copied"

The term "zero-copy" is useful but easy to abuse.

For a traditional user-space copy, the path may look like:

disk
  ↓
kernel
  ↓
user-space buffer
  ↓
kernel
  ↓
socket

A kernel-assisted path can avoid the user-space staging copy:

disk
  ↓
kernel
  ↓
socket

The exact mechanics depend on the operating system, device, filesystem, network stack, and system call being used.

So for production documentation, avoid claims such as:

"The data is never copied."

Prefer:

"The transfer can avoid unnecessary copies through user space."

That distinction matters when performance claims are being benchmarked or reviewed.

#8. io.CopyBuffer: When You Actually Need Control Over the Buffer

Most applications should start with:

io.Copy(dst, src)

If you need to supply and reuse a particular buffer, use:

buf := make([]byte, 64*1024)

_, err := io.CopyBuffer(dst, src, buf)

This can make sense when:

• buffer allocation must be controlled;
• a buffer is already available;
• several copy operations intentionally share a buffer;
• memory behavior needs to be explicitly bounded.

One important detail:

If either side provides WriterTo or ReaderFrom, the supplied buffer may not be used at all. io.CopyBuffer retains the optimized interface-based path rather than forcing every copy through the caller-provided buffer.

So CopyBuffer is not a way to disable the standard library's optimized transfer mechanisms.

#9. io.ReadAll: Convenient, and Better in Go 1.26

io.ReadAll is appropriate when the application genuinely needs the complete input:

data, err := io.ReadAll(r)
if err != nil {
    return err
}

Typical examples include:

• small configuration files;
• small HTTP responses;
• test fixtures;
• bounded request bodies;
• data that must be parsed as one complete byte sequence.

Go 1.26 significantly improved io.ReadAll. The implementation now uses less intermediate memory and produces a more appropriately sized final slice. The Go 1.26 release notes report that it is often about twice as fast and typically allocates around half as much total memory, with larger benefits for larger inputs.

That is a real improvement.

But it does not change the fundamental data model:

io.Reader
    │
    ▼
io.ReadAll
    │
    ▼
[]byte

The entire result still has to exist in memory.

Therefore the right production rule is not:

"Avoid io.ReadAll because it allocates too much."

It is:

Use io.ReadAll when the application needs an in-memory representation, and establish an input-size boundary when the input is not already bounded.

Go 1.26 makes ReadAll cheaper. It does not make an unbounded stream bounded.

#10. io.LimitReader: Put a Boundary Around External Input

Suppose an endpoint accepts at most 10 MiB:

r := io.LimitReader(body, 10<<20)

data, err := io.ReadAll(r)
if err != nil {
    return err
}

This prevents ReadAll from consuming an arbitrarily large amount of the underlying stream.

LimitReader is therefore useful as a boundary in a pipeline:

external input
      │
      ▼
LimitReader
      │
      ▼
application

But it is important to understand what it does not do.

It limits what the wrapped Reader exposes. It does not by itself tell the application whether the original stream contained additional bytes beyond the limit.

If the application must reject oversized input rather than merely stop reading after the limit, it needs an explicit overflow-detection strategy.

#11. io.ReadFull: When the Protocol Requires Exactly N Bytes

General stream reads may be short.

Protocols often are not.

Suppose a binary protocol defines a 32-byte header:

header := make([]byte, 32)

This is not enough:

_, err := r.Read(header)

Use:

_, err := io.ReadFull(r, header)
if err != nil {
    return err
}

The semantic difference is:

Read
    "Return some data from the stream."

ReadFull
    "Keep reading until this buffer is full,
     or until the operation cannot complete."

This distinction matters for:

• binary protocol headers;
• fixed-size records;
• framing;
• length-prefixed structures;
• cryptographic or serialization formats with fixed-width fields.

#12. io.MultiReader: Concatenate Streams

Suppose three Readers contain:

A
B
C

and the consumer should see:

A + B + C

Use:

r := io.MultiReader(a, b, c)

The resulting Reader presents the inputs as one logical stream:

A ──┐
B ──┼──→ MultiReader ──→ consumer
C ──┘

No intermediate combined []byte is required.

This is useful for:

• combining a header and body;
• concatenating files;
• constructing test streams;
• combining protocol fragments;
• composing generated and existing data.

#13. io.MultiWriter: Sequential Fan-Out, Not a Transaction

io.MultiWriter performs the opposite kind of composition:

w := io.MultiWriter(a, b, c)

A write is forwarded to each destination in order.

                 ┌──→ A
application ───→ MultiWriter
                 ├──→ B
                 └──→ C

The important part is the failure behavior.

If one Writer returns an error, MultiWriter stops and returns that error. It does not roll back previous writes or continue to later Writers.

Therefore:

A succeeds
B fails
C is not written

is a valid outcome.

MultiWriter should be understood as sequential fan-out, not atomic replication.

It also introduces latency coupling: a slow destination can slow down the whole write operation.

#14. io.TeeReader: Add a Synchronous Side Effect to a Read Path

TeeReader lets data be copied to another Writer while it is being read.

Conceptually:

                 ┌──→ consumer
                 │
source ──→ TeeReader
                 │
                 └──→ Writer

For example, calculating a SHA-256 digest while consuming a stream:

hash := sha256.New()

tee := io.TeeReader(src, hash)

if _, err := io.Copy(io.Discard, tee); err != nil {
    return err
}

sum := hash.Sum(nil)

No complete copy of the input is kept in memory.

But the side effect is synchronous.

The Writer is part of the read path.

If the Writer fails, the Reader operation reports that failure.

This means TeeReader is useful for:

• checksums;
• hashes;
• audit copies;
• instrumentation;
• streaming transformations.

It is not an asynchronous replication mechanism.

#15. io.Discard: A Real Writer for "Read and Ignore"

Sometimes the application needs to consume a stream but does not need the bytes.

Instead of allocating a buffer and manually discarding the data:

_, err := io.Copy(io.Discard, r)

io.Discard is an io.Writer whose writes succeed without retaining the data.

This is particularly useful when:

• draining a stream;
• benchmarking read performance;
• consuming data for side effects;
• using TeeReader only to compute a hash or metric.

It also makes the intent obvious:

read everything, keep nothing

#16. io.NopCloser: Adapt a Reader to a ReadCloser

Many APIs require:

io.ReadCloser

but the application only has:

io.Reader

For example:

r := strings.NewReader("hello")

A strings.Reader does not own an external resource that needs closing.

When an API still requires a ReadCloser, use:

rc := io.NopCloser(r)

Now:

rc.Read(...)
rc.Close()

works, and Close simply does nothing.

This is particularly useful when constructing values such as:

req.Body = io.NopCloser(bytes.NewReader(data))

io.NopCloser was added in Go 1.16 and is the standard adapter for this situation.

The important design point is:

NopCloser adapts an interface contract; it does not create resource ownership where none existed.

#17. bufio Is a Layer, Not a Replacement for io

bufio is best understood as a layer around an existing Reader or Writer.

For example:

file, err := os.Open("data.txt")
if err != nil {
    return err
}
defer file.Close()

r := bufio.NewReader(file)

The data path becomes:

os.File
   │
   ▼
bufio.Reader
   │
   ▼
application

The underlying file remains the actual source.

bufio.Reader adds buffering and higher-level access patterns.

#18. Why Buffering Exists

Suppose an application performs many small reads:

Read 1 byte
Read 1 byte
Read 1 byte
Read 1 byte
...

Calling the underlying I/O implementation for every operation may be unnecessarily expensive.

A buffered Reader can fetch a larger block:

underlying Reader
       │
       ▼
┌─────────────────┐
│ buffered bytes  │
└─────────────────┘
       │
       ├── application read
       ├── application read
       ├── application read
       └── application read

This is useful for:

• line-oriented input;
• text protocols;
• delimiter-based parsing;
• look-ahead;
• Peek;
• ReadByte;
• other small-granularity operations.

The key point is:

Buffering is primarily about I/O access patterns and granularity.

It is not a badge of performance.

#19. bufio.Reader: When You Need Fine-Grained Reads

For line-oriented data:

r := bufio.NewReader(file)

for {
    line, err := r.ReadString('\n')

    if len(line) > 0 {
        process(line)
    }

    if err == io.EOF {
        break
    }

    if err != nil {
        return err
    }
}

ReadString('\n') returns the delimiter when it is present.

That is an important semantic difference from Scanner.

bufio.Reader also provides:

ReadByte
ReadRune
ReadBytes
ReadString
ReadSlice
ReadLine
Peek
UnreadByte
UnreadRune

This makes it a better fit when the parser needs explicit control over boundaries and buffered data.

#20. bufio.Scanner: Convenient, but Its Token Model Matters

Scanner is not merely a "line reader."

Its default split function is ScanLines, but the split function can be changed:

scanner := bufio.NewScanner(r)
scanner.Split(bufio.ScanWords)

Or the application can provide its own SplitFunc.

The standard library provides split functions for:

ScanLines
ScanWords
ScanRunes

and custom tokenization is also supported.

The default is:

scanner := bufio.NewScanner(r)

which is equivalent to using ScanLines as the split function.

This makes Scanner a tokenization API rather than merely a line-reading API.

#21. Scanner and Reader Have Different Line Semantics

This distinction matters when parsing protocols.

bufio.ScanLines removes the line ending.

Its line-ending definition is:

\r?\n

So both:

LF
CRLF

are stripped from the returned token.

By contrast:

line, err := reader.ReadString('\n')

returns the delimiter as part of the returned string when the delimiter is found.

So:

input:
"Content-Length: 123\r\n"

can become:

Scanner:
"Content-Length: 123"

ReadString('\n'):
"Content-Length: 123\r\n"

That difference is easy to dismiss in ordinary text processing.

It is not necessarily harmless in protocol parsing.

When the exact wire representation matters, explicitly choose an API whose boundary semantics match the protocol.

#22. Scanner Also Has a Token-Size Boundary

Scanner stops when a token is too large for its configured buffer.

The default maximum token size is finite.

For inputs with potentially large records, configure it explicitly:

scanner := bufio.NewScanner(r)

scanner.Buffer(
    make([]byte, 64*1024),
    10*1024*1024,
)

The standard library documents Buffer as controlling both the initial buffer and the maximum buffer size that may be allocated during scanning.

There is another important difference from Reader:

Once Scanner stops because of EOF, an I/O error, or an oversized token, the scan is unrecoverable, and the underlying Reader may have advanced beyond the last token.

For applications that need tighter control over buffering, large tokens, or recovery behavior, bufio.Reader is often the better primitive.

#23. bufio.Writer: Batch Small Writes

The same principle applies on the write side.

Without buffering:

small write
small write
small write
small write
...

With bufio.Writer:

application
    │
    ▼
bufio.Writer
    │
    ▼
buffer
    │
    │ Flush
    ▼
underlying Writer

Example:

bw := bufio.NewWriter(w)

fmt.Fprintln(bw, "line 1")
fmt.Fprintln(bw, "line 2")
fmt.Fprintln(bw, "line 3")

if err := bw.Flush(); err != nil {
    return err
}

This is useful when the application naturally produces many small writes.

It is not automatically useful for every large stream copy.

For:

Reader → Writer

start with:

io.Copy(dst, src)

and introduce bufio when the access pattern justifies it.

#24. Flush Is Part of the Write Contract

This code is incomplete:

bw := bufio.NewWriter(w)

fmt.Fprintln(bw, "hello")

return nil

The write may still exist only inside the buffered Writer.

The final step is:

if err := bw.Flush(); err != nil {
    return err
}

This matters because:

Write to bufio.Writer
        ≠
Write to underlying destination

A successful buffered write means the data was accepted by the buffer.

Flush is where buffered data is pushed to the underlying Writer, and that operation can fail.

#25. bufio.Reader.Reset and Writer.Reset: Reuse the Buffer

Both bufio.Reader and bufio.Writer support Reset.

For a Reader:

br.Reset(r)

For a Writer:

bw.Reset(w)

Reset discards the previous buffered state and binds the existing object to a new underlying Reader or Writer.

This matters in high-throughput services where the same buffered object can safely be reused across independent operations.

For example:

var br bufio.Reader

br.Reset(r)
// use br

br.Reset(nextR)
// reuse the same buffer

The main benefit is not that Reset makes individual reads faster.

It is that the buffered object and, importantly, its backing storage can be reused instead of allocating another buffer for every operation.

#26. sync.Pool Can Reuse bufio Objects — But Only When It Pays

In a high-throughput server, a common pattern is to combine sync.Pool with Reset:

var readerPool = sync.Pool{
    New: func() any {
        return bufio.NewReaderSize(nil, 32*1024)
    },
}

func handle(r io.Reader) {
    br := readerPool.Get().(*bufio.Reader)
    defer readerPool.Put(br)

    br.Reset(r)

    // use br
}

The same idea applies to bufio.Writer.

The important engineering qualification is:

Do this because allocation and garbage collection have been shown to matter, not because every request needs a Pool.

sync.Pool is deliberately opportunistic. Objects may disappear from the pool, and retaining large buffers in a pool can increase memory usage.

A good progression is:

correct implementation
        ↓
benchmark / profile
        ↓
identify allocation pressure
        ↓
reuse with Reset / sync.Pool
        ↓
measure again

Do not turn pooling into a default coding convention.

#27. bufio and io.Copy Solve Different Problems

This code:

br := bufio.NewReader(src)
bw := bufio.NewWriter(dst)

_, err := io.Copy(bw, br)
if err != nil {
    return err
}

return bw.Flush()

can be correct.

But if the requirement is simply:

Reader → Writer

start with:

_, err := io.Copy(dst, src)

The distinction is:

io.Copy
    → moves a stream

bufio
    → changes the access pattern

Use bufio when you need:

• many small reads;
• many small writes;
• line-oriented access;
• look-ahead;
• delimiter parsing;
• explicit buffering;
• reuse of buffered state.

Do not add it simply because the input is large.

#28. os.File Fits Directly Into the I/O Model

An *os.File can participate directly in generic I/O code.

For example:

func copyData(dst io.Writer, src io.Reader) error {
    _, err := io.Copy(dst, src)
    return err
}

This function can work with:

*os.File
net.Conn
bytes.Buffer
HTTP body
compression stream
custom Reader/Writer

The concrete implementation is deliberately hidden behind the interface.

That is the practical value of the abstraction.

#29. bytes.Reader and bytes.Buffer: I/O in Memory

bytes.Reader turns a byte slice into a Reader:

r := bytes.NewReader(data)

This is useful when a function expects:

io.Reader

For example:

func parse(r io.Reader) error {
    // ...
}

err := parse(bytes.NewReader(data))

bytes.Buffer is useful as an in-memory destination:

var buf bytes.Buffer

if err := writeReport(&buf); err != nil {
    return err
}

data := buf.Bytes()

The same abstraction makes testing easier.

A function that accepts io.Reader does not need a real file or network connection merely to test its parsing logic.

#30. strings.Reader: Strings Can Enter the Same Pipeline

A string can also be exposed as an io.Reader:

r := strings.NewReader("hello world")

This lets the same processing code accept:

file
network
memory
string

without changing its interface.

That is exactly what small interfaces are supposed to accomplish.

#31. io.WriterTo and io.ReaderFrom: Optional Fast Paths

The interfaces behind io.Copy deserve special attention:

type WriterTo interface {
    WriteTo(w Writer) (n int64, err error)
}

and:

type ReaderFrom interface {
    ReadFrom(r Reader) (n int64, err error)
}

They allow concrete types to provide optimized transfer implementations.

For example, *os.File implements ReadFrom, and the implementation can select OS-specific mechanisms before falling back to a generic copy loop.

On Linux, the standard library contains explicit support for sendfile, splice, and other zero-copy-related paths.

This is the deeper reason to prefer:

io.Copy(dst, src)

over:

for {
    // custom Read/Write loop
}

The generic API preserves the implementation's opportunity to optimize the transfer underneath you.

#32. The Optimization Is Conditional

It is tempting to conclude:

"io.Copy always uses zero-copy."

It does not.

The specialized path depends on:

• the concrete source type;
• the concrete destination type;
• the operating system;
• the direction of transfer;
• file descriptor characteristics;
• the specific capabilities of the involved implementations.

When the specialized path cannot handle the operation, Go falls back to a generic implementation. The standard library explicitly exposes a handled result internally for this purpose.

This is an important production-performance principle:

Write against the high-level interface and let the concrete implementation select the lowest useful layer.

Do not hard-code an OS-specific optimization unless you have a concrete reason to do so.

#33. io.StringWriter: Another Optional Capability

The standard library also defines:

type StringWriter interface {
    WriteString(s string) (n int, err error)
}

Code can use:

io.WriteString(w, "hello")

and let the destination provide a specialized string-writing implementation when available.

This illustrates a broader Go design pattern:

small base interface
        +
optional capability interfaces
        =
composable abstraction

The standard library does not need one giant interface containing every possible operation.

#34. I/O Is Also About Resource Lifetime

A production I/O operation is not merely:

Read
Write

It is often:

Open
  ↓
Read / Write
  ↓
Flush
  ↓
Close

Network operations may add:

deadline
timeout
cancellation

For example:

file, err := os.Open(path)
if err != nil {
    return err
}
defer file.Close()

For writes, the application may also need to reason about:

buffered data
Flush errors
Close errors

For network operations, blocking I/O may need explicit deadlines or context-driven cancellation depending on the API.

The point is:

Resource lifetime is part of I/O correctness.

Moving the bytes correctly is only part of the job.

#35. HTTP Bodies Are Just Another Reader

Consider:

resp, err := http.Get(url)
if err != nil {
    return err
}
defer resp.Body.Close()

resp.Body is an io.ReadCloser.

That means it can directly participate in generic stream operations:

_, err := io.Copy(file, resp.Body)

The data path is:

HTTP server
     │
     ▼
HTTP response body
     │
     ▼
io.Reader
     │
     ▼
io.Copy
     │
     ▼
os.File

There is no HTTP-specific copy loop.

This is where the Reader abstraction becomes a real engineering advantage.

#36. Compression Streams Fit the Same Model

A gzip Reader can be inserted into the same pipeline:

gz, err := gzip.NewReader(src)
if err != nil {
    return err
}
defer gz.Close()

_, err = io.Copy(dst, gz)
return err

The pipeline becomes:

source
  │
  ▼
gzip.Reader
  │
  ▼
io.Reader
  │
  ▼
io.Copy
  │
  ▼
destination

The consumer does not need to know that decompression is happening.

This same model applies to:

• compression;
• encryption;
• hashing;
• framing;
• decoding;
• protocol adapters.

#37. Build I/O Pipelines Instead of Giant I/O Functions

Suppose an upload must:

1. enforce a maximum size;
2. calculate SHA-256;
3. write the data to storage.

The pipeline can be expressed directly:

func saveUpload(body io.Reader, dst io.Writer) ([]byte, error) {
    hash := sha256.New()

    limited := io.LimitReader(body, 100<<20)
    tee := io.TeeReader(limited, hash)

    if _, err := io.Copy(dst, tee); err != nil {
        return nil, err
    }

    return hash.Sum(nil), nil
}

The structure is visible:

external input
      │
      ▼
LimitReader
      │
      ▼
TeeReader ─────→ hash
      │
      ▼
   io.Copy
      │
      ▼
destination

No part of this function depends on HTTP, files, TCP, or a particular storage system.

That is what makes the code reusable.

#38. A More Complete Production Pipeline

A larger application might have:

HTTP Request Body
        │
        ▼
   size boundary
        │
        ▼
 compression decoder
        │
        ▼
    bufio.Reader
        │
        ▼
      parser
        │
        ▼
   business logic
        │
        ▼
      encoder
        │
        ▼
   bufio.Writer
        │
        ▼
     storage

Each layer has one job.

A maximum-size boundary can be added without changing the parser.

A compression layer can be inserted without changing the consumer.

A checksum can be added with TeeReader.

A destination can move from a local file to a network connection without changing the processing function.

That is the compositional model behind Go I/O.

#39. Common Mistake: Calling Read When You Mean ReadFull

Suspicious:

header := make([]byte, 32)

_, err := r.Read(header)

If the protocol requires 32 bytes, use:

_, err := io.ReadFull(r, header)

The code should express the protocol's requirement directly.

A fixed-size protocol field is not merely an arbitrary read buffer.

#40. Common Mistake: Calling ReadAll on Unbounded Input

This:

data, err := io.ReadAll(req.Body)

may be perfectly reasonable for a small, explicitly bounded request.

It is a poor default for an external body with no effective size limit.

If the application needs the whole body:

r := io.LimitReader(req.Body, maxSize)

data, err := io.ReadAll(r)
if err != nil {
    return err
}

If the application does not need the complete body:

_, err := io.Copy(dst, req.Body)

Go 1.26 makes ReadAll more efficient, but it does not change this architectural decision.

#41. Common Mistake: Forgetting Flush

This is incomplete:

bw := bufio.NewWriter(w)

fmt.Fprintln(bw, "hello")

return nil

The data may still be buffered.

Use:

if err := bw.Flush(); err != nil {
    return err
}

A successful write to bufio.Writer is not necessarily a successful write to the underlying destination.

#42. Common Mistake: Treating Every Read Error as EOF

Wrong:

for {
    n, err := r.Read(buf)

    if err != nil {
        break
    }

    process(buf[:n])
}

This turns actual I/O failures into normal termination.

The correct mental model is:

io.EOF
    → normal end of stream

other error
    → failed I/O

n > 0
    → process the returned data

#43. Common Mistake: Assuming MultiWriter Is Atomic

This:

w := io.MultiWriter(a, b, c)

does not provide transactional semantics.

If:

A succeeds
B fails

the state change in A remains.

If the application requires all-or-nothing behavior across destinations, that is an application-level consistency problem, not something MultiWriter solves.

#44. Common Mistake: Adding bufio Everywhere

This:

br := bufio.NewReader(src)
bw := bufio.NewWriter(dst)

io.Copy(bw, br)

is not automatically superior to:

io.Copy(dst, src)

The question is not:

"Can I add a buffer?"

The question is:

"Does the access pattern benefit from buffering at this layer?"

For ordinary stream copying, let io.Copy choose its path.

For many small operations or higher-level parsing, introduce bufio.

#45. Common Mistake: Pooling Everything

This is also a performance smell:

every request
    ↓
sync.Pool
    ↓
bufio.Reader
    ↓
bufio.Writer

without any measurement showing that allocations are a problem.

Pooling has costs:

• more complicated lifecycle management;
• retained memory;
• possible retention of unusually large buffers;
• harder reasoning about object state.

Use:

Reset
+
sync.Pool

when profiling shows that reuse is worthwhile.

Otherwise, ordinary construction is often the clearer choice.

#46. A Practical API Selection Guide

Start with the operation.

#Need to move a stream?

io.Copy(dst, src)

#Need a custom reusable copy buffer?

io.CopyBuffer(dst, src, buf)

#Need all data in memory?

io.ReadAll(r)

#Need to cap the readable input?

io.LimitReader(r, n)

#Need exactly N bytes?

io.ReadFull(r, buf)

#Need to concatenate Readers?

io.MultiReader(a, b, c)

#Need sequential fan-out?

io.MultiWriter(a, b, c)

#Need a synchronous side effect while reading?

io.TeeReader(r, w)

#Need to consume and discard data?

io.Copy(io.Discard, r)

#Need an io.ReadCloser around a plain Reader?

io.NopCloser(r)

#Need buffered, fine-grained reads?

bufio.NewReader(r)

#Need buffered writes?

bufio.NewWriter(w)

#Need token-oriented scanning?

bufio.NewScanner(r)

#Need an in-memory Reader?

bytes.NewReader(data)
strings.NewReader(s)

#Need an in-memory Writer?

bytes.Buffer

#Need to reuse an existing buffered object?

br.Reset(r)
bw.Reset(w)

The APIs become much easier to remember once the underlying operation is clear.

#47. A Production Decision Tree

A useful way to reason about an I/O problem is:

Need to process data
        │
        ▼
Do I need the complete data in memory?
        │
   ┌────┴────┐
  yes        no
   │          │
   ▼          ▼
ReadAll    Is this just Reader → Writer?
              │
          ┌───┴───┐
         yes       no
          │         │
          ▼         ▼
       io.Copy   Do I need an exact
                 number of bytes?
                      │
                  ┌───┴───┐
                 yes       no
                  │         │
                  ▼         ▼
              ReadFull   Do I need
                         fine-grained
                         buffered access?
                              │
                          ┌───┴───┐
                         yes       no
                          │         │
                          ▼         ▼
                       bufio      Reader

For external input, add:

Is the input size bounded?
        │
       no
        │
        ▼
Add an explicit size boundary.

For high-throughput services, add another question:

Have allocations been shown to matter?
        │
       yes
        │
        ▼
Consider Reset / sync.Pool

This is a much better optimization strategy than introducing pooling or buffering by habit.

#48. The Real Power of Go I/O Is Composition

The io package is easy to underestimate because the interfaces are tiny.

Its real strength comes from composition:

small interfaces
      +
small adapters
      +
clear contracts
      +
streaming

A Reader can be wrapped by another Reader.

A Writer can be wrapped by another Writer.

A stream can be:

limited
buffered
decompressed
hashed
decoded
copied
split
discarded

without changing the consumer's basic interface.

For example:

source
  │
  ▼
LimitReader
  │
  ▼
gzip.Reader
  │
  ▼
bufio.Reader
  │
  ▼
parser

Each layer remains focused.

#49. Think in Streams, Not Files

One of the most useful shifts in Go I/O is to stop thinking:

"I need to read this file."

and start thinking:

"I need to consume this stream."

The source might be:

file
socket
HTTP body
memory
pipe
compressed stream
custom Reader

If the consumer needs only:

io.Reader

all of them can fit the same processing path.

Likewise, an output may be:

file
socket
HTTP response
buffer
compression writer
custom Writer

if the consumer requires only:

io.Writer

This is why code built around io.Reader and io.Writer tends to compose well.

#50. The Performance Model Behind io.Copy

At the application level:

io.Copy(dst, src)

looks simple.

Underneath, the implementation can choose among several levels:

Application
    │
    ▼
io.Copy
    │
    ├── WriterTo
    │
    ├── ReaderFrom
    │
    ├── specialized net/os path
    │
    ├── sendfile / splice where supported
    │
    └── generic Read → Write loop

That layering is one of the strongest arguments for using the standard abstraction instead of replacing it with a hand-written loop.

The application states what it wants:

copy this stream

The concrete implementation decides how to move it efficiently.

That separation is a major part of Go's standard-library design.

#51. Production Rules Worth Keeping

If only a few rules survive from this article, keep these.

#1. Depend on io.Reader and io.Writer

Use the behavior you need, not the concrete source.

#2. Treat Read as a partial operation

Never assume one Read fills the supplied buffer.

#3. Process n even when err is non-nil

The returned bytes are still valid data.

#4. Treat io.EOF differently from other errors

EOF normally means the stream ended.

#5. Prefer io.Copy for ordinary stream transfers

It is not merely a shorter loop. It preserves specialized implementation and OS-level fast paths.

#6. Use io.ReadAll deliberately

Go 1.26 makes it significantly more efficient, but the result is still an in-memory []byte.

#7. Put explicit boundaries around external input

io.LimitReader is one useful building block.

#8. Use io.ReadFull when the protocol requires an exact byte count

Do not assume Read fills the buffer.

#9. Use bufio for an access pattern

Buffering is useful when reads or writes are small and frequent, or when higher-level buffered operations are needed.

#10. Treat Flush as part of correctness

Buffered data is not the same thing as data accepted by the underlying destination.

#11. Use Reset and sync.Pool only when reuse is justified

Measure first.

#12. Do not mistake MultiWriter for a transaction

Partial success is possible.

#13. Use NopCloser when adapting Reader to ReadCloser

It changes the interface contract without inventing resource ownership.

#14. Think about cancellation, timeouts, and resource lifetime

A correct I/O operation is more than moving bytes.

#Conclusion

Go's I/O library is small by design.

Its central interfaces are tiny, but they provide a common boundary between application logic and almost every kind of data source and destination in the standard library.

That is why the same code can process:

file
network connection
HTTP body
memory buffer
compressed stream
custom Reader

without knowing which one it received.

The most useful way to understand the package is therefore not as a catalog of functions:

ReadAll
Copy
LimitReader
ReadFull
MultiReader
MultiWriter
TeeReader

but as a set of composable operations on streams.

The engineering questions are:

Where does the data come from?
        ↓
How large can it become?
        ↓
Does it need to remain a stream?
        ↓
Does the consumer need the complete input?
        ↓
Does the protocol require exact byte counts?
        ↓
Does the access pattern justify buffering?
        ↓
Are there side effects or multiple destinations?
        ↓
Where can errors occur?
        ↓
When is the operation actually complete?

Once those questions are answered, choosing the API is usually straightforward.

And there is one deeper lesson worth remembering.

Go's I/O abstractions are not merely a convenient way to hide files and sockets. They form an optimization boundary.

At the top, application code sees:

io.Copy(dst, src)

Below that, concrete implementations can select:

WriterTo
ReaderFrom
buffered transfer
sendfile
splice
other platform-specific mechanisms

when the circumstances allow it.

That is why the best production Go I/O code is often surprisingly boring.

It uses the standard interfaces.

It keeps streams as streams.

It establishes explicit boundaries.

It handles errors and lifetimes deliberately.

And it lets the standard library decide how far down the stack the operation can safely be optimized.

Write the application against the stream. Let the standard library decide how to move the bytes.

That is the core of Go's I/O design.