gobase.net
  • English

      • Example: io.Copy

      Quick Example

      io.Copy acts like a direct, high-efficiency data pipeline. It streams data from a source (io.Reader) to a destination (io.Writer) without loading the entire content into memory. This makes it the perfect tool for handling large files or network streams.

      package main

import (
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	// 1. Initialize the source (Reader): a simulated string stream
	src := strings.NewReader("Hello, io.Copy from gobase.net.\n")
	
	// 2. Define the destination (Writer): standard output (console)
	dst := os.Stdout
	
	// 3. Start transferring data. It automatically loops until src returns io.EOF
	written, err := io.Copy(dst, src)
	if err != nil {
		fmt.Fprintf(os.Stderr, "Error during copy: %v\n", err)
		return
	}
	
	// 4. 'written' returns the actual number of bytes transferred
	fmt.Printf("\n[Transfer Complete] Successfully moved %d bytes.\n", written)
}

      Output:

      Hello, io.Copy from gobase.net.

[Transfer Complete] Successfully moved 31 bytes.
      Memory-Efficient Data Moving

      io.Copy is a zero-waste mover. Under the hood, it uses a default 32 KB temporary buffer to forward data streams, keeping memory consumption extremely low.

      If you need to fine-tune the buffer size for specific use cases, use io.CopyBuffer(dst Writer, src Reader, buf []byte):

      • dst / src: The destination writer and source reader.
      • buf: A pre-allocated byte slice that you provide. If you pass nil, the system defaults to a 32 KB buffer, behaving exactly like io.Copy.

      Copying Large Files with io.Copy

      When dealing with large files (like videos, zip archives, or massive logs) that span hundreds of megabytes or gigabytes, loading them completely into memory using os.ReadFile or io.ReadAll will likely trigger an OOM (Out of Memory) crash.

      Whether you are copying a 10 MB file or a 10 GB file, io.Copy keeps your memory footprint consistently low.

      package main

import (
	"fmt"
	"io"
	"os"
	"time"
)

func main() {
	start := time.Now()

	// 1. Open the source file (read-only)
	srcFile, err := os.Open("large_video.mp4")
	if err != nil {
		fmt.Fprintf(os.Stderr, "Failed to open source file: %v\n", err)
		return
	}
	defer srcFile.Close() // Ensure the file resource is released

	// 2. Create the destination file (truncates if exists, permission 0666)
	dstFile, err := os.Create("large_video_backup.mp4")
	if err != nil {
		fmt.Fprintf(os.Stderr, "Failed to create destination file: %v\n", err)
		return
	}
	defer dstFile.Close()

	// 3. Use io.Copy to establish the pipeline for streaming copy.
	// Only 32 KB of data cycles through memory at any given moment.
	written, err := io.Copy(dstFile, srcFile)
	if err != nil {
		fmt.Fprintf(os.Stderr, "File copy failed: %v\n", err)
		return
	}

	// 4. Print stats
	duration := time.Since(start)
	fmt.Printf("[Copy Success]\n")
	fmt.Printf("・ Data Transferred: %.2f MB\n", float64(written)/(1024*1024))
	fmt.Printf("— Time Elapsed: %v\n", duration)
}