#Go File Handling in Production: Ownership, Atomic Writes, and Durability

#TL;DR

Opening a file is easy.

Designing file operations that remain correct during crashes, concurrent access, and production failures is harder.

In production Go systems:

• *os.File is an owned operating-system resource.
• File lifetime should be explicit.
• O_CREATE does not mean exclusive creation.
• O_TRUNC can destroy existing data before replacement succeeds.
• Rename and Sync solve different problems.
• Atomic publication and durability are separate design concerns.

The hard part of file handling is not calling an API.

It is understanding what the operating system guarantees.

#1. os.File Is a Resource, Not Just a Filename

Many developers initially think about files as paths:

/etc/app/config.yaml

But after opening a file:

f, err := os.Open(path)
if err != nil {
    return err
}

defer f.Close()

the program no longer owns only a string.

It owns an operating-system resource.

Conceptually:

pathname
    |
    v
filesystem lookup
    |
    v
open file descriptor / handle
    |
    v
*os.File

The pathname and the opened resource are related, but they are not the same thing.

This distinction explains many production bugs.

#2. Resource Ownership Should Be Obvious

The code that acquires a resource should usually own its lifetime.

Good:

func loadConfig(path string) error {
    f, err := os.Open(path)
    if err != nil {
        return fmt.Errorf("open config: %w", err)
    }
    defer f.Close()

    return parseConfig(f)
}

The ownership is clear:

loadConfig()
    |
    +-- open file
    |
    +-- use file
    |
    +-- close file

Avoid helpers that return open files without clear ownership rules.

This creates ambiguity:

f, err := openConfig()

Who closes f?

The caller?

The helper?

Both?

Neither?

In production systems, unclear ownership eventually becomes a resource leak.

#3. Beware Deferred Close Inside Large Loops

This pattern looks clean:

for _, path := range paths {
    f, err := os.Open(path)
    if err != nil {
        return err
    }

    defer f.Close()

    process(f)
}

But defer executes when the surrounding function returns.

If there are 100,000 files:

function starts

open file 1
open file 2
open file 3
...
open file 100000

return

close everything

The program may exhaust file descriptors before cleanup happens.

Prefer:

func processFile(path string) error {
    f, err := os.Open(path)
    if err != nil {
        return err
    }
    defer f.Close()

    return process(f)
}

for _, path := range paths {
    if err := processFile(path); err != nil {
        return err
    }
}

The resource lifetime now matches one operation.

#4. Close Can Be Part of the Error Path

For reading:

f, err := os.Open(path)
if err != nil {
    return err
}

defer f.Close()

is normally enough.

Writing is different.

Example:

f, err := os.Create(path)
if err != nil {
    return err
}

if _, err := f.Write(data); err != nil {
    f.Close()
    return err
}

if err := f.Close(); err != nil {
    return err
}

Why?

Because some failures may only become visible during close.

A successful write does not always mean:

The complete operation succeeded.

The right question is:

What guarantee does this function promise when it returns nil?

#5. Open, Create, and OpenFile Express Different Intent

#Reading an existing file

f, err := os.Open(path)

Meaning:

Open an existing resource for reading.

#Creating or truncating

f, err := os.Create(path)

Equivalent idea:

create if missing
otherwise truncate

Be careful.

Truncation is destructive.

#Explicit semantics

f, err := os.OpenFile(
    path,
    os.O_WRONLY|os.O_CREATE|os.O_EXCL,
    0600,
)

The flags define concurrency behavior.

They are not just implementation details.

#6. O_CREATE Does Not Mean Exclusive Creation

A common mistake:

os.OpenFile(
    path,
    os.O_CREATE|os.O_WRONLY,
    0644,
)

Many developers read this as:

Create this file and fail if it already exists.

That is wrong.

It means:

Create the file if it does not exist.

If exclusive creation is required:

os.OpenFile(
    path,
    os.O_CREATE|os.O_EXCL|os.O_WRONLY,
    0600,
)

Now the filesystem performs:

check existence
        +
create file

as one atomic operation

#7. Avoid Check-Then-Act Filesystem Logic

This is dangerous:

if _, err := os.Stat(path); err == nil {
    return errors.New("already exists")
}

os.Create(path)

Because:

Stat()
 |
 | another process changes filesystem
 |
Create()

The assumption made by the first operation may no longer be true.

This is called a TOCTOU race:

Time Of Check
        |
        v
Time Of Use

Whenever possible:

Prefer:

one operating-system operation

over:

check
+
assume
+
act

#8. Why O_TRUNC Can Destroy Data

Consider:

f, err := os.OpenFile(
    "config.json",
    os.O_WRONLY|os.O_CREATE|os.O_TRUNC,
    0644,
)

The sequence:

open file
 |
truncate old content
 |
write new content
 |
process crashes

Possible result:

config.json

(empty)

or:

config.json

(partially written)

For temporary output this may be fine.

For:

• configuration files
• metadata
• checkpoints
• manifests
• user data

it is usually unsafe.

#9. Safer Pattern: Generate Then Publish

A safer update model:

old version
    |
    |
    v

create temporary file

    |
    v

write complete content

    |
    v

validate

    |
    v

publish with rename

Example:

tmp, err := os.CreateTemp(
    filepath.Dir(target),
    ".tmp-*",
)

Write the complete new version first.

Only after success:

os.Rename(tmp, target)

Readers see either:

old version

or:

new version

not:

half-written version

#10. Keep Temporary Files Near the Destination

Avoid:

/tmp/new-file
        |
        v
rename
        |
        v
/application/config.json

because they may exist on different filesystems.

Prefer:

/application/
    config.json
    .tmp-12345

Then:

os.Rename(
    ".tmp-12345",
    "config.json",
)

This makes the publication operation explicit.

#11. Rename and Sync Solve Different Problems

These two operations are often confused.

#Rename

Answers:

Can readers observe a partially published file?

It provides atomic namespace replacement.

#Sync

Answers:

Has data been pushed toward stable storage?

Example:

if err := f.Sync(); err != nil {
    return err
}

The difference:

Write succeeded

        |
        v

filesystem accepted data

        |
        v

durable after power failure

These are different guarantees.

#12. Production Checklist

Before shipping file-writing code:

#Ownership

• Who closes the file?
• Can an error path leak resources?

#Creation

• Should existing files be overwritten?
• Should creation be exclusive?

#Updates

• Can a crash leave partial data?
• Should you write a temporary file first?

#Durability

• Is "write succeeded" enough?
• Do you need Sync?

#Concurrency

• Are you doing check-then-act logic?
• Can another process modify the file?

#Conclusion

Production file handling in Go is not about remembering:

os.Open()
os.Create()
os.Write()

The important skill is understanding the relationship between:

path
 |
filesystem state
 |
open resource
 |
published state
 |
durability

Reliable Go programs make these boundaries explicit.

Use the operating system deliberately.

Do not rely on assumptions.