#Go os in Production: Atomic Updates, Durability, and Filesystem Mutation

#TL;DR

Most filesystem bugs in production are not caused by incorrect API usage. They come from misunderstanding what a filesystem operation actually guarantees.

Creating a file, replacing a file, syncing data, deleting a directory, and updating shared state are different operations with different failure modes.

A reliable Go program needs to distinguish between:

• making new data visible to readers;
• ensuring data survives crashes or power loss;
• coordinating multiple writers;
• safely mutating filesystem state.

The os package provides the building blocks:

• CreateTemp
• Rename
• Sync
• Remove
• RemoveAll
• OpenFile

But these APIs do not automatically provide transactions.

Production reliability comes from choosing the correct filesystem pattern for the failure model.

#1. Filesystems Are Mutable Shared State

A common beginner assumption is:

A file path points to a fixed object.

In reality, a pathname is a lookup operation inside a changing namespace.

For example:

/app/config.json

does not permanently identify one file.

At different moments:

time A:

config.json
    |
    v
inode 100


time B:

config.json
    |
    v
inode 200

Another process can:

• replace the file;
• rename it;
• delete it;
• change permissions;
• modify its contents.

This is why filesystem code must be designed around operations and guarantees, not just filenames.

A path is a reference.

It is not ownership.

#2. File Replacement Is Different From File Modification

Consider a configuration update:

func updateConfig(path string, data []byte) error {
    return os.WriteFile(path, data, 0644)
}

This looks simple.

But internally:

open existing file
        |
        v
truncate content
        |
        v
write new data

The dangerous moment is between truncation and completion.

A crash can produce:

config.json

(empty)

or:

config.json

(partially written)

For files that represent published state, this is usually unacceptable.

Examples:

• application configuration;
• deployment manifests;
• metadata indexes;
• generated certificates;
• checkpoint files.

The safer pattern is:

generate new version

        |

finish writing completely

        |

publish replacement

The important design decision is separating:

creating a new version

from:

making that version visible.

#3. Rename as a Publication Primitive

The classic Unix pattern is:

temporary file

      |

complete write

      |

rename into place

Example:

tmp, err := os.CreateTemp(
    filepath.Dir(target),
    ".config-*",
)
if err != nil {
    return err
}

defer os.Remove(tmp.Name())

if _, err := tmp.Write(data); err != nil {
    tmp.Close()
    return err
}

if err := tmp.Close(); err != nil {
    return err
}

return os.Rename(tmp.Name(), target)

Readers see either:

old config

or:

new config

They do not observe the intermediate writing process.

This is why rename is widely used for:

• configuration updates;
• package managers;
• editors;
• database checkpoints;
• build systems.

However, there is an important distinction:

atomic visibility

!=

durable storage

A rename can make a new filename visible while not guaranteeing survival after a sudden power failure.

#4. Rename Does Not Mean Transaction

A common oversimplification is:

Rename is atomic, therefore the update is safe.

The reality is more subtle.

Rename typically provides atomic namespace replacement:

before:

config.json
    |
    v
old content


after:

config.json
    |
    v
new content

Readers do not see:

half old
half new

But rename does not automatically guarantee:

• previous writes reached stable storage;
• metadata survived a power failure;
• multiple writers cannot overwrite each other.

A filesystem operation can be atomic without being a transaction.

A transaction would provide something closer to:

write data

+

update metadata

+

commit

+

rollback on failure

The filesystem API does not provide that automatically.

#5. Temporary Files Should Live Near the Destination

A common mistake:

tmp, err := os.CreateTemp("", "update-*")

Then later:

os.Rename(tmp.Name(), target)

This may cross filesystem boundaries.

For example:

/tmp
 |
 filesystem A


/app/data
 |
 filesystem B

Some filesystems cannot perform this rename directly.

The safer pattern:

dir := filepath.Dir(target)

tmp, err := os.CreateTemp(
    dir,
    ".update-*",
)

Now:

/app/data/config.tmp

        |

        rename

        |

/app/data/config.json

The operation remains within the same filesystem namespace.

#6. A Production Atomic Write Helper

A reusable helper usually looks like this:

func AtomicWrite(
    path string,
    data []byte,
    perm fs.FileMode,
) error {

    dir := filepath.Dir(path)

    f, err := os.CreateTemp(
        dir,
        ".tmp-*",
    )
    if err != nil {
        return err
    }

    tmp := f.Name()

    committed := false

    defer func() {
        if !committed {
            _ = os.Remove(tmp)
        }
    }()

    if err := f.Chmod(perm); err != nil {
        f.Close()
        return err
    }

    if _, err := f.Write(data); err != nil {
        f.Close()
        return err
    }

    if err := f.Close(); err != nil {
        return err
    }

    if err := os.Rename(tmp, path); err != nil {
        return err
    }

    committed = true

    return nil
}

The lifecycle is explicit:

create temporary resource

        |

write complete content

        |

close resource

        |

publish replacement

        |

remove temporary state on failure

The caller does not need to understand intermediate filesystem state.

#7. Sync and the Difference Between Writing and Durability

A successful Write does not necessarily mean that data has reached permanent storage.

Consider:

n, err := f.Write(data)

A successful return means the operating system accepted the write.

It does not necessarily mean:

• the filesystem metadata is committed;
• the storage device has persisted the data;
• the data will survive sudden power loss.

The path is more complicated:

application memory

        |

        v

kernel buffers

        |

        v

filesystem cache

        |

        v

storage device

File.Sync() exists for cases where the application needs a stronger guarantee:

if _, err := f.Write(data); err != nil {
    return err
}

if err := f.Sync(); err != nil {
    return err
}

However, durability is not a binary property.

The actual guarantee depends on:

• operating system;
• filesystem implementation;
• storage hardware;
• mount options;
• failure model.

A local SSD, a network filesystem, and a distributed storage system may all behave differently.

#8. Visibility and Durability Are Different Problems

Consider an application updating:

state.json

There are two separate questions.

#Question 1: Can readers observe incomplete data?

The solution is usually:

temporary file
        |
        v
complete write
        |
        v
rename

This provides atomic publication.

#Question 2: Will the update survive a crash?

The solution may require:

write

 |

Sync

 |

rename

 |

directory durability

These solve different failure modes.

A system can have:

• atomic visibility but weak crash durability;
• strong durability but unsafe concurrent updates.

Do not describe a filesystem operation as "safe" without specifying what kind of safety is required.

#9. Directory Durability After Rename

For applications that require crash consistency, syncing the file itself may not be enough.

The sequence:

write temporary file

        |

Sync file

        |

Rename file

updates the directory entry.

The directory metadata change may have its own persistence rules.

Some systems therefore also synchronize the containing directory.

Conceptually:

file contents

+

directory entry update

=

complete durable publication

The exact implementation is operating-system and filesystem dependent.

This is one reason databases usually implement their own storage protocols instead of relying on simple file replacement.

#10. Concurrent Writers Need a Separate Strategy

Atomic rename solves one problem:

partial file visibility

It does not solve:

two writers updating the same state

Example:

Writer A:

read config

modify value

write new version


Writer B:

read config

modify another value

write new version

Possible result:

A update lost

The filesystem correctly performed both writes.

The application design was incorrect.

Atomic replacement is not a locking mechanism.

#11. File Locking Is an Application Decision

When multiple processes modify shared state, choose an explicit coordination strategy.

Possible approaches:

#Single writer model

Example:

API server

    |

storage worker

    |

filesystem

Only one component writes.

#Advisory locking

The application agrees to respect a lock.

Examples:

• Unix file locks;
• Windows locking APIs.

The important word is:

advisory

The operating system does not automatically make unrelated programs obey your locking protocol.

#External coordination

For distributed systems:

• database transactions;
• leader election;
• distributed locks;
• queues.

A local filesystem lock does not automatically become a distributed consistency mechanism.

#12. Avoid Check-Then-Act During Mutation

A common filesystem race:

if _, err := os.Stat(path); err != nil {
    return err
}

os.Remove(path)

The check and action are separate.

Another process can change the filesystem between them.

The safer approach is usually:

err := os.Remove(path)

if errors.Is(err, os.ErrNotExist) {
    // already gone
}

Let the operating system perform the operation.

Then handle the result.

The general rule:

avoid:

check

 |

assume

 |

act


prefer:

attempt operation

 |

handle result

#13. Removing Files: Remove vs RemoveAll

Removing one path:

err := os.Remove(path)

is a targeted operation.

Recursive deletion:

err := os.RemoveAll(path)

is much more powerful.

It can mutate an entire subtree:

directory

 |
 +-- file A
 |
 +-- file B
 |
 +-- nested directory
        |
        +-- many files

Therefore:

os.RemoveAll(userInput)

should immediately raise a design question:

Is this path fully controlled?

#14. Destructive Operations Need Strong Boundaries

A safer design:

trusted root

    |

validate relative target

    |

perform operation

For example:

/srv/uploads

    |
    +-- user-a
    +-- user-b

The application should not construct arbitrary filesystem paths from external input and pass them to destructive operations.

Modern Go provides root-confined filesystem operations through:

os.Root

and:

os.OpenInRoot

when the requirement is:

this operation must remain below a specific filesystem root.

That is a filesystem boundary, not just string validation.

#15. Temporary Resources Need Cleanup

Temporary files are resources.

This pattern is common:

f, err := os.CreateTemp("", "job-*")
if err != nil {
    return err
}

defer os.Remove(f.Name())

But cleanup timing matters.

For long-running services:

• temporary files can accumulate;
• failed jobs can leave artifacts;
• disk usage can grow silently.

Production systems should define:

• who owns temporary files;
• when they expire;
• how cleanup failures are handled.

Temporary does not mean automatically removed.

#16. Error Handling for Filesystem Operations

Filesystem errors contain useful information.

Avoid:

strings.Contains(err.Error(), "not found")

Use:

if errors.Is(err, os.ErrNotExist) {
    return nil
}

Preserve errors:

return fmt.Errorf(
    "update config %q: %w",
    path,
    err,
)

The wrapped error still supports:

errors.Is()

and:

errors.As()

This matters because storage failures often need different handling:

missing file

        !=

permission denied

        !=

disk full

        !=

I/O failure

#17. Production File Update Pattern

For important generated state:

create temporary file

        |

write complete content

        |

validate content

        |

Sync if durability required

        |

close file

        |

rename into place

        |

cleanup failures

Examples:

• configuration generators;
• build metadata;
• deployment state;
• cache indexes;
• local checkpoints.

The key design principle:

Never use the destination file as the workspace for generating a new version.

#18. Production Checklist

Before shipping filesystem mutation code:

#Atomic Updates

• Does a crash leave a partial file?
• Should updates use temporary files?
• Is rename used only as publication?
• Is the temporary file created beside the destination?

#Durability

• Does success mean "write accepted" or "survives power loss"?
• Is Sync() required?
• Does the directory update need durability?

#Concurrency

• Can multiple writers update the same file?
• Is atomic replacement being confused with locking?
• Is there an ownership model?

#Temporary Resources

• Are temporary files cleaned up?
• Can failed operations leave disk usage growth?
• Is cleanup failure observable?

#Destructive Operations

• Is RemoveAll really necessary?
• Is the path trusted?
• Could external input influence the target?

#Error Handling

• Are filesystem errors checked semantically?
• Are wrapped errors preserving the original cause?
• Are different failure classes handled differently?

#Conclusion

The hardest filesystem problems are rarely about remembering an API.

They are about understanding what the API actually guarantees.

A filename is not a transaction.

A successful write is not automatically durable.

A rename is not a database commit.

A temporary file is not automatically cleaned up.

An atomic update is not a concurrency protocol.

Reliable Go systems treat filesystem operations as state transitions with explicit failure models.

The practical patterns are:

generate separately

        |

publish atomically

        |

sync when durability matters

        |

coordinate writers explicitly

        |

treat destructive operations carefully

The os package gives Go developers direct access to filesystem primitives.

Production engineering begins when those primitives are combined with the correct assumptions.