Go sync in Production
Go's sync package provides the low-level primitives used to coordinate shared state, one-time initialization, goroutine lifecycles, condition-based waiting, concurrent map access, and temporary object reuse.
Its API surface is deliberately small:
MutexRWMutexOnceOnceFuncOnceValueOnceValuesWaitGroupCondMapPool
The APIs are simple. The concurrency contracts built around them are not.
Production concurrency is rarely about making an individual operation "thread-safe." The harder problem is defining:
- who owns mutable state;
- which invariant must remain true;
- which operations must be atomic;
- how writes become visible to other goroutines;
- when goroutines may start and stop;
- how cancellation differs from completion;
- and whether synchronization is actually the performance bottleneck.
A mutex can eliminate a data race while leaving the application logically incorrect. A WaitGroup can prove that workers have stopped without providing a mechanism to stop them. sync.Map can make concurrent map operations safe while making multi-step invariants harder to express. sync.Pool can reduce allocation pressure while deliberately providing no guarantee that a pooled object will remain available.
[ PRODUCTION RULE ]
Synchronization is about invariants and lifecycle, not merely race avoidance.
Define the ownership boundary first. Choose the synchronization primitive that expresses that boundary second.
Part I — The Concurrency Mental Model
1. Start With Ownership
A useful production model separates four concerns:
sync primarily provides mechanisms for mutual exclusion, memory synchronization, one-time execution, waiting, and object reuse. It does not define ownership for you.
A good concurrent component therefore keeps its state and synchronization boundary together:
Callers should normally interact with the component through methods rather than acquiring its internal lock themselves. Whether the implementation later changes from RWMutex to Mutex, atomics, or another mechanism then remains an internal decision.
[ PRODUCTION RULE ]
Synchronization should usually be owned by the type that owns the state.
Do not turn an internal locking protocol into an application-wide convention unless the architecture explicitly requires it.
2. Protect the Invariant, Not Individual Fields
The most important synchronization boundary is the invariant boundary.
Suppose:
If items and active represent one logical state, protecting them independently can make individual memory accesses safe while allowing the combined state to become inconsistent.
The complete state transition should remain inside one critical section:
The same principle applies to compound operations:
Even if both methods are individually synchronized, the check-and-set operation is not atomic. Another goroutine can modify the cache between them.
The correct abstraction is one that represents the complete invariant:
[ PRODUCTION RULE ]
Protect the invariant, not individual fields.
Concurrency safety belongs at the boundary where the invariant is defined, not merely where the struct fields are declared.
3. A Lock Protects State Only While Its Contract Is Respected
Consider:
The map access is protected.
The returned *Item is not.
Once Get returns, the caller may mutate the object concurrently with another goroutine that also accesses it.
The mutex protected the registry's map, not the object referenced by the map.
This distinction matters for:
- pointers;
- slices;
- maps;
- nested structs containing pointers;
- buffers;
- mutable configuration objects.
Possible solutions include returning copies, publishing immutable objects, exposing narrowly scoped operations, or explicitly transferring ownership.
For example:
[ PRODUCTION RULE ]
Protecting a pointer is not the same as protecting the object it points to.
Synchronization boundaries must include object ownership, not just the container holding the object.
4. Synchronization and Lifecycle Are Different Problems
A concurrent service usually has at least two independent dimensions:
For example:
Calling cancel() does not mean workers have already stopped.
Calling Wait() does not tell workers that they should stop.
A typical shutdown sequence is:
The resulting lifecycle is explicit:
[ PRODUCTION RULE ]
Cancellation is a request. Completion is an observation.
context.Contextandsync.WaitGroupsolve different lifecycle problems and are often used together.
5. CPU Cache Lines and False Sharing
At ordinary application scale, synchronization cost is usually dominated by critical-section length and lock contention.
At very high operation rates, CPU cache-coherence behavior can become significant.
Two logically independent variables can occupy the same cache line. When different CPUs repeatedly modify those variables, cache-line ownership can bounce between cores even though the application never logically shares the variables.
This is false sharing.
For specialized hot structures, golang.org/x/sys/cpu provides CacheLinePad:
Manual padding such as:
is less explicit and makes architecture-dependent assumptions.
Padding is not a default optimization. It increases object size and can make locality worse.
[ PRODUCTION RULE ]
Do not optimize cache-line layout speculatively.
Introduce padding only when profiling demonstrates that false sharing is a real bottleneck.
Part II — Mutual Exclusion and State Protection
6. sync.Mutex: The Default Choice
For ordinary shared mutable state, sync.Mutex is usually the correct starting point:
The guarantee is more than mutual exclusion. Unlocking and a later successful locking operation establish the synchronization needed for safely publishing changes to protected memory.
A simple mutex with a clear ownership boundary is usually preferable to a more complicated synchronization mechanism whose performance benefits have not been demonstrated.
[ PRODUCTION RULE ]
Start with
Mutex. Optimize synchronization only after the workload justifies it.
7. Keep Critical Sections Small
Prefer:
to:
Do not hold locks unnecessarily across:
- network I/O;
- disk I/O;
- blocking channel operations;
- external callbacks;
- expensive computation;
- database calls.
Instead, perform independent work before acquiring the lock:
The question is not whether a critical section is "short" in absolute time.
The question is whether every operation inside it is required to preserve the protected invariant.
[ PRODUCTION RULE ]
Do not hold a lock across blocking or unrelated work unless the protected invariant requires it.
8. defer and Unlock Safety
For most production code:
is preferable to manually placing Unlock on every return path.
This becomes particularly valuable when a critical section contains validation, error handling, or multiple return paths.
The more important optimization is normally reducing work performed under the lock rather than prematurely eliminating defer.
If a specific hot path demonstrates a measurable overhead, optimize that path based on profiling.
9. Do Not Copy Synchronization Values After Use
Synchronization types must not be copied after first use.
This can happen accidentally through value parameters or value receivers:
Passing Cache by value copies the mutex.
Prefer:
This rule applies broadly to synchronization types including:
Mutex;RWMutex;Once;WaitGroup;Cond;Pool;- atomic wrapper types.
[ PRODUCTION RULE ]
Types containing synchronization state should generally be pointer-oriented and non-copyable after initialization.
10. sync.RWMutex: Use Only When the Workload Fits
RWMutex allows concurrent readers and exclusive writers:
It can make sense when reads substantially outnumber writes and the workload benefits from concurrent readers.
It is not automatically faster than Mutex. For very short critical sections, the additional reader/writer coordination may provide little benefit.
Start with Mutex. Introduce RWMutex when measurements demonstrate that concurrent readers materially improve the workload.
11. atomic.Pointer[T] for Immutable Snapshots
A particularly useful alternative to RWMutex is atomic publication of immutable state.
For configuration hot-reloading:
Readers perform an atomic pointer load instead of participating in lock coordination.
This can be significantly cheaper for extremely read-heavy workloads where writers replace the entire configuration rather than mutating it in place.
The critical requirement is that published objects must be treated as immutable.
atomic.Pointer[T] protects publication of the pointer. It does not make the object behind that pointer safe for concurrent mutation.
A typical design is:
[ PRODUCTION RULE ]
Use atomic snapshots when state can be replaced as an immutable whole.
Do not use an atomic pointer as a substitute for synchronization around mutable objects.
12. Avoid Lock Upgrades
This pattern is dangerous:
Another goroutine can change the state between the read unlock and write lock.
The original check is no longer part of the same atomic transition.
Prefer:
If the operation is logically one transaction, use one write-locked critical section.
13. Lock Ordering Is a System Invariant
Multiple locks create a potential circular wait:
If multiple locks are unavoidable, establish a global ordering:
and enforce it consistently.
For larger components, reducing the number of independently acquired locks or grouping related state behind a single ownership boundary is often safer than maintaining a complex lock hierarchy.
[ PRODUCTION RULE ]
Lock ordering is part of the concurrency contract.
14. Do Not Call Unknown Code While Holding a Lock
Avoid:
The callback may perform I/O, block, acquire another lock, or call back into the same component.
Prefer extracting the required state first:
This is particularly important for public extension points and callbacks whose implementation is outside the component's control.
15. Mutex Is Not Recursive
Go's sync.Mutex is not a recursive mutex.
This deadlocks:
A common internal design separates locked and unlocked implementations:
The naming should make the locking contract obvious.
16. TryLock Is Specialized
Mutex.TryLock and RWMutex.TryLock can be useful when failure to acquire a lock is itself a meaningful state.
For example:
But before using this pattern, verify that:
- failed acquisition has defined semantics;
- retry behavior is bounded;
- starvation is acceptable;
- and the operation is not merely hiding a deeper contention problem.
A non-blocking lock attempt is a primitive, not a general concurrency strategy.
17. Measure Mutex Contention
The runtime provides mutex profiling:
This enables sampling of mutex blocking events. The resulting profile can be examined through Go's profiling tooling.
For example:
Mutex profiling can answer questions that source inspection cannot:
- Which locks are actually contended?
- Which call paths spend time waiting?
- Is the critical section too large?
- Did an
RWMuteximprove the workload? - Did a recent change introduce contention?
Profiling should be enabled deliberately because it introduces runtime overhead.
[ PRODUCTION RULE ]
Do not optimize locks by intuition. Measure contention first.
Part III — Initialization, Waiting, and Goroutine Lifecycle
18. sync.Once: One-Time Execution
sync.Once provides exactly-once execution:
The initialization function runs once even when multiple goroutines call Do concurrently.
Successful completion of the initialization function is synchronized with callers returning from later Do calls.
This makes sync.Once appropriate for initialization whose result should remain stable.
19. Once Is Not a Retry Mechanism
Consider:
If connect() fails, the Once operation has still completed.
A later call does not retry.
There is a fundamental difference between:
and:
sync.Once implements the first model.
If initialization is retryable, the retry policy needs an explicit state machine.
20. OnceFunc, OnceValue, and OnceValues
The function-oriented APIs make common one-time initialization patterns more explicit:
For multiple return values:
For a one-time action:
These APIs reduce the amount of manually maintained package-level state while preserving the same one-time execution semantics.
21. WaitGroup: Completion, Not Cancellation
A WaitGroup answers:
Have all registered tasks completed?
Go 1.25 introduced WaitGroup.Go, and it is the preferred way to add a new goroutine-backed task in modern Go:
Go 1.22 and later give each range iteration its own loop variables, so there is no longer a need for the pre-1.22 job := job closure workaround.
The older form remains valid and is useful when task accounting is not directly tied to launching a goroutine:
But for ordinary goroutine launching, WaitGroup.Go expresses the intent more directly.
A WaitGroup still does not answer:
- Should workers stop?
- Why did a worker fail?
- Should sibling workers be cancelled?
- What error should the caller receive?
Those are separate concerns.
[ PRODUCTION RULE ]
Use
WaitGroupto observe completion. Do not mistake it for cancellation or error propagation.
22. WaitGroup.Go and Its Contract
WaitGroup.Go combines task accounting and goroutine creation:
The method:
- adds the task to the
WaitGroup; - starts
fin a new goroutine; - removes the task when
freturns.
Its function must not panic.
If a task may panic and the application needs controlled recovery, handle recovery inside the function:
WaitGroup.Go does not turn a WaitGroup into an error-aware task group. If errors and cancellation form one logical operation, use a higher-level abstraction.
23. The Production Standard for Coordinated Workers: errgroup
When a group of goroutines has a shared lifecycle, where one failure should cancel the remaining work and the caller needs an error, golang.org/x/sync/errgroup is generally a better abstraction than manually combining WaitGroup, error channels, and cancellation.
The important properties are composed in one abstraction:
When one function returns a non-nil error, the derived context is cancelled, allowing sibling workers to observe ctx.Done() and terminate.
This is particularly useful for:
- parallel request processing;
- fan-out/fan-in operations;
- concurrent startup;
- coordinated background work;
- service shutdown sequences.
WaitGroup remains appropriate when all you need is completion tracking and you deliberately do not want error-group semantics.
[ PRODUCTION RULE ]
Use
WaitGroupfor completion. Useerrgroup.WithContextwhen goroutines form one failure-coupled operation.
24. sync.Cond: Wait for a State Predicate
sync.Cond is useful when goroutines need to wait until shared state satisfies a condition.
A consumer waits while the predicate is false:
A producer changes the state and signals:
The central abstraction is the predicate, not the notification.
25. Cond.Wait Must Be Inside a Loop
This is unsafe:
The correct pattern is:
A goroutine can wake after another goroutine has already consumed the state change it was waiting for. More fundamentally, a notification means that the state may have changed; it does not establish that the desired predicate is true.
Wait releases the associated locker while sleeping and reacquires it before returning, so the predicate must be checked again after wake-up.
[ PRODUCTION RULE ]
The notification is not the condition. The shared state is the condition.
26. Signal vs Broadcast
Use Signal when one waiting goroutine should normally be sufficient.
Use Broadcast when a state transition may make progress possible for multiple waiters.
For example:
The choice depends on the predicate and workload.
Cond is not an event queue. Notifications are not durable messages, and a goroutine cannot rely on a notification that happened before it began waiting.
27. sync.Cond and Context Cancellation
A critical production limitation is that sync.Cond.Wait() has no context-aware form.
A goroutine blocked in:
cannot directly perform:
because Cond is not a channel-based waiting primitive.
This matters when the goroutine's lifecycle is controlled by:
- request deadlines;
- service shutdown;
- worker cancellation;
- parent contexts.
A context cancellation does not automatically wake a goroutine blocked in Cond.Wait().
It is possible to coordinate cancellation separately—for example, by having another goroutine call Broadcast() when the context is cancelled—but that adds another lifecycle mechanism that must itself be managed correctly.
For request-scoped or shutdown-sensitive waiting, channels are often a better fit because they compose directly with select:
[ PRODUCTION WARNING ]
sync.Condis not context-aware.If a goroutine must reliably respond to
context.Contextcancellation while waiting, prefer a channel-based design unless you have an explicit and correctly managed wake-up mechanism.
28. Channels vs sync.Cond
Channels are generally easier to reason about when the abstraction is value transfer:
sync.Cond is more appropriate when the abstraction is:
A practical decision table:
In modern service code, the combination of channels and context.Context often provides a clearer lifecycle model than sync.Cond.
Part IV — Specialized Primitives
29. sync.Map: Specialized, Not a General Map Replacement
sync.Map provides concurrent map operations without an external mutex:
It is designed for particular access patterns, especially workloads where entries are stable or reads dominate writes and where independent keys can be operated on concurrently.
Go 1.24 substantially improved the implementation of sync.Map, particularly for map modifications and large maps with operations on disjoint key sets. The modern performance model therefore should not be based on older claims that frequent key churn necessarily causes a dramatic performance collapse.
A standard typed map protected by Mutex or RWMutex is still often superior for application state because it provides:
- static type safety;
- simpler invariants;
- explicit ownership;
- easier compound operations;
- clearer state transitions.
[ PRODUCTION RULE ]
Choose
sync.Mapbecause the workload fits it, not because ordinary maps need synchronization.
30. sync.Map and Compound Operations
This is not atomic as a logical operation:
Another goroutine may store the same key between the two operations.
Use the compound operation when the invariant matches it:
Likewise, CompareAndSwap, CompareAndDelete, Swap, and LoadAndDelete should be preferred when they directly express the required state transition.
If the invariant spans multiple objects or operations, sync.Map alone may not provide the required atomicity. An external synchronization boundary may still be necessary.
[ PRODUCTION RULE ]
Concurrent individual operations do not automatically make a multi-step business operation atomic.
31. sync.Map and Type Safety
A conventional map expresses:
and the compiler verifies the value type.
sync.Map stores any:
so callers commonly need assertions:
This trade-off can be justified by the intended workload, but it should not be ignored.
For most application-level maps, a typed map plus explicit synchronization remains an excellent default.
The key question in modern Go is therefore less about whether sync.Map can handle a particular access pattern and more about whether its concurrency model is worth the loss of static typing and the increased difficulty of expressing larger invariants.
32. sync.Map and Key Churn
High key churn is no longer best described as a guaranteed sync.Map performance failure.
Go 1.24 changed the underlying implementation and specifically improved modification performance, including workloads involving disjoint sets of keys on larger maps.
The remaining production question is workload fit.
A workload that continuously creates, replaces, and removes keys should still be benchmarked against:
map[K]V+sync.Mutex;map[K]V+sync.RWMutex;- sharded maps;
- specialized concurrent structures.
The advantage of sync.Map depends on the actual operation mix, contention pattern, object lifetime, and invariant structure.
[ PRODUCTION RULE ]
Do not use historical
sync.Mapfolklore as a performance model. Benchmark the Go version and workload you actually deploy.
33. sync.Pool Is a Reuse Mechanism, Not a Cache
sync.Pool is designed for temporary objects that can be discarded:
Objects stored in a pool may be removed automatically by the runtime, including as part of garbage collection.
A future Get is therefore allowed to allocate a new object instead of returning one previously placed in the pool.
The correct mental model is:
not:
[ PRODUCTION RULE ]
If correctness depends on an object remaining in a pool, it does not belong in
sync.Pool.
34. sync.Pool and the Garbage Collector
The GC relationship explains an important part of sync.Pool semantics.
The pool exists to reduce the cost of repeatedly allocating temporary objects. It is deliberately integrated with runtime memory management and does not provide persistent object retention.
Under a workload with substantial temporary allocation, reuse can reduce allocation and GC pressure:
When workload characteristics change, pooled objects may disappear.
A correct program therefore behaves identically whether the pool returns:
- a previously used object;
- a newly allocated object;
- or no retained object at all.
35. Reset Pooled Objects Before Reuse
Pooled objects may contain state left by their previous users.
For example:
Without resetting it, a later caller may observe stale contents.
The same principle applies to:
bytes.Buffer;- slices;
- temporary request structures;
- encoders;
- decoders;
- scratch buffers.
sync.Pool does not establish object cleanliness.
The code owning the Get/Put lifecycle does.
36. Avoid Retaining Pathologically Large Objects
Suppose a request temporarily requires a large buffer:
Returning that buffer to the pool may retain significantly more memory than future requests need.
A production implementation can impose a reuse threshold:
The threshold should come from actual workload measurements.
The goal is not maximum reuse.
The goal is:
reduce allocation and GC pressure without creating excessive memory retention.
Part V — Production Verification and Failure Patterns
37. The Race Detector Is Necessary but Not Sufficient
For synchronization-heavy code:
The race detector is extremely valuable for finding conflicting unsynchronized memory accesses involving:
- maps;
- mutable configuration;
- caches;
- worker state;
- shutdown paths;
- shared object graphs.
But race detection does not prove logical correctness.
This can be race-free:
while still violating the application's "insert only if absent" invariant.
The distinction is:
Both are required.
38. Deterministic Concurrent Testing with testing/synctest
Time-based concurrency tests often rely on arbitrary sleeps:
This makes tests slower and potentially flaky.
Go 1.24 introduced testing/synctest experimentally, and Go 1.25 graduated it into the standard library with a revised API. In Go 1.26, the production API is:
Within the bubble:
- goroutines started by the test belong to the isolated test environment;
- the
timepackage uses a synthetic clock; - time advances when the bubble reaches a state where all goroutines are durably blocked;
synctest.Wait()can wait until background goroutines become blocked.
A simple example:
The test does not need to actually wait for one wall-clock second.
synctest is particularly useful for testing:
- timers;
- cancellation;
- goroutine coordination;
- asynchronous callbacks;
- retry loops;
- timeout behavior;
- shutdown sequences.
It should complement, not replace, the race detector and ordinary integration tests.
[ PRODUCTION RULE ]
Do not use
time.Sleepto guess when concurrent work has completed.Use explicit synchronization in production code and
testing/synctestto make asynchronous behavior deterministic in tests.
39. Common Production Failure Patterns
40. Production Review Checklist
When reviewing concurrent Go code in modern production environments, evaluate the architecture against four strict operational boundaries.
Ownership & State Boundaries
- Explicit Owner: Is it clear which struct or package owns the mutable state?
- Escape Prevention: Do methods return mutable references—pointers, slices, or maps—to state protected by an internal lock?
- Copying Safety: Are types containing synchronization primitives passed by pointer and prevented from being copied after first use?
- Publication Safety: If state is published through
atomic.Pointer, is the published object immutable after publication?
Invariants & Atomicity
- Complete Critical Sections: Does the lock boundary cover the entire logical transaction, or are check-and-set operations split across multiple synchronization operations?
- I/O Isolation: Are network requests, disk operations, blocking channel operations, or external callbacks executed inside critical sections?
- Deadlock Ordering: If multiple locks must be acquired, is there a documented and globally enforced lock acquisition hierarchy?
- Lock Upgrade Safety: Is any
RLockfollowed by a laterLockfor the same logical operation? If so, can the complete operation be performed under one write lock instead? - Compound Map Operations: Does
sync.Mapdirectly express the required atomic operation, such asLoadOrStoreorCompareAndSwap?
Lifecycle & Cancellation
- Goroutine Leakage: Is every launched goroutine guaranteed to exit under cancellation, context timeout, shutdown, or failure?
- Context-Aware Waiting: Are waiting mechanisms responsive to
context.Done()where required? Beware ofsync.Cond.Wait(), which has no context-aware form. - Coordinated Shutdown: Is cancellation separated from completion tracking—for example,
context.Contextfor cancellation andWaitGrouporerrgroupfor completion? - Failure Propagation: If one worker fails, is it clear whether sibling workers should continue or be cancelled?
- WaitGroup Contract: If
WaitGroup.Gois used, does the supplied function avoid panicking as required by its contract?
Specialized Primitives
-
RWMutexJustification: Is the read/write workload sufficiently read-heavy to justifyRWMutex? - Atomic Snapshot Fit: Can frequently read state be replaced as an immutable snapshot using
atomic.Pointer[T]? -
sync.MapFit: Does the workload match the characteristics for whichsync.Mapis intended? - Type Safety: Is the loss of compile-time map typing justified?
- Key Churn: Has a highly dynamic key workload been benchmarked against a typed map protected by a mutex or a specialized concurrent structure?
- Pool Safety: Are objects retrieved from
sync.Poolfully reset before reuse? - Pool Retention: Are pathologically large objects excluded from re-pooling when necessary?
- Pool Correctness: Would the program remain correct if the pool were always empty?
Profiling & Verification
- Race Detection: Is the code exercised with
go test -race ./...? - Deterministic Testing: Are asynchronous interactions tested with
testing/synctestwhere synthetic time and goroutine quiescence are useful? - Mutex Profiling: Has mutex contention been measured rather than inferred?
- Benchmark Evidence: Is the choice among
Mutex,RWMutex,sync.Map,atomic.Pointer, andsync.Poolbacked by workload-representative benchmarks? - Allocation Evidence: Has
sync.Pooldemonstrated a measurable reduction in allocation or GC pressure? - Cache Behavior: For extremely hot structures, has false sharing been considered and measured?
- Shutdown Testing: Are cancellation, timeout, worker failure, and shutdown paths explicitly tested?
The Core Principle
The sync package is deliberately minimal because synchronization primitives do not define architecture—they enforce precise constraints.
The core primitives provide narrowly defined guarantees:
Mutex→ exclusive access to protected stateRWMutex→ concurrent readers with exclusive writersatomic.Pointer[T]→ atomic publication of pointer-based stateOnce→ one-time executionOnceFunc/OnceValue/OnceValues→ reusable one-time execution and initialization patternsWaitGroup→ completion trackingWaitGroup.Go→ goroutine-backed task registration and completion trackingCond→ waiting for a shared-state predicateMap→ specialized concurrent map operationsPool→ temporary object reuse
Other parts of Go's concurrency toolbox solve different problems:
context.Context→ cancellation, deadlines, and request-scoped lifecycle- channels → communication, ownership transfer, and cancellable waiting
errgroup→ coordinated worker lifecycles, error propagation, and cancellationtesting/synctest→ deterministic testing of asynchronous behavior
Engineering correctness comes from establishing clear invariants and matching them to the appropriate primitive.
The primary question in production review is never:
"Is this struct thread-safe?"
It is:
"What invariant is being protected, who owns that invariant, where does the atomic state transition begin and end, and how does its lifecycle terminate?"
That is the real purpose of sync in production Go.