noCopy convention

Posted on Jul 28, 2024

In this post I briefly explain what is the noCopy convention in Go, and when/how to use it. This is nothing new, it dates back from 2016, but I think it is worth mentioning, as it is kind of a hack and the reasoning behind it is not that obvious.

Context

The noCopy convention is a way to tell linters that a structure should not be copied. End of the blog post. The rest of it is uninteresting details. Stop right there. You have been warned.

The `go vet` command

You probably already know go vet, but if you don’t, here is a little description. It is a linter directly integrated in the Go toolchain. It is used widely to catch common issues in Go code, and you probably are using it without realizing whenever you IDE checks for errors in your code. Here is how it describes itself:

Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string. Vet uses heuristics that do not guarantee all reports are genuine problems, but it can find errors not caught by the compilers.

— go vet documentation ¹

Although there are other, complementary or “meta” linters out there, go vet is the most common one and we’ll focus on it in this post. But keep in mind that the noCopy convention works with most linters out there.

The `copylock` analyzer

The vet linter works by running a set of analyzers on the code - each analyzer is responsible for a scope of potential issues. The one that is relevant here is the copylock analyzer. This analyzer goes through an AST to find instances of the sync.Locker interface that are copied by value.

// A Locker represents an object that can be locked and unlocked.
type Locker interface {
	Lock()
	Unlock()
}

The sync.Locker interface just describes two methods: Lock and Unlock. This is what mutexes implement, and here is what a mutex looks like:

type Mutex struct {
	state int32
	sema  uint32
}

The state attribute is 0 when it is unlocked, and 1 when it is locked. The sema attribute counts the number of consumers waiting on the mutex. That’s an approximation, but it is good enough.

The obvious issue with copying an instance of that structure is that these are just two integers values. Copying the value just copies the state, but the two copies are not synchronized, using different addresses in memory. Locking one copy does not lock the other, and you end up with a debugging nightmare, until you spot the issue.

Here is what the copylock analyzer looks for:

var lockerType *types.Interface

// Construct a sync.Locker interface type.
func init() {
	nullary := types.NewSignature(nil, nil, nil, false) // func()
	methods := []*types.Func{
		types.NewFunc(token.NoPos, nil, "Lock", nullary),
		types.NewFunc(token.NoPos, nil, "Unlock", nullary),
	}
	lockerType = types.NewInterface(methods, nil).Complete()
}

It just looks for instances of types that implement the sync.Locker interface, implementing the Lock and Unlock methods. If they are copied by value somewhere, it will insult you. Or at least it should.

The noCopy convention

From May 15, 2014 to Mar 1, 2016, the big brains of Google discussed the issue of “how we might detect accidental copies of types that shouldn’t be copyable”. Turns out the initial 2014 proposal got it almost right.

Here is the sync.noCopy structure:

// noCopy may be added to structs which must not be copied
// after the first use.
//
// See https://golang.org/issues/8005#issuecomment-190753527
// for details.
//
// Note that it must not be embedded, due to the Lock and Unlock methods.
type noCopy struct{}

// Lock is a no-op used by -copylocks checker from `go vet`.
func (*noCopy) Lock()   {}
func (*noCopy) Unlock() {}

It was since duplicated in sync/atomic and internal packages. The noCopy structure is no more than a dummy implementation of sync.Locker. You just define it somewhere, have a an attribute of that type in your non-copyable structures, and that’s it. The copylock analyzer will look after it.

Here is how it is used, for example, in atomic.Int32:

// An Int32 is an atomic int32. The zero value is zero.
type Int32 struct {
	_ noCopy
	v int32
}

The atomic.Int32 is just a wrapper around an int32 value, doing atomic Load/Store/Swap/CAS operations. It might not be obvious to the average user, that might think they can just move it around like some kind of magical item and have it work as intended. The noCopy attribute is here to tell linters that this is a no-no.

The key takeaway here is that you can do the same, if you ever have a structure that should not be copied after first use. Just define a noCopy structure somewhere in your package and use it as an attribute in your structure.

But why?

One can use noCopy to:

protect from data races and other synchronization issues,
avoid unnecessary large struct copies,
help enforce singleton patterns,
and whatever reason your imagination can come up with.

Although the noCopy convention sounds like good idea, I rarely come accross it in the wild, outside of the Go source code itself.

Before `noCopy`

That’s where things go awry. The sync.Cond structure tried to solve the same issue before noCopy become a thing. Here is how that looked like:

// copyChecker holds back pointer to itself to detect object copying.
type copyChecker uintptr

func (c *copyChecker) check() {
	// Check if c has been copied in three steps:
	// 1. The first comparison is the fast-path. If c has been initialized and not copied, this will return immediately. Otherwise, c is either not initialized, or has been copied.
	// 2. Ensure c is initialized. If the CAS succeeds, we're done. If it fails, c was either initialized concurrently and we simply lost the race, or c has been copied.
	// 3. Do step 1 again. Now that c is definitely initialized, if this fails, c was copied.
	if uintptr(*c) != uintptr(unsafe.Pointer(c)) &&
		!atomic.CompareAndSwapUintptr((*uintptr)(c), 0, uintptr(unsafe.Pointer(c))) &&
		uintptr(*c) != uintptr(unsafe.Pointer(c)) {
		panic("sync.Cond is copied")
	}
}

Yes, that’s a runtime check trying to figure out if the copyChecker address changed, hence copied. A copyChecker was added as an attribute to sync.Cond, and all methods would start with a check:

func (c *Cond) Broadcast() {
	c.checker.check()
	runtime_notifyListNotifyAll(&c.notify)
}

On the fast path, that’s a pointer comparison every time you call a method on sync.Cond. Since then, sync.Cond has a noCopy attribute, and the copyChecker is gone. Just kidding - it is still there, in addition to noCopy. I have no idea why this was not removed yet.

The trick copyChecker is using is simple: it is an uintptr to itself. Initially it is equal to zero, and is CAS’d on the first call to check from 0 to its own address - this sets up the copyChecker for subsequent calls. On susequent calls, it is checked against its own address. If it is different, then it means it was copied: the address of the structure changed since the first call to check. To recap, the fast path is a check of the uintptr value against its own address, if that fails, it is CAS’d from 0 to its own address to cover the initialization case, and then in a third step it is checked again against its own address. In case all three steps fail, it means the structure was copied after the initial call to check, as the copyChecker address changed.

Fun fact

The copylock analyzer contains dirty code to check for the presence of sync.noCopy instances.

	// In go1.10, sync.noCopy did not implement Locker.
	// (The Unlock method was added only in CL 121876.)
	// TODO(adonovan): remove workaround when we drop go1.10.
	if analysisutil.IsNamedType(typ, "sync", "noCopy") {
		return []string{typ.String()}
	}

As stated in the comments, this is because the noCopy structure in the sync package did not fully implement the sync.Locker interface until Go 1.11. Not sure the TODO will ever be removed.

The end

I hope this post was useful to you, and that you learned at least one thing from it. Now go have fun!

go vet command documentation ↩︎