Why does append on a Go sub-slice overwrite data in the original slice?

A slice is a small descriptor: a pointer into a backing array, a length and a capacity. Slicing copies the descriptor, not the data. head := buf[:3] points at the same array as buf, with length 3 and capacity 5, because capacity runs from the slice's first element to the end of the backing array.

append checks the capacity first. If there is room, it writes the new element into the existing array just past the current length and returns a descriptor with the larger length. Room existed here (3 is less than 5), so 0xFF went into array index 3, which is buf[3]. With three bytes the required length of 6 exceeds the capacity of 5, so append allocated a new array, copied the data and returned a slice of that; buf was not touched.

To prevent it, use the full slice expression with three indices: head := buf[:3:3]. The third index limits the capacity to 3, so any append has to reallocate and can never write into buf.

Or make an explicit copy: head := append([]byte(nil), buf[:3]...), or slices.Clone(buf[:3]) since Go 1.21. While debugging, print len(head), cap(head): it shows 3 5 for the original code and 3 3 for the three-index form, and that capacity is what decides whether the next append writes in place.

Joining has the same hazard. c := append(a, b...) writes the elements of b into the backing array of a if a has spare capacity, and c then shares memory with a. The classic bug is x := append(base, 1) followed by y := append(base, 2) when base has spare capacity: both write to the same slot, and x ends up seeing the 2.

For an independent result, allocate first: c := make([]byte, 0, len(a)+len(b)); c = append(c, a...); c = append(c, b...), or use slices.Concat(a, b) from Go 1.22, which always returns a new slice. And always assign the result of append; the descriptor it returns may differ from the one passed in.

The same reasoning applies to function calls, because a slice is passed as a copy of the descriptor. The callee can modify elements and the caller sees that, since the array is shared. But if the callee appends, the caller's length does not change, and if the append reallocated, the caller does not see the new elements at all.

So functions that grow a slice return it: func addCRC(p []byte) []byte { return append(p, crc(p)) }, called as p = addCRC(p). When append does reallocate, capacity grows geometrically, roughly doubling for small slices and by a smaller factor for large ones; the exact policy is not specified and has changed between releases. If the final size is known, preallocate with make([]byte, 0, n).

The reverse problem exists too: a small slice of a large array keeps the whole array alive. header := bigFile[:16] taken from a 100 MB read pins all 100 MB for as long as header is reachable, because the garbage collector works on the whole backing array. Copy the bytes you need when the small piece outlives the big buffer.

Arrays, as opposed to slices, are values: a [5]byte is copied in full on assignment and when passed to a function. For small fixed-size data that is sometimes the simplest way to get an independent copy: var hdr [3]byte; copy(hdr[:], buf) copies exactly three bytes, since copy stops at the shorter of the two lengths.