For Range Semantics

Jun 27, 2017


Prelude

These are good posts to read first to better understand the material presented in this post:

Index of the four part series:
1) Language Mechanics On Stacks And Pointers
2) Language Mechanics On Escape Analysis
3) Language Mechanics On Memory Profiling
4) Design Philosophy On Data And Semantics

The idea of value and pointer semantics are everywhere in the Go programming language. As stated before in those earlier posts, semantic consistency is critical for integrity and readability. It allows developers to maintain a strong mental model of a code base as it continues to grow. It also helps to minimize mistakes, side effects, and unknown behavior.

Introduction

In this post, I will explore how the for range statement in Go provides both a value and pointer semantic form. I will teach the language mechanics and show you how deep these semantics go. Then I will show a simple example of how easy it is to mix these semantics and the problems that can cause.

Language Mechanics

Start with this piece of code that shows the value semantic form of the for range loop.

https://play.golang.org/p/_CWCAF6ge3

Listing 1

01 package main
02
03 import "fmt"
04
05 type user struct {
06     name string
07     email string
08 }
09
10 func main() {
11     users := []user{
12         {"Bill", "bill@email.com"},
13         {"Lisa", "lisa@email.com"},
14         {"Nancy", "nancy@email.com"},
15         {"Paul", "paul@email.com"},
16     }
17
18     for i, u := range users {
19         fmt.Println(i, u)
20     }
21 }

In figure 1, the program declares a type named user, creates four user values and then displays information about each user. The for range loop on line 18 is using value semantics. This is because on each iteration a copy of the original user value from the slice is made and operated on inside the loop. In fact, the call to Println creates a second copy of the loops copy. This is what you want if value semantics are to be used for user values.

If you were to use pointer semantics instead, the for range loop would look like this.

Listing 2

18     for i := range users {
19         fmt.Println(i, users[i])
20     }

Now the loop has been modified to use pointer semantics. The code inside the loop is no longer operating on its own copy, instead it is operating on the original user value stored inside the slice. However, the call to Println is still using value semantics and is being passed a copy.

To fix this requires one more final change.

Listing 3

18     for i := range users {
19         fmt.Println(i, &users[i])
20     }

Now there is consistent use of pointer mechanics for user data.

For reference, listing 4 shows both value and pointer semantics side by side.

Listing 4

       // Value semantics.           // Pointer semantics.
18     for i, u := range users {     for i := range users {
19         fmt.Println(i, u)             fmt.Println(i, &users[i])
20     }                             }

Deeper Mechanics

The language mechanics go deeper than this. Take a look at this program below in listing 5. The program initializes an array of strings, iterates over those strings and on each iteration changes the string at index 1.

https://play.golang.org/p/IlAiEkgs4C

Listing 5

01 package main
02
03 import "fmt"
04
05 func main() {
06     five := [5]string{"Annie", "Betty", "Charley", "Doug", "Edward"}
07     fmt.Printf("Bfr[%s] : ", five[1])
08
09     for i := range five {
10         five[1] = "Jack"
11
12         if i == 1 {
13             fmt.Printf("Aft[%s]\n", five[1])
14         }
15     }
16 }

What is the expected output of this program?

Listing 6

Bfr[Betty] : Aft[Jack]

As you would expect, the code on line 10 has changed the string at index 1 and you can see that in the output. This program is using the pointer semantic version of the for range loop. Next, the code will use the value semantic version of the for range loop.

https://play.golang.org/p/opSsIGtNU1

Listing 7

01 package main
02
03 import "fmt"
04
05 func main() {
06     five := [5]string{"Annie", "Betty", "Charley", "Doug", "Edward"}
07     fmt.Printf("Bfr[%s] : ", five[1])
08
09     for i, v := range five {
10         five[1] = "Jack"
11
12         if i == 1 {
13             fmt.Printf("v[%s]\n", v)
14         }
15     }
16 }

On each iteration of the loop, the code once again changes the string at index 1. This time when the code displays the value at index 1 the output is different.

Listing 8

Bfr[Betty] : v[Betty]

You can see that this form of the for range is truly using value semantics. The for range is iterating over its own copy of the array. This is why the change is not seen in the output.

When ranging over a slice using the value semantic form, a copy of the slice header is taken. This is why the code in listing 9 does not panic.

https://play.golang.org/p/OXhdsneBec

Listing 9

01 package main
02
03 import "fmt"
04
05 func main() {
06     five := []string{"Annie", "Betty", "Charley", "Doug", "Edward"}
07
08     for _, v := range five {
09         five = five[:2]
10         fmt.Printf("v[%s]\n", v)
11     }
12 }

Output:
v[Annie]
v[Betty]
v[Charley]
v[Doug]
v[Edward]

If you look at line 09, the slice value is reduced to a length of 2 inside the loop, but the loop is operating on its own copy of the slice value. This allows the loop to iterate using the original length without any problems since the backing array is still in tact.

If the code uses the pointer semantic form of the for range, the program panics.

https://play.golang.org/p/k5a73PHaka

Listing 10

01 package main
02
03 import "fmt"
04
05 func main() {
06     five := []string{"Annie", "Betty", "Charley", "Doug", "Edward"}
07
08     for i := range five {
09         five = five[:2]
10         fmt.Printf("v[%s]\n", five[i])
11     }
12 }

Output:
v[Annie]
v[Betty]
panic: runtime error: index out of range

goroutine 1 [running]:
main.main()
	/tmp/sandbox688667612/main.go:10 +0x140

The for range took the length of the slice before iterating, but during the loop that length changed. Now on the third iteration, the loop attempts to access an element that is no longer associated with the slice’s length.

Mixing Semantics

Here is an example that is a complete horror show. This code is mixing semantics for the user type and it’s causing a bug.

https://play.golang.org/p/L_WmUkDYFJ

Listing 11

01 package main
02
03 import "fmt"
04
05 type user struct {
06     name  string
07     likes int
08 }
09
10 func (u *user) notify() {
11     fmt.Printf("%s has %d likes\n", u.name, u.likes)
12 }
13
14 func (u *user) addLike() {
15     u.likes++
16 }
17
18 func main() {
19     users := []user{
20         {name: "bill"},
21         {name: "lisa"},
22     }
23
24     for _, u := range users {
25         u.addLike()
26     }
27
28     for _, u := range users {
29         u.notify()
30     }
31 }

This example isn’t so contrived. On line 05 the user type is declared and pointer semantics are chosen to implement the method set for the user type. Then in the main program, value semantics are used in the for range loop to add a like to each user. Then a second loop is used to notify each user, again using value semantics.

Listing 12

bill has 0 likes
lisa has 0 likes

The output shows that no likes have been added. I can’t stress enough that you should choose a semantic for a given type and stick to it for everything you do with data for that type.

This is how the code should look to stay consistent with pointer semantics for the user type.

https://play.golang.org/p/GwAnyBNqPz

Listing 13

01 package main
02
03 import "fmt"
04
05 type user struct {
06     name  string
07     likes int
08 }
09
10 func (u *user) notify() {
11     fmt.Printf("%s has %d likes\n", u.name, u.likes)
12 }
13
14 func (u *user) addLike() {
15     u.likes++
16 }
17
18 func main() {
19     users := []user{
20         {name: "bill"},
21         {name: "lisa"},
22     }
23
24     for i := range users {
25         users[i].addLike()
26     }
27
28     for i := range users {
29         users[i].notify()
30     }
31 }

// Output:
bill has 1 likes
lisa has 1 likes

Conclusion

Value and pointer semantics are a big part of the Go programming language and as I have shown, integrated into the for range loop. When using the for range, validate you are using the right form for the given type you are iterating over. The last thing you want is to mix semantics and it’s easy to do this with the for range if you are not paying attention.

The language is giving you this power to chose a semantic and work with it cleanly and consistently. This is something you want to take full advantage over. I want you to decide what semantic each type is using and be consistent. The more consistent you are with the semantic for a piece of data, the better off your code base will be. If you have a good reason to change the semantic, then document this extensively.


Ultimate Go Programming LiveLessons

Ultimate Go Programming LiveLessons provides an intensive, comprehensive, and idiomatic view of the Go programming language. This course focuses on both the specification and implementation of the language, including topics ranging from language syntax, design, and guidelines to concurrency, testing, and profiling. This class is perfect for anyone who wants a jump-start in learning Go or wants a more thorough understanding of the language and its internals.

Learn more

Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our Go, Web and Data Science classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Learn More

To learn about Corporate training events, options and special pricing please contact:

William Kennedy
ArdanLabs (www.ardanlabs.com)
bill@ardanlabs.com