Pitfalls With Closures In Go

Jun 20, 2014


Introduction
Closures in Go are a very powerful construct but they can also be the cause of bugs if you don’t understand how they work. In this post I am going to pull a small piece of code from Chapter 2 from the Go In Action book that discusses a pitfall you can run into when using closures. The full code example can be found in the Github repository for the book. Chapter 2 discusses this code example in full detail.

The Closure Pitfall
First let’s look at the piece of code:

search/search.go

29  // Launch a goroutine for each feed to find the results.
30  for _, feed := range feeds {
31     // Retrieve a matcher for the search.
32     matcher, exists := matchers[feed.Type]
33     if !exists {
34        matcher = matchers["default"]
35     }
36
37     // Launch the goroutine to perform the search.
38     go func(matcher Matcher, feed *Feed) {
39        Match(matcher, feed, searchTerm, results)
40        waitGroup.Done()
41     }(matcher, feed)
42  }

This code sample starts out on line 30 iterating over a slice of Feed values. The value of the feed variable declared within the for range loop is changing with each iteration. Then on line 32 the code is checking a map for a value that matches the specified key for the value of the feed.Type field. If the key does not exist, a default value for the matcher variable is then assigned. Just like the feed variable, the value of the matcher variable also changes with each iteration of the for range loop.

Now we can jump to lines 38 through 41 which still exist within the for range loop. Here we are declaring an anonymous function and launching that function as a goroutine. The anonymous function is being declared to accept two parameters. The first parameter is a value of type Matcher and the second parameter is a pointer to a value of type Feed. On line 41 we can see the value of the matcher and feed variables being passed into the anonymous function.

The implementation of the anonymous function on line 39 is where things get interesting. Here we see a call to a function named Match. This function accepts four parameters and if you look closely at the function call, you will notice the first two parameters are the variables we declared as the function parameters. The last two parameters however were not declared within the scope of the anonymous function. Here we are seeing two variables being used by the anonymous function via closures.

search/search.go

37     // Launch the goroutine to perform the search.
38     go func(matcher Matcher, feed *Feed) {
39        Match(matcher, feed, searchTerm, results)
40        waitGroup.Done()
41     }(matcher, feed)
42  }

The searchTerm and results variables are declared within the scope of the outer function yet we are able to use them within the scope of the anonymous function without the need to pass them in as parameters. A question this raises is why are we passing in the values of the matcher and feed variables as parameters but using closures for the searchTerm and results variables?

I pointed out in the beginning how the values of the matcher and feed variables were changing with every iteration of the for range loop. The values of the searchTerm and results variable are not changing with each iteration. Their values remain constant throughout the lifetime of each goroutine that is launched based on the declaration of the anonymous function. What does this have to do with anything?

When we use a variable in an anonymous function via closures, we are not passing the value of the variable at the time the anonymous function is declared. We are sharing the actual variable which means changes to that variable’s value will be reflected within the scope of the anonymous function and in our case the running goroutine. If we were to share the matcher and feed variables via closures with the anonymous function and not pass the value of these variables into the function, most of the goroutines would be processing the very last value in the slice.

In this program all of the goroutines will be running concurrently and not in parallel. By the time the first or even second goroutine is given time to run, the for range loop will be complete and the value of the matcher and feed variables will contain values for the last iteration of the loop. This mean the majority if not all of the goroutines will be processing the same values for these variables. This is ok for the searchTerm and results variables since they do not change.

Conclusion
Luckily we can declare anonymous functions that accept parameters and these types of closure problems can be avoided. In our example above, when each anonymous function is declared within the scope of the for range loop, the values of the matcher and feed variables are locked in by passing them as parameters. The code remains clean and readable by leveraging closures for the remaining variables the anonymous function requires. Before using closures to share a variable with an anonymous function, ask yourself if the value of that variable will be changing and how that affects the function when it is called to run.

To learn more details about this piece of code and the entire code sample, please take the time to download and read the Go In Action book. Chapter 2 is available for download here.


Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our Go, Web and Data Science classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Learn More

To learn about Corporate training events, options and special pricing please contact:

William Kennedy
ArdanLabs (www.ardanlabs.com)
bill@ardanlabs.com