Manage Dependencies With GODEP

Oct 26, 2013


Introduction

If you are using 3rd party packages, (packages that you don’t own or control), you will want a way to create a reproducible build every time you build your projects. If you use 3rd party packages directly and the package authors change things, your projects could break. Even if things don’t break, code changes could create inconsistent behavior and bugs.

Keith Rarick’s tool godep is a great step in the right direction for managing 3rd party dependencies and creating reproducible builds. The godep tool gives you two options for managing dependencies. The first option creates a dependency file with version control information and then with some godep magic, the code is built against those versions. You can also Vendor your 3rd party packages inside your projects as well. You never need to change a single source code file and everything is accomplished in conjunction with the go tooling.


Downloading Godep

Download godep using go get and make sure your $GOPATH/bin directory is in your PATH.

go get github.com/kr/godep
export PATH=$PATH:$GOPATH/bin


Create A Project

Build your project using the 3rd party packages as you normally would. Since godep does not require you to change any import paths in the code, ‘go get’ the code you need and import those packages directly. To keep the post simple, I am going to use an existing program called News Search that uses one 3rd party dependency.

export GOPATH=$HOME/example
go get github.com/goinggo/newssearch


After ‘go get’ completes, I have the following code on disk inside the GOPATH.

The News Search program is using code from a different Going Go repository, which for this project is a 3rd party package. Since ‘go get’ was successful, the code built and is installed.

Dependency Management

Once you have a project that can build and install properly, you can use godep to create the Godeps dependency file. Change to the root location for the project and run the godep save command with the -copy=false option:

cd $GOPATH/src/github.com/goinggo/newssearch
godep save -copy=false

Once the save is complete, godep creates a file called Godeps. You will save this file with your project:

{
    "ImportPath": "github.com/goinggo/newssearch",
    "GoVersion": "go1.1.2",
    "Deps": [
        {
            "ImportPath": "github.com/goinggo/utilities/workpool",
            "Rev": "7e6141d61b2a16ae83988907308f8e09f703a0d0"
        }
    ]
}

The Godeps file contains everything godep needs to create a reproducible build. The Godep file lists each 3rd party package and the git commit number for that version of code to use.


Now remove the 3rd party package from its original location and perform a build using the godep tool:

godep go build


If you remove the original 3rd party code, you can’t use ‘go build’ directly anymore, the imports don’t exist. To build the project use ‘go build’ from the godep tool.

You can also use ‘go install’ and ‘go test’ as long as you run those commands through godep.

godep go build
godep go install
godep go test

To make this work, godep performs a bit of magic. It uses a working directory and manipulates the GOPATH underneath.

Run the godep path command from inside the project folder:

cd $GOPATH/src/github.com/goinggo/newssearch
godep path

You should see the following output:

/var/folders/8q/d2pfdk_x4qd4__l6gypvzsw40000gn/T/godep/rev/7e/6141d61b2a16ae83988907308f8e09f703a0d0

If you open that folder you will see the code for that version. This code is being used to build the project:


The godep tool will continue to use the code from this location to perform future builds if it exists. Calling godep go build will download the version of code specified in the Godeps file if it doesn’t already exist.

If you open any of your source code files you will see the imports have not changed. The way godep works, it doesn’t have to change a thing. This is one of the really awesome things about godep.

Updating Dependencies

When it is time to update one of your 3rd party libraries just ‘go get’ it. Then you just need to run godep save once again to update the Godeps file. Because the imports paths in the source code files are not changed, godep will find and update the dependencies.

I have changed the 3rd party package and pushed it up to GitHub:


Now I ‘go get’ the code changes and update the Godeps file:

go get github.com/goinggo/utilities
cd $GOPATH/src/github.com/goinggo/newssearch
godep save

If I open the Godeps file the dependencies have changed:

{
    "ImportPath": "github.com/goinggo/newssearch",
    "GoVersion": "go1.1.2",
    "Deps": [
        {
            "ImportPath": "github.com/goinggo/utilities/workpool",
            "Rev": "8ecd01ec035e29915aa6897a3385ee4f8d80cc05"
        }
    ]
}

Now I use godep to build the code:

godep go build


The godep tool downloaded the new version and built the code successfully.

Vendoring

Vendoring is the act of making your own copy of the 3rd party packages your project is using. Those copies are traditionally placed inside each project and then saved in the project repository. The godep tool supports Vendoring and will place the copies inside the project that are using them.

To Vendor code with godep, don’t use any options with the save command. First clean the workspace and download a new version of the News Search program:

export GOPATH=$HOME/example
go get github.com/goinggo/newssearch
cd $GOPATH/src/github.com/goinggo/newssearch

Now issue the godep save command again but this time without the copy option:

godep save


This time you will have a Godeps folder with a special workspace subfolder and the Godeps file. All the 3rd party packages are copied into the workspace folder under src. This is setup to work with GOPATH.

Version control files are removed and no import paths are changed in any of the source code files.
Next remove the original code for the 3rd party package and perform the build:

godep go build

The build is successful and everything is ready to be pushed back into the repository.

Performing an update is as simple as downloading the new version of the 3rd party package and running godep save again.

Conclusion

The godep tool solves many of the problems that exist with creating reproducible builds. It is incredibly easy to use and sits on top of the go tooling. It doesn’t change anything about go, how you write go programs or how you import 3rd party packages. The only drawback is that godep does not support Bazaar using the non-vendored option.

For the public packages your are publishing, you can include a Godeps file to provide your "stable build" configuration. Package users can choose to use it or not. That is really cool. Build the code with go directly or through godep.

In the end, godep is a tool that:

1. Supports a Vendor and Non-Vendor solution that provides a reproducible build
2. Maintains backwards compatible with all existing Go packages
3. Provides a way to publish and access the "stable build" configuration of a product
4. Easy to update package dependencies when new package versions are available


Ultimate Go Programming LiveLessons

Ultimate Go Programming LiveLessons provides an intensive, comprehensive, and idiomatic view of the Go programming language. This course focuses on both the specification and implementation of the language, including topics ranging from language syntax, design, and guidelines to concurrency, testing, and profiling. This class is perfect for anyone who wants a jump-start in learning Go or wants a more thorough understanding of the language and its internals.

Learn more

Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our Go, Web and Data Science classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Learn More

To learn about Corporate training events, options and special pricing please contact:

William Kennedy
ArdanLabs (www.ardanlabs.com)
bill@ardanlabs.com