Go Package Management For 2014

Jan 16, 2014


Introduction
In October 2013 I sent out a call to action to the Go community. I wanted to form a group of Gophers that would come together and help write a specification and build a working implementation of a package management tool. We are not there yet, but the group did accomplish a few things:

  • We started a mailing list called Go package management [go-pm] where people could discuss ideas and get feedback on existing and new tools.
  • We wrote a goals document that outlined what we wanted to accomplish along with guidelines and help for package management tool authors.
  • We identified and categorized the existing set of tools that are available for use.
  • We developed a set of user stories that describe the different scenarios people were facing.
  • We started a list of packages that tool developers can use to test their tools against.
  • We helped some of the tool developers find and fix bugs.
Overall I think the last 3 months have been productive and I am pleased with the results.

Building Tools
I have come to appreciate that there isn’t going to be a silver bullet or a one tool fits all solution. I believe a majority of the use-cases that have been defined in the goals document can be solved and building tools to handle these use-cases is worth the time and effort. If you’re thinking about building a tool, please consider these guidelines which are outlined in the goals document:

Work with the Go language as defined in the Go 1 spec of March 28, 2012.
Don’t implement solutions that require feature changes or build tools that change the way Go works.

Provide backwards compatibility with go get and the convention of using VCS repositories in the import paths.
The existing set of programs, build processes and workflows can’t break. You must respect the existing environments and allow them to continue to function.

Prevent version skewing.
Don’t build into the solution the potential for version skewing to occur. Such as requiring semantic versioning in the import path urls. Imports should not need to be changed to access the latest or different versions of a package.

Work on all operating systems and architectures that Go currently supports.
One of the great things about Go is that programs can be built on all these different operating systems and architectures. Your tool should not exclude a platform or make use of a specific operating system construct like symlinks.

Interoperate in such a way that ‘go get’ continues to work for package authors who do not wish to participate, or for existing Go source code that cannot be changed. Also, do not force package authors to choose between making their code go getable or using the proposed solution.
No one should be required to use a tool in order to share or import a repository. The tooling must continue to work for everyone.

All these guidelines are important because they will allow others to try and use your tools without the need to refactor any existing code. They also guarantee that existing projects continue to build and install with the Go tooling on all platforms.

Choices
I would love to see the community rally around a few tools and help improve them. There are two classes of tools that I think work well with Go, Vendoring and Revision Locking.

Vendoring takes the 3rd party source code that is referenced in your project and makes a copy of that code inside a new folder within the project. All the code your project needs is inside the one project repository. Vendoring also provides a performance enhancement on getting the code because only one url call is required.

Revision Locking creates a dependency file that references specific commits in the different version control systems the code is located in. Just like vendoring, the RL tool is used to get, build and install your project. One advantage is that your project repository continues to only contain the specific project code.

Choosing how to handle package management it is a matter of personal preference. The Go team recommends vendoring, which can be found in the Go FAQ. They mention Keith Rarick’s tool goven as an option. Keith has abandoned** goven for his other tool godep, which provides both vendoring and revision locking.

** After talking with Keith he has stated that he has not totally abandoned goven, it is just "finished". He continues to maintain the package and merge bug fixes when necessary.

New Call To Action
For 2014 I would like to see the Go community play a greater role in helping the package management tool authors. There are several ways I think this can be done:
  • Participate in the go-pm group. Give advice and help to those who have questions.
  • Submit packages that the tool authors can use to help test their tools.
  • Report bugs and feature requests.
  • Continue to add user stories that are missing in the goals document.
  • Submit your new tools and ideas to the go-pm group.
  • Work with the existing tool authors to improve the tools that we have today.
I hope to see all of you participating in the go-pm mailing list this year. I love Go and only want to see it improve for everyone.