Improving Go build time with wercker cache
I have added my Go projects to wercker, which now builds, test and deploys my software. In this post we want to take you through speeding up your build time for golang projects with caching present on wercker.
The build
Here are the steps including their duration from one of the builds:
- get code (0 sec)
- setup environment (15 sec)
- environment variables (0 sec)
- setup golang (0 sec)
- go get (55 sec)
- go build (0 sec)
- go test (1 sec)
- saving build output (0 sec)
Total time: 1 min 14 sec
This means that the go get step constitutes 75% of my build time.
Wercker cache
Wercker has a per project build cache directory that is shared betweens builds.
A build can update the cache by writing to the $WERCKER_CACHE_DIR
directory. If the build succeeds, this directory is saved for future
builds. We can levarage this to cache for our golang
workspace.
Go workspace
The golang box that I have
created in order to add Go support to wercker sets up a Go workspace in $HOME/go. This workspace contains the three required directories at its root:
- src contains Go source files organized into packages (one package per directory),
- pkg contains package objects, and
- bin contains executable commands.
When a build is finished, this workspace is filled with:
- The source code and compiled end result of your project.
- The source code and compiled end result of the dependencies
The go get of future builds could be improved if this workspace is shared.
Storing the workspace in cache
I added a script step to the end of my wercker.yml that rsync's the Go full workspace, but excludes the source directory.
- script:
name: Store cache
code: |-
rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$GOPATH/" "$WERCKER_CACHE_DIR/go-pkg-cache/"
Populate cache
I added a script step that first checks if the cache directory is
present, and if this is the case, it rsync's it into the Go workspace at the $GOPATH.
- script:
name: Populate cache
code: |-
if test -d "$WERCKER_CACHE_DIR/go-pkg-cache"; then rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$WERCKER_CACHE_DIR/go-pkg-cache/" "$GOPATH/" ; fi
The result
The go get now has a duration of 0 seconds, the actual time is of
course a handful of milliseconds, but the bottom-line is that we
improved the build time with 55 seconds!
- get code (0 sec)
- setup environment (14 sec)
- environment variables (0 sec)
- setup golang (0 sec)
- go get (0 sec)
- go build (0 sec)
- go test (1 sec)
- saving build output (0 sec)
What is next
Next up is wrapping this logic in wercker steps, which I will release as a part 2 of this post, so stay tuned.
