Posts Categorized “programming”

Improving Go build time with wercker cache

I have added my Go projects to wercker, which now builds, test and deploys my software. In this post we want to take you through speeding up your build time for golang projects with caching present on wercker.

The build

Here are the steps including their duration from one of the builds:

  1. get code (0 sec)
  2. setup environment (15 sec)
  3. environment variables (0 sec)
  4. setup golang (0 sec)
  5. go get (55 sec)
  6. go build (0 sec)
  7. go test (1 sec)
  8. saving build output (0 sec)

Total time: 1 min 14 sec

This means that the go get step constitutes 75% of my build time.

Wercker cache

Wercker has a per project build cache directory that is shared betweens builds. A build can update the cache by writing to the $WERCKER_CACHE_DIR directory. If the build succeeds, this directory is saved for future builds. We can levarage this to cache for our golang workspace.

Go workspace

The golang box that I have created in order to add Go support to wercker sets up a Go workspace in $HOME/go. This workspace contains the three required directories at its root:

  • src contains Go source files organized into packages (one package per directory),
  • pkg contains package objects, and
  • bin contains executable commands.

When a build is finished, this workspace is filled with:

  • The source code and compiled end result of your project.
  • The source code and compiled end result of the dependencies

The go get of future builds could be improved if this workspace is shared.

Storing the workspace in cache

I added a script step to the end of my wercker.yml that rsync's the Go full workspace, but excludes the source directory.

- script:
    name: Store cache
    code: |-
      rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$GOPATH/" "$WERCKER_CACHE_DIR/go-pkg-cache/"

Populate cache

I added a script step that first checks if the cache directory is present, and if this is the case, it rsync's it into the Go workspace at the $GOPATH.

- script:
    name: Populate cache
    code: |-
        if test -d "$WERCKER_CACHE_DIR/go-pkg-cache"; then rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$WERCKER_CACHE_DIR/go-pkg-cache/" "$GOPATH/" ; fi

The result

The go get now has a duration of 0 seconds, the actual time is of course a handful of milliseconds, but the bottom-line is that we improved the build time with 55 seconds!

  1. get code (0 sec)
  2. setup environment (14 sec)
  3. environment variables (0 sec)
  4. setup golang (0 sec)
  5. go get (0 sec)
  6. go build (0 sec)
  7. go test (1 sec)
  8. saving build output (0 sec)

What is next

Next up is wrapping this logic in wercker steps, which I will release as a part 2 of this post, so stay tuned.

Install Go on Ubuntu

Here is a quick gist that installs Go on Ubuntu.

  • Go root will be in usr/local/go
  • Go path will be in $HOME/go
wget https://go.googlecode.com/files/go1.1.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.1.2.linux-amd64.tar.gz
rm go1.1.2.linux-amd64.tar.gz

mkdir -p "$HOME/go/"{src,pkg,bin}
echo 'export GOPATH="$HOME/go"' >> "$HOME/.profile"

echo 'export GOROOT="/usr/local/go"' >> "$HOME/.profile"
echo 'export PATH="$GOROOT/bin:$GOPATH/bin:$PATH"' >> "$HOME/.profile"

source "$HOME/.profile"

Build and deploy status notification with the cctray feed

At wercker we have added build and deployment status output in the cctray.xml format, an RSS-like continuous integration server status feed, which was originally developed for CruiseControl.net. In this post I would like to share the details and how I leverage this to get notified on important wercker status changes via CCMenu.

image

Configuration

Wercker offers two endpoints per project that serve status information in the cctray.xml format. One for the build status, and one for the deployment status.

Buids status feed

The feed url that contains the build status from an project is:

https://app.wercker.com/api/v2/applications/{PROJECT-ID}/cc/build

You need to replace {PROJECT-ID} with the project id of the project you want to monitor.

Deploy status feed

The feed url that contains the deployment status of a project's deploy target is:

https://app.wercker.com/api/v2/applications/{PROJECT-ID}/cc/deploytargets/{DEPLOY-TARGET-NAME}

You need to replace {PROJECT-ID} with the project id of the project you want to monitor and {DEPLOY-TARGET-NAME} with the name of the deploy target. Read more about deploy targets on ou dev center

Tools

Here is a list of tools that allow you to monitor cctray.xml feeds and that can be used to monitor your build and deployment statuses of your wercker projects.

Using CCMenu

There are a lot of tools that support the cc status format. As I'm mostly working on a Mac I decided to use CCMenu. It allows you to monitor multiple projects at the same time, and identifies the overall build status with just a glance at the menu bar.

image

Download and install the latest version of ccmenu from the CCMenu project.

Finding your project id

Navigate to your project page at app.wercker.com and copy your project id from the url.

image

Add builds to CCMenu

Open CCMenu by clicking on the icon in the top menu bar and open preferences.

image

Enter feed details

Click on the plus sign in the preference window and enter your project cc feed url. It is important to follow the following steps closely because CCMenu was acting a bit weird when I entered my details.

  1. Enter your feed url
  2. Click Use URL as entered above option
  3. Enter your wercker credentials
  4. Check The server requires authentication
  5. Click Continue
  6. See the status of your project in your menu bar!

image

Other feeds

You can repeat the previous steps for other feeds as well.

Notification center

CCMenu supports Mac's Notification Center

image

Final result

image

Earn some stickers

Let me know about the applications you build with wercker. Don't forget to tweet out a screenshot of your first green build with #wercker and I'll send you some @wercker stickers.