Go ships with a buildin function append that can append elements to the end of a slice and returns the updated slice. Here is an example:

data := []int{ 1,2,3 }
data = append(data, 4)

fmt.Println(data)

This will print the following:

[1 2 3 4]

Now I was browsing to some Go code and found something like this:

data := []int{ 1,2,3 }
next := []int{ 4,5,6 }

for _ , v := range next {
    data = append(data, v)
}

This is not a very good way to append a slice to another slice. Actually it is bad and here is why. The append function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated.

The important part here is that it will allocate a new underlying array everytime there is insufficient capacity. Because the code above only appends a single value at the time, the runtime does not know what to expect, otherwise that it should grow to make room for an single extra value. The whole idea behind slices is to prevent unnecessary allocations.

An easy way to fix this is to pass all the data that needs to be appended at once. This way the runtime has all the information needed to grow only once, if needed.

data := []int{ 1,2,3 }
next := []int{ 4,5,6 }

data = append(data, next...)

Notice the suffix ... added to next. This is Go's way of telling to a variadict that we want to pass multiple arguments instead of a single value. If we remove the ... we will receive an compile error that next an invalid type to append to data. This makes sense because we can't append an []int value to an []int, only int values are allowed.

I guess that the programmer used an for loop to resolve this compile error instead of passing multiple arguments at once.

After upgrading my Go 1.1.1 to 1.1.2, 9o fails to compile packages - or have dependencies - that are already compiled with errors like this:

specs/client/AppendingToAStream_test.go:4: import /Users/pjvds/dev/mygo/pkg/darwin_amd64/github.com/smartystreets/goconvey/convey.a: object is [darwin amd64 go1.1.1 X:none] expected [darwin amd64 go1.1.2 X:none]

The last part in the errors tells me that this is because the code has previously been compiled with Go 1.1.1. Which makes perfect sense since I just updated. After a short journey of trial and error I found the solution that worked for me.

Since building is very fast with Go, the easiest thing to do is to remove the compiled packages.

rm -rf $GOPATH/{bin,pkg}

Happy coding!

The go get command helps to download the packages to the right location for you. Currently this command does not support cloning a git repository over SSH, which makes it harder to retrieve private packages in an automated fassion. In this blogpost I want to share my solution for retrieving private packages from GitHub or Bitbucket from your build at wercker using deploy keys.

Go dependencies are quite simple. All code should be available in the Go workspace. This means that you must have a copy of the source code of all your dependencies stored in the local file system. With the go get command you retrieve these from remote source control systems. Go knowns populair hostings services like GitHub and BitBucket. It supports git, mercurial, svn and bazzaar.

No SSH support

Although the documentation states otherwise, the current version of go does not support the retrieval of code from private repositories via SSH. We need to do this manually in the build pipeline. It will allways try to retrieve the code via https. Https is not the protocoll you want to use for this because it requires a username and password from an existing GitHub or BitBucket user account which will also effect the limits of your plan. When using SSH you can use deploy keys to gain access to a repository.

Deploy Keys

A deploy key is simply a password-less SSH key that grants read-only access to a repository. Both GitHub and Bitbucket support deploy keys by adding the public part of a key. Deploy keys have the following features and limitations:

• Deployment keys do not apply to your plan limit.
• You can add the same deployment key to multiple repositories.
• The deployment key must be unique — it cannot also be associated with an account.

Adding a key to your application

Wercker allows you to generate SSH keys for your application, which we can be used as a deploy key. Open your application on wercker and go to the application settings tab. Here you find the key management section. Use the generate new key pair button to create a new key.

Add variable for key

Before we can use the key in our build pipeline, we need to expose it as a variable. This can be done in the pipeline section of the application settings tab. Press the add new variable button, pick the SSH Key pair option and select your key.

This will expose the key as two variables, in my case: MYPACKAGE_KEY_PUBLIC and MYPACKAGE_KEY_PRIVATE. The first one holds the public part of the key, and the latter the private part.

Add key as deploy key

You can copy the public key from the previous step and add this as a deploy key to the private git repository that contains the package. This option can be found in the settings page of your repository at either GitHub or Bitbucket.

Add key to build pipeline

We can leverage the add-ssh-key step to write the key as an identity file.

box: wercker/golang
build:
  steps:
    - add-ssh-key:
        keyname: MYPACKAGE_KEY

Getting the dependency

Due to a bug in go get we cannot get the private package via ssh. Until this is fixed we need to clone the repository ourself. As said in the intro, an import path denotes a package stored in the local file system. If we clone the package into the correct location, go get will see it is already installed and will not try to get it. It will also be able to update the package, since go get will execute a git fetch with the default remote that is added by our clone.

To clone the package to the correct path we need to understand the structure for code hosting services. Here are the conventions for GitHub and Bitbucket, a full list is available via the go help remote command:

Bitbucket (Git, Mercurial)
        import "bitbucket.org/user/project"

GitHub (Git)
        import "github.com/user/project"

These import paths are relative from the $GOPATH/src/ path. Here is an updated version of my wercker.yml that clones the private package into the correct directory:

box: wercker/golang
build:
  steps:
    - add-ssh-key:
        keyname: MYPACKAGE_KEY
    - setup-go-workspace
    - script:
        name: Clone private packages
        code: |-
          git clone git@github.com:pjvds/private-package.git $GOPATH/src/github.com/pjvds/private-package

The GOPATH environment variable is created by the setup-go-workspace step.

Final pipeline

We can now execute a go get to get the rest of the dependencies, if any. Executing a go build and a go test finalizes the build pipeline. Here is the final wercker.yml I used:

box: wercker/golang
build:
  steps:
    - add-ssh-key:
        keyname: MYPACKAGE_KEY
    - setup-go-workspace
    - script:
        name: Clone private packages
        code: |-
          git clone git@github.com:pjvds/private-package.git $GOPATH/src/github.com/pjvds/private-package
    - script:
        name: Go build
        code: go build -v
    - script:
        name: Go test
        code: go test -v

You could also execute only the go test command in the build pipeline, since it will also build the code if needed. But by spliting these steps you better understanding which of these two failed.

Earn some stickers!

Let me know about the applications you build with wercker. Don't forget to tweet out a screenshot of your first green build with #wercker and we'll send you some @wercker stickers.