Code coverage can be an important metric to watch. It gives you insight in which parts of your code are covered well by tests and which might need some extra attention. In this post I will explain how I leverage gocov in adding test coverage reports to the build pipeline of one of my go projects.

Executing tests with gocov

To measure test coverage in Go we can use gocov created by Andrew Wilkins. It has a test command that executes the test with the default go test test runner and generates the coverage details in json format. The gocov test command respects the go test command and still prints output from it to the console. Here is an example of a script step that runs the tests for all packages and subpackages of the working directory and writes the coverage results to a file called coverage.json:

- script:
    name: Test
    code: |-
        # Get gocov package
        go get github.com/axw/gocov/gocov

        # Execute actual tests and store coverage result
        gocov test ./... > coverage.json

Next to the coverage.json that will be created, it writes the following output:

ok    github.com/pjvds/go-cqrs/sourcing 0.070s
ok    github.com/pjvds/go-cqrs/storage  0.023s
ok    github.com/pjvds/go-cqrs/storage/eventstore 0.022s
ok    github.com/pjvds/go-cqrs/storage/memory 0.012s
?     github.com/pjvds/go-cqrs/storage/serialization  [no test files]
ok    github.com/pjvds/go-cqrs/tests  0.021s
?     github.com/pjvds/go-cqrs/tests/domain [no test files]
?     github.com/pjvds/go-cqrs/tests/events [no test files]

You can see the actual step result on wercker: go-cqrs / 6c8cd61 / test.

Generating the report

The coverage.json created by the previous step can now be used as input for the report command. This commands generates a textual report based on the coverage.json. It makes sence to do this in a seperate step to seperate the test output and the coverage output. Here is an example of a script step that executes the gocov report command:

  - script:
        name: Coverage
        code: gocov report coverage.json

This step will write the following output:

github.com/pjvds/go-cqrs/storage/eventstore/Log.go         InitLogging            100.00% (3/3)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  EventStore.ReadStream   0.00% (0/27)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  EventStore.WriteStream  0.00% (0/22)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  downloadEvent           0.00% (0/13)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  processFeed             0.00% (0/9)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  linksToMap              0.00% (0/4)
github.com/pjvds/go-cqrs/storage/eventstore/EventStore.go  DailEventStore          0.00% (0/1)
github.com/pjvds/go-cqrs/storage/eventstore                ----------------------  3.80% (3/79)

github.com/pjvds/go-cqrs/tests/Log.go  InitLogging   100.00% (3/3)
github.com/pjvds/go-cqrs/tests         -----------   100.00% (3/3)

You can see the actual step result on wercker: go-cqrs / 6c8cd61 / coverage.

Writing to the artifact dir

The coverage.json can also be used to create a nice self sufficient html report. We can use gocov-html tool for this. Here is how I enchanced the previous step and added html reporting that is stored in the artifact directory.

- script:
    name: Coverage
    code: |-
        go get github.com/matm/gocov-html
        gocov report coverage.json
        gocov-html coverage.json > $WERCKER_REPORT_ARTIFACTS_DIR/coverage.html

When this step succeeds you can download the artifacts package and open coverage.html.

What is next

Now wercker gives insight in your test coverage for your golang projects. The next step is to use the coverage as a number to pass or fail the build. Stay tuned for an update where I will create a step to do this.

Working with docker is great, but sometimes I feel the need to cleanup my containers. Currently docker has no easy way for doing this. So let me share some commands with you I use frequently.

There has been some talk on Github about a docker clean command. If you are interested I suggest you to watch this issue: Implement a 'clean' command. Until this feature is released, you can use one of the following command to clean up all your containers.

Remove all containers

# WARNING: This will remove all your docker containers!
docker ps -a | tail -n +2 | awk '{print $1}' | xargs docker rm

Remove containers older than a week

# WARNING: This will remove all docker containers older than one week!
docker ps -a | grep 'week ago' | awk '{print $1}' | xargs docker rm

Remove containers from specific image

# WARNING: This will remove all containers from matching image(s)
docker ps -a | awk '{$2 ~ /my-image-name/} {print $1}' | xargs docker rm

I have added my Go projects to wercker, which now builds, test and deploys my software. In this post we want to take you through speeding up your build time for golang projects with caching present on wercker.

The build

Here are the steps including their duration from one of the builds:

1. get code (0 sec)
2. setup environment (15 sec)
3. environment variables (0 sec)
4. setup golang (0 sec)
5. go get (55 sec)
6. go build (0 sec)
7. go test (1 sec)
8. saving build output (0 sec)

Total time: 1 min 14 sec

This means that the go get step constitutes 75% of my build time.

Wercker cache

Wercker has a per project build cache directory that is shared betweens builds. A build can update the cache by writing to the $WERCKER_CACHE_DIR directory. If the build succeeds, this directory is saved for future builds. We can levarage this to cache for our golang workspace.

Go workspace

The golang box that I have created in order to add Go support to wercker sets up a Go workspace in $HOME/go. This workspace contains the three required directories at its root:

• src contains Go source files organized into packages (one package per directory),
• pkg contains package objects, and
• bin contains executable commands.

When a build is finished, this workspace is filled with:

• The source code and compiled end result of your project.
• The source code and compiled end result of the dependencies

The go get of future builds could be improved if this workspace is shared.

Storing the workspace in cache

I added a script step to the end of my wercker.yml that rsync's the Go full workspace, but excludes the source directory.

- script:
    name: Store cache
    code: |-
      rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$GOPATH/" "$WERCKER_CACHE_DIR/go-pkg-cache/"

Populate cache

I added a script step that first checks if the cache directory is present, and if this is the case, it rsync's it into the Go workspace at the $GOPATH.

- script:
    name: Populate cache
    code: |-
        if test -d "$WERCKER_CACHE_DIR/go-pkg-cache"; then rsync -avzv --exclude "$WERCKER_SOURCE_DIR" "$WERCKER_CACHE_DIR/go-pkg-cache/" "$GOPATH/" ; fi

The result

The go get now has a duration of 0 seconds, the actual time is of course a handful of milliseconds, but the bottom-line is that we improved the build time with 55 seconds!

1. get code (0 sec)
2. setup environment (14 sec)
3. environment variables (0 sec)
4. setup golang (0 sec)
5. go get (0 sec)
6. go build (0 sec)
7. go test (1 sec)
8. saving build output (0 sec)

What is next

Next up is wrapping this logic in wercker steps, which I will release as a part 2 of this post, so stay tuned.