Nov 30, 2016

Continuously delivering a Go microservice with Wercker on DC/OS

Currently, I am really into the field of building cloud native applications and the associated technology stacks. Normally I would use Java as a primary language to implement such an application. But since everyone seems to be using Go at the moment, I figured it's about time to learn a new language to see how it fits into the whole cloud native universe.

So let's implement a small microservice written in Go, build a Docker image and push it to Docker hub. We will be using the Docker based CI platform Wercker to continuously build and push the image whenever we change something in the code. The complete example source code of this article can be found on Github here.

Before you start

Make sure you have all the required SDKs and tools installed. Here is the list of things I used for the development of this showcase:
  • Visual Studio Code with Go language plugin installed
  • The Go SDK using Brew
  • The Docker Toolbox or native Docker, whatever you prefer
  • The Make tool (optional)
  • The Wercker CLI, for easy local development (optional)

Go micro service in 10 minutes

If you are new to the Go language, make sure you read the Go Bootcamp online book

To build the micro service, we will only be using the 'net/http' and 'encoding/json' standard libraries that come with Go. We define the response structure of our endpoint using a plain Go struct. The main function registers the handler function for the '/api/hello' endpoint and then listens on port 8080 for any incoming HTTP requests. The handler function takes two parameters: a response writer and a pointer the original HTTP request. All we do in here is to create and initialize the response structure, marshall this structure to JSON and finally write the data to the response stream. Per default, the Go runtime will use 'text/plain' as content type, so we also set the 'Content-Type' HTTP header to the expected value for the JSON formatted response.

package main

import (
    "encoding/json"
    "net/http"
)

// Hello response structure
type Hello struct {
    Message string
}

func main() {
    http.HandleFunc("/api/hello", hello)
    http.ListenAndServe(":8080", nil)
}

func hello(w http.ResponseWriter, r *http.Request) {

    m := Hello{"Welcome to Cloud Native Go."}
    b, err := json.Marshal(m)

    if err != nil {
        panic(err)
    }

    w.Header().Add("Content-Type", "application/json;charset=utf-8")
    w.Write(b)
}

Now it is time to trigger the first Go build for our micro service. Open a terminal, change directory into you project folder and issue the following command:

go build -o cloud-native-go

You should now have an executable called 'cloud-native-go' in your project directory which you can use to run the micro service. You should also be able to call the '/api/hello' HTTP endpoint on localhost, e.g. curl http://localhost:8080/api/hello. Done.

Go CI/CD pipeline using Wercker

Wercker is a Docker native CI/CD automation platform for Kubernetes, Marathon and general microservice deployments. It is pretty easy to use, allows local development and is free for community use. For the next step, make sure you have the Wercker CLI tools installed. The instructions can be found here.

Create a file called 'wercker.yml' in the root directory of your project and add the following code snippet to it to define the local development build pipeline. We specify the Docker base box to use for the build as well as the commands to build and run the app.

dev:
  # The container definition we want to use for developing our app
  box: 
    id: golang:1.7.3-alpine
    cmd: /bin/sh
  steps:
    - internal/watch:
        code: |
          CGO_ENABLED=0 go build -o cloud-native-go
          ./cloud-native-go
        reload: true

In order to continuously build and run our Go microservice locally, and also watch for changes to the sources, you only have to issue the following Wercker CLI command:

wercker dev --publish 8080

This will download the base box, and then build and run the app inside the container. Om case of changes Wercker will rebuild and restart the application automatically. You should now be able to call the '/api/hello' endpoint via the IP address of your local Docker host and see the result message, e.g. curl http://192.168.99.100:8080/api/hello.

Once the application and the development build are working, it is time to define the pipelines to build the application and to push the image to Docker hub. The first pipeline does have 3 basic steps: first call Go Lint, then build the application and finally copy the build artifacts to the Wercker output folder for the next pipeline to use as inputs. The following code excerpt should be pretty self-explanatory.

build:
  # The container definition we want to use for building our app
  box: 
    id: golang:1.7.3-alpine
    cmd: /bin/sh
  steps:
    - wercker/golint
    - script:
        name: go build
        code: |
          CGO_ENABLED=0 go build -o cloud-native-go
    - script:
        name: copy binary
        code: cp cloud-native-go "$WERCKER_OUTPUT_DIR"

The final pipeline will use the outputs from the previous pipeline, build a new image using a different base box and then push the final image to Docker hub. Again, there is not much YAML required to do this. But wait, where is the Dockerfile required to do this? If you pay close attention you will notice that some of the attributes of the 'interna/docker-push' step resemble the different Dockerfile keywords.

deploy:
  # The container definition we want to use to run our app
  box: 
    id: alpine:3.4
    cmd: /bin/sh
  steps:
    - internal/docker-push:
        author: "M.-L. Reimer <mario-leander.reimer@qaware.de>"
        username: $USERNAME
        password: $PASSWORD
        repository: lreimer/cloud-native-go
        tag: 1.0.0 $WERCKER_GIT_COMMIT latest
        registry: https://registry.hub.docker.com
        entrypoint: /pipeline/source/cloud-native-go
        ports: "8080"

Once you have saved and pushed the 'wercker.yml' file to Github, create a new Wercker application and point it to this Github repo. Next, define the build pipeline using the Wercker web UI. Also make sure that you define the $USERNAME and $PASSWORD variables as secure ENV variables for this application and that you set them to your Docker Hub account. After the next 'git push' you will see the pipeline running and after a short while the final Docker images should be available at Docker Hub. Sweet!

Wercker is also capable of deploying the final Docker image to a cluster orchestrator such as Kubernetes, Marathon or Amazon ECS. So as a final step, we will enhance our pipeline with the automatic deployment to a DC/OS cluster running Marathon.

    - script:
        name: generate json
        code: chmod +x marathon.sh && ./marathon.sh
    - script:
        name: install curl
        code: apk upgrade && apk update && apk add curl
    - wercker/marathon-deploy:
        marathon-url: $MARATHON_URL
        app-name: $APP_NAME
        app-json-file: $APP_NAME.json
        instances: "3"
        auth-token: $MARATHON_AUTH_TOKEN

First, we execute a shell script that generates the Marathon service JSON definition from a template enhanced with some Wercker ENV variables. Then we install 'curl' as this tool is required by the next step and it's not included in the Alpine base image. Finally, we will use the built-in Wercker step to deploy 3 instances of our microservice to a DC/OS cluster. We use several ENV variables here, which need to be set on a deployment pipeline level. Important here are $MARATHON_URL and $MARATHON_AUTH_TOKEN, which are required to connect and authenticate to the Marathon REST API.

Summary and Outlook

Implementing simple microservices in Go is pretty straight forward. However, things like service discovery, configuration, circuit breakers or metrics aren't covered by the current showcase application yet. For real cloud native Go applications we will have a closer look at libraries such as Go-Kit or Go-Micro in the next instalment.

Stay tuned. To be continued ...

References


No comments:

Post a Comment