Legacy to the Edge

Wading into Go

Jason Stone -

I first took notice of Go last year when I read this article about how Iron.io went from 30 servers down to 2 by converting from Ruby to Go. Since then I keep peeking back at it, reading a bit more doco here, trying out the tour, and eventually attending the local Go meetups. There I finally got my burst of inspiration to build a proof of concept for replacing an old enterprise backend I work with, built on Classic ASP, with Go.

The experience was quite fascinating for me because my primary language is Javascript, and my experience with modern server-side web frameworks is pretty shallow. For example, the way I keep thinking of pointers vs values in Go is the same way I think of objects vs strings in Javascript. It probably oversimplifies the difference, but it works for me.

The Plan

OK. We're porting a legacy web application to a new platform. What's the plan?

It'd be naive to say "X is better than Y, therefore we shall port all our Y to X immediately!" and leave it at that. There's other questions we need to answer:

  • How long will it take?
  • What does all the X you've written do while you're still dependant on Y?
  • What if there's just some things that Y can do which X can't (yet)?

The answers should be:

  • "A long time" cause that's the truth
  • "It gets used in production" because code has no value until it's used, and
  • "Then you keep using Y for those things" because, as they say, "you don't throw out the baby with the bath water".

The plan is to use the new technology as a reverse proxy which, initially, takes all requests and forwards them on to your legacy back end. Then, over time, you start porting features from the legacy platform to the new one. If something goes wrong, you can turn off that route handler and let the legacy platform pick it up again. You may recognise this as the "Strangler Application" pattern.

Doing this in Go is trivial. The core library already comes with a simple reverse proxy that suits our needs:

import (
    "net/http"
    "net/http/httputil"
    "net/url"
    "fmt"
)

func main() { 
    // New functionality written in Go
    http.HandleFunc("/new", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "New function")
    })

    // Anything we don't do in Go, we pass to the old platform
    u, _ := url.Parse("http://old.mydomain/")
    http.Handle("/", httputil.NewSingleHostReverseProxy(u))

    // Start the server
    http.ListenAndServe(":8080", nil)
}

The Martini Web Framework

While the Go core library is very complete, especially for the building of web applications, there is still plenty of room to build libraries on top. For example, while Go comes with the "testing" library and the "go test" and "go cover" commands, there's space for third-party libraries to define their own DSLs for writing tests, such as GoConvey. In the same vein, there's plenty room for different web frameworks to implement different opinions and architectures. My personal favourite is Martini.

Martini is a micro-framework, similar to Sinatra and Express. It doesn't give you bells and whistles to plug into your application (well, not out of the box), but it does give you a simple way to define chains of request handlers (like filters, if you're coming from Java servlets) and pass values to handlers through dependency injection.

You can have common handlers, called middleware, which run against every request, and handlers for routes. So you can have a common Authentication handler that identifies the user for every request, and then specific Authorization handlers for different routes that make sure the user has the required access before running the final handler. Here's an example:

package main

import (
    "github.com/go-martini/martini"
)

func main() {
    // Create a new Martini server
    m := martini.New()

    // Add middleware that logs incoming requests, and how long they take to handle
    m.Use(martini.Logger())

    // Add middleware which recovers from any panics, preventing the whole server from
    // crashing over one bad request.
    m.Use(martini.Recovery())

    // Creates a new martini router and sets it as the handler that runs after all middleware
    r := martini.NewRouter()
    m.Action(r.Handle)

    // Everything so far could be replaced by one line: m := martini.Classic()

    // Register middleware which authenticates the client
    m.Use(Authentication)

    // Public route
    r.Get("/", func() string {
        return "Hello World!"
    })

    // Private route
    r.Get("/secret", Authorization("secret access"), func() string {
        return "Hello Secret Agent!"
    })

    // Start the server
    m.Run()
}

// Type which, through reflection, Martini uses to dependency inject the user
// found by the Authentication middleware
type User interface{}

// Identifies the user for the request
func Authentication(c martini.Context) {
    // ...
    c.MapTo(user, (*User)(nil))
}

// Creates a handler function for checking that the user has specific access
func Authorization(access string) func() {
    return func(u User) {
        // ...
    }
}

Here's what a request to this code looks like, with the flow of control through the request and response:

But how does the Authorization handler know what the result of the Authentication handler was? That's where Martini's dependency injection comes into play.

It uses Go's reflect package to determine what types a handler is expecting as parameters. By default it knows about the http.Request and *http.ResponseWriter objects, which is why Martini also works with any handlers designed for http.HandleFunc().

You can add your own injectables or services by calling the MapTo() method on the martini.Context (Another default injectable) for the request. MapTo() takes a variable and the type that it should be injected for. You can also use this to wrap existing services, as Martini does with the *http.ResponseWriter.

Impressions of Go

Now that you've got an idea of what I've been working on, here's some of the impressions that Go made on me.

A well stocked box of goodies

Out of the box, Go comes with a standard library to be proud of. Take for example the fact that it comes with a simple reverse web proxy, ready to go. It also comes with a set of packages for compression, including gzip, and 11 different packages for encoding, including JSON, XML, and Base64. Normally I'd expect to go to third-party libraries or roll my own for at least a few of these. Not in Go.

And let's not forget the 'go' command line tool itself, which can:

  • Build
  • Test
  • Benchmark
  • Calculate code coverage from tests
  • Format code
  • Fix code written for older versions of Go
  • Host a web server for serving code documentation
  • and retrieve dependant packages from Git and Mercurial repositories

The convention of using the repository domain as the package namespace is a great idea which keeps dependency management nice and simple. The only drawback I've heard is that it doesn't support the ability to set specific revisions as a dependency, which will make it awkward if a third-party package introduces non-backwards compatible changes. But there are examples of the community stepping up to fill the gap.

Another favourite feature of mine is "go fmt", and the way the formatting conventions are enforced by the compiler. I am tired of arguments over code formatting, and I'm glad that the creators of Go have cut those arguments off at the knees, saying "This is how it is - deal with it. Now, back to the show!".

Third-party support isn't perfect, yet

I had one third-party technology requirement for my Go program; It had to support MS SQL, because that's what the legacy platform was using for persistence.

go-wiki's list of supported SQL drivers included an ODBC driver which is cross-platform, using Free TDS for Mac and Linux, so I thought this wouldn't be a problem. Sure enough, when I first tested it on Windows, everything seemed fine. Then I did some benchmarks and realised every stored procedure call was taking a lot longer than it should. When I tried the same calls with an ADODB driver (which I didn't want to use because it only works on Windows), I was getting much better speeds.

I haven't exhausted all my options yet to fix the issue yet. For example I've only tested it on Windows, and I haven't explored whether connection pooling will help (I'm currently opening and closing a connection on each query). But it's an unhealthy reminder that there's likely to be issues like this when moving from legacy platforms to something new like Go. If I were using Postgres or a NoSQL database, there'd be no problem. But I'm not, and so there is.

Types: The good, the bad, and the "what the?"

Go uses strict typing, which can be a bit of a shock coming from a dynamic typing language like Javascript. Most of the time it makes perfect sense and you wouldn't have it any other way. Here's a few common examples:

The simplest example possible - a function that expects a number rather than, say, a string.

func Triple(input int) int {
    // This line would be really awkward if we were allowed to accept a string
    return input * 3
}

Unlike Java but similar to Scala, Go treats functions as first class objects that you can pass around as parameters to other functions, and return them as results. But you don't want just any old function, you want one which matches your callback schema: message string as a parameter, and an error (if any) as the result.

func DoSomething(callback func(message string) error) {
    // Good thing I know this function will take a message string and return a single error response
    err := callback("Did something good!")
    if err != nil {
        panic(err)
    }
}

One thing that's unique to Go is the way it handles interfaces. They're implemented implicitly. So an interface becomes a way of saying "I need 'something' which can do X" rather than "I need a Y, which is 'something' that can do X". Anything can say "I can do X" will be accepted, and can be declared without even know what a "Y" is.

// A Doer is anything that will Do something
type Doer interface {
    Do()
}

// X needs a Doer
func X(doer Doer) {
    doer.Do()
}

// Y will Do something, and is therefore a Doer
type Y string
func (y Y) Do() {
    // Does something
}

func main() {
    var y Y = "meep"
    X(y)
}

Interfaces also give us a way to cheat the typing system entirely. An empty interface can be implemented by anything. So if we have a function which expects an empty interface, we can pass anything to it. Or we can have a slice (Equivalent of a Javascript Array) of empty interfaces, which can store anything. But that just means we need to use type assertions when we try to do something with that value, otherwise we could end up trying to get the substring of a number.

// X needs anything
func X(anything interface{}) {
    // Nothing comes for free - need to see what I got
    switch value := anything.(type) {
    case string:
        fmt.Printf("I got a string!: %s", value)
    default:
        fmt.Print("I got... something!")
    }
}

func main() {
    X("meep")
}

Here's an example which did my head in a bit at first.
The Rows.Scan() method in the database/sql package expects a slice of empty interfaces, which are the destination for the values found in the current row. Empty interfaces can be anything, right? So a slice of empty interfaces can contain anything. That makes sense because you could have numbers, strings, dates, etc. all returned in the same row.

The SQL driver I'm currently using returns everything as strings. So I thought "OK, I can just pass in a slice of strings. Strings can be empty interfaces, so therefore a slice of strings can be a slice of empty interfaces, right?

Wrong.

It's not the containers that can be anything, it's the contents. A container for strings and a container for anything are two different things; If I ask for one, I don't expect the other.
In this example of Rows.Scan(), you can get around it by loading a slice of empty interfaces with the pointers from a slice of strings.

cols, err := rows.Cols()

// rows.Scan wants '[]interface{}' as an argument, but we want the values
// loaded into a '[]string'.  So we must copy the references into such a slice
// See http://code.google.com/p/go-wiki/wiki/InterfaceSlice for details
scanArgs := make([]interface{}, len(cols))
values := make([]string, len(cols))
for i := range values {
    scanArgs[i] = &values[i]
}

if rows.Next() {
    // Load the row data into scanArgs, which references values
    err := rows.Scan(scanArgs...)
}

Thankfully this example is the exception, not the rule. And it demonstrates that if the typing system does start to restrict you, there are ways to bend it to your will. But it's always better to discover a problem at compile time than run time.

New language insecurity

This isn't really related to Go, it just happened to be the unknown language that I was trying.

I found that doing things in Go took 3-4 times longer than it would have done in Javascript. When working on something in Javascript I would simply plan it out, and then implement it. In Go I would plan it out, implement half of it, then go back to the plan, then start implementing from scratch, then back to the plan, then redo the original implementation.

This isn't because there's anything wrong with Go or right with Javascript. It's because I'm experienced with Javascript and inexperienced with Go. Experience doesn't just bring competence, it also brings confidence. When I was restarting my work in Go, it's not that it wasn't working, it's that I would start second guessing the design. "Should that have been a type? Or maybe an interface? Maybe I should just split these into separate packages... or, maybe less packages?".

Sometimes you need to put your head down and say "If it compiles and my tests pass, that's good enough for me!". You can always comes back to it later when you've: a) Got a more experienced Go developer to critique your work, or b) enough hours reading other people's Go code to form your own opinions.

Conclusion

Coming from Javascript world, Go is an intriguing place that I'd like to get to know better. The core library is very complete and powerful, though somewhat dry at times. But this allows the community to build on top of it with their own opinions, like how Martini builds on top of net/http, or GoConvey on top of testing.

If you're interested in Go but haven't found the excuse to give it a try, my advice is: Find that excuse. Rewrite something, anything, just to get a taste for it. If people at your company are dismissive of the idea, then practices a little "Constructive Disobedience" and do it anyway. "You decide your own level of involvement".

Cheers,
Jason Stone

(Original posted 2 Mar 2014 at legacytotheedge.blogspot.com.au)