Legacy to the Edge

GopherJS: Go to Javascript Transpiler

Jason Stone -

In my last entry I talked about the idea of migrating from a legacy platform to a modern platform by implementing the new platform as a reverse proxy. That way you can keep your existing platform active while gradually migrating over to the new platform with minimal risk; No need to maintain 2 code bases, and no need to keep your new code on the shelf collecting dust until you've finished porting everything else. Then I demonstrating how you can do this using Go.

Amongst the feedback from the community (For which I say, thank you to everyone who read, shared, and commented. It was much appreciated. Let no one doubt how nice the online Go community is) was the question "What about your common libraries?". It's a good point. The reverse proxy solves the problem nicely when porting code route by route. But you're likely to have some common libraries shared across multiple routes. If you've only migrated some of those routes, you're going to be dual-maintaining both an old and a new version of that library until all your routes have been ported.

Maybe that's okay. Maybe you're comfortable with the pace of your migration vs the need to make maintenance changes. But if you're not in that position, you can use a transpiler to convert your code from the new language back to the old one. For my situation of moving from Javascript to Go, that means using GopherJS.

GopherJS

GopherJS transpiles Go source code to Javascript, which means you can get all the development and build time advantages of Go (eg. Static type checking, built in code coverage tool, etc.) and then run it in a Javascript library like NodeJS or the browser. The creator, Richard Musiol, has even setup a GopherJS Playground so you can give it a try online and immediately execute the code in your browser. You can even use AngularJS. In fact, that's what the playground uses, along with an AngularJS wrapper library, go-angularjs.

Naturally, there are some limitations. Like you can't run anything anything that requires cgo or anything that needs low level access to the OS (unless you build an adapter for NodeJS). Even so, it's a pretty impressive list of core packages which are compatible. And don't let the lack of "net/http" or "database/sql" access scare you away - you can still use existing Javascript libraries to fill those gaps, and I'm going to show you how.

Another thing you need to be aware of is that GopherJS produces Javascript suitable for an ECMAScript 5 (or ES5) compliant environment. So if you're building code for an older Javascript environment, like IE8 or Windows Host Script, you're going to need shims for at least Object.keys(), Object.defineProperty(), Object.getOwnPropertyNames(), and the various Typed Arrays.

And last, you need to be aware that it's not an all access border between Go and Javascript. If you want to pass a struct with methods from Go, you need to use "js.MakeWrapper()" to make those methods safe. Similarly, you can't implement Go interfaces with Javascript objects. You'll need an intermediary that accesses the Javascript object as a "*js.Object".

What does Go look like in Javascript?

First I'm going to show you what Go code looks like as Javascript. We eventually want to take an existing Javascript library and convert it to Go, so we need to know what changes (if any) we should make to our code before porting it to Go.

package main

import (
    "github.com/gopherjs/gopherjs/js"
    "github.com/rolaveric/pet"
)

func main() {
    js.Global.Set("pet", map[string]interface{}{
        "New": pet.New,
    })
}

package pet

import "github.com/gopherjs/gopherjs/js"

type Pet struct {
    name string
}

func (p *Pet) Name() string {
    return p.name
}

func (p *Pet) SetName(newName string) {
    p.name = newName
}

func New(name string) *js.Object {
    return js.MakeWrapper(&Pet{name})
}

You can see I've created a (rather contrived) Go package called 'pet' which defines a simple 'Pet' struct type, and a factory method called 'New()'. Since 'Pet' includes a method, 'New()' uses 'js.MakeWrapper()' to make the methods safe to use in Javascript. Then in 'main' I'm importing 'pet' and the 'github.com/gopherjs/gopherjs/js' package, which gives me access to Javascript context objects like the global scope. So I attach the 'New()' factory under the namespace 'pet'.

Here's the result when built with GopherJS.

1470 LOC and 45kb, uncompressed and unminified. The bulk of which is the builtin library.
It will only compile what it needs to. So if you declare types that are never used, they won't show up in the resulting code. This goes for core packages too. If I change that code so it requires "fmt", the result explodes to 12845 LOC and 624kb ("fmt" imports a LOT of stuff).

Lets take a look at what the code we wrote looks like:

// Lines 8-13
var go$global;
if (typeof window !== "undefined") {
    go$global = window;
} else if (typeof GLOBAL !=\= "undefined") {
    go$global = GLOBAL;
}

// Lines 1425-1461
go$packages["github.com/rolaveric/gopherjs/pet"] =     (function() {
    var go$pkg = {}, Pet, New;
    Pet = go$pkg.Pet = go$newType(0, "Struct", "pet.Pet", "Pet", "github.com/rolaveric/gopherjs/pet",     function(name_) {
        this.go$val = this;
        this.name = name_ !== undefined ? name_ : "";
    });
    Pet.Ptr.prototype.Name = function() {
        var p;
        p = this;
        return p.name;
    };
    Pet.prototype.Name = function() { return this.go$val.Name(); };
    Pet.Ptr.prototype.SetName = function(newName) {
        var p;
        p = this;
        p.name = newName;
    };
    Pet.prototype.SetName = function(newName) { return this.go$val.SetName(newName); };
    New = go$pkg.New = function(name) {
        return new Pet.Ptr(name);
    };
    go$pkg.init = function() {
        (go$ptrType(Pet)).methods = [["Name", "", [], [Go$String], false, -1], ["SetName", "", [Go$String], [], false, -1]];
        Pet.init([["name", "name", "github.com/rolaveric/gopherjs/pet", Go$String, ""]]);
    }
    return go$pkg;
})();
go$packages["C:\GoCode\src\github.com\rolaveric\gopherjs"] = (function() {
    var go$pkg = {}, js = go$packages["github.com/gopherjs/gopherjs/js"], pet = go$packages["github.com/rolaveric/gopherjs/pet"], main;
    main = go$pkg.main = function() {
        var _map, _key;
        go$global.pet = go$externalize((_map = new Go$Map(), _key = "New", _map[_key] = { k: _key, v: new (go$funcType([Go$String], [(go$ptrType(pet.Pet))], false))(pet.New) }, _map), (go$mapType(Go$String, go$emptyInterface)));
    };
    go$pkg.init = function() {
    }
    return go$pkg;
})();

You can easily recognise the "pet" package in that extract. I wouldn't get too worried about what it's doing there. The important thing is that it's there.

One thing I will draw your attention to is our main method at the end there, starting with main = go$pkg.main = function() {.
It's creating a map and setting the value of "New" to the function "pet.New()". It's then passing that to "go$externalize" which is a helper method GopherJS uses for turning Go types into primitive Javascript types. Take maps as an example. In Javascript, map keys can only be strings. But in Go, they can be anything. So GopherJS uses it's own special "Go$Map()" type internally, and then tries to convert it to a standard Javascript object when passed to "go$externalize".

Then it's assigning our externalised map to "go$global.pet". "go$global" is an internal variable for referencing the global scope object in Javascript. You can see it being declared on line 2. If used in a browser, it will be equivalent to "window". Otherwise, it's whatever the "GLOBAL" variable currently is. If you're using a Javascript runtime that doesn't include either of these, you'll need to manually declare "GLOBAL" yourself.

Porting a Javascript library to Go

Now we've got an idea of what our Go code will look like when it's converted to Javascript, we can start thinking about how we're going to port a part of our Javascript code to Go without breaking the rest of our Javascript code.

Lets say we've got a 'User' model object which uses the global variable 'DB' to make SQL database calls:

// User Type
function User(name, id) {
    this.name = name;
    this.id = id;
  
    this.save = function () {
        DB.query('UPDATE User SET name = ? WHERE id = ?', this.name, this.id);
    }
}

// Factory for creating a new user
User.new = function (name) {
    DB.query('INSERT INTO User (name) VALUES (?)', name);
    var id = DB.query('SELECT @@IDENTITY').nextRow()[0];
    return new User(name, id);
};

// Retrieves a user from the database
User.get = function (id) {
    var result = DB.query('SELECT name FROM User WHERE id = ?', id);
    if (result.rowCount === 0) {
        return null;
    }
    var name = result.nextRow()[0];
    return new User(name, id);
};

// Retrieves all users from the database
User.all = function () {
    var users = [];
    var result = DB.query('SELECT name, id FROM User');
    for (var x = 0; x < result.rowCount; x++) {
        var row = result.nextRow();
        users.push(new User(row[0], row[1]));
    }
    return users;
};

// Use case that still needs to use this model
function myHttpHandler(request) {
    if (request.method === 'GET') {
        var id = request.params['userid'];
        if (id) {
            return User.get(id);
        } else {
            return User.all();
        }
    } else if (request.method === 'POST') {
        return User.new(request.params['name']);
    }
}

Couple of things we know will be different when we convert this to Go.

  1. The method names will start with an uppercase letter, otherwise they won't be exported.
    This isn't idiomatic for Javascript, but that's OK because we're not writing Javacript. We're writing Go that runs as Javascript.
  2. DB will need to be an interface, with a new function for registering a DB implementation.
    That way we can switch implementations for Go and Javascript.
  3. In Go, "User" will be a type of struct, not a type of function.
    And while we can create methods for type instances, we can't create static methods like "User.new()". They'll need to go into the package namespace.
  4. While it's possible to wraps all "User" objects with "js.MakeWrapper()" so we can access "user.Save()", that means we also have to create getters and setter for the regular properties. Rather than add the extra boilerplate, "Save()" will be moved to the package namespace and take the "user" as a parameter.

With that in mind, here's what the refactored API looks like:

// Namespace created within an IIFE for private scope
var user = (function () {
    // Variable for holding the injected DB interface
    var DB;
  
    // User Type
    function User(name, id) {/* ... */}
  
    return {
        // Expose a function for setting the DB interface
        "registerDB": function (db) {
            DB = db;
        },
        "new": function (name) {
            // The "DB" type's methods will be capitalised
            DB.Query('INSERT INTO User (name) VALUES (?)', name);
      
            // Lets be a bit more type safe and specify that we expect an int
            var id = DB.Query('SELECT @@IDENTITY').NextRow().GetInt(0);
      
            return new User(name, id);
        },
        "get": function (id) {/* ... */},
        "all": function () {/* ... */},
        "save": function () {/* ... */}
    };
})();

The main difference is the addition of the "registerDB()" method for registering a DB interface implementation, rather than finding it on the global scope.

Now to the Go code:

package user

// Interface for a database result row
type DBRow interface {
    GetInt(colnum int) int
    GetString(colnum int) string
}

// Interface for a database result
type DBResult interface {
    NextRow() DBRow
    RowCount() int
}

// Interface for an object which can be used to make database queries
type DB interface {
    Query(query string, params ...interface{}) DBResult
}

// Private package variable for the registered DB interface
var db DB

// Method for registering a DB interface
func RegisterDB(newDb DB) {
    db = newDb
}

// User type
type User struct {
    Name string
    ID   int
}

// Save method for the User type
func Save(u *User) {
    db.Query("UPDATE User SET name = ? WHERE id = ?", u.Name, u.ID)
}

// Function for creating a new User
func New(name string) *User {
    db.Query("INSERT INTO User (name) VALUES (?)", name)
    id := db.Query("SELECT @@IDENTITY").NextRow().GetInt(0)
    return &User{name, id}
}

// Function for getting a single User
func Get(id int) *User {
    result := db.Query("SELECT name FROM User WHERE id = ?", id)
    if result.RowCount() == 0 {
        return nil
    }
    name := result.NextRow().GetString(0)
    return &User{name, id}
}

// Function for getting all users
func All() []*User {
    result := db.Query("SELECT name, id FROM User")
    users := make([]*User, result.RowCount())
    for x, c := 0, result.RowCount(); x < c; x++ {
        row := result.NextRow()
        users[x] = &User{row.GetString(0), row.GetInt(1)}
    }
    return users
}

package main

import (
    "github.com/gopherjs/gopherjs/js"
    "github.com/rolaveric/gopherjs-demo/user"
    "github.com/rolaveric/gopherjs-demo/user/js/db"
)

// Starting point for compiling JS code
func main() {
    js.Global.Set("user", map[string]interface{}{
        "registerDB": RegisterDBJS,
        "new":        user.New,
        "get":        user.Get,
        "all":        user.All,
        "save":       SaveJS,
    })
}

// Takes a DB adapter written in Javascript and wraps it as a DB interface
func RegisterDBJS(o *js.Object) {
    user.RegisterDB(db.JSDB{o})
}

// Takes a JS object and wraps it as a User struct
func SaveJS(o *js.Object) {
    user.Save(&user.User{o.Get("Name").String(), o.Get("ID").Int()})
}

The "Save()" method required a "SaveJS()" wrapper to bridge the JS <-> Go barrier for the "user" object, and "RegisterDBJS()" does the same for the database adapter. You can find the full working code examples at: https://github.com/rolaveric/gopherjs-demo

There you have it: A Javascript library written in Go with only a little tweaking to the original API.
And without compromising on the quality of our Go code either.

Conclusion

GopherJS bridges that gap between Go and Javascript quite nicely without compromising on quality.
There is a cost in the size of the generating code, but lets remember just how little is provided by Javascript's standard library compared to Go's core library. And once you get past that initial bootstrap, there's definitely no issue with performance.

So if you're looking for a way to port away from Javascript to Go without dual-maintaining libraries, or if you're so enamoured with Go that you can't bare to write Javascript even for the browser, then GopherJS is for you.

UPDATE: GopherJS has matured since I originally wrote this article. So with Richard Musiol's help, I've updated the examples to be more conscious of the Go <-> JS barriers. I've also created a github repo with the examples so they can be tested from end to end.

(Originally posted 11 Mar 2014 at legacytotheedge.blogspot.com.au)