Sunday, 20 January 2013

Golang: Overflowing JSON

Go's json.Unmarshal function works perfectly by taking a JSON blob and attempting to create a known structure from the data.

type S struct {
    A int
    B string

var s S

json.Unmarshal([]byte(`{"A": 42, "B": "b","C": "c"}`),&s) 

I recently had to construct a structure in this way, but also needed to store data that overflowed the structure.

I came up with this little function to do just that.

func UnmarshalJSON(src []byte, dst interface{}) (remainder []byte, err error) {
    var m map[string]interface{}
    o := make(map[string]interface{})
    _ = json.Unmarshal(src, &m)

    // put anything that doesnt match dst into a map
    rv := reflect.ValueOf(dst).Elem()
    for k, v := range m {
        if rv.FieldByName(k).IsValid() == false {
            o[k] = v

    // marshal the map to JSON
    remainder, _ = json.Marshal(o)

    // now fill the dst
    err = json.Unmarshal(src, dst)


Now if you set src bytes to the remainder bytes in the call you can use this function to 'consume' structures from JSON blobs.

src, _ = UnmarshalJSON(src,&myStruct)

Sunday, 6 January 2013

GOBing Down Secure Websockets

Golang's gob package allows you to do something interesting when designing message protocols. 

Consider these structures:

Now I can construct a message and assign whichever message body is required. 

The gob package handles the passing of the populated and mismatched fields during the encode / decode.

Below is a noddy example of passing such message structures through a secure websocket. The code not only shows the gob package in action but also highlights how trivial golang makes the coding of a secure websocket for both the client and server.