Sunday, 20 January 2013

Golang: Overflowing JSON

Go's json.Unmarshal function works perfectly by taking a JSON blob and attempting to create a known structure from the data.

type S struct {
    A int
    B string

var s S

json.Unmarshal([]byte(`{"A": 42, "B": "b","C": "c"}`),&s) 

I recently had to construct a structure in this way, but also needed to store data that overflowed the structure.

I came up with this little function to do just that.

func UnmarshalJSON(src []byte, dst interface{}) (remainder []byte, err error) {
    var m map[string]interface{}
    o := make(map[string]interface{})
    _ = json.Unmarshal(src, &m)

    // put anything that doesnt match dst into a map
    rv := reflect.ValueOf(dst).Elem()
    for k, v := range m {
        if rv.FieldByName(k).IsValid() == false {
            o[k] = v

    // marshal the map to JSON
    remainder, _ = json.Marshal(o)

    // now fill the dst
    err = json.Unmarshal(src, dst)


Now if you set src bytes to the remainder bytes in the call you can use this function to 'consume' structures from JSON blobs.

src, _ = UnmarshalJSON(src,&myStruct)