Go and Protocol Buffers (Quick Tutorial)

A very short example of how to use Protocol Buffers in Go. This is proto3, i.e. the 3rd version of the protocol, but the example is generally valid for the second version as well.

Preparation

Go to this link to read, download, and install (everything is simple there).

Installation instructions in Go can be found here:

go get -u github.com/golang/protobuf/{proto,protoc-gen-go}

You may need to use -f if you have something like this in ~/.gitconfig:

[url "ssh://[email protected]/"]
insteadOf = https://github.com/

Example

For this example, we will save an array of numbers and a string, and then read them back. Furthermore, we will assume that we are in the root of our new project.

The proto-file will look like this:

msg/msg.proto

// comments follow a style C/C++
/*
   and multiline too
*/
syntax = "proto3";

// package name, this will be saved in the resulting go-file
package msg;

// type of data to be saved
message msg {
	// type field_name = field_number
	string key = 1;
	// repeated means slice
	repeated int64 value = 2;
}
/*
 In the third version, there are no required fields and extensions.
 Instead of extensions, the type `Any` is implemented (more on that later)
*/

Now, we need to compile the proto file:

protoc --go_out=. msg/*.proto

The result will be a file like this:

msg/msg.pb.go

package msg

import proto "github.com/golang/protobuf/proto"

var _ = proto.Marshal

/*
The structure looks like this. Note that tags for JSON have been added automatically
*/
type Msg struct {
	Key   string  `protobuf: "bytes,1,opt,name=key" json: "key,omitempty"`
	Value []int64 `protobuf: "varint,2,rep,name=value" json: "value,omitempty"`
}

// methods are needed to make the structure conform to the proto.Message interface
func (m *Msg) Reset()         { *m = Msg{} }
func (m *Msg) String() string { return proto.CompactTextString(m) }
func (*Msg) ProtoMessage()    {}

func init() {
}

Now let's create a structure, write its bytes, and read it back:

main.go

package main

import (
	"log"
	"./msg"
	"github.com/golang/protobuf/proto"
)

func main() {
	// create a new "message"
	msg1 := &msg.Msg{
		Key: "Hello Protocol Buffers",
		Value: []int64{1, 2, 3, 4},
	}

	// structure to bytes
	data, err := proto.Marshal(msg1)
	if err != nil {
		log.Fatal("marshaling error: ", err)
		return
	}

	// how much memory does it take?
	log.Printf("data length: %d", len(data))

	// bytes into the structure
	msg2 := new(msg.msg)
	err = proto.Unmarshal(data, msg2)
	if err != nil {
		log.Fatal("unmarshaling error: ", err)
	}

	// now both structures must be equal
	if msg1.Key != msg2.Key {
		log.Printf("unexpected value, expected '%s', got '%s'", msg1.Key, msg2.Key)
	}

	for i := 0; i < 4; i++ {
		if msg1.Value[i] != msg2.Value[i] {
			log.Printf("unexpected value, expected %d, got %d", msg1.Value[i], msg2.Value[i])
		}
	}

	log.Println("Done")
}

As you can see, it's easy. If we dig deeper, let's say there is a desire to create a database that stores "messages" - so that the type of "message" is not initially defined, and to store these "messages" in some structure. In other words, to have a library that will store what we give it in a certain format. In proto3 type Any is implemented to store any type.

Type Any looks like this:

message Any {
	string type_url = 1; // type
	bytes value = 2; // type content in bytes
}

Links

• Protocol Buffers GitHub Releases
• Protocol Buffers Issue Tracker
• One of the Protocol Buffers implementations for Go
• Guide to Protocol Buffers v3