1,139 reads

What is Protobuf in Hyperledger Fabric? [Explained]

by Deeptiman PattnaikApril 28th, 2020

Too Long; Didn't Read

Protobuf is Google’s serializing structured data with the language-neutral, platform-neutral and extensible mechanism. The serialize structured data is compiled in bytes and smaller, faster, simpler compared to traditional data formats like XML or. JSON data format is compiled directly to the protobuf message format invokes the SmartContracts. The data is stored in the CouchDB as a world state record. The objects are stored as key-value pair in the. data format in the data format.

Company Mentioned

featured image - What is Protobuf in Hyperledger Fabric? [Explained]

In this article, I’ll explain the technique used in serializing structured data using Protobuf in Hyperledger Fabric. Protobuf makes simpler in data processing and formatting. It structures the data with specially generated source code to easily write and read data within the same smartcontract.

Chaincode and SmartContract

In hyperledger fabric, Chaincode is a specific program that handles the core business logic agreed by the group of participants in a blockchain network. Hyperledger fabric also uses a data format technique called SmartContract defined for a specific set of a data model within a Chaincode. Chaincode can have multiple sets of SmartContract to work as a transaction logic that controls the different data models. In simple words, SmartContract governs the transactions, whereas Chaincode governs how smartcontract are packaged for deployment.

Example: If a few user records need to be stored into the ledger then we need to have a SmartContract defining all required data fields of a single

User SmartContract

type User struct {

	ID    		string 		`json:"id"`
	Email 		string 		`json:"email"`	
	Name  		string 		`json:"name"`
	Mobile 		string 		`json:"mobile"`
	Age    		string 		`json:"age"`	 
}

In this SmartContract, there are ID, Email, Name, Mobile, Age data fields relates to a single user record. Similarly, if we need to store the financial records for each user then we will have another smartcontract called Financial.

Financial SmartContract

type Financial struct {

	ID    		  	string 		`json:"id"`
	BankName 	  	string 		`json:"bankName"`	
	IFSCCode  		string 		`json:"ifscCode"`
	AccNumber	  	string		`json:"accNumber"`	
	CreatedDate		string		`json:"createdDate"`
}

So, these two smartcontracts will be deployed into the Chaincode. And will handle the transaction logic for both ledger state — WorldState, Blockchain.

SmartContract mainly performs Put, Get, Delete and GetHistory in the world state.

1. PutState — It creates new objects per distinct key or will overwrite to the existing objects.

2. GetState — It retrieves the object for a distinct key from the ledger state.

3. DelState — It removes the object from the world state of the ledger.

4. GetHistoryForKey — Returns all the transaction history records for a key across time.

All the records are stored in the CouchDB as a world state record. The objects are stored as key-value pair in JSON data format. CouchDB is faster in querying JSON collections from the database. In Blockchain state, all these records are stored in the byte and it’s immutable.

Protobuf

Protocol buffer (short as protobuf) is Google’s serializing structured data with the language-neutral, platform-neutral, extensible mechanism. The serialize structured data is compiled in bytes and smaller, faster, simpler compared to traditional data formats like XML or JSON.

Why use Protobuf?

As we have seen, there is two smartcontracts User and Financial to store the information into the ledger of the same user, which means User stores the basic bio information and Financial stores the banking details of the user.

But, if we look into the smartcontracts in terms of querying purpose there is no relation among two data collection. We cannot define the same ID as key for User and Financial data model because ledger data stores in key-value pair and for the same key the information will be overridden.

Both records will be stored with two different IDs in the ledger state

To solve this problem, Protobuf provides a faster and flexible solution. We just need to write a .proto file describing the data structure, in this case, the Financial data structure we want to store.

So, the byte result of the protobuf message format invokes directly to the User SmartContract and removing Financial SmartContract completely.

How Protobuf works?

We will see here, how to setup protobuf compiler and generating protobuf message format.

Installation

First, we need to follow a few installation processes to use protobuf-compiler.

$ go get github.com/golang/protobuf
$ go get github.com/golang/protobuf/proto
$ go get -u github.com/golang/protobuf/protoc-gen-go
$ export PATH=$PATH:$GOPATH/bin

Now, install the protobuf-compiler

$ sudo apt  install protobuf-compiler

Now, type protoc’ in the command line. It should say ‘Missing input file’, which means the protobuf-compiler is successfully installed.

Example

First, we need to create a financial.proto file. It consists of a message format of type Financial, which has four fields, bank name, ifsc code, account number, created date.

financial.proto

syntax="proto3";

package main;

message Financial {
      string bankName = 1;
      string ifscCode = 2;
      string accNumber = 3;
      string createdDate = 4;
}

Compile the proto file to generate the protobuf data model file for Financial message format.

$ protoc --go_out=. *.proto

You can see a protobuf file has generated as financial.pb.go. This file is a compatibility data model with the proto package and it will be used to convert the proto message format into bytes.

financial.pb.go

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: financial.proto

package main

import (
	fmt "fmt"
	proto "github.com/golang/protobuf/proto"
	math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type Financial struct {
	BankName             string   `protobuf:"bytes,1,opt,name=bankName,proto3" json:"bankName,omitempty"`
	IfscCode             string   `protobuf:"bytes,2,opt,name=ifscCode,proto3" json:"ifscCode,omitempty"`
	AccNumber            string   `protobuf:"bytes,3,opt,name=accNumber,proto3" json:"accNumber,omitempty"`
	CreatedDate          string   `protobuf:"bytes,4,opt,name=createdDate,proto3" json:"createdDate,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

func (m *Financial) Reset()         { *m = Financial{} }
func (m *Financial) String() string { return proto.CompactTextString(m) }
func (*Financial) ProtoMessage()    {}
func (*Financial) Descriptor() ([]byte, []int) {
	return fileDescriptor_a283ebe7677acfbc, []int{0}
}

func (m *Financial) XXX_Unmarshal(b []byte) error {
	return xxx_messageInfo_Financial.Unmarshal(m, b)
}
func (m *Financial) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
	return xxx_messageInfo_Financial.Marshal(b, m, deterministic)
}
func (m *Financial) XXX_Merge(src proto.Message) {
	xxx_messageInfo_Financial.Merge(m, src)
}
func (m *Financial) XXX_Size() int {
	return xxx_messageInfo_Financial.Size(m)
}
func (m *Financial) XXX_DiscardUnknown() {
	xxx_messageInfo_Financial.DiscardUnknown(m)
}

var xxx_messageInfo_Financial proto.InternalMessageInfo

func (m *Financial) GetBankName() string {
	if m != nil {
		return m.BankName
	}
	return ""
}

func (m *Financial) GetIfscCode() string {
	if m != nil {
		return m.IfscCode
	}
	return ""
}

func (m *Financial) GetAccNumber() string {
	if m != nil {
		return m.AccNumber
	}
	return ""
}

func (m *Financial) GetCreatedDate() string {
	if m != nil {
		return m.CreatedDate
	}
	return ""
}

func init() {
	proto.RegisterType((*Financial)(nil), "main.Financial")
}

func init() { proto.RegisterFile("financial.proto", fileDescriptor_a283ebe7677acfbc) }

var fileDescriptor_a283ebe7677acfbc = []byte{
	// 136 bytes of a gzipped FileDescriptorProto
	0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4f, 0xcb, 0xcc, 0x4b,
	0xcc, 0x4b, 0xce, 0x4c, 0xcc, 0xd1, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0xc9, 0x4d, 0xcc,
	0xcc, 0x53, 0x6a, 0x66, 0xe4, 0xe2, 0x74, 0x83, 0xc9, 0x08, 0x49, 0x71, 0x71, 0x24, 0x25, 0xe6,
	0x65, 0xfb, 0x25, 0xe6, 0xa6, 0x4a, 0x30, 0x2a, 0x30, 0x6a, 0x70, 0x06, 0xc1, 0xf9, 0x20, 0xb9,
	0xcc, 0xb4, 0xe2, 0x64, 0xe7, 0xfc, 0x94, 0x54, 0x09, 0x26, 0x88, 0x1c, 0x8c, 0x2f, 0x24, 0xc3,
	0xc5, 0x99, 0x98, 0x9c, 0xec, 0x57, 0x9a, 0x9b, 0x94, 0x5a, 0x24, 0xc1, 0x0c, 0x96, 0x44, 0x08,
	0x08, 0x29, 0x70, 0x71, 0x27, 0x17, 0xa5, 0x26, 0x96, 0xa4, 0xa6, 0xb8, 0x24, 0x96, 0xa4, 0x4a,
	0xb0, 0x80, 0xe5, 0x91, 0x85, 0x92, 0xd8, 0xc0, 0x4e, 0x32, 0x06, 0x04, 0x00, 0x00, 0xff, 0xff,
	0x44, 0x01, 0xf8, 0x14, 0xa5, 0x00, 0x00, 0x00,
}

Now, we will create an extra data field financial in the User smartcontract.

type User struct {

	ID    		string 		`json:"id"`
	Email 		string 		`json:"email"`	
	Name  		string 		`json:"name"`
	Mobile 		string 		`json:"mobile"`
	Age    		string 		`json:"age"`
	Financial	string		`json:"financial"`
}

Financial message format reference

financial := &Financial {	
	  ID: 	        "F1",
	  BankName:     "Hellenic Bank",
	  IFSCCode:     "1234",
	  AccNumber:    "8765",
  	  CreatedDate : "12/12/08,
}

While adding the user records into the ledger, you can also add financial message format reference into the same User smartcontract.

package main

import (
    "fmt"
    "log"
    "github.com/golang/protobuf/proto"
)

func main() {

    financial := &Financial {	
      BankName:	    "Hellenic Bank",
      IFSCCode:	    "1234",
      AccNumber:	"8765",
      CreatedDate : "12/12/08,
    }
     financialdata, err := proto.Marshal(financial)
     if err != nil {
        log.Fatal("marshaling error: ", err)
     }
    userdata := &User {
      ID: 	      "1",
      Email:	  "[email protected]",
      Name:	      "James",
      Mobile:     "8765432",
      Age: 	      "34",
      Financial:  string(financialdata), 
     }
  
      userDataJSONasBytes, err := json.Marshal(userdata)
      if err != nil {
        return shim.Error(err.Error())
      }
      indexName := "id"
      userNameIndexKey, err := stub.CreateCompositeKey(indexName, []string{userdata.ID})
      if err != nil {
        return shim.Error(err.Error())
      }
      err = stub.PutState(userNameIndexKey, userDataJSONasBytes)
      if err != nil {
        return shim.Error(err.Error())
      } 
}

Parsing Protobuf

Parsing protobuf data is very simple, as it stores the record in bytes, it can easily be unmarshal using the Financial proto reference.

financialByteData, err := proto.Marshal(financialData)
    if err != nil {
       log.Fatal("marshaling error: ", err)
    }

//Parsing the financial byte data
financial := &Financial{}
err = proto.Unmarshal(financialByteData, financial)
    if err != nil {
        log.Fatal("unmarshaling error: ", err)
    }

fmt.Println("BankName : "+financial.GetBankName())
fmt.Println("IFSCCode : "+financial.GetIfscCode())
fmt.Println("AccNumber : "+financial.GetAccNumber())
fmt.Println("CreatedDate : "+financial.GetCreatedDate())

Conclusion

Protobuf makes simpler in data processing and formatting. It structures the data with specially generated source code to easily write and read data within the same smartcontract.

References

1. https://developers.google.com/protocol-buffers

2. https://developers.google.com/protocol-buffers/docs/gotutorial

So, this is the basic overview of implementing Protobuf into a SmartContract in Hyperledger Fabric.

I hope you find this article useful :-)

Thanks