In conversation with ProtocolBuffers

Question:- What are basic semantics to work with protocol-Buffers ?

Answer:- The ‘message’ is term used here with Protobuf, which shall be transferred over the network. Basically, this is a type of Object that we define. We can have numerous ‘Message’ inside one ‘.proto’ file.

Question:- What are advantages of working with Protocol-Buffers ?

Answer :- Following are the advantages with ProtoBuff :-

  • It’s language-agnostic.

Question:- Which all companies have used Protocol Buffers ? Why is this euphoria ?

Question:- How does Protocol-buffers works at all ?

Using protocol-buffers, we can have same serialisation and deserialisation for all the languages.

Question:- Can you explain, how does a simplest ‘.proto’ looks like ?

Question:- How does process of Serialisation & Deserialisation happens actually, while using Protocol-Buffers ?

Part #1.) Here is how the process of serialisation happens. The Message-format(in .proto) along with the message gets converted to Byte-Array.

Part #2.) Here is how the process of de-serialisation happens. The Message is formed using the Byte-Array and Message-format(in .proto).

Question:- Which are the supported field types with Protocol-Buffers ?

Answer:- Following are the supported field-types with Protocol-Buffers :-

  • Supported scalar type NUMBERS are as following: int32, int64, double, float, sint32, sint64, uint32, uint64, fixed32, fixed64, sfixed32, sfixed64. Pl note here that, fixed32 uses 4 bytes constantly, whereas int32 and sint32 uses variable encoding, wherein, if it can use less space, it shall use for small values.

Question:- How does a sample-proto file looks like and explain it’s basic working ?

  • Here, we have a “Greeting” message.

Question:- What’s the fundamental ideology with which Protocol-Buffers works ?

Question:- Demonstrate somewhat more complex MessageType using .proto file ?

syntax = "proto3";/*
Human represents a User of our system.
*/
message Human {
int32 age = 1;
string first_name = 2;
string last_name = 3;
bytes small_picture = 4;
bool is_profile_verified = 5;
float height = 6;
repeated string phone_numbers = 7;
enum EyeColor {
UNKNOWN_COLOR = 0;
GREEN = 1;
BROWN = 2;
BLACK = 3;
}
EyeColor eyecolor = 8; my.Date.Date birthday = 9; message Address {
string address_line_1 = 0;
string address_line_2 = 1;
string zip_code = 2 ;
string city = 3;
string country = 4;
}
repeated Address addressOfHuman = 10;
}

In above ‘message’, we also created a nested message of type Address. This is very much possible with protobuf.

Question:- In above “Human” message, we have defined “my.Date.Date”, can you explain something about the same ?

Answer:- Pl observe following points :-

  • We are creating the DATE ‘message’ in an separate file called as date.proto and then importing this DATE type of message to the aforesaid ‘Human’ message.
syntax = "proto3";package my.Date;message Date {
// Year of date. Must be from 1 to 9999, or 0 if specifying a date without
// a year.
int32 year = 1;

// Month of year. Must be from 1 to 12.
int32 month = 2;

// Day of month. Must be from 1 to 31 and valid for the year and month, or 0
// if specifying a year/month where the day is not significant.
int32 day = 3;
}

Pl note here that, there is no way for the constraints on the ranges to be adhered by proto-buf, its something code has to take care of. For example → Value of month can very well go beyond the value of 12 as well, but protobuf has no option to take control of the same.

Question:- Explain about advanced data-types supported by the Protobuf ?

Answer:- Following are advanced data-types

1.) map → It can be used to map scalars(except float/double) to values of any type. Map fields can not be repeated. example →

map<string, CustomValue> = 2

2.) Timestamp → Below is an example of usage of TimeStamp, which comes ready-made from google :-

syntax = "proto3";import "google/protobuf/timestamp.proto";package example.simple;message SimpleMessage {
int32 id = 1;
google.protobuf.Timestamp created_date = 2;
}

Question:- Whats a TAG in Protocol-Buffers ?

Answer:- Here, pay attention to the numbers used for every field of the ‘message’ object. This number is also called as TAG.

  • Range of values that TAG can take is: {1 TO 53,68,70,911}.

Question:- What is ‘protoc’ ?

Answer:- The ‘protoc’ is a way for us to generate the code. We specify the .proto files as input and we can generate code for following languages: C++/Java/C#/Python/Ruby/PHP/Objective-C/etc.

Question:- Demonstrate an example of generating the code from ‘.proto’ files and writing that MessageObject to the File.

Answer:-

Step #1.) Below is a simple.proto file, where we have defined a SimpleMessage :-

syntax = "proto3";package example.simple;message SimpleMessage {
int32 id = 1;
bool is_simple = 2;
string name = 3;
repeated int32 sample_list = 4;
}

Step #2.) Upon compiling this code, it shall auto-generate the java source-code for us.

Question:- Demonstrate a SimpleMessage object and write it to a text file. We shall then read the same object.

Answer:- Please note here that, object once written to the file can be read in any language.

Step #1.) Let’s first generate the message and write it to a file.

public static void main(String[] args) throws IOException {
System.out.println("Write message to file..");
SimpleMessage.Builder simpleMessageBuilder = SimpleMessage.newBuilder();
simpleMessageBuilder
.setId(4567)
.setIsSimple(true)
.setName("Honesty is the best policy.")
.addAllSampleList(Arrays.asList(1,2,3));
SimpleMessage simpleMessage = simpleMessageBuilder.build();
FileOutputStream fileOutputStream = new
FileOutputStream("simpleMessage_bin");
simpleMessage.writeTo(fileOutputStream);
}

Step #2.) Let’s now read back the message from this file :-

public static void main(String[] args) throws IOException {
System.out.println("Reading message now...!");
FileInputStream fileInputStream = new FileInputStream("simpleMessage_bin");
SimpleMessage messageAsReadFromInputStream = SimpleMessage.parseFrom(fileInputStream);
System.out.println(messageAsReadFromInputStream);
}

Question:- Let’s now see an example of an ENUM proto and its usage.

Step #1.) Below is the sample proto file we have created. Using protoc compiler, it shall auto-generate the java source code for us.

syntax = "proto3";
package example.enumerations;
message WeekDay {
int32 id = 1;
DayOfTheWeek day_of_the_week = 2;
}
enum DayOfTheWeek {
UNKNOWN = 0;
MONDAY = 1;
TUESDAY = 2;
WEDNESDAY = 3;
THURSDAY = 4;
FRIDAY = 5;
SATURDAY = 6;
SUNDAY = 7;
}

Step #2.) Below is how we shall be using the auto-generated code of POJO :-

WeekDay.Builder weekDayBuilder = WeekDay.newBuilder();
weekDayBuilder
.setId(1)
.setDayOfTheWeek(DayOfTheWeek.SATURDAY);

WeekDay weekDayBuiltIs = weekDayBuilder.build();
System.out.println(weekDayBuiltIs);
}

Following are two additional notes :-

Note #1.) We can also enforce the java package-name of the auto-generate class using below command inside the proto files :-

option java_package = "com.example.options";

Note #2.) If our 1 proto file contains multiple Messages and for every ‘Message’, we want to auto-generate the different/separate POJO, we can use the below command inside the proto files :-

option java_multiple_files = true;

Question:- Let’s assume that, we have to now upgrade our message that we defined above, as some of the fields needs to be added to the Message. Is that supported with protocol-buffers ?

Answer:- Yes, this is very much possible.

  • We might have a scenario, where one application is writing the Message Object in enhanced format and some application is still reading the old format of the ‘Message’.

Question:- What are the Rules for updating the Protocol (i.e. format of Message inside the .proto file) ?

Answer:- Following are the rules to update the contract :-

  • Numeric tags for any existing fields must not be changed. Now, when we add newer fields, older-code would just ignore those newer fields.

V. Imp. Point to note here is, never-ever remove the reserved tags, as it can cause conflicts in case future developer uses the reserved tag-number.

Question:- Can we also define the Services using Protocol-Buffer-Services ?

Answer:- Yes, using Protocol-buffers, we can also define the services.

Question:- What is a Service in very simple words ?

Answer:- Here is the understanding of Service :-

  • A Service is a set of end-points exposed by server.

Question:- Why Protocol-buffers format is more appreciated over JSON ?

Answer:- Following are the advantages of using Protocol-Buffers over JSON :-

  • First, Protobuf formats are more efficient in terms of the payload-size. We know all of this data has to travel over the network eventually and it can help us to save the lot of Bandwidth.

Question:- Demonstrate the comparison of payload-size between JSON & Protobuffs :-

Answer:- Say we have this JSON :- The size of below JSON is around 55 bytes.

{
"age":29,
"firstname": "Aditya",
"lastname":"Goel"
}

In contrast say, we have this protobuf format. The size of protobuf is 20 bytes.

message Person {
int32 age = 1;
string first_name = 2;
string last_name = 3;
}

Question:- Let’s now conclude our findings with Protocol Buffer (protobuf3).

Answer:- Here are our learnings in nutshell regarding Protocol-Buffers :-

  • The API can be defined in a simple and easy manner.

That’s all in this section. If you liked reading this blog, kindly do press on clap button multiple times, to indicate your appreciation. We would see you in next series.

--

--

Software Engineer for Big Data distributed systems

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store