gRPC service in Node.js: Tutorial, Examples and Best practices

gRPC is a new technology here for building APIs to achieve high load time and fast network latency. In this article, we will demonstrate how to build a gRPC-powered API in Nodejs. First, let's learn what gRPC is.

What is gRPC?

gRPC is an open-source remote procedure call(RPC) framework created by Google. It is an inter-process communication technology based on HTTP/2, that is used for client-server and duplex streaming of data, and this data streaming is highly efficient because of the use of protocol buffers.

Protocol buffer is a library that helps us serialize structured data built by Google. It is platform-, and language-neutral, it currently supports generated code in Java, Python, Objective-C, and C++. The latest proto3 version supports more languages. The protocol buffers are where we define our service definitions and messages. This is written in IDL(Interface Definition Language) language, this will be like a contract or common interface between the client and server on what to expect from each other; the methods, types, and returns of what each operation would bear.

A gRPC service contains, the server, protocol buffer, and the client

1. The server has the procedures or subroutines or methods that perform different actions. They could be to remove an item, add an item, edit an item, or anything the dev wants to do. The point here is that any action is carried out in the subroutines/methods. The server adds a service to itself using the service declared in the proto file. Methods/procedures will be created in the service to match the methods the service exports in the proto file.
2. The protocol buffer states the types, and shape of each request and response.
3. The client uses the protocol buffer to get a service and then connect to it via the server's URL and port. From here now, the client can call the methods set in the server. The awesome thing here is that the server can be built in any language and the client can be built also in any language, only the proto is written in IDL.
   

In this article, we will build a gRPC service in Nodejs.

gRPC in Node.js

Nodejs is the world's most popular and used runtime. Many popular startups and companies use Nodejs. So it will be very informative and useful to know how to set up a gRPC service in Nodejs. Let's begin.

We will build a News service. With this service, we can perform CRUD actions on it. We can add news, edit a piece of news, delete a piece of news, get all the news and get a piece of particular news.

Prerequisites

We will need a basic knowledge of:

• JavaScript
• How Nodejs works
• REST
• API
• Client to server communication.

We will need the following tools installed in our machine:

• Nodejs: You can download Nodejs binary from here
• NPM/Yarn: NPM comes bundled with Nodejs binary. Yarn can be installed using NPM npm i yarn -g.
  

So that's it.

Scaffold Node.js project

So we create a grpc-nodejs folder:

mkdir grpc-nodejs

Move inside the folder:

cd grpc-nodejs

Initialize the Node environment:

npm init -y

Now, we will need the dependency:

• @grpc/grpc-js: This is a gRPC library for the Nodejs. It enables us to create gRPC service in the Nodejs runtime.
• @grpc/proto-loader: A utility package for loading .proto files for use with gRPC, using the latest Protobuf.js package.
  

So we install it:

npm i @grpc/grpc-js

Now, we create a news.proto, server.js, and client.js files.

touch news.proto server.js client.js

Adding news item to proto

The news.proto will contain the RPC services and the data types the services are going to accept and return.

A single news item will have:

• id: contains the unique identifier of the news item.
• title: the title of the news
• body: the body of the news. This is the news this item is conveying.
• postImage: the header image of the news.
  

The above is how the news item will be sent and received in our service. We will represent it in a message format in protocol buffers.

syntax = "proto3";

message News {
    string id = 1;
    string title = 2;
    string body = 3;
    string postImage = 4;
}

The News defines our news request message format, where each news request has a unique id, the title of the news, the body of the news, and the image of the news. All of the fields are string types or texts.

Define the NewsService in proto

Now, we will define an RPC service. It will be a news service. This service will have the CRUD methods. The service name will be NewsService. We will define a NewsService RPC service in the proto file, the protocol buffer compiler will generate service interface code and stubs in our chosen language.

...
service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
}

message Empty {}

message NewsList {
   repeated News news = 1;
}

message is like a type in statically typed languages. The Empty denotes void or nothing, which means nothing is expected. The NewsList is the shape of the returned news list will take, it will have a news field which is of repeated type and each item in the list will be a News type. rpc sets the GetAllNews to be a method in the NewsService service. The rpc keyword is used to denote a method in a service in proto. So, the GetAllNews method requires an empty object as argument and it returns an array/list of news. The repeated denotes the variable is an Array or a List. So the news field is an Array of News.

The service is used to set an RPC in a proto buffer. To set another service e.g chat service it will be this:

service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
}

service ChatService {
    joinChat() returns () {}
}

Protobuf is quite easy to learn and use. So let's see the full code of our news.proto file:

syntax = "proto3";

message News {
    string id = 1;
    string title = 2;
    string body = 3;
    string postImage = 4;
}

service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
}

message Empty {}

message NewsList {
   repeated News news = 1;
}

Set up the server

We will create our server in the server.js. For now, we will use an in-memory array to store our news items.

const grpc = require("@grpc/grpc-js");
const PROTO_PATH = "./news.proto";
var protoLoader = require("@grpc/proto-loader");

const options = {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
};
var packageDefinition = protoLoader.loadSync(PROTO_PATH, options);
const newsProto = grpc.loadPackageDefinition(packageDefinition);

const server = new grpc.Server();
let news = [
  { id: "1", title: "Note 1", body: "Content 1", postImage: "Post image 1" },
  { id: "2", title: "Note 2", body: "Content 2", postImage: "Post image 2" },
];

server.addService(newsProto.NewsService.service, {
  getAllNews: (_, callback) => {
    callback(null, news);
  },
});

server.bindAsync(
  "127.0.0.1:50051",
  grpc.ServerCredentials.createInsecure(),
  (error, port) => {
    console.log("Server running at http://127.0.0.1:50051");
    server.start();
  }
);

First, we imported the @grpc/grpc-js and @grpc/proto-loader libraries. Also, we store the path of our news.proto file in PROTO_PATH const variable. Next, we define the options object for our proto file. Then, we load the proto file using the loadSync method, and the package definition gotten from it is loaded into the gRPC via the loadPackageDefinition method. Next, we set up our server instance by calling new grpc.Server().

Our in-memory array news database is defined in the news array. Next, we call the addService method in the gRPC Server instance. This method adds services to the server. Here, we have NewsService service, we can have as many services as we can e.g Analytics service, chat service, Auth service, etc all of them will be added to the gRPC server via this addService method. So here, we added our NewsService to the gRPC server. Services declared in the proto are referenced from the package definition loaded from the protoLoader, it becomes the first arg of the addService method, and the next arg is an object that contains the methods defined in the NewsService in the proto file. We have GetAllNews in the proto, so we defined getAllNews function in the object arg.

The function props args is the request object, and the second arg is the callback function, this callback is used to pass the results of the action back to the client. The first arg in the callback is the error and the second arg is the payload. See in the getAllNews function, we called the callback function passing in null as an error and the news array as the payload. Next, we called the bindAsync method in the server, passing in the port, the credentials, and a callback function. The URL and port are in 127.0.0.1 and 50051 respectively. In the callback, we start the server by calling the start method in the server instance.

So, let's start our server, run the command:

➜ node server
Server running at http://127.0.0.1:50051

Create our client stub

The client is where we will call the methods defined in the Server's addService method as the second param. We only have getAllNews, so let's see how we can call it. Open the client.js file and paste the code:

const grpc = require("@grpc/grpc-js");
var protoLoader = require("@grpc/proto-loader");
const PROTO_PATH = "./news.proto";

const options = {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
};

var packageDefinition = protoLoader.loadSync(PROTO_PATH, options);

const NewsService = grpc.loadPackageDefinition(packageDefinition).NewsService;

const client = new NewsService(
  "localhost:50051",
  grpc.credentials.createInsecure()
);

This is similar to what we have in our server.js but not so exactly. Here, we load the proto definitions compiled from the news.proto (Remember, this is the file that holds the interface or contract between the server and client communication). From the package definition, we reference the NewsService and store it in the NewsService variable. This enables us to call the procedures in the NewsService in the gRPC server. If we had other services this is where we can reference them if we want to call their procedures.

Next, the instance of NewsService is created using the new keyword and is stored in the client variable. So to use the procedures/methods in a service, the instance is first created using the new keyword, and from the instance, the methods can be called.

Get all news

Now, we call the getAllNews method in the server from the client variable.

...
const client = new NewsService(
  "localhost:50051",
  grpc.credentials.createInsecure()
);


client.getAllNews({}, (error, news) => {
  if (!error) throw error
    console.log(news);
});

See it, the getAllNews is called from the client. We passed an empty object literal in the first param, this will be the request body and it will be taken by the _ arg in the getAllNews in the server. The second param is the callback function that will be called in the server to pass back results to the client. See in the callback function we just log the result or the error if any. So now, run the client.js file:

➜ node client.js

{
  news: [
    {
      id: '1',
      title: 'Note 1',
      body: 'Content 1',
      postImage: 'Post image 1'
    },
    {
      id: '2',
      title: 'Note 2',
      body: 'Content 2',
      postImage: 'Post image 2'
    }
  ]
}

Voila!! :)

We have successfully executed a gRPC service method from a gRPC client. We can export the NewsService instance in client and we can import it in another file to call the methods.

const grpc = require("@grpc/grpc-js");
var protoLoader = require("@grpc/proto-loader");
const PROTO_PATH = "./news.proto";

const options = {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
};

var packageDefinition = protoLoader.loadSync(PROTO_PATH, options);

const NewsService = grpc.loadPackageDefinition(packageDefinition).NewsService;

const client = new NewsService(
  "localhost:50051",
  grpc.credentials.createInsecure()
);

module.exports = client;

Now, we can import the client in another file. We will create a file for each action we want to perform. We will create a get_news.js file, it will import the client and call the getAllNews method.

// get_news.js
const client = require("./client");

client.getAllNews({}, (error, news) => {
  if (!error) throw error;
  console.log(news);
});

Insert a news

We will add a method that inserts a news into the database. To add a new method to a service, we will have to go to the service in the proto file and add a rpc with the name of the method. So, in this case, we want our method name to be AddNews, so we go to the service NewsService and add it there.

...
service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
    rpc AddNews (News) returns (News) {}
}
...

It accepts a News message as a request and returns the new News object as a response. So we go over to the server.js file, there we will add addNews function to the second param object in the addService(...) call.

...
server.addService(newsProto.NewsService.service, {
  getAllNews: (_, callback) => {
    callback(null, news);
  },
  addNews: (call, callback) => {
    const _news = { id: Date.now(), ...call.request };
    news.push(_news);
    callback(null, _news);
  },
});
...

Voila! The client can now call the addNews method from a NewsService instance. The call arg will contain the arg passed during the call, the callback is a function that is called with the result so the client receives it as a response.

Delete a news

So the same way, we add a method for deleting a piece of news.

...
service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
    rpc DeleteNews (NewsId) returns (Empty) {}
    rpc AddNews (News) returns (News) {}
}

message NewsId {
    string id = 1;
}
...

The NewsId is a message object with the id as the id of the news. The NewsId is the request and it returns an Empty message as a response. Then, on the server.js:

...
  deleteNews: (_, callback) => {
    const newsId = _.request.id;
    news = news.filter(({ id }) => id !== newsId);
    callback(null, {});
  },
...

The news is filtered out of the news array.

Edit an existing news

To edit a piece of news, we add a method in the proto file:

...
service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
    rpc DeleteNews (NewsId) returns (Empty) {}
    rpc EditNews (News) returns (News) {}
    rpc AddNews (News) returns (News) {}
}
...

EditNews is our new rpc method exactly for that. It accepts a News message as a request and it returns the edited News object. Let's see the server.js:

...
  editNews: (_, callback) => {
    const newsId = _.request.id;
    const newsItem = news.find(({ id }) => newsId == id);
    newsItem.body = _.request.body;
    newsItem.postImage = _.request.postImage;
    newsItem.title = _.request.title;
    callback(null, newsItem);
  },
...

Here, the news item is retrieved from the news array using the id in the request object _. The body, title, and image are set to the ones in the _ request param. The callback function param callback is called with the edited news item.

Get a news

Let's set a method in the proto file:

...
service NewsService {
    rpc GetAllNews (Empty) returns (NewsList) {}
    rpc GetNews (NewsId) returns (News) {}
    rpc DeleteNews (NewsId) returns (Empty) {}
    rpc EditNews (News) returns (News) {}
    rpc AddNews (News) returns (News) {}
}
...

The GetNews method requires a NewsId as the request message and returns a News message. We implement it in the server.js file:

...
  getNews: (_, callback) => {
    const newsId = _.request.id;
    const newsItem = news.find(({ id }) => newsId == id);
    callback(null, newsItem);
  },
...

We get the id from the _ param object. The id is used to retrieve the corresponding news item from the news array. The callback function is called with the retrieved news item passed as param, this makes the client gets the news item.

Test

Now let's test our service procedures.

// test.js
// get all news
const client = require("./client");

client.getAllNews({}, (error, news) => {
  if (!error) throw error;
  console.log(news);
});

// add a news
client.addNews(
  {
    title: "Title news 3",
    body: "Body content 3",
    postImage: "Image URL here",
  },
  (error, news) => {
    if (error) throw error;
    console.log("Successfully created a news.");
  }
);

// edit a news
client.editNews(
  {
    id: 2,
    body: "Body content 2 edited.",
    postImage: "Image URL edited.",
    title: "Title for 2 edited.",
  },
  (error, news) => {
    if (error) throw error;
    console.log("Successfully edited a news.");
  }
);

// delete a news
client.deleteNews(
  {
    id: 2,
  },
  (error, news) => {
    if (error) throw error;
    console.log("Successfully deleted a news item.");
  }
);

Now, run the file:

node test

Use with Http server

We now have a server, proto, and client all built and ready. We can attach a Noder server to the client.js so an endpoint in our server will call the procedure in the gRPC news service. In doing so we are exposing our gRPC news service to be run via an HTTP request over the internet.

We will build a Node server and create endpoints to call the gRPC service procedures.

• /news GET endpoint will call the getAllNews sub-routine to get all the news in the database.
• /news/:id GET endpoint will call the getNews sub-routine to get a news item.
• /news POST endpoint will call the addNews sub-routine to create a new news item.
• /news/:id PUT will call the editNews sub-routine to edit/update a news item.
• /news/:id DELETE will call the deleteNews sub-routine to delete a news item.
  

The code in Nodejs will be this:

const http = require("http");
const client = require("./client");

const host = "localhost";
const port = 8000;

const requestListener = function (req, res) {
  const url = req.url.split("/");
  const method = req.method;

  switch (method) {
    case "GET":
      if (url[0] == "news") {
        if (url.length > 1 && url[1]) {
          client.getNews(
            {
              id: url[1],
            },
            (error, news) => {
              if (!error) throw error;
              res.end(news);
            }
          );
        }
        client.getAllNews({}, (error, news) => {
          if (!error) throw error;
          res.end(news);
        });
        res.end();
      }
      break;
    case "PUT":
      client.editNews(
        {
          id: url[1],
          body: req.body.body,
          postImage: req.body.postImage,
          title: req.body.title,
        },
        (error, news) => {
          if (error) throw error;
          res.end(news);
        }
      );

      break;
    case "DELETE":
      client.deleteNews(
        {
          id: url[1],
        },
        (error, news) => {
          if (error) throw error;
          res.end({ msg: "Successfully deleted a news item." });
        }
      );

      break;
    case "POST":
      client.addNews(
        {
          body: req.body.body,
          postImage: req.body.postImage,
          title: req.body.title,
        },
        (error, news) => {
          if (error) throw error;
          res.end({ data: news, msg: "Successfully created a news." });
        }
      );
      break;
    default:
      res.end("");
      break;
  }
};

const server = http.createServer(requestListener);
server.listen(port, host, () => {
  console.log(`Server is running on http://${host}:${port}`);
});

We use Nodejs http library to create a server. Next, we imported and the client.js file and stores the NewsService instance it exports in the client variable. The requestListener is where the main code lies. We listen for URL, params, and methods and from there we execute the sub-routines in the client accordingly.

In the "GET" condition, we call the getNews method in the client when there is an id param in the URL. We retrieve the id value from the URL and call the getNews method with the id as param. This will return us a news item and we send it back as a response via the res.end call.

If the "GET" condition is just "news", the getAllNews in the client is called to get all the news in the service. The result is sent back as a response to the caller.

The /news "PUT" calls the editNews with the id param and the body payload passed as param to the function.

The /news "DELETE" calls the deleteNews passing in the id as param.

The /news "POST" calls the addNews with the payload in the body of the request as param in the method. The newly created news item is returned and sent as a response via res.end(...).

Last, we create our server passing in the requestListener as param, then, the server is started via the listen(...) method.

Now, start the server:

node nodeServer

Source code

The full source code of the example in this tutorial can be found here.

Conclusion

gRPC is awesome!

We covered a lot in this tutorial. We started by introducing gRPC to elaborating on how it works. We explained the protocol buffer and how gRPC uses it to create a contract/interface to which its server and client must agree. Next, we demonstrated how to create a gRPC service in Nodejs. We installed dependencies and then proceeded to create the server and client stubs. Last, we tested the methods in the gRPC service we created and also showed how to create an API in Nodjes that connects to the methods in the gRPC service. There is a lot more you can do with gRPC, read its docs to find out more.

gRPC resources

• gRPC docs
• Creating a CRUD API with Node, Express, and gRPC
• Building gRPC Service Server Note CRUD API with node.js