gRPC performance benchmark in ASP.NET Core 3

Sun, 17 Nov 2019 00:00:00 +0000

In the previous post, I have explored the new gRPC service template from Visual Studio.

However, I haven’t done any check of the performance of the gRPC, which I think is an interesting thing to look at. The performance benefit is a “selling point” of gRPC, so it will be good to verify how big the gain is.

Results

I’ll start with the results as they are probably the most interesting thing.

For a quick overview, see the charts below.

For details, click here to view raw data

API Type	Message	Time [ms]
gRPC	small-ints	2505
gRPC-CodeFirst	small-ints	3080
System.Text.Json	small-ints	3589
Newtonsoft.Json	small-ints	3488
gRPC	small-mixed	1982
gRPC-CodeFirst	small-mixed	2770
System.Text.Json	small-mixed	3825
Newtonsoft.Json	small-mixed	3625
gRPC	big-list	2813
gRPC-CodeFirst	big-list	4678
System.Text.Json	big-list	5338
Newtonsoft.Json	big-list	38329

API types

As you see I’ve decided to compare 4 variants:

gRPC - official suite of gRPC libraries that are associated with new Visual Studio template
gRPC-CodeFirst - protobuf-net.Grpc library, which works also for .NET Framework
System.Text.Json - the new JSON serialization API from .NET Core 3
Newtonsoft.Json - probably doesn’t need introduction :)

Why these 4?

Firstly, I wanted to see how gRPC performs against API based on JSON. RPC over HTTP with JSON serialization is the most prevalent thing these days.

Secondly, I also wanted to look at different implementations of both gRPC & JSON serialization, mostly out of curiosity.

Test messages

Test messages differ by size and data types used within them:

small ints - a class with a few int properties
small-mixed - a class with a few int/string properties and a nested class
big-list - a big list of items (1000 elements, 190kB JSON file)

Test procedure

In the test, I’ve done the following:

run the API methods sequentially 1000 times on localhost, on my machine
all APIs were using HTTPS
all APIs were running on ASP.NET Core 3
all gRPC calls were unary

That’s not rigid science, but gives something roughly comparable.

The charts report total runtime of all calls.

You can review benchmark code on my GitHub repo

Conclusions

gRPC beats JSON by quite a big margin - from 30% to 50%.

2 different gRPC flavours perform pretty close to each other. Official gRPC libraries seems to be faster.

Newtonsoft.Json seems to be having problems with the serialization of big messages, which are not present in any other tested API. Or perhaps it’s System.Text.Json that’s optimised close to the metal with all new memory-efficient framework constructs.

So, if you send large JSONs, then migrate to System.Text.Json to get something like a 6x speedup of serialization.

If you fight for milliseconds in your APIs, then switch from JSON to gRPC is worth consideration.

Trying out gRPC in ASP.NET Core 3

Fri, 18 Oct 2019 00:00:00 +0000

gRPC has received first class support in ASP.NET Core 3 from Microsoft. It’s a binary RPC protocol based on Protocol Buffers created by Google that works on top of HTTP/2. It offers good performance and integration with many programming languages. Let’s try out together this new feature by looking at code generated by Visual Studio gRPC project template.

You can find code described in this post on my GitHub repo.

Basic project structure

After we create project from the gRPC service template, we have following files structure:

│   appsettings.Development.json
│   appsettings.json
│   Program.cs
│   Startup.cs
│   TryingOut.gRPC.Service.csproj
│
├───Properties
│       launchSettings.json
│
├───Protos
│       greet.proto
│
└───Services
        GreeterService.cs

Proto file

gRPC is based on proto files, so let’s start by looking at greet.proto:

First thing is syntax version declaration - proto3 is the latest one
csharp_namespace option says that all generated classes will be placed in TryingOut.gRPC.Service namespace
package specifier is equivalent of C# namespace concept in protocol buffers
The rest of file defines single method of a service, its input and output types

It’s worth noting that fields in messages have to be ordered explicitly. The order should not be changed after publishing the service otherwise we ask for backward compatibility issues.

All in all, I’d say that file looks nice and clean, there’s little boilerplate and RPC contract is outlined in a concise way.

syntax = "proto3";

option csharp_namespace = "TryingOut.gRPC.Service";

package Greet;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply);
}

// The request message containing the user's name
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings.
message HelloReply {
  string message = 1;
}

A glance at .csproj file

Let’s go to the .csproj file. Here we have Protobuf item with GrpcServices="Server" property which instructs msbuild task to generate server-side code of gRPC service.

  <ItemGroup>
    <Protobuf Include="Protos\greet.proto" GrpcServices="Server" />
  </ItemGroup>

Generated code

After build we can peek at the generated code, which is in obj directory.

obj\Debug\netcoreapp3.0\Greet.cs
obj\Debug\netcoreapp3.0\GreetGrpc.cs

Service class

We consume generated code by inheriting from base service class. We just have to put our logic into overrides.

The base class doesn’t offer much methods or properties to use. Whole info about context of the request is passed through ServerCallContext parameter.

    public class GreeterService : Greeter.GreeterBase
    {
        // ...

        public override Task<HelloReply> SayHello(HelloRequest request, ServerCallContext context)
        {
            return Task.FromResult(new HelloReply
            {
                Message = "Hello " + request.Name
            });
        }
    }

Configuration in Startup

To make this work we have to configure gRPC in Startup class by calling AddGrpc and mapping all gRPC services. We have to do that by using Endpoint Routing, which got a lot of attention in the release of ASP.NET Core 3. One of the goals of Endpoint Routing is to allow seamless integration of multiple endpoint handlers in single application - we can freely mix MVC controllers and gRPC services in same project.

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddGrpc();
        }
        
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            // ...

            app.UseRouting();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapGrpcService<GreeterService>();

                // ...
            });
        }
        

Creating a client

Let’s make a little console app that will consume the service.

The first step to accomplish that is adding necessary nuget packages:

Grpc.Tools - MSBuild tasks which generate client code
Google.Protobuf - protocol buffers implementation
Grpc.Net.Client - gRPC client

  <ItemGroup>
    <PackageReference Include="Google.Protobuf" Version="3.10.0" />
    <PackageReference Include="Grpc.Net.Client" Version="2.23.2" />
    <PackageReference Include="Grpc.Tools" Version="2.24.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>

The next thing which we have to do is linking .proto file from server and defining GrpcServices=Client property to configure client code generation. I guess the better way to handle that in the long run would be defining proto files in some shared project or directory, so that we emphasize through project structure that those files are something meant to be reused.

  <ItemGroup>
    <Protobuf Include="..\TryingOut.gRPC.Service\Protos\greet.proto" Link="greet.proto" GrpcServices="Client" />
  </ItemGroup>

After those preliminary steps we can use generated code to communicate with the server.

    static async Task Main()
    {
        var channel = Grpc.Net.Client.GrpcChannel.ForAddress("https://localhost:5001");
        var client = new Greeter.GreeterClient(channel);
        var response = await client.SayHelloAsync(new HelloRequest
        {
            Name = "Paweł"
        });
        Console.WriteLine($"Response from server: {response.Message}");
    }

gRPC doesn’t work with IIS (yet)

As of today, IIS doesn’t support gRPC due to the issue with HTTP/2 trailing headers implementation in the http.sys Windows driver. The problem affects also Azure App Service as it uses IIS under the hood.

As David Fowler says it’s not going to be a quick fix, because it involves Windows update process. See the issue on GitHub AspNetCore repository for all details.

I’d say it’s one more instance of the problems stemming from Microsoft move away from single integrated ecosystem approach. It seems like the question “Will this work on IIS / Windows / Azure?” is becoming more and more relevant with the new features of the ASP.NET Core.

gRPC for .NET Framework

If you would like to use gRPC on .NET Framework 4.X, then you should take a look at protobuf-net.Grpc, which allows to do that. It also uses code-first approach, no need to write .proto files!

Closing thoughts

In a layered view of an application gRPC services sits in a similar place as controllers in the Api projects. However, HTTP is used purely as a transport protocol and nothing like RESTful principles applies here. This simplification takes away some design options, but I’d say it makes sense for RPC approach and can make development faster. We don’t even have to think of resource names for our API methods or their HTTP verbs. We just need to define proto files and consume generated code.

By choosing gRPC we sacrifice interoperability and plain-text advantages for sake of performance. However, good tooling which comes with gRPC reduces a lot of the usual friction related to working with binary protocols. I guess gRPC will find its place in .NET space mostly in internal APIs which require minimal serialization overhead.

pawelkmiec.net