--- title: gRPC client-side load balancing author: jamesnk description: Learn how to make scalable, high-performance gRPC apps with client-side load balancing in .NET. monikerRange: '>= aspnetcore-3.0' ms.author: wpickett ms.date: 05/11/2023 uid: grpc/loadbalancing --- # gRPC client-side load balancing [!INCLUDE[](~/includes/not-latest-version.md)] By [James Newton-King](https://twitter.com/jamesnk) Client-side load balancing is a feature that allows gRPC clients to distribute load optimally across available servers. This article discusses how to configure client-side load balancing to create scalable, high-performance gRPC apps in .NET. Client-side load balancing requires: * .NET 5 or later. * [`Grpc.Net.Client`](https://www.nuget.org/packages/Grpc.Net.Client) version 2.45.0 or later. ## Configure gRPC client-side load balancing Client-side load balancing is configured when a channel is created. The two components to consider when using load balancing: * The resolver, which resolves the addresses for the channel. Resolvers support getting addresses from an external source. This is also known as service discovery. * The load balancer, which creates connections and picks the address that a gRPC call will use. Built-in implementations of resolvers and load balancers are included in [`Grpc.Net.Client`](https://www.nuget.org/packages/Grpc.Net.Client). Load balancing can also be extended by [writing custom resolvers and load balancers](#write-custom-resolvers-and-load-balancers). Addresses, connections and other load balancing state is stored in a `GrpcChannel` instance. A channel must be reused when making gRPC calls for load balancing to work correctly. > [!NOTE] > Some load balancing configuration uses dependency injection (DI). Apps that don't use DI can create a instance. > > If an app already has DI setup, like an ASP.NET Core website, then types should be registered with the existing DI instance. `GrpcChannelOptions.ServiceProvider` is configured by getting an from DI. ## Configure resolver The resolver is configured using the address a channel is created with. The [URI scheme](https://wikipedia.org/wiki/Uniform_Resource_Identifier#Syntax) of the address specifies the resolver. | Scheme | Type | Description | | -------- | ----------------------- | ----------- | | `dns` | `DnsResolverFactory` | Resolves addresses by querying the hostname for [DNS address records](https://wikipedia.org/wiki/List_of_DNS_record_types#A). | | `static` | `StaticResolverFactory` | Resolves addresses that the app has specified. Recommended if an app already knows the addresses it calls. | A channel doesn't directly call a URI that matches a resolver. Instead, a matching resolver is created and used to resolve the addresses. For example, using `GrpcChannel.ForAddress("dns:///my-example-host", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure })`: * The `dns` scheme maps to `DnsResolverFactory`. A new instance of a DNS resolver is created for the channel. * The resolver makes a DNS query for `my-example-host` and gets two results: `127.0.0.100` and `127.0.0.101`. * The load balancer uses `127.0.0.100:80` and `127.0.0.101:80` to create connections and make gRPC calls. #### DnsResolverFactory The `DnsResolverFactory` creates a resolver designed to get addresses from an external source. DNS resolution is commonly used to load balance over pod instances that have a [Kubernetes headless services](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services). ```csharp var channel = GrpcChannel.ForAddress( "dns:///my-example-host", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure }); var client = new Greet.GreeterClient(channel); var response = await client.SayHelloAsync(new HelloRequest { Name = "world" }); ``` The preceding code: * Configures the created channel with the address `dns:///my-example-host`. * The `dns` scheme maps to `DnsResolverFactory`. * `my-example-host` is the hostname to resolve. * No port is specified in the address, so gRPC calls are sent to port 80. This is the default port for unsecured channels. A port can optionally be specified after the hostname. For example, `dns:///my-example-host:8080` configures gRPC calls to be sent to port 8080. * Doesn't specify a load balancer. The channel defaults to a pick first load balancer. * Starts the gRPC call `SayHello`: * DNS resolver gets addresses for the hostname `my-example-host`. * Pick first load balancer attempts to connect to one of the resolved addresses. * The call is sent to the first address the channel successfully connects to. ##### DNS address caching Performance is important when load balancing. The latency of resolving addresses is eliminated from gRPC calls by caching the addresses. A resolver will be invoked when making the first gRPC call, and subsequent calls use the cache. Addresses are automatically refreshed if a connection is interrupted. Refreshing is important in scenarios where addresses change at runtime. For example, in Kubernetes a [restarted pod](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/) triggers the DNS resolver to refresh and get the pod's new address. By default, a DNS resolver is refreshed if a connection is interrupted. The DNS resolver can also optionally refresh itself on a periodic interval. This can be useful for quickly detecting new pod instances. ```csharp services.AddSingleton( sp => new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(30))); ``` The preceding code creates a `DnsResolverFactory` with a refresh interval and registers it with dependency injection. For more information on using a custom-configured resolver, see [Configure custom resolvers and load balancers](#configure-custom-resolvers-and-load-balancers). #### StaticResolverFactory A static resolver is provided by `StaticResolverFactory`. This resolver: * Doesn't call an external source. Instead, the client app configures the addresses. * Is designed for situations where an app already knows the addresses it calls. ```csharp var factory = new StaticResolverFactory(addr => new[] { new BalancerAddress("localhost", 80), new BalancerAddress("localhost", 81) }); var services = new ServiceCollection(); services.AddSingleton(factory); var channel = GrpcChannel.ForAddress( "static:///my-example-host", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure, ServiceProvider = services.BuildServiceProvider() }); var client = new Greet.GreeterClient(channel); ``` The preceding code: * Creates a `StaticResolverFactory`. This factory knows about two addresses: `localhost:80` and `localhost:81`. * Registers the factory with dependency injection (DI). * Configures the created channel with: * The address `static:///my-example-host`. The `static` scheme maps to a static resolver. * Sets `GrpcChannelOptions.ServiceProvider` with the DI service provider. This example creates a new for DI. Suppose an app already has DI setup, like an ASP.NET Core website. In that case, types should be registered with the existing DI instance. `GrpcChannelOptions.ServiceProvider` is configured by getting an from DI. ## Configure load balancer A load balancer is specified in a [`service config`](https://github.com/grpc/grpc/blob/master/doc/service_config.md) using the `ServiceConfig.LoadBalancingConfigs` collection. Two load balancers are built-in and map to load balancer config names: | Name | Type | Description | | ------------- | ------------------------------- | ----------- | | `pick_first` | `PickFirstLoadBalancerFactory` | Attempts to connect to addresses until a connection is successfully made. gRPC calls are all made to the first successful connection. | | `round_robin` | `RoundRobinLoadBalancerFactory` | Attempts to connect to all addresses. gRPC calls are distributed across all successful connections using [round-robin](https://www.nginx.com/resources/glossary/round-robin-load-balancing/) logic. | `service config` is an abbreviation of service configuration and is represented by the `ServiceConfig` type. There are a couple of ways a channel can get a `service config` with a load balancer configured: * An app can specify a `service config` when a channel is created using `GrpcChannelOptions.ServiceConfig`. * Alternatively, a resolver can resolve a `service config` for a channel. This feature allows an external source to specify how its callers should perform load balancing. Whether a resolver supports resolving a `service config` is dependent on the resolver implementation. Disable this feature with `GrpcChannelOptions.DisableResolverServiceConfig`. * If no `service config` is provided, or the `service config` doesn't have a load balancer configured, the channel defaults to `PickFirstLoadBalancerFactory`. ```csharp var channel = GrpcChannel.ForAddress( "dns:///my-example-host", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure, ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() } } }); var client = new Greet.GreeterClient(channel); var response = await client.SayHelloAsync(new HelloRequest { Name = "world" }); ``` The preceding code: * Specifies a `RoundRobinLoadBalancerFactory` in the `service config`. * Starts the gRPC call `SayHello`: * `DnsResolverFactory` creates a resolver that gets addresses for the hostname `my-example-host`. * Round-robin load balancer attempts to connect to all resolved addresses. * gRPC calls are distributed evenly using round-robin logic. ## Configure channel credentials A channel must know whether gRPC calls are sent using [transport security](xref:grpc/security#transport-security). `http` and `https` are no longer part of the address, the scheme now specifies a resolver, so `Credentials` must be configured on channel options when using load balancing. * `ChannelCredentials.SecureSsl` - gRPC calls are secured with [Transport Layer Security (TLS)](https://tools.ietf.org/html/rfc5246). Equivalent to an `https` address. * `ChannelCredentials.Insecure` - gRPC calls don't use transport security. Equivalent to an `http` address. ```csharp var channel = GrpcChannel.ForAddress( "dns:///my-example-host", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure }); var client = new Greet.GreeterClient(channel); var response = await client.SayHelloAsync(new HelloRequest { Name = "world" }); ``` ## Use load balancing with gRPC client factory [gRPC client factory](xref:grpc/clientfactory) can be configured to use load balancing: ```csharp var builder = WebApplication.CreateBuilder(args); builder.Services .AddGrpcClient(o => { o.Address = new Uri("dns:///my-example-host"); }) .ConfigureChannel(o => o.Credentials = ChannelCredentials.Insecure); builder.Services.AddSingleton( sp => new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(30))); var app = builder.Build(); ``` The preceding code: * Configures the client with a load-balancing address. * Specifies channel credentials. * Registers DI types with the app's . ## Write custom resolvers and load balancers Client-side load balancing is extensible: * Implement `Resolver` to create a custom resolver and resolve addresses from a new data source. * Implement `LoadBalancer` to create a custom load balancer with new load balancing behavior. > [!IMPORTANT] > The APIs used to extend client-side load balancing are experimental. They can change without notice. ### Create a custom resolver A resolver: * Implements `Resolver` and is created by a `ResolverFactory`. Create a custom resolver by implementing these types. * Is responsible for resolving the addresses a load balancer uses. * Can optionally provide a service configuration. ```csharp public class FileResolver : PollingResolver { private readonly Uri _address; private readonly int _port; public FileResolver(Uri address, int defaultPort, ILoggerFactory loggerFactory) : base(loggerFactory) { _address = address; _port = defaultPort; } public override async Task ResolveAsync(CancellationToken cancellationToken) { // Load JSON from a file on disk and deserialize into endpoints. var jsonString = await File.ReadAllTextAsync(_address.LocalPath); var results = JsonSerializer.Deserialize(jsonString); var addresses = results.Select(r => new BalancerAddress(r, _port)).ToArray(); // Pass the results back to the channel. Listener(ResolverResult.ForResult(addresses)); } } public class FileResolverFactory : ResolverFactory { // Create a FileResolver when the URI has a 'file' scheme. public override string Name => "file"; public override Resolver Create(ResolverOptions options) { return new FileResolver(options.Address, options.DefaultPort, options.LoggerFactory); } } ``` In the preceding code: * `FileResolverFactory` implements `ResolverFactory`. It maps to the `file` scheme and creates `FileResolver` instances. * `FileResolver` implements `PollingResolver`. `PollingResolver` is an abstract base type that makes it easy to implement a resolver with asynchronous logic by overriding `ResolveAsync`. * In `ResolveAsync`: * The file URI is converted to a local path. For example, `file:///c:/addresses.json` becomes `c:\addresses.json`. * JSON is loaded from disk and converted into a collection of addresses. * Listener is called with results to let the channel know that addresses are available. ### Create a custom load balancer A load balancer: * Implements `LoadBalancer` and is created by a `LoadBalancerFactory`. Create a custom load balancer and factory by implementing these types. * Is given addresses from a resolver and creates `Subchannel` instances. * Tracks state about the connection and creates a `SubchannelPicker`. The channel internally uses the picker to pick addresses when making gRPC calls. The `SubchannelsLoadBalancer` is: * An abstract base class that implements `LoadBalancer`. * Manages creating `Subchannel` instances from addresses. * Makes it easy to implement a custom picking policy over a collection of subchannels. ```csharp public class RandomBalancer : SubchannelsLoadBalancer { public RandomBalancer(IChannelControlHelper controller, ILoggerFactory loggerFactory) : base(controller, loggerFactory) { } protected override SubchannelPicker CreatePicker(List readySubchannels) { return new RandomPicker(readySubchannels); } private class RandomPicker : SubchannelPicker { private readonly List _subchannels; public RandomPicker(List subchannels) { _subchannels = subchannels; } public override PickResult Pick(PickContext context) { // Pick a random subchannel. return PickResult.ForSubchannel(_subchannels[Random.Shared.Next(0, _subchannels.Count)]); } } } public class RandomBalancerFactory : LoadBalancerFactory { // Create a RandomBalancer when the name is 'random'. public override string Name => "random"; public override LoadBalancer Create(LoadBalancerOptions options) { return new RandomBalancer(options.Controller, options.LoggerFactory); } } ``` In the preceding code: * `RandomBalancerFactory` implements `LoadBalancerFactory`. It maps to the `random` policy name and creates `RandomBalancer` instances. * `RandomBalancer` implements `SubchannelsLoadBalancer`. It creates a `RandomPicker` that randomly picks a subchannel. ## Configure custom resolvers and load balancers Custom resolvers and load balancers need to be registered with dependency injection (DI) when they are used. There are a couple of options: * If an app is already using DI, such as an ASP.NET Core web app, they can be registered with the existing DI configuration. An can be resolved from DI and passed to the channel using `GrpcChannelOptions.ServiceProvider`. * If an app isn't using DI then create: * A with types registered with it. * A service provider using . ```csharp var services = new ServiceCollection(); services.AddSingleton(); services.AddSingleton(); var channel = GrpcChannel.ForAddress( "file:///c:/data/addresses.json", new GrpcChannelOptions { Credentials = ChannelCredentials.Insecure, ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new LoadBalancingConfig("random") } }, ServiceProvider = services.BuildServiceProvider() }); var client = new Greet.GreeterClient(channel); ``` The preceding code: * Creates a `ServiceCollection` and registers new resolver and load balancer implementations. * Creates a channel configured to use the new implementations: * `ServiceCollection` is built into an `IServiceProvider` and set to `GrpcChannelOptions.ServiceProvider`. * Channel address is `file:///c:/data/addresses.json`. The `file` scheme maps to `FileResolverFactory`. * `service config` load balancer name is `random`. Maps to `RandomLoadBalancerFactory`. ## Why load balancing is important HTTP/2 multiplexes multiple calls on a single TCP connection. If gRPC and HTTP/2 are used with a network load balancer (NLB), the connection is forwarded to a server, and all gRPC calls are sent to that one server. The other server instances on the NLB are idle. Network load balancers are a common solution for load balancing because they are fast and lightweight. For example, Kubernetes by default uses a network load balancer to balance connections between pod instances. However, network load balancers are not effective at distributing load when used with gRPC and HTTP/2. ### Proxy or client-side load balancing? gRPC and HTTP/2 can be effectively load balanced using either an application load balancer proxy or client-side load balancing. Both of these options allow individual gRPC calls to be distributed across available servers. Deciding between proxy and client-side load balancing is an architectural choice. There are pros and cons for each. * **Proxy**: gRPC calls are sent to the proxy, the proxy makes a load balancing decision, and the gRPC call is sent on to the final endpoint. The proxy is responsible for knowing about endpoints. Using a proxy adds: * An additional network hop to gRPC calls. * Latency and consumes additional resources. * Proxy server must be setup and configured correctly. * **Client-side load balancing**: The gRPC client makes a load balancing decision when a gRPC call is started. The gRPC call is sent directly to the final endpoint. When using client-side load balancing: * The client is responsible for knowing about available endpoints and making load balancing decisions. * Additional client configuration is required. * High-performance, load balanced gRPC calls eliminate the need for a proxy. ## Additional resources *