Aller au contenu principal
How systems communicate — TCP/IP, HTTP, DNS, load balancing, and security.

Networking

How systems communicate — TCP/IP, HTTP, DNS, load balancing, and security.

gRPC, Protobuf & RPC Internals

A Remote Procedure Call (RPC) makes a function call on a remote machine look like a function call on your local one. The idea is older than the web. gRPC is the production-grade modernisation: an Interface Definition Language (Protobuf) defines the contract, HTTP/2 carries the bytes, and a code generator emits typed clients and servers in eleven languages. The result is that two services in different ecosystems call each other with compiler-checked signatures instead of hand-rolled JSON clients drifting apart silently.

This topic covers what gRPC actually does on the wire, and when it should replace REST.

Why an IDL Beats Hand-Rolled Schemas

The original sin of REST-as-ad-hoc is that every client re-implements the schema. Field names drift between server and client documentation, types are guessed from examples, optional fields live in folklore. An IDL puts the schema at the centre:

service CartService {
  rpc AddItem (AddItemRequest) returns (AddItemResponse) {}
}

message AddItemRequest {
  string cart_id = 1;
  string sku      = 2;
  int32  quantity = 3;
}
message AddItemResponse {
  string cart_id = 1;
  int32 item_count = 2;
}

From this .proto, protoc emits client and server stubs in Go, Java, Python, TS, Rust, … Whoever changes the schema re-runs protoc; every consumer fails to compile until they update. The schema is the contract. No folklore.

Protobuf on the Wire

Protocol Buffers is a binary encoding. Properties:

  • Compact — typed field numbers (not field names) on the wire; quantity = 3 takes 3 bytes at the binary level, not the 30 bytes of "quantity": 3.
  • Forward / backward compatible — new optional fields can be added without breaking old readers (they ignore unknown tag numbers). Field numbers are the stable identity; field names are not.
  • Schema-bound — there is no “empty value for everything” trick; optional fields stay distinguishable from absent fields.

The cost: not human-readable. Debugging requires tooling (protoc --decode_raw, Buf, grpcurl). The binary compactness vs the JSON readability is the central trade-off of gRPC vs REST.

gRPC over HTTP/2 Mechanics

gRPC runs on HTTP/2 — a streaming, multiplexed protocol that solves the head-of-line blocking problem of HTTP/1.1.

HTTP/2 propertyWhat it gives gRPC
Multiplexed streamsMany concurrent RPCs on one TCP connection; no per-request connection cost
Binary framingSmaller on the wire; gRPC adds its own length-prefixed frames for streaming payloads
Server pushNot used by gRPC, but the underlying HPACK header compression shrinks every repeated metadata header
Flow control per streamBackpressure per RPC instead of per-connection

A gRPC message is sent as HEADERS frame (with :path, :authority, method, custom metadata), then one or more DATA frames carrying the length-prefixed protobuf bytes, terminated by a HEADERS frame carrying the trailing status. This pattern is gRPC’s six-message-prefix format — a one-time read cost; the framing machinery lives in the gRPC library, not the application.

Unary, Server, Client, Bidi

gRPC’s strength over REST is its streaming RPCs, leveraged directly from HTTP/2 streams:

RPC typeCallerServerUse
UnaryOne requestOne responseThe REST replacement: GET /cart/{id} → GetCart(req)
Server streamingOne requestMany responsesA watch subscription, a result stream, a chunked download
Client streamingMany requestsOne responseUpload a batch, collect an aggregate acknowledgement
Bidi streamingMany requests, interleavedMany responses, interleavedChat, collaborative editing, telemetry back-and-forth

Streaming RPCs are stateful connections — the client and server share one HTTP/2 stream and exchange frames in either direction. This is closer to a TCP socket than to a REST request, and it is the unique capability REST cannot replicate cleanly.

Deadlines, Cancellation, Metadata

Two production-critical features:

  • Deadlines are first-class. A caller passes context.WithTimeout(ctx, 500 * time.Millisecond). The deadline propagates across every downstream RPC; the entire call tree aborts at the deadline, not at individual-hop timeouts. This implements the deadline-propagation pattern from backpressure.
  • Cancellation is causal. A cancelled context at the root cancels every in-flight downstream. There is no orphan work; the savings potential of gRPC cancellation over REST’s ungraceful client.close() is the single biggest influence on tail latency at scale.

Metadata is the gRPC equivalent of HTTP headers: arbitrary key-value pairs sent alongside the RPC. Authentication tokens, tracing headers (traceparent), request IDs, custom context — all metadata. The standard practice is propagate the trace headers at the gateway so a single trace spans the entire call tree.

REST vs gRPC vs GraphQL

The most-asked architectural question of the last decade. The honest answer is they are not exclusive — most platforms run all three.

PropertyRESTgRPCGraphQL
SchemaHand-rolled or OpenAPIProtobuf IDL (compiler-checked)GraphQL SDL (compiler-checked)
Wire formatJSON (human readable)Protobuf (binary, compact)JSON over HTTP (human readable)
TransportHTTP/1.1 commonHTTP/2 onlyHTTP/1.1 or HTTP/2
StreamingServer-Sent Events hacksFirst-class streaming RPCsSubscriptions via websockets
Browser supportExcellent via fetchgrpc-web proxy neededExcellent via Apollo, Relay
Best use casePublic APIs, partner integrationsInternal service-to-serviceClient-driven aggregates (mobile, dashboards)

The platform shape that emerges in practice — Google, Netflix, Slack, Stripe — uses gRPC internally (typed, performance-tight, deadline-aware), REST externally (ubiquitous, browser-native, debuggable with curl), and GraphQL at the orchestration edge (mobile clients fetching shaped trees, BFF aggregating internal gRPC services). The three are different layers, not competitors.

Practice Trajectory

  1. Write a 4-method Protobuf IDL for a tiny LibraryService (AddBook, ListBooks, WatchNewBooks, RemoveBook). Generate stubs in two languages; call one from the other.
  2. Implement unary Get and server-streaming Watch. Compare the developer ergonomics — what became different about handling streaming in your codebase?
  3. Replace a REST service-to-service endpoint with gRPC. Compare message size on the wire (protobuf vs JSON), tail latency, and the bandwidth of cancellation propagation in your tracing bind.
  4. In a service that uses REST today and suffers from client schema drift, prototype the IDL equivalent in either gRPC or GraphQL. Note what the schema migration costs and what it cracks.
  5. Run a load test bench comparing REST vs gRPC for the same operation (GetCart). At 1000 QPS, what changes — message size, latency, CPU?

When It’s the Right Tool

SituationTakeaway
Internal service-to-service callsgRPC — typed contracts, deadline propagation, lower wire cost
Public API for third-party consumersREST — ubiquitous, debuggable, browser-native
Mobile/dashboard aggregations over many microservicesGraphQL at the BFF layer; gRPC behind it
High-volume server-pushed telemetry or watchgRPC server-streaming as the natural fit
Bulk import jobs that care about client-streaming bandwidthgRPC client-streaming or HTTP/2 chunked uploads