tRPC – What is tRPC?

Docs: Introduction

Mon, 01 Jan 0001 00:00:00 +0000

What is tRPC

tRPC is a high-performance RPC framework that designed based on the concept of pluggable，the overall design follows the following principles:

Simple: Users develop service very simply based on the framework.
High-Performance: The performant of the framework can be applicable to massive scenario on the internet.
Pluggable: The framework is layered and modularized in architectural design and implementation, and each core module be pluggable and evolves independently.

By using tRPC, you can:

Build services (tRPC/HTTP(s)/gRPC, etc.) with multiple ports that support multiple protocols (one port only support one protocol), and can handle client requests synchronously/asynchronously.
Access various protocol backend services (tRPC/HTTP(s)/gRPC, etc.) in a synchronous, asynchronous, and one-way.
Streaming RPC programming, currently supports tRPC streaming, gRPC streaming, HTTP streaming, etc., to implement streaming application services such as Push, File Upload/Download, and AI Serving.
Support various protocols and service governance systems, such as: customized protocols, various name-service/metrics systems/tracing systems/config-center systems/log system, etc., to facilitate service interoperability and operation.

Features

Works across languages：Implements cross-language service communication based on Protocol Buffers.
Support multi-protocols：Supports multiple protocols and interoperates with different frameworks (such as gRPC).
Streaming RPC: Supports streaming RPC, which is better suited for various application scenarios such as large file upload/download, message push, AI speech recognition/video understanding, etc.
Rich plugin ecosystem: Provides a large number of plugins that docking to microservice components (such as Consul/Promethues/OpenTelemetry, etc.) to facilitate users to build their own service governance system.
Scalability: Based on the pluggable design of the framework, users can develop secondary to expand the framework capabilities, such as: parameter validation, authentication, log replay, etc.
Flow & Overload protection: provides flow control and overload protection plugins in a variety of application scenarios to prevent services from being overloaded and unavailable due to burst traffic.

Supported languages

To start using tRPC

Per-language quickstart guides and tutorials can be found in the tRPC website . Code examples are available in the examples directory.

To start developing tRPC

Contributions are welcome!

Please read How to contribute which will guide you through the entire workflow of how to build the source code, how to run the tests, and how to contribute changes to the tRPC codebase.

Docs: Archtecture Design

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This article introduces the background of tRPC, pluggable design, and overall architecture.

Background

In the early days of the development of the Internet, business scenarios varied greatly and iterations were rapid. This results in more or less differences in the language technology stacks, development frameworks, application protocols, service governance systems, operation and maintenance platforms, etc. used by their backend services.

At the same time, with the development of cloud native technology, businesses are increasingly using open source technology and cloud components. Embracing cloud native has become a mainstream trend.

The above situation also exists within Tencent, and because of its large scale and diverse business types, it is more difficult to solve and must be solved. tRPC was born in this context.

Pluggable Design

It must not only be interconnected with the existing technology system, but also adapt to cloud native technology and develop rapidly. This requires that the development framework must be open and extensible. In order to achieve this goal, tRPC adopts pluggable design ideas in architectural design.

The tRPC pluggable design ideas are as follows:

First, tRPC layers and modularizes the entire framework, and abstracts the core functional modules into independent plugins. The framework is then responsible for the concatenation and assembly of these independent plugins to achieve the features that the framework wants to support.

In terms of specific implementation, the following key technologies are used:

plugin factory based on interface mechanism.
filter based on AOP idea.

Plugin Factory

The overall implementation idea of the plugin factory is that the framework only defines the standard interface of the plugin and provides registration capabilities without specific implementation. When interoperating with external services or connecting with a certain service governance system, you only need to develop the corresponding specific plugins.

For example: tRPC supports multiple protocols by defined a unified Codec. Different protocols only need to be implemented according to the Codec interface.

For example: tRPC connects to different naming service systems by defined unified Registry and Selector abstract interfaces. When connecting to different naming service systems, you only need to follow the Registry and Selector interfaces to implement it.

The design of the plugin factory can bring the following benefits:

For the framework side: the framework only defines standard interfaces, without any plugin implementation, and is completely decoupled from the specific platform;
For the platform side: You only need to implement the plugin according to the standard interface of the framework plugin, and integrate the capabilities of the platform into the framework;
For the user side: business development only needs to be used through configuration, which is transparent to users;

Filter

In order to make the tRPC framework more scalable and meet personalized business needs (such as Metrics, Log Collection, Tracing, Parameter Verification, Log Replay, Fault Injection, etc.), tRPC use java’s aspect-oriented programming(AOP) idea to support filter.

The overall idea of filter implementation is to set up buried points in the framework’s process of processing RPC requests, and then insert a series of filters into the buried points.

The filter workflow is as follows:

The ultimate goal of filter is to decouple business logic from the framework and allow them to develop cohesively. It can dynamically add or replace personalized functions to business programs without modifying the framework code.

Pluggable Archtecture

With the above key technical support for pluggable, let’s take a look at how the tRPC pluggable architecture is designed.

Overall Architectural Design

The overall architecture design of tRPC is as follows:

The overall architecture consists of two parts: “Framework Core” and “Plugin”. The dotted line box is tRPC, the red solid line box in the middle is the core of the framework, and the blue box is the pluggable part.

The core of the framework can be divided into three layers:

Communication Layer: responsible for data transmission and protocol encoding and decoding. the framework has built-in support for communication protocols such as tcp and udp. The transmission protocol uses the tRPC protocol based on Protocol Buffers to carry RPC messages. It also supports other transmission protocols through codec plugins;
Service Governance Layer: responsible for abstracting service governance functions into plugins and connecting them with service governance systems by calling plugins to realize Service Discovery, LoadBalance, Metrics, Tracing, etc.
Call Layer: encapsulates services and service proxy entities, provides RPC interfaces, and supports business calls between services using synchronous, asynchronous, one-way and streaming calls;

In addition, the framework also provides an Admin management interface, so that users or the operation platform can manage services by calling the Admin interface. The management interface includes functions such as updating configurations, viewing versions, modifying log levels, viewing framework runtime information, etc. At the same time, the framework also supports user-defined management interfaces to meet business customization needs.

Plugins are the bridge that connects the framework core and external service governance systems. They are roughly divided into the following plugin types according to their functions:

Codec: provides interfaces related to protocol encoding and decoding, allowing expansion of business protocols, serialization methods, data compression methods through plugins;
Naming: provides service registration (Registry), service discovery (Selector), load balance, circuit breaker and other capability encapsulation, used to connect with various naming service systems;
Config: provides interfaces to read local configuration files, remote configuration center configurations, etc., allows plugin extensions to support configuration files in different formats, different configuration centers, and supports reload and watch configuration updates;
Metrics: provides interfaces to report monitor data, supports common single-dimensional reporting, such as counter, gauge, etc., and also supports multi-dimensional reporting;
Logging: Provides a general log collection interface, allowing the log implementation to be extended through plugins and output to remote locations;
Tracing: Provides distributed tracing capabilities, allowing reporting to the call chain system through plugins;
Telemetry: provides the ability to collect and report telemetry data. It is a plugin that integrates Tracing, Metrics and Logging;

When implementing a specific plugin, one needs to implement the plugin according to the standard interface of the plugin, register it in the core of the framework, and complete plugin instantiation; on the other side, the specific plugin also needs to implement features (such as service discovery, load balance) based on using the SDK or API of the external service governance system.

Specific Architecture Design

Before talking about the specific architecture design, let’s first look at the process of rpc call. From a developer’s perspective, it allows you to make cross-node function call like local function call. Usually a complete RPC process is as follows:

The above figure describes the steps that an RPC must go through. Based on this commonality, we divide the framework into server and client horizontally and layer it vertically. Among them, the vertical server is roughly divided into Transport, Codec, Service and Filter layers, and the client is divided into Transport, Codec, ServiceProxy and Filter layers. The specific design is as follows:

In this picture, we have added a new layer of filter layer. Its main purpose is to use the idea of AOP to personalize the needs of the business (such as parameter verification, log replay, fault injection, etc.) and service governance (such as Metrics, Tracing, Logging, Authentication, etc.) that are inserted into the request/response processing process in a cross-cutting manner. This design enhances the framework’s scalability.

At the same time, tRPC modularizes each layer and adopts pluggable implementation. The framework connects the entire process of RPC through the idea of interface-based programming. For some modules, the framework also adopts a more fine-grained module splitting. For example: the Selector module is subdivided into sub-modules such as service discovery, service routing, load balanc and etc., and the Codec layer is also subdivided into three sub-modules: encode/decode, serialization and compression.

Through the above overall layered design, pluggable implementation of specific modules and fine-grained module splitting, the framework has strong scalability and openness. Businesses can flexibly replace plugins to achieve connecting with different systems, and can also implement personalized business capabilities.

Docs: Protocol Design

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This article introduces the tRPC protocol.

Why design tRPC protocol

First of all, tRPC is a multi-language RPC framework that requires a unified transmission protocol for communication and some functional alignment during communication.

Secondly, why tRPC does not choose the http/http2 protocol? The main reason here is performance issue.

Finally, in addition to forward compatibility, tRPC also has strong scalability requirements in the communication protocol. tRPC not only emphasizes forward compatibility but also has a strong focus on extensibility in its communication protocol. It is designed to support various business requirements and has the ability to be extended accordingly.

The following table shows the capabilities and requirements of tRPC for communication protocols.

Capabilities	Demands
RPC	remote procedure call like local interface call
Stream	the protocol should be able to solve the problem of transmitting large data packets and meet the scenario of streaming data transmission.
High Performance	the protocol design and implementation need to consider performance issues
Compatibility & Scalability	the protocol design and implementation should have forward compatibility and scalability
Support Multiple Serialization	the protocol design should support multiple serialization methods, such as Protobuf, JSON, etc.
Support multiple Decompression	the protocol design should support multiple decompression methods.
Request Timeout Control	The protocol should provide the ability to control request timeout
Passing Tracing Information	The protocol should provide the ability to pass tracing information
Passing Auth Information	The protocol should provide the ability to pass auth information
Passing Dyeing Information	The protocol should provide the ability to pass dyeing information
Passing Custom Metadata	The protocol should provide the ability to pass custom metadata

In order to support these capabilities and requirements, the overall design of the tRPC protocol is as follows:

Protocol header (custom design) + Protocol body (use Protobuf by default, extensible)

Design And Implementation

Protocol Design

The figure below is the specific design of the tRPC protocol.

The tRPC protocol supports two transmission methods: unary and stream.

for unary RPC(one-request-one-response), the entire data packet = Fix Header（16 bytes）+ Unary Header + Unary Body.

for stream RPC, the entire data packet = Fix Header（16 bytes）+ Stream Frame(Init/Data/Feeback/Close)

the complete protocol definition, please see trpc.proto

Protocol Details

The first 2 bytes: The first two bytes of the protocol are magic numbers, indicating the start of the trpc frame, which is 0x930.
The 3rd byte: the data frame type of the packet

// Two types are currently supported:
// 1. The data frame type for unary(one-response-one-response)
// 2. The data frame type for stream
enum TrpcDataFrameType {
TRPC_UNARY_FRAME = 0x00;

TRPC_STREAM_FRAME = 0x01;
}

The 4th byte: streaming frame type, only valid when the data frame type is TRPC_STREAM_FRAME

// Four types are currently supported:
// `INIT` frame: FIXHEADER + TrpcStreamInitMeta
// `DATA` frame: FIXHEADER + body (business serialized data)
// `FEEDBACK` frame: FIXHEADER + TrpcStreamFeedBackMeta (triggered strategy: high/low water level and timer)
// `CLOSE` frame: FIXHEADER + TrpcStreamCloseMeta
enum TrpcStreamFrameType {
TRPC_UNARY = 0x00;

TRPC_STREAM_FRAME_INIT = 0x01;

TRPC_STREAM_FRAME_DATA = 0x02;

TRPC_STREAM_FRAME_FEEDBACK = 0x03;

TRPC_STREAM_FRAME_CLOSE = 0x04;
}

The 5-8 bytes, the total size of the packet

For unary(one response one response), total data size = fixed header size + header size + body size

For stream, total data size = fixed header size + stream frame size

The 9-10 bytes

For unary(one response one response), header size

For stream, set 0

The 11-14 bytes

For unary(one response one response), unique-id

For stream, stream-id

stream-id needs to be unique in a single connection

The 15 byte, the version of trpc protocol
The 16 byte, reserved field

The Header of Unary

After 16 bytes, for unary(one-response-one-response), it corresponds to the header of the protocol, which is divided into request header and response header.

Request Header

// The request header for unary
message RequestProtocol {
  // The version of protocol
  // The specific value corresponds to `TrpcProtoVersion`
  uint32    version                     = 1;

  // Call type
  // eg: unary, one-way
  // The specific value corresponds to `TrpcCallType`
  uint32    call_type                   = 2;

  // The unique id of the request(on the conneciton)
  uint32    request_id                  = 3;

  // The timeout of the request(ms)
  uint32    timeout                     = 4;

  // Caller name
  // The specification format: trpc.application_name.server_name.proto_service_name, 4 segments
  bytes     caller                      = 5;

  // Callee name
  // The specification format: trpc.application_name.server_name.proto_service_name[.interface_name]
  bytes     callee                      = 6;

  // Interface name of callee
  // The specification format: /package.service_name/interface_name
  bytes     func                        = 7;

  // The message type of the transparent transmission information
  // such as tracing, dyeing key, gray, authentication, multi-environment, set name, etc.
  // The specific value corresponds to `TrpcMessageType`
  uint32    message_type                = 8;

  // The information key-value pair transparently transmitted by the framework
  // Currently divided into two parts:
  // 1 part is the information to be transparently transmitted by the framework layer,
  // and the name of the key must be started with `trpc-``
  // 2 part is the information to be transparently transmitted by the business layer,
  // and the business can set it by itself, it is recommended to start with `app-``, not `trpc-`
  // Note: The key-value pair in trans_info will be transparently transmitted through the whole link, please use it carefully for business.
  map<string, bytes> trans_info         = 9;

  // The serialization type of the request data
  // eg: proto/json/.., default proto
  // The specific value corresponds to `TrpcContentEncodeType`
  uint32    content_type                = 10;

  // The compression type of the requested data
  // eg: gzip/snappy/..., not used by default
  // The specific value corresponds to `TrpcCompressType`
  uint32    content_encoding            = 11;

  // The size of attachment data
  uint32    attachment_size             = 12;
}

Response Header

// The response header for unary
message ResponseProtocol {
  // The version of protocol
  // The specific value corresponds to `TrpcProtoVersion`
  uint32    version                     = 1;

  // Call type
  // eg: unary, one-way
  // The specific value corresponds to `TrpcCallType`
  uint32    call_type                   = 2;

  // The unique id of the request(on the conneciton)
  uint32    request_id                  = 3;

  // Error code
  // The specific value corresponds to `TrpcRetCode`
  int32     ret                         = 4;

  // The error code of the interface
  // 0 means success, other means failure
  int32     func_ret                    = 5;

  // The result information when the call fails
  bytes    error_msg                    = 6;

  // The message type of the transparent transmission information
  // such as tracing, dyeing key, gray, authentication, multi-environment, set name, etc.
  // The specific value corresponds to `TrpcMessageType`
  uint32    message_type                = 7;

  // The information key-value pair transparently transmitted by the framework
  // Currently divided into two parts:
  // 1 part is the information to be transparently transmitted by the framework layer,
  // and the name of the key must be started with `trpc-``
  // 2 part is the information to be transparently transmitted by the business layer,
  // and the business can set it by itself, it is recommended to start with `app-``, not `trpc-`
  map<string, bytes> trans_info         = 8;

  // The serialization type of the request data
  // eg: proto/json/.., default proto
  // The specific value corresponds to `TrpcContentEncodeType`
  uint32    content_type                = 9;

  // The compression type of the requested data
  // eg: gzip/snappy/..., not used by default
  // The specific value corresponds to `TrpcCompressType`
  uint32    content_encoding            = 10;

  // The size of attachment data
  uint32    attachment_size             = 12;
}

Stream Frame

After 16 bytes, for stream, it is followed by the stream frame

Divided into init, data, feedback, close frame.

Init Frame is used for stream initialization
Data Frame is used to transmit streaming data
Feedback Frame is used to transmit flow control messages
Close Frame is used to close the stream

Init Frame

// The message definition of stream `INIT` frame
message TrpcStreamInitMeta {
  // request meta information
  TrpcStreamInitRequestMeta request_meta = 1;

  // response meta information
  TrpcStreamInitResponseMeta response_meta = 2;

  // The window size is notified by the receiver to the sender
  uint32 init_window_size = 3;

  // The serialization type of the request data
  // eg: proto/json/.., default proto
  // The specific value corresponds to `TrpcContentEncodeType`
  uint32 content_type = 4;

  // The compression type of the requested data
  // eg: gzip/snappy/..., not used by default
  // The specific value corresponds to `TrpcCompressType`
  uint32 content_encoding = 5;
}

// The request meta information definition of stream `INIT` frame
message TrpcStreamInitRequestMeta {
  // Caller name
  // The specification format: trpc.application_name.server_name.proto_service_name, 4 segments
  bytes caller = 1;

  // Callee name
  // The specification format: trpc.application_name.server_name.proto_service_name[.interface_name]
  bytes callee = 2;

  // Interface name of callee
  // The specification format: /package.service_name/interface_name
  bytes func = 3;

  // The message type of the transparent transmission information
  // such as tracing, dyeing key, gray, authentication, multi-environment, set name, etc.
  // The specific value corresponds to `TrpcMessageType`
  uint32 message_type = 4;

  // The information key-value pair transparently transmitted by the framework
  // Currently divided into two parts:
  // 1 part is the information to be transparently transmitted by the framework layer,
  // and the name of the key must be started with `trpc-``
  // 2 part is the information to be transparently transmitted by the business layer,
  // and the business can set it by itself, it is recommended to start with `app-``, not `trpc-`
  // Note: The key-value pair in trans_info will be transparently transmitted through the whole link, please use it carefully for business.
  map<string, bytes> trans_info = 5;
};

// The response meta information definition of stream `INIT` frame
message TrpcStreamInitResponseMeta {
  // Error code
  // The specific value corresponds to `TrpcRetCode`
  int32 ret = 1;

  // The result information when the call fails
  bytes error_msg = 2;
};

Data Frame

The data frame corresponds to the message body of the actual request

Feedback Frame

// The meta information definition of stream `FEEDBACK` frame
message TrpcStreamFeedBackMeta {
  // increased window size
  uint32 window_size_increment = 1;
}

Close Frame

Close frame is divided into normal Close and abnormal Close (Reset)

// The closed type of trpc stream protocol
enum TrpcStreamCloseType {
  // normal closes unidirectional flow
  TRPC_STREAM_CLOSE = 0;

  // Exception closes bidirectional stream
  TRPC_STREAM_RESET = 1;
}

// The meta information definition of trpc stream protocol for closing stream
message TrpcStreamCloseMeta {
  // The type of stream closure, close one end, or close all
  int32 close_type = 1;

  // Error code
  // The specific value corresponds to `TrpcRetCode`
  int32 ret = 2;

  // The result information when the call fails
  bytes msg = 3;

  // The message type of the transparent transmission information
  // such as tracing, dyeing key, gray, authentication, multi-environment, set name, etc.
  // The specific value corresponds to `TrpcMessageType`
  uint32    message_type  = 4;

  // The information key-value pair transparently transmitted by the framework
  // Currently divided into two parts:
  // 1 part is the information to be transparently transmitted by the framework layer,
  // and the name of the key must be started with `trpc-``
  // 2 part is the information to be transparently transmitted by the business layer,
  // and the business can set it by itself, it is recommended to start with `app-``, not `trpc-`
  // Note: The key-value pair in trans_info will be transparently transmitted through the whole link, please use it carefully for business.
  map<string, bytes> trans_info = 5;

  // The error code of the interface
  // 0 means success, other means failure
  int32 func_ret = 6;
}

3 IDL

tRPC’s IDL is based on Protobuf

Unary IDL

syntax = "proto3";

package proto;

service StreamService {
    rpc List(Request) returns ( Response) {};
    rpc Record( Request) returns (Response) {};
    rpc Route( StreamRequest) returns ( Response) {};
}


message Point {
  string name = 1;
  int32 value = 2;
}

message Request {
  Point pt = 1;
}

message Response {
  Point pt = 1;
}

Stream IDL

Support:

Server-side streaming RPC
Client Streaming RPC
Bidirectional streaming RPC

syntax = "proto3";

package proto;

service StreamService {
    rpc List(StreamRequest) returns (stream StreamResponse) {};
    rpc Record(stream StreamRequest) returns (StreamResponse) {};
    rpc Route(stream StreamRequest) returns (stream StreamResponse) {};
}


message StreamPoint {
  string name = 1;
  int32 value = 2;
}

message StreamRequest {
  StreamPoint pt = 1;
}

message StreamResponse {
  StreamPoint pt = 1;
}

Docs: Plugin Ecosystem

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This article introduces the plugin ecosystem of tRPC.

Maturity classification

In terms of maturity, tRPC plugins are currently divided into the following 4 levels:

Maturity	Description
stable	Already used on a large scale
on trial	It has been tested and has been used in certain businesses
tested	It has been tested and has not been used in business yet.
archived	Archived, no longer maintained, and not recommended for use

Plugin ecosystem in each language

The plugin types of tRPC mainly include Codec, Filter, Naming, Config, Metrics, Tracing, Logging, Telemetry, etc.

tRPC-Cpp plugin ecosystem

Type	Plugin	Maturity	Doc
Codec	HTTP	stable	link
Codec	gRPC	stable	link
Serialiazation	Protobuf	stable	link
Serialiazation	JSON	stable	link
Serialiazation	text	stable	link
Serialiazation	binary	stable	link
Compressor	Gzip	stable	link
Compressor	lz4	stable	link
Compressor	Snappy	stable	link
Compressor	zlib	stable	link
Naming	MeshPolaris	tested	link
Config	etcd	tested	link
Metrics	Prometheus	stable	link
Tracing	Jaeger	stable	link
Logging	CLS	stable	link
Telemetry	OpenTelemetry	stable	link
Filter	concurrency_limiter	stable	link

tRPC-Go plugin ecosystem

Type	Plugin	Maturity	Doc
Codec	HTTP	stable	link
Codec	gRPC	stable	link
Naming	MeshPolaris	tested	link
Config	etcd	tested	link
Metrics	Prometheus	stable	link
Tracing	Jaeger	stable	link
Logging	CLS	stable	link
Telemetry	OpenTelemetry	stable	link
Filter	debuglog	stable	link
Filter	degrade	stable	link
Filter	filterextensions	stable	link
Filter	hystrix	stable	link
Filter	jwt	stable	link
Filter	masking	stable	link
Filter	mock（client mock）	stable	link
Filter	recover ）	stable	link
Filter	referer	stable	link
Filter	slime	stable	link
Filter	transinfo-blocker	stable	link
Filter	tvar	stable	link
Filter	validation	stable	link

tRPC-Java plugin ecosystem

Type	Plugin	Maturity	Code
Codec	HTTP	stable	link
Codec	TRPC	stable	link
Registry	MeshPolaris	stable	link
Registry	Consul	tested	link
Registry	Nacos	tested	link
Registry	Zookeeper	tested	link
Selector	MeshPolaris	stable	link
Selector	Consul	tested	link
Selector	Nacos	tested	link
Selector	Zookeeper	tested	link
Config	Nacos	tested	link
Metrics	Opentelemetry	tested	link
Tracing	Opentelemetry	tested	link
Logging	Opentelemetry	tested	link
Limiter	Sentinel	stable	link
Transport	HTTP	stable	link
Transport	Netty	stable	link
Validation	PGV	stable	link
-	Code Generator	stable	link

tRPC – What is tRPC?

Docs: Introduction

What is tRPC

Features

Supported languages

To start using tRPC

To start developing tRPC

Docs: Archtecture Design

Overview

Background

Pluggable Design

Plugin Factory

Filter

Pluggable Archtecture

Overall Architectural Design

Specific Architecture Design

Docs: Protocol Design

Overview

Why design tRPC protocol

Design And Implementation

Protocol Design

Protocol Details

Fixed Header

The Header of Unary

Request Header

Response Header

Stream Frame

Init Frame

Data Frame

Feedback Frame

Close Frame

3 IDL

Docs: Plugin Ecosystem

Overview

Maturity classification

Plugin ecosystem in each language

tRPC-Cpp plugin ecosystem

tRPC-Go plugin ecosystem

tRPC-Java plugin ecosystem