Does gRPC compression configuration take effect on CDC?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: grpc压缩配置对cdc是否生效

| username: TiDBer_FFBbFX9x

[Test Environment for TiDB] Testing
[TiDB Version] 6.5.3
Our production cluster is deployed in a 3 AZ high availability zone.

We found that about 60% of the cross-availability zone traffic is from TiKV to TiCDC. Then we saw the gRPC compression configuration for TiKV: server.grpc-compression-type: gzip.
In the test cluster, using the tpchmark dataset, we changed the server.grpc-compression-type each time we wrote data. After changing it, there was basically no impact, with a difference of less than 10%.

Then I briefly looked at the TiKV code (I don’t understand Rust). The gRPC compression configuration is only used as shown below, and it seems to have nothing to do with CDC. I’m wondering if it’s simply not supported. :disappointed_relieved:

image
image1380×406 57.6 KB

2
| username: xfworld | Original post link

CDC captures the incremental change data of TiKV, which is relatively small, so configuring compression won’t make much difference.

3
| username: TiDBer_FFBbFX9x | Original post link

So how about incrementally importing several gigabytes of data?

4
| username: xfworld | Original post link

gRPC is quite efficient. You can set up a service locally and test this protocol by transmitting several gigabytes of data to see if there’s any noticeable delay…

If you enable streaming mode, it will feel even smoother… :face_with_spiral_eyes:

5
| username: 有猫万事足 | Original post link

gRPC is based on HTTP/2, which is a binary HTTP protocol. This results in gRPC having very compact packet sizes. Therefore, configuring gzip won’t compress much.

As you mentioned, after configuring gzip, the difference is less than 10%, which should be a normal result.

6
| username: TiDBer_FFBbFX9x | Original post link

I captured a packet and found that this parameter is not effective for CDC.

7
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.