Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: tidb集群中的cdc添加新配置参数后,无法正常启动cdc组件,报错:component TiCDC server’s config file conf/cdc.toml contained unknown configuration options: sorter.cache-size-in-mb, sorter.sorter-dir
[TiDB Usage Environment] Production Environment
[TiDB Version] v6.5.0
[Reproduction Path] Add new configuration parameters to the cdc in the tidb cluster using
# tiup cluster edit-config cluster_name

[Encountered Problem: Problem Phenomenon and Impact] After adding new configuration parameters to the cdc in the tidb cluster, the cdc component cannot start normally, and an error is reported:
component TiCDC server's config file conf/cdc.toml contained unknown configuration options: sorter.cache-size-in-mb, sorter.sorter-dir
Check the configuration file of the cdc-server, and the parameters have been loaded normally:
Questions:
- After normally modifying and updating the configuration parameters, the cdc component shows normal when starting, but in fact, the cdc service on the backend cdc server did not start successfully.
- The version of the cdc in the tidb cluster is also: v6.5.0, and the official cdc_server configuration does indeed have these two parameters:
docs-cn/ticdc/ticdc-server-config.md at release-6.5 · pingcap/docs-cn · GitHub
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
The error you encountered is because cdc does not recognize the parameter.
tiup cdc version
Starting component cdc: /root/.tiup/components/cdc/v8.1.0/cdc version
Release Version: v8.1.0
Git Commit Hash: 137ed650cc3fcd04ed31b4e7978dccebc087330d
Git Branch: HEAD
UTC Build Time: 2024-05-20 13:05:21
Go Version: go version go1.21.10 linux/amd64
Failpoint Build: false
Use a similar syntax to check the version of cdc in the corresponding deployment directory.
To add variables in tiup, many need to be placed under config. You can refer to the specific details at 通过 TiUP 部署 TiDB 集群的拓扑文件配置 | PingCAP 文档中心
Well, I tried version 7.5, and this method does work. It also reloads successfully.
On the control machine, execute the command to check the CDC version information as follows:
[root@wbjdnchzttdb01 bj_sjzt_to_slave]# tiup cdc version
tiup is checking updates for component cdc ...
Starting component `cdc`: /root/.tiup/components/cdc/v6.5.0/cdc version
Release Version: v6.5.0
Git Commit Hash: 9e91cff866d240ab6c1737680c17f5c5d0586911
Git Branch: heads/refs/tags/v6.5.0
UTC Build Time: 2022-12-23 08:43:57
Go Version: go version go1.19.3 linux/amd64
Failpoint Build: false
On the remote CDC server component’s server, check the CDC version as follows:
[root@wtj7vpztnctcd01 cdc-8300]# ./bin/cdc version
Release Version: v6.5.0
Git Commit Hash: 9e91cff866d240ab6c1737680c17f5c5d0586911
Git Branch: heads/refs/tags/v6.5.0
UTC Build Time: 2022-12-23 08:43:57
Go Version: go version go1.19.3 linux/amd64
Failpoint Build: false
I didn’t quite understand the modification suggestion in this passage. Could you provide an example of a configuration change? Thanks.
Some versions do not support certain parameter configuration methods.
It looks like version 650 does not support this parameter. It seems the documentation was written too early…
You can ignore that part. I tried it according to your method, and version 7.5.1 can correctly recognize and reload. So it’s still a version issue, possibly a new feature introduced in a minor version of 6.5. 6.5.0 doesn’t have it.
After checking the time when the configuration was added, it seems that the variable was introduced in version 6.5.3, so you need to upgrade the version to use it.
PS: While flipping through the release notes, I found that before version 6.5.4, it seems that each version of CDC introduced some new configuration items, with significant performance improvements. Therefore, when using it, you indeed need to pay attention to the version.
Okay, it seems that the only remaining option is to upgrade the version.