TiDB CDC Synchronization Error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb cdc 同步报错

| username: Chen1234

TiDB version: 5.4.1, TiCDC version: 5.4.1
Using TiCDC to synchronize TiDB data to another TiDB instance.
Testing TiCDC synchronization speed with the command insert into table select * from table_a limit 50000;, encountered an error:

“error”: {
“addr”: “10.101.22.89:8300”,
“code”: “CDC:ErrProcessorUnknown”,
“message”: “[CDC:ErrReachMaxTry]reach maximum try: 8: [CDC:ErrMySQLTxnError]MySQL txn error: Error 1105: context canceled”
},

TiCDC error log:
[2022/09/14 08:38:29.202 +00:00] [WARN] [mysql.go:537] [“execute DMLs with error, retry later”] [error=“[CDC:ErrMySQLTxnError]MySQL txn error: Error 1105: context canceled”] [errorVerbose=“[CDC:ErrMySQLTxnError]MySQL txn error: Error 1105: context canceled
github.com/pingcap/errors.AddStack
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174
github.com/pingcap/errors.(*Error).GenWithStackByArgs
\tgithub.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/normalize.go:164
github.com/pingcap/tiflow/pkg/errors.WrapError
\tgithub.com/pingcap/tiflow/pkg/errors/helper.go:30
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries.func1.3
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:602
github.com/pingcap/tiflow/cdc/sink.(*Statistics).RecordBatchExecution
\tgithub.com/pingcap/tiflow/cdc/sink/statistics.go:111
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries.func1
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:574
github.com/pingcap/tiflow/pkg/retry.run
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:54
github.com/pingcap/tiflow/pkg/retry.Do
\tgithub.com/pingcap/tiflow/pkg/retry/retry_with_opt.go:32
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLWithMaxRetries
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:566
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).execDMLs
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:736
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run.func3
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:114
github.com/pingcap/tiflow/cdc/sink.(*mysqlSinkWorker).run
\tgithub.com/pingcap/tiflow/cdc/sink/mysql_worker.go:134
github.com/pingcap/tiflow/cdc/sink.(*mysqlSink).createSinkWorkers.func1
\tgithub.com/pingcap/tiflow/cdc/sink/mysql.go:382
runtime.goexit
\truntime/asm_amd64.s:1371”]
[2022/09/14 08:39:22.245 +00:00] [WARN] [etcd_worker.go:414] [“Etcd transaction took too long”] [duration=1.608856534s]
[2022/09/14 08:41:13.697 +00:00] [WARN] [etcd_worker.go:414] [“Etcd transaction took too long”] [duration=1.512278782s]

Parameters have been set to:
performance.txn-total-size-limit: 5048576000

| username: xfworld | Original post link

TiCDC does not support large transactions… Please read the precautions carefully… :rofl:

| username: Chen1234 | Original post link

For the same data, I can synchronize it to MySQL through TiCDC, and it can handle up to 100,000 records. But when synchronizing to TiDB, it fails at 50,000 records. Why is that?

| username: xfworld | Original post link

Please refer to it yourself.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.