Sync_diff_inspector Data Comparison Abnormal Exit

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: sync_diff_inspector数据对比异常退出

| username: TiDBer_pkQ5q1l0

【TiDB Usage Environment】Poc
【TiDB Version】5.2.1
【Encountered Problem: Phenomenon and Impact】
sync_diff_inspector always exits automatically halfway through the data comparison between the primary and standby clusters.
【Attachments: Screenshots/Logs/Monitoring】


| username: dba远航 | Original post link

It feels like it’s caused by insufficient memory.

| username: TiDBer_pkQ5q1l0 | Original post link

I observed that the memory is sufficient.

| username: 小龙虾爱大龙虾 | Original post link

Use a higher version of sync-diff, the lower version may have bugs.

| username: tidb菜鸟一只 | Original post link

Is this table very large? How long did it take to report the error?

| username: TiDBer_pkQ5q1l0 | Original post link

Changed to the latest version, still getting errors. The exit time is quite random, sometimes at 4%, sometimes at over 20%.

| username: Jellybean | Original post link

How long does it take for sync_diff_inspector to report an error during this comparison?

Check if there are any abnormal logs in the corresponding TiDB server of the upstream and downstream clusters.

Additionally, you can enable the tool’s debug logs to print more information for troubleshooting.

| username: TiDBer_pkQ5q1l0 | Original post link

Error within 10 seconds

| username: songxuecheng | Original post link

Take a look at the configuration file.

| username: TiDBer_pkQ5q1l0 | Original post link

| username: Fly-bird | Original post link

I suspect that the issue might be caused by differences in table structures or incompatible data types. Could it be that the case sensitivity of the fields was overlooked? Check the case sensitivity of the fields in both upstream and downstream, and manually compare them first.

| username: songxuecheng | Original post link

Check the logs of 37.17pd.

| username: TiDBer_pkQ5q1l0 | Original post link

The structure appears to be consistent.

| username: TiDBer_pkQ5q1l0 | Original post link

There are no ERROR logs in pd.

| username: songxuecheng | Original post link

Execute it once more. Please send the PD logs for the corresponding time.