Sync-diff-inspector Only Compares Table Structure and Total Table Data

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: sync-diff-inspector仅对比表结构和表数据总量

| username: TiDBer_JZFHyklQ

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] After DM migration is completed, use the sync-diff-inspector tool to verify upstream and downstream data.
[Encountered Problem: Problem Phenomenon and Impact] When only comparing table structures, the output shows that the number of upstream and downstream is 0. It is necessary to add a comparison of the total amount of upstream and downstream data based on the table structure.
[Resource Configuration]
[Attachment: Screenshot/Log/Monitoring]
The configuration is relatively simple, hoping to only compare table structures and total table data, not each row of data. Screenshot:
image
The result is as follows:

Part of the log is as follows:


It remains the same after analysis.

| username: hey-hoho | Original post link

Please upload the accompanying documents, commands, and log files, otherwise, everyone will have no way to help you analyze.

| username: TiDBer_JZFHyklQ | Original post link

Added, thank you.

| username: Jellybean | Original post link

Comparing the total amount of data, it seems that data comparison needs to be enabled. You can try turning on the data comparison switch to see.

| username: TiDBer_JZFHyklQ | Original post link

The idea is to compare only the data volume, not the specific content of each row.

| username: 小龙虾爱大龙虾 | Original post link

Does sync-diff have this feature? You can use sync-diff to compare table structures and write SQL to compare row counts yourself.

| username: zhanggame1 | Original post link

It’s very simple to write some code to compare this yourself.

| username: hey-hoho | Original post link

Currently, sync-diff probably can’t compare just the row count without comparing the data. However, I had a similar need before and wrote a tool for comparison, which now supports various common database types.

On which platform do you need to run it? I can send you a package to try.

| username: hey-hoho | Original post link

I asked the AI assistant, and it said it can be used like this. You can give it a try (ps. I didn’t find this parameter in the documentation :joy:)

sync_diff_inspector -C ./config.yaml --ignore-row-check

| username: TiDBer_JZFHyklQ | Original post link

Haha, it reported an error. In fact, there is no such parameter as ignore-row-check.

| username: TiDBer_JZFHyklQ | Original post link

Thank you, I can just write SQL queries directly.

| username: TiDBer_JZFHyklQ | Original post link

Well, it can only be like this.

| username: 随缘天空 | Original post link

You can directly compare using the Navicat client, but it will affect online usage.

| username: dba远航 | Original post link

We all write our own SQL scripts to do this.

| username: lichunzhu-PingCAP | Original post link

I checked the code and there is currently no such configuration. You can temporarily write an SQL query to perform the check. Later, you can submit a feature request issue in the sync-diff repo.