This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: lightning

| username: TiDBer_mGBoAnW9

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachment: Screenshot/Log/Monitoring]

The tidb lightning exit successfully log shows success, and there are no errors in all logs, but there is a significant difference in the data volume. What could be the reason?

| username: linnana | Original post link

Please post the configuration file for us to take a look.

| username: TiDB_C罗 | Original post link

How is there a significant difference? When importing from the source cluster to the target cluster, is the target cluster having more or less data? Are there any other services writing or deleting data on the source cluster?

| username: TiDBer_mGBoAnW9 | Original post link

The target was reduced. From over 300 million data entries to 3 million, it succeeded and automatically stopped. The log just printed “success.”

| username: TiDB_C罗 | Original post link

Try importing another table and see if it works. What version are you using? I used dumpling+light for migration and haven’t encountered similar issues.

| username: TiDBer_mGBoAnW9 | Original post link

I have the same setup as you, just dumpling+light with the same configuration for other tables. Could it be due to memory issues?

| username: okenJiang | Original post link

It is estimated that the schema_pattern is written incorrectly.

| username: okenJiang | Original post link

Which version? The new version uses file-pattern.

| username: TiDBer_mGBoAnW9 | Original post link


| username: yilong | Original post link

You can check the lightning logs to see if any files have been filtered. Alternatively, check if there are any error messages.

| username: redgame | Original post link

When importing data, some data may be filtered out or not correctly imported due to incorrect filtering conditions or configuration issues. Please check the filtering conditions and configuration files before and after the import to ensure that all necessary data is correctly imported.