After successful import with tidb-lightning, the target cluster has no data

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb-lightning 成功导入后,目标集群没有数据

| username: 不输土豆

[TiDB Usage Environment] Test Environment
Data Migration
[TiDB Version] Old Cluster v4.0.16, Target Cluster v6.5.0
[Reproduction Steps]

  1. Use the tidb dumpling tool to export the test database
    tiup dumpling --database tidbtest -h 127.0.0.1 -P 4200 --output . --snapshot "441869824295174146" --filetype csv --consistency=snapshot
    After exporting, you can see the following content:

    The content of the *.csv file meets expectations.
  2. Use tidb-lighting to import into the target cluster
    tiup tidb-lightning -config tidb-lightning.toml
    (1) tidb-lightning.toml configuration file
[lightning]
# Log
level = "debug"
file = "tidb-lightning.log"

[tikv-importer]
# Choose the import mode
backend = "local"
# Set the temporary storage location for sorted key-value pairs, the target path needs to be an empty directory
sorted-kv-dir = "/home/zx/wmh_workroot/sorted-kv-dir"
pause-pd-scheduler-scope = "global"

[mydumper]
# Source data directory.
data-source-dir = "/home/zx/wmh_workroot/source_cluster_data/"
# Configure wildcard rules, the default rules will filter out all tables under the mysql, sys, INFORMATION_SCHEMA, PERFORMANCE_SCHEMA, METRICS_SCHEMA, INSPECTION_SCHEMA system databases
# If this item is not configured, an "unable to find schema" exception will occur when importing system tables
# filter = ['*.*', '!mysql.*', '!sys.*', '!INFORMATION_SCHEMA.*', '!PERFORMANCE_SCHEMA.*', '!METRICS_SCHEMA.*', '!INSPECTION_SCHEMA.*']
filter = ['tidbtest.*']

[checkpoint]
# Whether to enable checkpoint continuation.
# When importing data, TiDB Lightning will record the progress of the current table import.
# Therefore, even if TiDB Lightning or other components exit abnormally, it can avoid re-importing completed data upon restart.
enable = false

# [mydumper.csv]
# separator = ','
# delimiter = '"'

[tidb]
# Information of the target cluster (Tidb server port)
host = "127.0.0.1"
port = 4300
user = "root"
password = ""
# Table schema information is obtained from the "status port" of TiDB.
status-port = 28080
# Address of the cluster pd
pd-addr = "127.0.0.1:2570"
# pd-urls = "127.0.0.1:2570"

Execution result:

[Encountered Problem: Problem Phenomenon and Impact]
After the operation is completed, the corresponding table in the target cluster has no data:
Image

[Resource Configuration]

The output log file is as follows
tdib-lightning.log (44.4 KB)

| username: zhanggame1 | Original post link

Can you see any data when you open the CSV file?

| username: 不输土豆 | Original post link

There is data, as follows!

| username: 不输土豆 | Original post link

Found the reason, the configuration was wrong:

[tidb]
# Information of the target cluster (Tidb service port)
host = "127.0.0.1"
port = 4300
user = "root"
password = ""
# Table schema information is obtained from TiDB's "status port".
status-port = 28080
# Address of the cluster pd
pd-addr = "127.0.0.1:2570"
# pd-urls = "127.0.0.1:2570"

Changed to:

[tidb]
# Information of the target cluster (Tidb service port)
host = "127.0.0.1"
port = 4300
user = "root"
password = ""
# Table schema information is obtained from TiDB's "status port".
status-port = 29080
# Address of the cluster pd
pd-addr = "127.0.0.1:2570"
# pd-urls = "127.0.0.1:2570"

Everyone take heed, don’t make mistakes in small places! :joy:

| username: zhanggame1 | Original post link

Try commenting out the filter.

| username: 不输土豆 | Original post link

Found the reason, the tidb-server status-port was filled in incorrectly; it was filled with the old cluster’s tidb-server port! :joy:
Thank you, expert, for helping to check the issue!

| username: zhanggame1 | Original post link

It turned out to be a simple question.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.