Tidb-lightning is slow

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb-lightning 速度慢

| username: 不输土豆

[TiDB Usage Environment] Production Environment / Testing / Poc
[TiDB Version] v6.5.0
[Reproduction Path]
(1) Use the tidb-dumpling tool to dump data from a table.
(2) Use the tidb-lightning tool to import data.

Shortly after tidb-lightning starts, it remains in import mode continuously. The import speed feels too slow; it has taken 3 hours to import 113GB of data and it is still not completed.

The restore file process, however, was very fast, taking only 3 minutes to complete. Then it remains in Import Model, state=writing mode.

[Encountered Problem: Problem Phenomenon and Impact] tidb-lightning is slow and cannot meet the migration schedule.
[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]
This is the configuration file (the server has 256 virtual cores)

| username: xingzhenxiang | Original post link

Try switching to a different logic import mode.

| username: 不输土豆 | Original post link

I think Lightning is stuck. I canceled the task at 23:33 last night, but the process is still running:

| username: linnana | Original post link

How long does logic import approximately take?

| username: Hacker_xUwtuKxa | Original post link

Is the sort_dir placed on an NVMe disk? Check if the space in that directory is full?

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.