When importing SQL files using TiDB Lightning, an error occurs: tidb lightning encountered error: [Lightning:Config:ErrInvalidConfig] '/hdpdata01/nnnx_dashboard_taginfo.sql': `mydumper.data-source-dir` does not exist

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用tidb lightning导入sql文件的时候,报错tidb lightning encountered error: [Lightning:Config:ErrInvalidConfig]‘/hdpdata01/nnnx_dashboard_taginfo.sql’: mydumper.data-source-dir does not exist

| username: 朵拉大虾

[Test Environment for TiDB]
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]


| username: 胡杨树旁 | Original post link

How about removing the “sql” at the end of the source directory? /hd*** like this?

| username: Billmay表妹 | Original post link

When TiDB Lightning processes SQL files, it cannot quickly split a single file, so the import speed of a single file cannot be improved by increasing concurrency. Therefore, when exporting data to SQL files, it is advisable to avoid having a single SQL file that is too large. Typically, a single file around 256MiB can achieve optimal performance.

| username: 朵拉大虾 | Original post link

I changed the path a bit. This can only keep the directory path, not the file path.

| username: 朵拉大虾 | Original post link

My SQL file is 20GB.

| username: Billmay表妹 | Original post link

Are these two versions inconsistent?

| username: 朵拉大虾 | Original post link

I followed the official documentation to install Lightning, using tiup directly.

| username: 朵拉大虾 | Original post link

This is not a problem, just wait, right?

| username: WalterWj | Original post link

Specify the directory, do not specify the file.

| username: 朵拉大虾 | Original post link

Is it looking for the file to be imported from the specified directory?

| username: 啦啦啦啦啦 | Original post link

Yes, just like mydumper, you can specify the folder name. The export will also generate a folder containing the split SQL files.

| username: 朵拉大虾 | Original post link

Okay, thank you.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.