Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: lighting导入外部数据参数data-source-dir配置
[TiDB Usage Environment] Production Environment / Testing / PoC
[TiDB Version]
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact] How to configure the data-source-dir parameter when importing an SQL file from another server using lighting
[Resource Configuration]
[Attachment: Screenshot/Log/Monitoring]
NFS shared data directory or copy the data to the current machine.
Why do you need to import the SQL file from another machine? Can’t you just copy it over?
The data volume is too large, with over 1 billion records in a single table. The server can’t handle it, so we’re using an intermediate server.
Then consider using shared storage devices and have the operations colleagues mount it to a specific path for you.
The value of data-source-dir supports S3 addresses, so you can also consider placing it on S3.
In that case, mount the remote directory locally using NFS, but the import will be affected by the network bandwidth between the two servers.
If you want to extend further, you can set up a MinIO on the intermediate server to act as S3 storage, or directly transfer TiDB-Lightning to the intermediate server and import it there.
|
username: TiDBer_FMWXa7ja
|
username: 
|
username: 
|
username: 
|
username: