Unable to import sql.gz file exported with compression parameters using dumpling in lightning?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: dumpling使用压缩参数导出的sql.gz文件lightning无法导入?

| username: Woody

Using dumpling to export MySQL data, planning to import it into TiDB. Since the table is relatively large, I used the compression method [–compress gzip].

– The export statement is roughly as follows, and the exported SQL file has a .gz compressed suffix

cd /apps/svr/tidb-community-toolkit-v6.1.0-linux-amd64
./dumpling -u myuser -p’mypass’ -P 3306 -h 172.16.0.11 -T test.mytab
–filetype sql -t 4 -r 200000 -F256MiB --compress gzip --no-schemas
-o /tmp/test

– Using lightning to import, but it actually failed to import successfully.

nohup /apps/svr/tidb-community-toolkit-v6.1.0-linux-amd64/tidb-lightning -no-schema -config /apps/tidb-lightning_test.toml > nohup_tidb-lightning_test.out &

The solution is to decompress all the sql.gz files exported by dumpling using gunzip, and then initiate the above lightning again.

How can lightning support direct import using sql.gz compressed files?

2
| username: Mark | Original post link

Generally, it’s SQL/csv. You can take a look at toml.

3
| username: tidb狂热爱好者 | Original post link

You use shell command pipelines to pass parameters and do it.

4
| username: 长安是只喵 | Original post link

In this case, wouldn’t it be better to not compress during export, saving a step of compression and decompression?

5
| username: buchuitoudegou | Original post link

It doesn’t support it, right? Why should Lightning support importing compressed files?

6
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.