Data Corruption Occurs When Importing Dumpling Exported Data Using Lightning

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 利用lighting导入dumpling导出的数据时变成乱码

| username: danghuagood

[TiDB Usage Environment]
Production Environment

[TiDB Version]
Old version v3.0.0
New version v6.1.0

[Encountered Problem]
Using dumpling to export data from v3.0.0, and then using lighting’s Logical Import Mode to import into the v6.1.0 environment, it was found that the table creation statements’ comments and the imported data became garbled.

The dumpling tool version is compatible with v3.0.0.
The lighting tool version is compatible with v6.1.0.

[Problem Phenomenon and Impact]
[Attachment]

Export command:

Exported file:


Lighting configuration parameters:

Import command:

After importing into the database:
image

| username: db_user | Original post link

Could you provide the metadata, schema, create table, and insert SQL statements for a minimal table? If version 3.0.0 still supports it, could you create it, insert a few rows, and then export it?

| username: alfred | Original post link

Chinese garbled text is generally a problem related to language environment and character set.

| username: forever | Original post link

Check the encoding format of the exported file to see if it is UTF-8. In MySQL, if the file format is incorrect, it may result in garbled characters during import. (On Windows, you can see this by opening the file and saving it as a different format; on Linux, you can use vim to open the file and use the command :set fileencoding).

| username: danghuagood | Original post link

The exported file in that column is the file generated after exporting in version 3.0.0. Using the cat or more command to view it displays Chinese characters normally.

| username: danghuagood | Original post link

The same terminal displays Chinese correctly when querying version 3.0.0, but shows garbled text when querying version 6.1.0.

| username: danghuagood | Original post link

The image you provided is not accessible. Please provide the text you need translated.

| username: xiaohetao | Original post link

Check what the system’s LANG or nls_lang is during the import?

| username: xiaohetao | Original post link

If it’s not UTF-8, try changing it to UTF-8.

| username: danghuagood | Original post link

No settings, default is empty

| username: danghuagood | Original post link

After setting export LANG=en_US.UTF-8 to keep the environment consistent with the one exported from v3.0.0, the Chinese display in the imported v6.1.0 is normal.

| username: xiaohetao | Original post link

It seems to be due to the environment variables.

| username: danghuagood | Original post link

Yes, after setting LANG, importing Chinese works fine.

| username: alfred | Original post link

It seems that the language environment has had an impact.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.