Is there a way to handle column mismatch when importing data with lighting?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: lighting导入数据时,列数不匹配有办法处理吗?

| username: TiDBer_QHSxuEa1

When testing the inconsistency in the number of table fields between the upstream and downstream during the Lightning migration, it results in an error. Is there a way to solve this?

| username: tidb菜鸟一只 | Original post link

Does the target table’s corresponding field not have a default value? Set it to default null.

| username: 有猫万事足 | Original post link

It looks like a field is missing in the downstream.
This is a bit tricky.

| username: 胡杨树旁 | Original post link

Check the upstream and downstream table structures of the error table. If the downstream is missing fields, you can add fields to match the upstream, and then re-import. If the upstream is missing fields and the extra fields in the downstream can default to empty, you can add a parameter when exporting with dumpling, something like insert full. You can look up the specifics, re-export, and then re-import.

| username: TiDBer_QHSxuEa1 | Original post link

When the field is empty by default, the SQL file generated by dumpling uses insert into table_name values().... The insert statement does not include field names, which causes the issue. However, I am not sure how to handle this.

| username: TiDBer_QHSxuEa1 | Original post link

There is an additional field downstream, and it seems that it cannot follow the data source table structure like DM. So I would like to ask if anyone has any solutions for this kind of issue.

| username: TiDBer_QHSxuEa1 | Original post link

Thank you, it’s resolved. I didn’t notice before that Dumpling has this parameter.

| username: 昵称想不起来了 | Original post link

Which parameter was it, boss? I couldn’t seem to find it. Could you please share it?

| username: TiDBer_QHSxuEa1 | Original post link

The parameter --complete-insert can add field names to the insert into statements in the SQL file exported by dumpling.

| username: 昵称想不起来了 | Original post link

Okay, thank you~ (Although I still didn’t find it in the Dumpling documentation, I’ll try again later)

| username: TiDBer_QHSxuEa1 | Original post link

The official documentation does not introduce this parameter. You can find it directly by typing the command ./dumpling --help.

| username: 昵称想不起来了 | Original post link

Alright~ Thank you, boss.

| username: zhanggame1 | Original post link

Learned.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.