Exporting from Hive to TiDB using Sqoop: Newly added fields through ALTER in TiDB are NULL

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: sqoop 从hive导出到tidb,tidb建表后通过alter新增的字段为null

| username: baofengyu

[TiDB Usage Environment] Production Environment
[TiDB Version] v6.5.0
[Reproduction Path] After creating a table in TiDB, add a new column through ALTER, and use Sqoop to export from Hive to TiDB.
[Encountered Problem: Symptoms and Impact]
After exporting with Sqoop, the fields created during table creation in TiDB have values, but the newly added fields are null.
[Expected Problem to be Solved]

  1. What causes this issue?
| username: Billmay表妹 | Original post link

The issue might be caused by the following reasons:

  1. Data type mismatch: In Hive, the data type of a certain field might not match the corresponding field’s data type in TiDB. When Sqoop exports data from Hive to TiDB, data type conversion might occur, resulting in the new field’s value being NULL. Please ensure that the data type of the new field in Hive matches the corresponding field’s data type in TiDB.
  2. Data missing: Before exporting data, ensure that the new field in the Hive table has valid data. If the new field has no value in the Hive table, its value will be NULL when exported to TiDB.
  3. Sqoop configuration issue: Ensure that you have correctly configured field mapping and data conversion rules when exporting data with Sqoop to ensure that the new field’s data can be correctly exported to TiDB.

To better assist you in resolving the issue, I suggest you check the above reasons and ensure data type matching, data integrity, and correct Sqoop configuration. If the issue persists, please provide more detailed information, such as table structure, Sqoop export command, and data samples, so that we can better understand and analyze the problem.