Tidb-lightning Data Import Failed error="batch split regions failed: split region failed

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb-lightning 导入数据失败 error="batch split regions failed: split region failed

| username: xihong08

[TiDB Usage Environment] Production\Test Environment\POC
[TiDB Version] 6.2
[Encountered Problem]
Failed to import CSV format data using tidb-lightning, error log as follows
[Problem Phenomenon and Impact]
[Attachment]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IBZmY5YWFmY2L/NDgzMGIyMDX/MjgwOTgwYzn/MzU2YjU3ZmP/AAAAAAAAAAD3] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IBZmZjMzNmMGT/M2I0NjNiZWL/Zjk3N2EwMzf/MGJiNjRjMjj/AAAAAAAAAAD3] [encodedKey=dIAAAAAAAAD/Sl9yAWZmYzP/M2YwZP8zYjT/NjNiZWL/Zjn/NzdhMDM3/zD/YmI2NGMyOP//AAAAAAAAAAD/9wAAAAAAAAD4] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IBZmZjMzNmMGT/M2I0NjNiZWL/Zjk3N2EwMzf/MGJiNjRjMjj/AAAAAAAAAAD3] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IBZmZlYzE0YWX/ZTI2NTA1ZDb/MDEzZTdiZDj/OGM3MTU3ZjH/AAAAAAAAAAD3] [encodedKey=dIAAAAAAAAD/Sl9yAWZmZWP/MTRhZf9lMjb/NTA1ZDb/MDH/M2U3YmQ4/zj/YzcxNTdmMf//AAAAAAAAAAD/9wAAAAAAAAD4] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IBZmZlYzE0YWX/ZTI2NTA1ZDb/MDEzZTdiZDj/OGM3MTU3ZjH/AAAAAAAAAAD3] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IB5LmQ5bGx5aT/p+S9m+aXheb/uLjmlLvnlaX/77yM5rOo5oT/j+S6i+mhue//vIzlh7rmuLj/5YmN5LiA5a7/muimgeaPkOX/iY3lh4blpIf/5pS755Wl77z/jOWIq+iiq+X/v73mgqDkuob/AAAAAAAAAAD3] [encodedKey=dIAAAAAAAAD/Sl9yAeS5kOX/sbHlpP+n5L3/m+aXheb/uLj/5pS755Wl/+//vIzms6jmhP//j+S6i+mhue///7yM5Ye65rj/uP/liY3kuID/5a7/muimgeb/j5Dl/4mN5Yf/huWkh//mlLv/55Wl77z/jOX/iKvooqvl/7//veaCoOS6hv//AAAAAAAAAAD/9wAAAAAAAAD4] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IB5LmQ5bGx5aT/p+S9m+aXheb/uLjmlLvnlaX/77yM5rOo5oT/j+S6i+mhue//vIzlh7rmuLj/5YmN5LiA5a7/muimgeaPkOX/iY3lh4blpIf/5pS755Wl77z/jOWIq+iiq+X/v73mgqDkuob/AAAAAAAAAAD3] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IB5paw6bKc55r/hOajo+ajoyD/IOKAiyIAAAD8] [encodedKey=dIAAAAAAAAD/Sl9yAeaWsOn/spznmv+E5qP/o+ajoyD/IOL/gIsiAAAA/AD+] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IB5paw6bKc55r/hOajo+ajoyD/IOKAiyIAAAD8] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IB6LSf6LSj5Lv/u+OAgeacieb/i4XlvZPvvJ//AAAAAAAAAAD3] [encodedKey=dIAAAAAAAAD/Sl9yAei0n+j/tKPku/+744D/geacieb/i4X/5b2T77yf/wD/AAAAAAAAAPf/AAAAAAAAAAD3] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IB6LSf6LSj5Lv/u+OAgeacieb/i4XlvZPvvJ//AAAAAAAAAAD3] [startKey=] [endKey=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:551] [“need split”] [splitKey=dIAAAAAAAABKX3IB8KCDiui/mOf/qoHnhLbooqv/UeaXpeWQjuj/p4TliJLnmoT/5oiRIOKIkSj/776f0JTvvp//KSAg4oCLIgD/] [encodedKey=dIAAAAAAAAD/Sl9yAfCgg4r/6L+Y5/+qgef/hLbooqv/Ueb/l6XlkI7o/6f/hOWIkueahP//5oiRIOKIkSj//+++n9CU777/n/8pICDigIv/IgD/AAAAAAD6] [“region start”=] [“region end”=]
[2022/09/28 14:10:11.182 +08:00] [DEBUG] [localhelper.go:528] [“get key for split region”] [key=dIAAAAAAAABKX3IB8KCDiui/mOf/qoHnhLbooqv/UeaXpeWQjuj/p4TliJLnmoT/5oiRIOKIkSj/776f0JTvvp//KSAg4oCLIgD/] [startKey=] [endKey=]
[2022/09/28 14:10:11.187 +08:00] [WARN] [split_client.go:351] [“fail to split region”] [region=“{ID=2,startKey=endKey=epoch="conf_ver:23 version:46 ",peers="id:3 store_id:1 ,id:277 store_id:5 ,id:278 store_id:7 "}”] [regionErr="message:"EpochNotMatch [region 2] 3 epoch changed conf_ver: 23 version: 48 != conf_ver: 23 version: 46, retry later" epoch_not_match:<current_regions:<id:2 start_key:"t\200\000\000\000\000\000\000\377J_r\001\360\240\203\212\377\350\277\230\347\377\252\201\347\377\204\266\350\242\253\377Q\346\377\227\245\345\220\216\350\377\247\377\204\345\210\222\347\232\204\377\377\346\210\221 \342\210\221(\377\377\357\276\237\320\224\357\276\377\237\377) \342\200\213\377\"\000\377\000\000\000\000\000\372" region_epoch:<conf_ver:23 version:48 > peers:<id:3 store_id:1 > peers:<id:277 store_id:5 > peers:<id:278 store_id:7 > > > "]
[2022/09/28 14:10:11.188 +08:00] [WARN] [localhelper.go:255] [“split regions”] [error=“batch split regions failed: split region failed: err=message:"EpochNotMatch [region 2] 3 epoch changed conf_ver: 23 version: 48 != conf_ver: 23 version: 46, retry later" epoch_not_match:<current_regions:<id:2 start_key:"t\200\000\000\000\000\000\000\377J_r\001\360\240\203\212\377\350\277\230\347\377\252\201\347\377\204\266\350\242\253\377Q\346\377\227\245\345\220\216\350\377\247\377\204\345\210\222\347\232\204\377\377\346\210\221 \342\210\221(\377\377\357\276\237\320\224\357\276\377\237\377) \342\200\213\377\"\000\377\000\000\000\000\000\372" region_epoch:<conf_ver:23 version:48 > peers:<id:3 store_id:1 > peers:<id:277 store_id:5 > peers:<id:278 store_id:7 > > > : [BR:Restore:ErrRestoreSplitFailed]fail to split region”] [“retry time”=0] [region_id=2]
[2022/09/28 14:10:11.190 +08:00] [INFO] [table_restore.go:350] [“import whole table completed”] [table=contents_db.contents] [takeTime=7m26.892521307s]
[2022/09/28 14:10:11.190 +08:00] [INFO] [backend.go:451] [“engine close start”] [engineTag=contents_db.contents:-1] [engineUUID=a773bf64-2023-5b59-9133-1c8dffa52432]

| username: tidb狂热爱好者 | Original post link

Use mysql < 1.sql to import

| username: 数据小黑 | Original post link

Using CSV as an intermediate medium for data import can encounter various issues. It is recommended to use Spark to read data from HBase and directly import it into TiDB through TiSpark. Some reference materials for Spark: 关于TiSpark的几种使用方式以及常见错误 - TiDB 的问答社区