What are the best practices for using TiDB as a data warehouse and performing ETL with KETTLE?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 将 TiDB 作为数仓，使用 KETTEL 进行 ETL ，有哪些最佳实践？

| username: 望海崖2084

Our company uses TiDB as a data warehouse, but a colleague mentioned that when using KETTLE for batch data import, locks significantly affect the efficiency of data import. Currently, we can only import data one by one. I would like to ask, how should TiDB be configured to best handle this type of issue? Are there any best practices?

2
| username: 数据小黑 | Original post link

Generally, specific problems require specific analysis. The best implementation in a broad sense also has its limitations. Analyze your current situation. What kind of lock are you referring to that affects the ETL process? How is the ETL process currently organized?

3
| username: wfxxh | Original post link

Importing one by one? What is your data source?

4
| username: xfworld | Original post link

KETTEL can also perform batch inserts, and KETTEL handles this through JDBC.

Is it possible that KETTEL is not configured properly?
It doesn’t have much to do with TiDB.

5
| username: Billmay表妹 | Original post link

Here are some recommended articles related to data warehousing that you can check out:

6
| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.