What are the recommendations or guidelines for data migration operations?

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 数据迁移操作有什么建议或者指引？

| username: TiDBer_0FQv9uCC

[TiDB Usage Environment] Production Environment
[TiDB Version]: Server version: 5.7.25-TiDB-v5.3.0
[Reproduction Path] create table t2 like t1; insert into t2 select * from t1;
[Encountered Problem: Problem Phenomenon and Impact]: t1 is a table with millions of records. During data migration, it is observed that memory consumption is very high, directly leading to memory overflow.

2
| username: zhanggame1 | Original post link

A million-level is not big at all, just change the parameters to 8g. At the same time, use the system variable tidb_mem_quota_query to configure the memory usage threshold during the execution of a single SQL query.
https://docs.pingcap.com/zh/tidb/stable/configure-memory-usage#如何配置一条-sql-执行过程中的内存使用阈值

3
| username: zhanggame1 | Original post link

If the table is at the tens of millions level, add a WHERE condition to insert in batches
or use batch non-transactional inserts

4
| username: Kongdom | Original post link

Just refer to the official guide:

5
| username: 昵称想不起来了 | Original post link

You can refer to the TiDB Memory Control Documentation to try to alleviate the OOM issue.

6
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.