GC life time is shorter than transaction duration, GC duration is shorter than table analysis duration

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: GC life time is shorter than transaction duration ，GC时长短于表分析时长

| username: TI表弟

[TiDB Usage Environment] Production Environment
[TiDB Version] V6.1.2
[Reproduction Path]
[Encountered Problem: Problem Phenomenon and Impact]
There is an error message in the log

r\": null\n}"]
[2023/01/12 11:34:32.551 +08:00] [ERROR] [update.go:1218] ["[stats] auto analyze failed"] [sql="analyze table `remote_vehicle_prod`.`history_wti_alarm` partition `p1`, `p2`, `p3`, `p4`, `p5`, `p6`, `p7`, `p8`, `p9`, `p10`, `p11`, `p12`, `p13`, `p14`, `p15`, `p16`, `p17`, `p18`, `p19`, `p20`, `p21`, `p22`, `p23`, `p24`, `p25`, `p26`, `p27`, `p28`, `p29`, `p30`, `p31`, `p32`, `p33`, `p34`, `p35`, `p36`, `p37`, `p38`, `p39`, `p40`, `p41`, `p42`, `p43`, `p44`, `p45`, `p46`, `p47`, `p48`, `p49`, `p50`"] [cost_time=20m3.380803402s] [error="[tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2023-01-12 11:14:29.123 +0800 CST, GC safe point is 2023-01-12 11:22:09.173 +0800 CST"]

[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]

image
image1380×452 125 KB

2
| username: WalterWj | Original post link

Increase the GC time?

3
| username: 裤衩儿飞上天 | Original post link
  1. If it is a super large table, you can manually analyze it periodically;
  2. You can try enabling tidb_enable_fast_analyze;
  3. You can appropriately increase the GC time.
4
| username: tidb菜鸟一只 | Original post link

It is recommended to deploy manual tasks for large tables (outside of business hours, small sample collection, multi-threaded) for collection. Otherwise, no matter how long you adjust the GC, as long as the table is large enough, it will still time out…

5
| username: tidb狂热爱好者 | Original post link

tidb_enable_fast_analyze or disable auto analyze and do it manually.
It will be much faster for a single partition
analyze table t partition p1;

Automatic Update

When insert, delete, and update statements occur, TiDB will automatically update the total number of rows in the table and the number of modified rows.

System Variable Name Default Value Function
tidb_auto_analyze_ratio 0.5 Auto update threshold
tidb_auto_analyze_start_time 00:00 +0000 Start time for auto update during the day
tidb_auto_analyze_end_time 00:59 +0000 End time for auto update during the day
6
| username: tidb狂热爱好者 | Original post link
tidb_auto_analyze_ratio 0.0 Automatic update threshold
tidb_auto_analyze_start_time 00:00 +0000 Start time for automatic updates during the day
tidb_auto_analyze_end_time 08:59 +0000 End time for automatic updates during the day
Generally, table statistics are updated when there are fewer users online, such as from midnight to 8 AM. No updates are performed during the day.
7
| username: h5n1 | Original post link

6.1 isn’t it optimized for analyze and gc, with gc time adjusted according to analyze, and analyze has a max time?

8
| username: dba-kit | Original post link

You can look forward to the actual effect of this parameter introduced in version 6.5. Currently, analyzing partitioned tables is done partition by partition, and the default 10 minutes is easily exceeded. It is still recommended to increase the tidb_gc_life_time parameter. I have set it to 6 hours, so if the business does something foolish, there is still room for recovery. At present, it does not have much impact on query efficiency.

9
| username: dba-kit | Original post link

PS: If you have used the dynamic partition pruning feature, be sure to remind the business whether they have encountered this bug.

10
| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.