TiDB Dashboard Traffic Graph Hotspot is a Dropped Table

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb dashborad流量图 热点是已经drop的表

| username: foxchan

[TiDB Usage Environment] Production Environment
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Issue Phenomenon and Impact]
The hotspot table in the traffic graph was dropped at 09:54, but it still shows up.

(user:tidbdba time: 09:54)[db: yixintui_operate]drop table yixintui_operate_arch.ad_manage_log_20230411 ;
Query OK, 0 rows affected (0.52 sec)
image

The region can be found through PD, but the table name cannot be found.

Suspecting if it is affected by gc lifetime.
image

| username: tidb菜鸟一只 | Original post link

Drop/Truncate Table and Drop Index will first write the Ranges into the TiDB system table (mysql.gc_delete_range). TiDB’s GC worker periodically checks if the Safepoint has passed, then takes out these Ranges and concurrently sends them to TiKV to delete the SST files. The concurrency is not related to the concurrency setting but is directly sent to each TiKV. The deletion is done directly without waiting for compaction. After completing the Delete Ranges, it will be recorded in the TiDB system table mysql.gc_delete_range_done, and the content in the table will be cleared after 24 hours:

mysql> select * from gc_delete_range_done;
Check if it has been GC’d.

| username: foxchan | Original post link

The GC lifetime is set to 16 hours, so it hasn’t reached the time yet. I don’t know what is causing the hotspot due to frequent reads. Or maybe it’s the TiDB GC worker periodically checking if the Safepoint has passed, but this frequency is too high.

| username: tidb狂热爱好者 | Original post link

Just ignore it and wait for the data to update.

| username: 胡杨树旁 | Original post link

I saw in the documentation that you can use SQL to query the TiKV addresses of the top 3 Regions with the largest WRITTEN_BYTES:

SELECT
  address,
  tikv.address,
  region.region_id
FROM
  TIKV_STORE_STATUS tikv,
  TIKV_REGION_PEERS peer,
  (SELECT * FROM tikv_region_status ORDER BY written_bytes DESC LIMIT 3) region
WHERE
  region.region_id = peer.region_id
  AND peer.is_leader = 1
  AND peer.store_id = tikv.store_id;

Can these tables also be used to query read hotspots?

| username: foxchan | Original post link

After the GC expires, the hotspot is gone.
Why does the table have a hotspot after being dropped, and will it affect cluster performance?

| username: tidb菜鸟一只 | Original post link

You can check the setting of the tidb_gc_run_interval parameter. This variable is used to specify the interval at which garbage collection (GC) runs, and it can be set to be slightly larger.

| username: foxchan | Original post link

It’s still the default value. I’ll increase it and observe further.

| username: knull | Original post link

Hello, I would like to know what version of TiDB you are using.
Additionally, can you provide the DDL information on the DDL job?
Also, could you provide a more complete screenshot of the heatmap, including the period before the drop?

| username: foxchan | Original post link

tidb: 6.1.5
The heatmap disappeared after a long time.
After the GC time passed, the heatmap returned to normal.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.