As long as the cardinality of the grouped aggregate field data is large enough, it can be written to disk. If the memory allocated is too small, it might be used up by other processes before reaching the grouped aggregation, causing an OOM-killer. For example, the memory occupied by the data cached from TiDB might trigger the session to be killed. As you can see here, it can be written to disk, but note that writing to disk will significantly slow down performance:
mysql> explain analyze select count(*) from orders group by o_custkey;
+---------------------------+-------------+----------+-----------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------+----------+---------+
| id | estRows | actRows | task | access object | execution info | operator info | memory | disk |
+---------------------------+-------------+----------+-----------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------+----------+---------+
| HashAgg_9 | 4008704.67 | 2999660 | root | | time:10m36.6s, loops:2931 | group by:tpch.orders.o_custkey, funcs:count(Column#11)->Column#10 | 395.5 MB | 10.7 GB |
| └─TableReader_10 | 4008704.67 | 37114427 | root | | time:77.6ms, loops:1215, cop_task: {num: 1214, max: 36.6ms, min: 554.7µs, avg: 12.4ms, p95: 23.1ms, max_proc_keys: 51200, p95_proc_keys: 51200, tot_proc: 13.1s, tot_wait: 42ms, rpc_num: 1214, rpc_time: 15s, copr_cache: disabled, build_task_duration: 58.1µs, max_distsql_concurrency: 15} | data:HashAgg_5 | 12.4 MB | N/A |
| └─HashAgg_5 | 4008704.67 | 37114427 | cop[tikv] | | tikv_task:{proc max:34ms, min:0s, avg: 11ms, p80:18ms, p95:21ms, iters:36586, tasks:1214}, scan_detail: {total_process_keys: 37438464, total_process_keys_size: 5686354813, total_keys: 37439678, get_snapshot_time: 66.8ms, rocksdb: {key_skipped_count: 37438464, block: {cache_hit_count: 199034}}} | group by:tpch.orders.o_custkey, funcs:count(1)->Column#11 | N/A | N/A |
| └─TableFullScan_8 | 37327872.00 | 37438464 | cop[tikv] | table:orders | tikv_task:{proc max:29ms, min:0s, avg: 8.75ms, p80:15ms, p95:17ms, iters:36586, tasks:1214} | keep order:false | N/A | N/A |
+---------------------------+-------------+----------+-----------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------+----------+---------+
4 rows in set (10 min 38.10 sec)