Error scanning data from region when querying data in TiKV

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv查询数据时出现Error scanning data from region

| username: TIDB救我狗命

[TiDB Usage Environment] Production Environment / Test / PoC
[TiDB Version] 6.5.3
[Reproduction Path] Querying data using TiKV
[Encountered Problem: Problem Phenomenon and Impact] Encountered “Error scanning data from region”
This issue occurs occasionally; sometimes it appears, and sometimes it works normally.

| username: TIDB救我狗命 | Original post link

The image link you provided appears to be broken or inaccessible. Please provide the text you need translated.

| username: redgame | Original post link

When monitoring shows this period: Are resources tight? Is there a network issue?

| username: 大飞哥online | Original post link

Observe the monitoring at the time when the exception occurred and check if there are any anomalies in various indicators such as TiKV, TiDB, memory, disk IO, etc.

| username: TIDB救我狗命 | Original post link

Currently, this issue has not reoccurred. If it happens again in the future, I will get the specific monitoring logs from the operations team.

| username: TIDB救我狗命 | Original post link

This situation has occurred again. Some tables are abnormal.

| username: TIDB救我狗命 | Original post link

All metrics are normal.

| username: TIDB救我狗命 | Original post link

I have placed the images of the metrics below. They should all be normal.

| username: TIDB救我狗命 | Original post link

Here is the complete error log

| username: 大飞哥online | Original post link

This is from before, right? Looking at the store_id, it was sent at the beginning.

| username: 大飞哥online | Original post link

Let’s check the record corresponding to this store_id.

| username: 像风一样的男子 | Original post link

Use pd-ctl to check the detailed information of this error region.

| username: TIDB救我狗命 | Original post link

This is the complete error log, but it is not the latest error log.

| username: TIDB救我狗命 | Original post link

I checked the schema information of the latest error regionId.

| username: TIDB救我狗命 | Original post link

The regionId is already gone, I need to wait for the issue to reappear before I can check it.

| username: TIDB救我狗命 | Original post link

The problem has occurred again, and I found that the log contains store_id instead of region_id. How should I troubleshoot this issue…

| username: TIDB救我狗命 | Original post link

The image you provided is not visible. Please provide the text you need translated.

| username: TIDB救我狗命 | Original post link

I found the region_id of the abnormal table and executed pd-ctl. The result is shown in the following image:

| username: TIDB救我狗命 | Original post link

Is it because of the leader node switch… The store_id held by the client is 171615401, but the server has already switched to 11979412891?

| username: TIDB救我狗命 | Original post link

The latest situation has been posted below, you can take a look.