Inconsistent Query Results Between TiKV and TiFlash

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: TiKV和TiFlash查询结果不一致

| username: TiDBer_VxIYCcXM

[TiDB Usage Environment] Production Environment
[TiDB Version] v5.4.1
[Reproduction Path] Honestly, not sure how it happened
[Encountered Problem: Symptoms and Impact]
In short, the data queried from TiKV and TiFlash is inconsistent.

Specifically:

  1. The following SQL query returns three rows using the TiFlash engine.

  2. Using coupe_code as the query condition, only 2 rows are returned, missing the one with id 2042783592.

    From the execution plan, it shows that the data is queried using TiKV.
  3. Directly querying the record with id 2042783592
    The execution plan shows rocksdb’s delete_skipped_count as 1 and key_skipped_count as 1.
    So, does this mean that this data in TiKV has been deleted and thus cannot be found?
  4. Trying different engines
    The same SQL, using Hint to force TiKV engine, returns only 2 records, while TiFlash returns 3 records. So, is this a bug?
    image
  5. Other attempts
    Checked the TIFLASH_REPLICA table and found no issues.

Checked Github issues and found no similar issues.
Checked the community, this TiFlash数据不一致 - TiDB 的问答社区 is somewhat similar but not the same.

Request: Please help confirm if this is a bug, thank you.

[Resource Configuration]

  • 3 PD: Total 24 CPU, 48G memory
  • 8 TiDB: Total 148 CPU, 1 TB+ memory
  • 24 TiKV: Total 168 CPU, 3 TB+ memory
  • 2 TiFlash: Total 56 CPU, 1TB+ memory

[Attachments: Screenshots/Logs/Monitoring]

| username: 江湖故人 | Original post link

I have seen this issue before.
Try adding the TiFlash replica to this table again and see if it works. If not, you may need to upgrade.

| username: tidb菜鸟一只 | Original post link

It’s normal to have bugs when adding partition tables to TiFlash, especially since the version is still 5.4.

| username: caiyfc | Original post link

Check this table to see if there are any issues:
ADMIN CHECK [TABLE|INDEX] | PingCAP Documentation Center

| username: TiDBer_VxIYCcXM | Original post link

It might really be a bug. TiFlash Batch Deletion Scenario May Cause Data Inconsistency | TiDB Books

| username: IanWong | Original post link

Your situation is very similar to the one in the link above. Why not test it according to the link posted above?

| username: zhang_2023 | Original post link

It looks like it was caused by a bug.

| username: TiDBer_RjzUpGDL | Original post link

Upgrade it.

| username: TiDBer_aaO4sU46 | Original post link

Report a bug.

| username: YuchongXU | Original post link

Suggest opening a bug.

| username: 随便改个用户名 | Original post link

Someone mentioned a similar issue before.

| username: WinterLiu | Original post link

After reading everyone’s analysis, it feels like a version bug.

| username: zhaokede | Original post link

I haven’t encountered this before. Could it be that the synchronization is not complete?

| username: IanWong | Original post link

There is a bug in the lower version, it is recommended to upgrade the version.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.