Warning When Scanning Data with tikc-ctl

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikc-ctl 扫描数据的时候警告

| username: 我家有个臭皮崽

The command is as follows: ./tikv-ctl --data-dir ./import/tikv1 scan --from “z”
When using tikv-ctl to scan SST files, there will be such a warning:
[config.rs:712] [“compaction guard is disabled due to region info provider not available”]
Then the data cannot be scanned out.

| username: Fly-bird | Original post link

Are all TiKV nodes in a normal state?

| username: xfworld | Original post link

It seems that the issue is caused by improper configuration of TiKV. It is recommended to provide additional cluster-related configurations, including but not limited to hardware, software versions, detailed configurations, etc.

| username: 我家有个臭皮崽 | Original post link

The TiKV node status is normal.

| username: 我家有个臭皮崽 | Original post link

The default configuration for TiKV:

nohup ./pd-server -name=pd1 -data-dir=./import/pd1 -client-urls="http://127.0.0.1:12379" -peer-urls="http://127.0.0.1:12380" -log-file=./import/pd1.log &
echo "start pd done"
sleep 2
nohup ./tikv-server --pd-endpoints="127.0.0.1:12379" --addr="127.0.0.1:20161" --data-dir=./import/tikv1 --log-file=./import/tikv1.log &
nohup ./tikv-server --pd-endpoints="127.0.0.1:12379" --addr="127.0.0.1:20162" --data-dir=./import/tikv2 --log-file=./import/tikv2.log &
nohup ./tikv-server --pd-endpoints="127.0.0.1:12379" --addr="127.0.0.1:20163" --data-dir=./import/tikv3 --log-file=./import/tikv3.log &

For more details, visit: https://tikv.org/docs/6.5/deploy/install/test/

| username: ajin0514 | Original post link

Check if the status of all nodes is normal.

| username: 我家有个臭皮崽 | Original post link

From the process perspective, all nodes, including pd-server and tikv-server, are functioning normally.

| username: xfworld | Original post link

Okay, standard manual operation, directly entering hard mode… :melting_face: :melting_face: :melting_face:

I recommend using PD CLI to scan the region information via the command line to see if it matches your operations.

If the region information in PD does not match, it indicates there might be an issue with your operations, and you may need to clean all regions and start over.
If the region information in PD matches, you can use this command line to scan the keys to see if they meet your expectations. If not, you might need to consider cleaning the data and reloading it (refer more to the official TiKV mode demo).

| username: 我家有个臭皮崽 | Original post link

Ok, I’ll give it a try.

| username: 我家有个臭皮崽 | Original post link

After writing data, there is no problem querying through pd-ctl, but there are still the following warnings when scanning through tikv-ctl:

[root@VM-210-105-tikv]# ./tikv-ctl --data-dir ./import/tikv1 scan --from "z"
[2023/10/07 14:27:37.047 +08:00] [WARN] [config.rs:604] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:37.047 +08:00] [WARN] [config.rs:712] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:37.115 +08:00] [WARN] [pipe_builder.rs:273] ["The last log file is corrupted but ignored: Append:1, Corruption: Log item offset is smaller than log batch header length"]
please pass "to" or "limit"
[root@VM-210-105-tikv]# ./tikv-ctl --data-dir ./import/tikv2 scan --from "z"
[2023/10/07 14:27:52.521 +08:00] [WARN] [config.rs:604] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:52.521 +08:00] [WARN] [config.rs:712] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:52.577 +08:00] [WARN] [pipe_builder.rs:273] ["The last log file is corrupted but ignored: Append:1, Corruption: Log item offset is smaller than log batch header length"]
please pass "to" or "limit"
[root@VM-210-105-tikv]# ./tikv-ctl --data-dir ./import/tikv3 scan --from "z"
[2023/10/07 14:27:56.767 +08:00] [WARN] [config.rs:604] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:56.767 +08:00] [WARN] [config.rs:712] ["compaction guard is disabled due to region info provider not available"]
[2023/10/07 14:27:56.828 +08:00] [WARN] [pipe_builder.rs:273] ["The last log file is corrupted but ignored: Append:1, Corruption: Log item offset is smaller than log batch header length"]
please pass "to" or "limit"

And no data is scanned out. After re-splitting, the data written still cannot be found through the scan.

| username: 我家有个臭皮崽 | Original post link

This is the result of scanning SST files under TiKV.

| username: xfworld | Original post link

PD can recognize Regions and Keys to ensure that TiKV can identify them.

Maybe the usage is not quite right. It’s best to refer to the official documentation for operations and handling…

| username: 我家有个臭皮崽 | Original post link

Is there a WeChat group, or can I add you on WeChat? Solving problems this way is too slow.

| username: xfworld | Original post link

Please add billmay on WeChat @Billmay’s cousin, she will arrange it for you.

| username: Billmay表妹 | Original post link

There is a community group, but community assistance is based on mutual help among community members. If you want to ensure the efficiency of problem-solving, you might need commercial support.

| username: Billmay表妹 | Original post link

When using tikv-ctl to scan SST files, a warning [config.rs:712] ["compaction guard is disabled due to region info provider not available"] appears. This is because the tikv-ctl tool cannot obtain the correct Region information.

To resolve this issue, you can try the following methods:

  1. Ensure TiKV nodes are running normally: Check if the TiKV nodes are properly started and running. You can use the pd-ctl command to check the status of TiKV nodes, such as pd-ctl -u <pd-address> store.

  2. Check if the PD address is correct: Ensure that the PD address used when executing the tikv-ctl command is correct. You can obtain the correct PD address through the pd-ctl command or the TiDB Dashboard interface.

  3. Check the TiKV configuration file: Inspect the TiKV configuration file tikv.toml to ensure that the PD address is configured correctly. You can modify the configuration file and restart the TiKV process to apply the changes.

  4. Use the latest version of the tikv-ctl tool: Make sure you are using the latest version of the tikv-ctl tool. You can try updating the TiKV version or recompiling the tikv-ctl tool.

If the above methods still do not resolve the issue, it is recommended to refer to the official TiDB documentation.

| username: 我家有个臭皮崽 | Original post link

The issue has been resolved. It turns out the data was written into writecf. I was using raw KV mode, so I couldn’t find it. Thank you, everyone!!!

| username: 我家有个臭皮崽 | Original post link

It’s already added, thank you. The problem has been solved. It turns out that the data was written into writecf. I was using raw kv mode, so I couldn’t find it. Thanks, everyone!!!

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.