JavaClient KvClient scan

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: JavaClient KvClient scan

| username: T-TRz879

No matter how you use the method public List<KvPair> scan(ByteString startKey, long version, int limit), it will always throw an error:

org.tikv.common.exception.TiClientInternalException: Error scanning data from region.

	at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:113)
	at org.tikv.common.operation.iterator.ConcreteScanIterator.hasNext(ConcreteScanIterator.java:106)
	at java.util.Iterator.forEachRemaining(Iterator.java:115)
	at org.tikv.txn.KVClient.scan(KVClient.java:134)
	at fs.server.tikv.TikvScanTest.testTIKVTxn3(TikvScanTest.java:313)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.lang.IndexOutOfBoundsException: current cache size = 4, larger than 2
	at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:100)
	... 28 more
2
| username: xfworld | Original post link

IndexOutOfBoundsException, the index is out of bounds. Is there not that much data?

Which version of the client are you using?

3
| username: T-TRz879 | Original post link

There are 6 pieces of data, but I only want 4. According to the method, it means taking Math.min(limit, 10240) starting from the startKey.

4
| username: T-TRz879 | Original post link

The client is KvClient version 3.3.0, and I am using Txn to store data.

5
| username: xfworld | Original post link

The startKey should be recorded. If a range scan is performed, both the startKey and endKey must be known.

image
image751×706 29.4 KB

Additionally, the startKey of the region can also be obtained through the local cache.

image
image717×944 40.6 KB

I feel like it’s out of range…
You can use PD-CLI to check the region and the startKey and endKey related to the region to verify if the request is reasonable.

6
| username: T-TRz879 | Original post link

Thank you for your guidance. I used to use the RawClient with a lower version, but this time I upgraded the version and need to use Txn transactions for control. I will go and study it.

7
| username: T-TRz879 | Original post link

I just checked, and it is like this. I called the method scan(ByteString startKey, long version, int limit), and I want to get the data starting from startKey with a limit of limit. Isn’t this method supposed to be used this way?

8
| username: xfworld | Original post link

I suggest you take a look at this:
https://tikv.github.io/client-java/examples/txnkv.html

Try running it according to the example first.

9
| username: T-TRz879 | Original post link

I have seen this before. The scan here includes startKey and endKey. KVClient provides three types of scans:

  1. startKey, endKey
  2. startKey, limit
  3. startKey
10
| username: xfworld | Original post link

Here’s the source code for you. All of them have at least two parameters and cannot be omitted.

At the very least, you need to get the version from PD.

11
| username: T-TRz879 | Original post link

All three provided methods require passing in startKey and version, and I have passed them. If you modify the scan in the official test case to not specify endKey but specify limit, it will report an error regardless of what limit is passed.

12
| username: xfworld | Original post link

Well, that’s it. We need to check the source code to see if there are any bugs. :rofl:

13
| username: T-TRz879 | Original post link

:sweat_smile: Hahaha, let me see if I can raise an issue.

14
| username: system | Original post link

This topic was automatically closed 1 minute after the last reply. No new replies are allowed.