Help: Error Listing Keys with a Certain Prefix - tikv aborts txn: Error(Txn(Error(Codec(KeyPadding)))

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 求助:列出某前缀的 keys 时报错 tikv aborts txn: Error(Txn(Error(Codec(KeyPadding))))

| username: TiDBer_QBsB7GOW

Bug Report
Clearly and accurately describe the issue you found. Providing any steps to reproduce the issue can help the development team address it promptly.
[TiKV Version]
TiKV 7.4.0 (not using TiDB)
[Impact of the Bug]
Unable to list keys with the specified prefix
[Possible Steps to Reproduce]
Uncertain how to reproduce
(If possible, could someone explain what this error means? I’ve searched for a long time but couldn’t find relevant information)
[Observed Unexpected Behavior]
Listing failed with the following error:
[errorVerbose=“tikv aborts txn: Error(Txn(Error(Codec(KeyPadding))))\ngithub.com/tikv/client-go/v2/error.ExtractKeyErr\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/error/error.go:301\ngithub.com/tikv/client-go/v2/txnkv/txnlock.ExtractLockFromKeyErr\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/txnlock/lock.go:27\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.(*Scanner).getData\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/txnsnapshot/scan.go:291\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.(*Scanner).Next\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/txnsnapshot/scan.go:138\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.newScanner\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/txnsnapshot/scan.go:86\ngithub.com/tikv/client-go/v2/txnkv/txnsnapshot.(*KVSnapshot).Iter\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/txnsnapshot/snapshot.go:754\ngithub.com/tikv/client-go/v2/internal/unionstore.(*KVUnionStore).Iter\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/internal/unionstore/union_store.go:119\ngithub.com/tikv/client-go/v2/txnkv/transaction.(*KVTxn).Iter\n\t/Users/apap/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.7/txnkv/transaction/txn.go:241\ngithub.com/iaff/starstor/core/core/tikv.(*TikvStore).ListPrefixedObjects.func1\n\t/Users/apap/starstor/uttc/core/tikv/tikv_store.go:433\ngithub.com/iaff/starstor/core/core/tikv.(*TxnWrapper).RunInTxn\n\t/Users/apap/starstor/uttc/core/tikv/tikv_store.go:548\ngithub.com/iaff/starstor/core/core/tikv.(*TikvStore).ListPrefixedObjects\n\t/Users/apap/starstor/uttc/core/tikv/tikv_store.go:432\ngithub.com/iaff/starstor/core/core.(*CoreStoreWrapper).ListPrefixedObjects\n\t/Users/apap/starstor/uttc/core/corestore_wrapper.go:377\ngithub.com/iaff/starstor/core/core.(*Core).runListDirectoryObjects\n\t/Users/apap/starstor/uttc/core/core.go:327\ngithub.com/iaff/starstor/core/core.(*Core).runListValidObjects\n\t/Users/apap/starstor/uttc/core/core_search.go:101\ngithub.com/iaff/starstor/core/core.(*Core).runListPatternMatchedObjects\n\t/Users/apap/starstor/uttc/core/core_search.go:70\ngithub.com/iaff/starstor/core/core.(*Core).StreamListDirectoryObjects\n\t/Users/apap/starstor/uttc/core/core_search.go:58\ngithub.com/iaff/starstor/core/server.(*CoreServer).ListObjects\n\t/Users/apap/starstor/uttc/server/core_grpc_server.go:60\ngithub.com/iaff/starstor/core/pb/core_pb._StarStorCore_ListObjects_Handler\n\t/Users/apap/starstor/uttc/pb/core_pb/core_grpc.pb.go:454\ngoogle.golang.org/grpc.(*Server).processStreamingRPC\n\t/Users/apap/go/pkg/mod/google.golang.org/grpc@v1.56.2/server.go:1631\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/Users/apap/go/pkg/mod/google.golang.org/grpc@v1.56.2/server.go:1718\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/Users/apap/go/pkg/mod/google.golang.org/grpc@v1.56.2/server.go:959\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650”]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x2d90d77]

[Expected Behavior]
List normally
[Related Components and Specific Versions]
go sdk
[Other Background Information or Screenshots]
Rocky Linux 9.2 5.14.0-284.30.1.el9_2.x86_64
wx: nicoljiang
email: nicol@soft.org

| username: Billmay表妹 | Original post link

Raw KV? Not using TiDB?

| username: TiDBer_QBsB7GOW | Original post link

Yes, Raw Kv

| username: xfworld | Original post link

Can you describe how it was operated?
From the logs, it looks like the transaction request was abandoned. It would be best if you could provide a scenario description.

| username: xfworld | Original post link

The raw kv mode does not initiate txn requests. Maybe you are using the wrong method…

| username: Fly-bird | Original post link

The region may have undergone changes.

| username: ti-tiger | Original post link

Check the versions of TiKV and client-go to ensure they are compatible.

| username: TiDBer_QBsB7GOW | Original post link

Thanks a lot. After troubleshooting last night, it should be caused by txn.Iter(lastKey, nil). I’ll keep looking into it.

| username: TiDBer_QBsB7GOW | Original post link

@xfworld @Fly-bird @ti-tiger Can TiKV’s key allow bytes like ESC and \n to exist?

The [bytes] lastKey we generated contains an ESC and a newline.

image

| username: xfworld | Original post link

Not quite sure, I suggest using escaping to avoid unrecognizable characters :slightly_smiling_face:

| username: TiDBer_QBsB7GOW | Original post link

After investigation, it seems that when the endkey is empty, the client is querying the wrong endkey (it is possible that an incorrect region is returned).

| username: TiDBer_QBsB7GOW | Original post link

There is no region returning an error.
The error is triggered when the client calculates the end key given a start key and the end key is nil.
@xfworld @ti-tiger @Fly-bird Do you have any tips or suggestions?

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.