Failed to Start TiKV

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 启动tikv失败

| username: Alan

[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:79] [“Welcome to TiKV”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Release Version: 6.1.0”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Edition: Community”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Git Commit Hash: 080d086832ae5ce2495352dccaf8df5d40f30687”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Git Commit Branch: heads/refs/tags/v6.1.0”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“UTC Build Time: Unknown (env var does not exist when building)”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Rust Version: rustc 1.60.0-nightly (1e12aef3f 2022-02-13)”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Enable Features: jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [lib.rs:84] [“Profile: dist_release”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [mod.rs:74] [“cgroup quota: memory=Some(9223372036854771712), cpu=None, cores={4, 6, 10, 1, 8, 11, 15, 3, 13, 0, 12, 2, 5, 9, 7, 14}”]
[2022/08/18 09:41:13.891 +08:00] [INFO] [mod.rs:81] [“memory limit in bytes: 68742515712, cpu cores quota: 16”]
[2022/08/18 09:41:13.894 +08:00] [WARN] [lib.rs:537] [“environment variable TZ is missing, using /etc/localtime”]
[2022/08/18 09:41:13.894 +08:00] [WARN] [server.rs:1533] [“check: kernel”] [err=“kernel parameters net.core.somaxconn got 128, expect 32768”]
[2022/08/18 09:41:13.894 +08:00] [WARN] [server.rs:1533] [“check: kernel”] [err=“kernel parameters net.ipv4.tcp_syncookies got 1, expect 0”]
[2022/08/18 09:41:13.894 +08:00] [WARN] [server.rs:1533] [“check: kernel”] [err=“kernel parameters vm.swappiness got 60, expect 0”]
[2022/08/18 09:41:13.894 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:13.897 +08:00] [INFO] [] [“TCP_USER_TIMEOUT is available. TCP_USER_TIMEOUT will be used thereafter”]
[2022/08/18 09:41:13.917 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:13.917 +08:00] [WARN] [client.rs:163] [“validate PD endpoints failed”] [err=“Other("[components/pd_client/src/util.rs:570]: PD cluster failed to respond")”]
[2022/08/18 09:41:14.218 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:14.219 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:14.520 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:14.522 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:14.822 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:14.824 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.124 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.126 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.426 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.428 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.728 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:15.730 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.030 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.032 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.332 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.334 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.634 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.636 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.936 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.938 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:16.938 +08:00] [WARN] [client.rs:163] [“validate PD endpoints failed”] [err=“Other("[components/pd_client/src/util.rs:570]: PD cluster failed to respond")”]
[2022/08/18 09:41:17.238 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:17.240 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:17.541 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:17.543 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:17.843 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:17.845 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.145 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.147 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.447 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.449 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.749 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:18.751 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.052 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.054 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.354 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.356 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.656 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.658 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.958 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.960 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:19.960 +08:00] [WARN] [client.rs:163] [“validate PD endpoints failed”] [err=“Other("[components/pd_client/src/util.rs:570]: PD cluster failed to respond")”]
[2022/08/18 09:41:20.260 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:20.262 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:20.562 +08:00] [INFO] [util.rs:575] [“connecting to PD endpoint”] [endpoints=192.168.110.13:2379]
[2022/08/18 09:41:20.564 +08:00] [INFO] [util.rs:537] [“PD failed to respond”] [err=“Grpc(RpcFailure(RpcStatus { code: 12-UNIMPLEMENTED, message: "unknown service pdpb.PD", details: }))”] [endpoints=192.168.110.13:

2
| username: Alan | Original post link

The log files all contain this error.

3
| username: Billmay表妹 | Original post link

Version of the issue
Steps to reproduce
What is it?

4
| username: Alan | Original post link

The issue has been resolved in version 6.1. The reason was that my 2380 port was occupied. However, after starting, a new error appeared: failed to create the server"] [error="failed to cleanup stale Unix socket file /tmp/tidb-4000.sock: dial unix /tmp/tidb-4000.sock: connect: permission denied

5
| username: Alan | Original post link

All other services have started, but TiKV did not start. Here is the last error message in the log: [2022/08/18 13:55:28.028 +08:00] [FATAL] [main.go:726] [“failed to create the server”] [error=“failed to cleanup stale Unix socket file /tmp/tidb-4000.sock: dial unix /tmp/tidb-4000.sock: connect: permission denied”] [stack=“main.createServer\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:726\nmain.main\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:203\nruntime.main\t/usr/local/go/src/runtime/proc.go:250”] [stack=“main.createServer\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:726\nmain.main\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:203\nruntime.main\t/usr/local/go/src/runtime/proc.go:250”]

6
| username: Alan | Original post link

Here is the startup information:

  • [ Serial ] - StartCluster
    Starting component pd
    Starting instance 192.168.110.13:2479
    Start instance 192.168.110.13:2479 success
    Starting component tikv
    Starting instance 192.168.110.13:20160
    Start instance 192.168.110.13:20160 success
    Starting component tidb
    Starting instance 192.168.110.13:4000
    Start instance 192.168.110.13:4000 success
    Starting component tiflash
    Starting instance 192.168.110.13:9000
    Start instance 192.168.110.13:9000 success
    Starting component prometheus
    Starting instance 192.168.110.13:9090
    Start instance 192.168.110.13:9090 success
    Starting component grafana
    Starting instance 192.168.110.13:3000
    Start instance 192.168.110.13:3000 success
    Starting component alertmanager
    Starting instance 192.168.110.13:9093
    Start instance 192.168.110.13:9093 success
    Starting component node_exporter
    Starting instance 192.168.110.13
    Start 192.168.110.13 success
    Starting component blackbox_exporter
    Starting instance 192.168.110.13
    Start 192.168.110.13 success
  • [ Serial ] - UpdateTopology: cluster=tidb-test
    Started cluster tidb-test successfully
    Failed to set root password of TiDB database to ‘^5Z$VxNba62H@-e097’
7
| username: Billmay表妹 | Original post link

Okay, then please create a new post for the new question~

8
| username: Alan | Original post link

The issue has been resolved, from the post → Failed to set root password of TiDB database to - #13，来自 Hi70KG - TiDB 的问答社区

9
| username: Billmay表妹 | Original post link

Great~ It’s resolved, that’s wonderful~

10
| username: system | Original post link

This topic will be automatically closed 60 days after the last reply. No new replies are allowed.