[TiDB Usage Environment] Production Environment
[TiDB Version] 7.5.0
[Reproduction Path] None
[Encountered Problem: Problem Phenomenon and Impact] Deployed TiDB using k8s+kubesphere+operator, one day found that a TiKV failed to start
[Resource Configuration]
[Attachments: Screenshots/Logs/Monitoring]
Another question is that the default deployed TiDB cluster is in UTC time zone. I want to change it to Shanghai time zone. Do I need to restart kv, pd, and tidb? What is the restart order?
Sure, thank you for your reply. Regarding the issue of TiKV failing to start, how can I resolve it? After deployment, I haven’t manually restarted it. It should be automatically scheduled and started by k8s. There were always 3 KV nodes before.
The TiKV error seems to be a configuration file issue. It is recommended to check whether the configuration files under the node are different from those under other nodes.
Sorry to bother you. I have another question. The built-in dashboard panel in the cluster sometimes doesn’t open when scheduled to other nodes because there are three nodes deployed. So, I manually deployed an independent dashboard, but when I open the webpage, I get this prompt:
{
"error": true,
"message": "Invalid Semantic Version",
"code": "common.internal",
"full_text": "Invalid Semantic Version\n
at github.com/pingcap/tidb-dashboard/util/rest.Error()\n\t/go/src/github.com/pingcap/tidb-dashboard/util/rest/context_helpers.go:19\n
at github.com/pingcap/tidb-dashboard/pkg/apiserver/info.(*Service).infoHandler()\n\t/go/src/github.com/pingcap/tidb-dashboard/pkg/apiserver/info/info.go:86\n
at github.com/gin-gonic/gin.(*Context).Next()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/context.go:165\n
at github.com/pingcap/tidb-dashboard/util/rest.ErrorHandlerFn.func1()\n\t/go/src/github.com/pingcap/tidb-dashboard/util/rest/error.go:70\n
at github.com/gin-gonic/gin.(*Context).Next()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/context.go:165\n
at github.com/gin-contrib/gzip.Gzip.func2()\n\t/go/pkg/mod/github.com/gin-contrib/gzip@v0.0.1/gzip.go:47\n
at github.com/gin-gonic/gin.(*Context).Next()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/context.go:165\n
at github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/recovery.go:99\n
at github.com/gin-gonic/gin.(*Context).Next()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/context.go:165\n
at github.com/gin-gonic/gin.(*Engine).handleHTTPRequest()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/gin.go:489\n
at github.com/gin-gonic/gin.(*Engine).ServeHTTP()\n\t/go/pkg/mod/github.com/gin-gonic/gin@v1.7.4/gin.go:445\n
at github.com/pingcap/tidb-dashboard/pkg/apiserver.(*Service).handler()\n\t/go/src/github.com/pingcap/tidb-dashboard/pkg/apiserver/apiserver.go:244\n
at net/http.HandlerFunc.ServeHTTP()\n\t/usr/local/go/src/net/http/server.go:2109\n
at github.com/pingcap/tidb-dashboard/pkg/utils.(*ServiceStatus).NewStatusAwareHandler.func1()\n\t/go/src/github.com/pingcap/tidb-dashboard/pkg/utils/service_status.go:67\n
at net/http.HandlerFunc.ServeHTTP()\n\t/usr/local/go/src/net/http/server.go:2109\n at net/http.(*ServeMux).ServeHTTP()\n\t/usr/local/go/src/net/http/server.go:2487\n
at net/http.serverHandler.ServeHTTP()\n\t/usr/local/go/src/net/http/server.go:2947\n at net/http.(*conn).serve()\n\t/usr/local/go/src/net/http/server.go:1991\n
at runtime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"
}
It looks like the version information was not correctly retrieved.
Can you confirm the configuration of kubesphere? Are all pods interconnected, and are all ports open?