Port 4000 fails to start when launching TiDB

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 在启动TIDB的时候 4000端口无法起来

| username: bulibuqiyaoyao133

【TiDB Usage Environment】Production\Test Environment\POC
【TiDB Version】tidb6.0
【Encountered Issue】Port 4000 fails to start on the first launch of TiDB
【Reproduction Path】First-time use, start command: tiup cluster start bigbirdmeddb --init
【Issue Phenomenon and Impact】

Startup error

TIDB%20%E5%90%AF%E5%8A%A8%E6%8A%A5%E9%94%99
TIDB%20%E5%90%AF%E5%8A%A8%E6%8A%A5%E9%94%99970×410 26.5 KB

Error: failed to start tidb: failed to start: 10.0.1.145 tidb-4000.service, please check the instance’s log(/mnt/data/tidb-deploy/tidb-4000/log) for more detail.: timed out waiting for port 4000 to be started after 2m0s
Logs:
tiup-cluster-debug-2022-06-28-11-18-22.log (530.7 KB)

2
| username: ddhe9527 | Original post link

Check if port 4000 is occupied: netstat -anp | grep 4000

3
| username: xuexiaogang | Original post link

I’ve encountered this before. As long as it’s not being occupied, it should start up. I had the same issue when I failed the first installation and tried to install again. You need to completely clear it out.

4
| username: bulibuqiyaoyao133 | Original post link

The image you provided is not visible. Please provide the text you need translated.

5
| username: bulibuqiyaoyao133 | Original post link

I installed the system from scratch and this is the first time I’m installing it. It fails to start at port 4000. The firewall is already disabled.

6
| username: bulibuqiyaoyao133 | Original post link

The image you provided is not visible. Please provide the text you need translated.

7
| username: bulibuqiyaoyao133 | Original post link

The image is not visible. Please provide the text you need translated.

8
| username: ddhe9527 | Original post link

You need to check the tidb.log file under /mnt/data/tidb-deploy/tidb-4000/log.

9
| username: db_user | Original post link

Check the TiDB log to see what the specific error is.

10
| username: bulibuqiyaoyao133 | Original post link

[2022/06/28 11:23:27.687 +08:00] [INFO] [base_client.go:350] [“[pd] switch leader”] [new-leader=http://10.0.1.145:2379] [old-leader=]
[2022/06/28 11:23:27.687 +08:00] [INFO] [base_client.go:105] [“[pd] init cluster id”] [cluster-id=7110132895032081485]
[2022/06/28 11:23:27.687 +08:00] [INFO] [client.go:687] [“[pd] tso dispatcher created”] [dc-location=global]
[2022/06/28 11:23:27.688 +08:00] [INFO] [store.go:80] [“new store with retry success”]
[2022/06/28 11:23:27.696 +08:00] [INFO] [tidb.go:72] [“new domain”] [store=tikv-7110132895032081485] [“ddl lease”=45s] [“stats lease”=3s] [“index usage sync lease”=0s]
[2022/06/28 11:23:27.709 +08:00] [WARN] [info.go:240] [“init TiFlashPlacementManager”] [“pd addrs”=“[10.0.1.145:2379]”]
[2022/06/28 11:23:27.724 +08:00] [INFO] [domain.go:171] [“full load InfoSchema success”] [currentSchemaVersion=0] [neededSchemaVersion=32] [“start time”=10.980018ms]
[2022/06/28 11:23:27.724 +08:00] [INFO] [domain.go:431] [“full load and reset schema validator”]
[2022/06/28 11:23:27.724 +08:00] [INFO] [ddl.go:378] [“[ddl] start DDL”] [ID=c99977dd-653b-44fb-8c74-e86edf1cd523] [runWorker=true]
[2022/06/28 11:23:27.724 +08:00] [INFO] [manager.go:189] [“start campaign owner”] [ownerInfo=“[ddl] /tidb/ddl/fg/owner”]
[2022/06/28 11:23:27.725 +08:00] [INFO] [ddl.go:367] [“[ddl] start delRangeManager OK”] [“is a emulator”=false]
[2022/06/28 11:23:27.725 +08:00] [INFO] [ddl_worker.go:161] [“[ddl] start DDL worker”] [worker=“worker 2, tp add index”]
[2022/06/28 11:23:27.725 +08:00] [INFO] [ddl_worker.go:161] [“[ddl] start DDL worker”] [worker=“worker 1, tp general”]
[2022/06/28 11:23:27.733 +08:00] [WARN] [coprocessor.go:981] [“other error”] [txnStartTS=434211810887335945] [regionID=38] [storeAddr=10.0.1.145:20160] [error=“other error: [components/tidb_query_datatype/src/expr/ctx.rs:86]: evaluation failed: unknown or incorrect time zone: Asia/Beijing”]
[2022/06/28 11:23:27.733 +08:00] [FATAL] [terror.go:292] [“unexpected error”] [error=“other error: [components/tidb_query_datatype/src/expr/ctx.rs:86]: evaluation failed: unknown or incorrect time zone: Asia/Beijing”] [stack=“github.com/pingcap/tidb/parser/terror.MustNil
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:292
main.createStoreAndDomain
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:299
main.main
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:204
runtime.main
\t/usr/local/go/src/runtime/proc.go:250”] [stack=“github.com/pingcap/tidb/parser/terror.MustNil
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:292
main.createStoreAndDomain
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:299
main.main
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:204
runtime.main
\t/usr/local/go/src/runtime/proc.go:250”]
[2022/06/28 11:23:42.934 +08:00] [INFO] [cpuprofile.go:115] [“parallel cpu profiler started”]
[2022/06/28 11:23:42.934 +08:00] [INFO] [printer.go:34] [“Welcome to TiDB.”] [“Release Version”=v6.0.0] [Edition=Community] [“Git Commit Hash”=36a9810441ca0e496cbd22064af274b3be771081] [“Git Branch”=heads/refs/tags/v6.0.0] [“UTC Build Time”=“2022-03-31 10:33:28”] [GoVersion=go1.18] [“Race Enabled”=false] [“Check Table Before Drop”=false] [“TiKV Min Version”=v3.0.0-60965b006877ca7234adaced7890d7b029ed1306]
[2022/06/28 11:23:42.935 +08:00] [INFO] [printer.go:48] [“loaded config”] [config="{"host":"0.0.0.0","advertise-address":"10.0.1.145","port":4000,"cors":"","store":"tikv","path":"10.0.1.145:2379","socket":"/tmp/tidb-4000.sock","lease":"45s","run-ddl":true,"split-table":true,"token-limit":1000,"oom-use-tmp-storage":true,"tmp-storage-path":"/tmp/1000_tidb/MC4wLjAuMDo0MDAwLzAuMC4wLjA6MTAwODA=/tmp-storage","oom-action":"cancel","mem-quota-query":1073741824,"tmp-storage-quota":-1,"enable-batch-dml":false,"lower-case-table-names":2,"server-version":"","version-comment":"","tidb-edition":"","tidb-release-version":"","log":{"level":"info","format":"text","disable-timestamp":null,"enable-timestamp":null,"disable-error-stack":null,"enable-error-stack":null,"file":{"filename":"/mnt/data/tidb-deploy/tidb-4000/log/tidb.log","max-size":300,"max-days":0,"max-backups":0},"enable-slow-log":true,"slow-query-file":"/mnt/data/tidb-deploy/tidb-4000/log/tidb_slow_query.log","slow-threshold":300,"expensive-threshold":10000,"query-log-max-len":4096,"record-plan-in-slow-log":1},"security":{"skip-grant-table":false,"ssl-ca":"","ssl-cert":"","ssl-key":"","require-secure-transport":false,"c

11
| username: songxuecheng | Original post link

Change Asia/Beijing to Asia/Shanghai.

12
| username: db_user | Original post link

Timezone issue, it looks like you have already identified it yourself. :+1:

13
| username: bulibuqiyaoyao133 | Original post link

Sorry, I can’t assist with that.

14
| username: bulibuqiyaoyao133 | Original post link

After changing the time zone and restarting the server, it still cannot start.

15
| username: db_user | Original post link

What error is reported in the log this time?

16
| username: bulibuqiyaoyao133 | Original post link

[terror.go:292] [“unexpected error”] [error=“other error: [components/tidb_query_datatype/src/expr/ctx.rs:86]: evaluation failed: unknown or incorrect time zone: Asia/Beijing”] [stack=“github.com/pingcap/tidb/parser/terror.MustNil
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:292
main.createStoreAndDomain
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:299
main.main
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:204
runtime.main
\t/usr/local/go/src/runtime/proc.go:250”] [stack="github.com/pingcap/tidb/parser/terror.MustNil
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/parser/terror/terror.go:292
main.createStoreAndDomain
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/tidb-server/main.go:299
main.main
\t/home/jenkins/agent/workspace/build-com

17
| username: db_user | Original post link

Could you please check if the time_zone configuration is displayed? It seems that it has not been changed yet.
The tidb-related part of edit-config:
/home/tidb/.tiup/storage/cluster/clusters/cluster_name/meta.yaml
/home/tidb/.tiup/storage/cluster/clusters/cluster_name/cache-config

18
| username: bulibuqiyaoyao133 | Original post link

meta.yaml (2.1 KB) tidb-10.0.1.145-4000.service (309 bytes) tidb-10.0.1.145-4000.toml (352 bytes)

19
| username: db_user | Original post link

Try removing this configuration in edit-config to see if it can restart. If it can’t restart, check for any errors. This is not a TiDB parameter.

20
| username: bulibuqiyaoyao133 | Original post link

Cannot start, still an issue with port 4000.