Failed to start tidb-dashboard with tiup cluster start tidb-join --init

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tiup cluster start tidb-join --init tidb-dashboard启动失败

| username: Hacker_5YEF49J7

cat tidb_dashboard_stderr.log

bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)
| username: tidb菜鸟一只 | Original post link

Check the glibc version of each one using ldd --version and see the glibc version requirements corresponding to your TiDB cluster version.

| username: Hacker_5YEF49J7 | Original post link

My TiDB is 6.5.0, and I don’t know which version to install.

| username: Billmay表妹 | Original post link

Search it, and you’ll find it immediately.

| username: Hacker_5YEF49J7 | Original post link

It means that you need to compile and install glibc-2.28.tar.gz on each machine first.

| username: Hacker_5YEF49J7 | Original post link

[tidb@db33 tmp]$ ldd --version
ldd (GNU libc) 2.28
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

| username: Hacker_5YEF49J7 | Original post link

tidb_dashboard is not installed on the PD server. Does it need to be installed on the same machine as one of the PDs?

I deleted the tidb_dashboard_stderr.log and tidb_dashboard.log files. Now, when I initialize, even the tidb_dashboard logs are not being generated.

| username: Hacker_5YEF49J7 | Original post link

I destroyed the cluster and redeployed it. Now even PD cannot start, and there are no log files under PD.

| username: Hacker_5YEF49J7 | Original post link

bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by bin/tidb-dashboard)
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bin/tidb-dashboard)

I manually executed the script. Then it prompted this.

I manually executed the script on the 31 machine, and pd can start normally.

| username: Hacker_5YEF49J7 | Original post link

glibc-2.29.tar.gz
glibc-2.32.tar.gz
glibc-2.33.tar.gz
After installation, it only prompts
bin/tidb-dashboard: /lib64/libm.so.6: version `GLIBC_2.29’ not found (required by bin/tidb-dashboard)

But installing glibc-2.34.tar.gz caused the system to crash
ls: symbol lookup error: /lib64/libpthread.so.0: undefined symbol: __libc_pthread_init, version GLIBC_PRIVATE
All commands prompt this. SSH cannot connect either.

| username: Hacker_5YEF49J7 | Original post link

My other cluster is also 2.17, why can it run?
It’s just that the tidb_dashboard is not separated and is version 5.4.0.

| username: SabaPing | Original post link

Hi there,

Starting from v6.5.0, TiDB Dashboard has become an independent component. The TiDB Operator v1.4.0 on K8S already supports using the TiDBDashboard CRD. TiUP is expected to support independent deployment of TiDB Dashboard in v6.6.0.

Regarding the issue mentioned by the original poster, during our CD process, we used a different base image builder from the core components like PD and TiDB, resulting in a glibc version that is incompatible with the kernel’s minimum required version.

We are currently working on resolving this and will replace the v6.5.0 TiDB Dashboard image as soon as possible. In the meantime, you can use the Dashboard built into PD (this Dashboard will be retained for some time and will not be removed from PD in the short term).

For those who are capable, you can also compile the dashboard image yourself by referring to contributing.md and dockerfile.

Thank you!

| username: Billmay表妹 | Original post link

The fixed TiDB Dashboard image for v6.5.0 is expected to be released this week. It will directly replace the images under the following two tags:

  • pingcap/tidb-dashboard:latest
  • pingcap/tidb-dashboard:v6.5.0