Backup and Restore Error: Encountered "Column Too Many" Error

Translated
1

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: br 备份恢复报错，遇到column too manys的错误

| username: Hacker_9LYnzJhP

[TiDB Version] TIDB 5.4

[Encountered Problem: Phenomenon and Impact]

The TiDB backup database instance has a wide table with more than 1017 columns. Before executing the restore, the table-column-count-limit of TiDB was adjusted to 4096. However, executing br restore still encountered the “column too many” error.

Manually creating this wide table in the target database with the table-column-count-limit adjusted to 4096 is not an issue. So, are there any additional parameters that need to be set for BR backup?

[Attachment: Screenshot/Logs/Monitoring]
The error log is as follows:
I0220 06:32:24.743893 8 restore.go:90] [2023/02/20 06:32:24.743 +00:00] [ERROR] [main.go:58] [“br failed”] [error=“[ddl:1117]Too many columns”] [errorVerbose=“[ddl:1117]Too many columns\ngithub.com/pingcap/errors.AddStack\n\t/nfs/cache/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/nfs/cache/mod/github.com/pingcap/errors@v0.11.5-0.20211224045212-9687c2b0f87c/juju_adaptor.go:15\ngithub.com/pingcap/tidb/ddl.checkTableInfoValidExtra\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/ddl/ddl_api.go:1711\ngithub.com/pingcap/tidb/ddl.(*ddl).CreateTableWithInfo\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/ddl/ddl_api.go:2077\ngithub.com/pingcap/tidb/br/pkg/gluetidb.(*tidbSession).CreateTable\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/gluetidb/glue.go:146\ngithub.com/pingcap/tidb/br/pkg/restore.(*DB).CreateTable\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/db.go:129\ngithub.com/pingcap/tidb/br/pkg/restore.(*Client).createTable\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/client.go:415\ngithub.com/pingcap/tidb/br/pkg/restore.(*Client).GoCreateTables.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/client.go:468\ngithub.com/pingcap/tidb/br/pkg/restore.(*Client).createTablesWithDBPool.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/client.go:523\ngithub.com/pingcap/tidb/br/pkg/utils.(*WorkerPool).ApplyWithIDInErrorGroup.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/utils/worker.go:82\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/nfs/cache/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371”] [stack=“main.main\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/cmd/br/main.go:58\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225”]

2
| username: tidb菜鸟一只 | Original post link

It is estimated that it is a problem with the BR version. It is recommended to upgrade the BR version.

3
| username: Hacker_9LYnzJhP | Original post link

If it’s a version issue, could you post the related issues of TiDB?

4
| username: WalterWj | Original post link

It feels unexpected. It looks like the database restrictions have been adjusted, but BR has its own restriction checks. :thinking:

5
| username: Hacker_9LYnzJhP | Original post link

Where is the adjustment? I checked the source code, and before executing the DDL operation, it loads the target end’s config without modifying the relevant parameters.

6
| username: WalterWj | Original post link

I couldn’t find it either. I recommend you go to the feedback section and report it to see if there is indeed an issue with the current product.

If there is an issue but no workaround, we’ll have to wait for the product team to address it. :thinking:

7
| username: YuJuncen | Original post link

It looks like the table-column-count-limit parameter is a local configuration of the tidb-server (rather than a GLOBAL VARIABLE type), and this check does not seem to be done on the DDL owner (judging by the stack trace, it’s BR).
The BR process embeds a tidb-server to perform DDL-related operations, but currently, the configuration of this server does not synchronize with other servers (and probably there’s no good way to synchronize it, I guess), so increasing the table-column-count-limit on other TiDB nodes does not solve this problem.
As a workaround, you can consider creating the table you need in advance through a TiDB node with the modified table-column-count-limit. This way, BR will not attempt to create this table, and it should not report an error.

8
| username: YuJuncen | Original post link

Another possible workaround is to use the RESTORE SQL on the TiDB node where the table-column-count-limit has been modified.