Timeout Issue Occurred When Exporting MySQL Data Using Dumpling

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用dumpling导出mysql数据时,出现执行超时问题

| username: yejinlei

[TiDB Usage Environment] Production Environment / Test / Poc
[TiDB Version]
[Reproduction Path] What operations were performed when the issue occurred
[Encountered Issue: Problem Phenomenon and Impact]
[Resource Configuration]
[Attachments: Screenshots / Logs / Monitoring]

[2023/11/29 18:00:00.738 +08:00] [INFO] [status.go:37] [progress] [tables=“100/198 (50.5%)”] [“finished rows”=4168702] [“estimate total rows”=17865436] [“finished size”=3.961GB] [“average speed(MiB/s)”=15.773540740537307] [“recent speed bps”=16539734.92853311] [“chunks progress”=“50.51 %”]
[2023/11/29 18:01:06.077 +08:00] [WARN] [writer_util.go:194] [“fail to dumping table(chunk), will revert some metrics and start a retry if possible”] [database=fas] [table=bo_fire_dispatch] [“finished rows”=1760117] [“finished size”=1146033265] [error=“Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded”]
[2023/11/29 18:01:06.077 +08:00] [WARN] [writer_util.go:194] [“fail to dumping table(chunk), will revert some metrics and start a retry if possible”] [database=fas] [table=bo_fire_warning] [“finished rows”=979474] [“finished size”=924222876] [error=“context canceled”]
[2023/11/29 18:01:06.078 +08:00] [WARN] [writer_util.go:194] [“fail to dumping table(chunk), will revert some metrics and start a retry if possible”] [database=fas] [table=bo_fire_info] [“finished rows”=1134590] [“finished size”=1230988271] [error=“context canceled”]
[2023/11/29 18:01:06.078 +08:00] [WARN] [writer_util.go:194] [“fail to dumping table(chunk), will revert some metrics and start a retry if possible”] [database=fas] [table=bo_fire_warning_wide_info] [“finished rows”=54494] [“finished size”=91906779] [error=“context canceled”]
[2023/11/29 18:01:06.078 +08:00] [INFO] [collector.go:224] [“units canceled”] [cancel-unit=0]
[2023/11/29 18:01:06.078 +08:00] [INFO] [collector.go:225] [“backup failed summary”] [total-ranges=1] [ranges-succeed=0] [ranges-failed=1] [unit-name=“dump table data”] [error=“Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded”] [errorVerbose=“Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/juju_adaptor.go:15\ngithub.com/pingcap/tidb/dumpling/export.(*rowIter).Error\n\tgithub.com/pingcap/tidb/dumpling/export/ir_impl.go:42\ngithub.com/pingcap/tidb/dumpling/export.WriteInsert\n\tgithub.com/pingcap/tidb/dumpling/export/writer_util.go:285\ngithub.com/pingcap/tidb/dumpling/export.FileFormat.WriteInsert\n\tgithub.com/pingcap/tidb/dumpling/export/writer_util.go:667\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).tryToWriteTableData\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:243\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).WriteTableData.func1\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:228\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry.func1\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:57\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetryV2[…]\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:75\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:56\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).WriteTableData\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:192\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).handleTask\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:115\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).run\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:93\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).startWriters.func4\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:376\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.3.0/errgroup/errgroup.go:75\nruntime.goexit\n\truntime/asm_amd64.s:1650”]
[2023/11/29 18:01:06.078 +08:00] [ERROR] [main.go:78] [“dump failed error stack info”] [error=“Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded”] [errorVerbose=“Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/errors@v0.11.5-0.20221009092201-b66cddb77c32/juju_adaptor.go:15\ngithub.com/pingcap/tidb/dumpling/export.(*rowIter).Error\n\tgithub.com/pingcap/tidb/dumpling/export/ir_impl.go:42\ngithub.com/pingcap/tidb/dumpling/export.WriteInsert\n\tgithub.com/pingcap/tidb/dumpling/export/writer_util.go:285\ngithub.com/pingcap/tidb/dumpling/export.FileFormat.WriteInsert\n\tgithub.com/pingcap/tidb/dumpling/export/writer_util.go:667\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).tryToWriteTableData\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:243\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).WriteTableData.func1\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:228\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry.func1\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:57\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetryV2[…]\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:75\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:56\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).WriteTableData\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:192\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).handleTask\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:115\ngithub.com/pingcap/tidb/dumpling/export.(*Writer).run\n\tgithub.com/pingcap/tidb/dumpling/export/writer.go:93\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).startWriters.func4\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:376\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.3.0/errgroup/errgroup.go:75\nruntime.goexit\n\truntime/asm_amd64.s:1650”]

dump failed: Error 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded

| username: h5n1 | Original post link

Is there an execution time setting?

| username: yejinlei | Original post link

Setting max_execution_time to 0 for the session still results in an error.

| username: h5n1 | Original post link

Use the --params parameter to specify this variable.

| username: dba远航 | Original post link

SET SESSION max_execution_time = 0; is a current session variable and will not take effect for other session connections. You should use SET GLOBAL max_execution_time = 0;.

| username: Kongdom | Original post link

Try SET GLOBAL max_execution_time = 0;

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.