ERROR 1105 (HY000): runtime error: index out of range [2048] with length 2048

translator_bot · June 23, 2024, 12:46pm

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: ERROR 1105 (HY000): runtime error: index out of range [2048] with length 2048

| username: Hacker_ojLJ8Ndr

【 TiDB Version】
v6.1.0
【Encountered Problem】
Upgraded from 5.4.1 to 6.1.0, enabled dynamic pruning mode, and executed the SQL ‘ANALYZE TABLE table_name ALL COLUMNS;’. This error occurred.
【Problem Phenomenon and Impact】
After rebuilding the table, the show stats_meta command shows the aggregated global statistics information, but inserting data and then performing analyze still results in an error.

translator_bot · June 23, 2024, 12:46pm

| username: yilong | Original post link

Does the default static mode report an error?
Please upload the complete table structure.
Please upload the complete error information from the tidb.log log.

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

No errors in static mode

Table structure:

CREATE TABLE `weibo_user` (
  `id` bigint(20) unsigned NOT NULL,
  `name` varchar(45) NOT NULL,
  `homeurl` varchar(1024) NOT NULL,
  `description` varchar(500) DEFAULT NULL,
  `profile_image_url` varchar(1024) DEFAULT NULL,
  `location` varchar(45) DEFAULT NULL,
  `province` varchar(20) DEFAULT NULL,
  `city` varchar(20) DEFAULT NULL,
  `gender` varchar(45) DEFAULT NULL,
  `birthday` date DEFAULT NULL,
  `age` varchar(45) DEFAULT NULL,
  `interests` text DEFAULT NULL,
  `followers_count` int(11) NOT NULL DEFAULT '0',
  `friends_count` int(11) NOT NULL DEFAULT '0' ,
  `statuses_count` int(11) NOT NULL DEFAULT '0' ,
  `buy_power` tinyint(4) DEFAULT NULL,
  `life_stage` varchar(10) DEFAULT NULL,
  `career_label` varchar(20) DEFAULT NULL,
  `is_navy` tinyint(4) DEFAULT NULL,
  `navy_probability` float(9,8) DEFAULT NULL,
  `verified` tinyint(1) DEFAULT '0',
  `verified_type` smallint(6) DEFAULT NULL,
  `verifiyType` tinyint(4) NOT NULL DEFAULT '4',
  `verified_reason` varchar(100) DEFAULT NULL,
  `verified_detail` json DEFAULT NULL,
  `verified_level` tinyint(4) DEFAULT NULL,
  `active` tinyint(1) DEFAULT '0',
  `created_at` datetime DEFAULT NULL,
  `spidertime` datetime NOT NULL,
  `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `inx_name` (`name`),
  KEY `idx_create_time` (`create_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
PARTITION BY HASH (`id`) PARTITIONS 128;

tidb.log:

[2022/06/28 16:36:05.847 +08:00] [INFO] [analyze.go:252] ["analyze table `mbase_test`.`weibo_user` has finished"] [partition=p127] ["job info"="analyze table all columns with 256 buckets, 500 topn, 0.7949470276207958 samplerate"] ["start time"=2022/06/28 16:15:30.120 +08:00] ["end time"=2022/06/28 16:36:05.829 +08:00] [cost=20m35.708522053s]
[2022/06/28 16:37:12.601 +08:00] [ERROR] [conn.go:1056] ["connection running loop panic"] [conn=2401122288830536617] [lastSQL="ANALYZE TABLE mbase_test.weibo_user ALL COLUMNS"] [err="runtime error: index out of range [2048] with length 2048"] [stack="github.com/pingcap/tidb/server.(*clientConn).Run.func1\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1059\
runtime.gopanic\
\t/usr/local/go/src/runtime/panic.go:838\
github.com/pingcap/tidb/executor.(*ExecStmt).Exec.func1\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:371\
runtime.gopanic\
\t/usr/local/go/src/runtime/panic.go:838\
runtime.goPanicIndex\
\t/usr/local/go/src/runtime/panic.go:89\
github.com/pingcap/tidb/util/chunk.(*Column).GetFloat64\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/chunk/column.go:557\
github.com/pingcap/tidb/util/chunk.Row.GetFloat64\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/chunk/row.go:68\
github.com/pingcap/tidb/util/chunk.Compare\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/chunk/compare.go:195\
github.com/pingcap/tidb/util/chunk.(*Chunk).LowerBound.func1\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/chunk/compare.go:227\
sort.Search\
\t/usr/local/go/src/sort/search.go:66\
github.com/pingcap/tidb/util/chunk.(*Chunk).LowerBound\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/util/chunk/compare.go:226\
github.com/pingcap/tidb/statistics.(*Histogram).locateBucket\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/statistics/histogram.go:481\
github.com/pingcap/tidb/statistics.(*Histogram).equalRowCount\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/statistics/histogram.go:440\
github.com/pingcap/tidb/statistics.MergePartTopN2GlobalTopN\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/statistics/cmsketch.go:781\
github.com/pingcap/tidb/statistics/handle.(*Handle).mergePartitionStats2GlobalStats\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/statistics/handle/handle.go:514\
github.com/pingcap/tidb/statistics/handle.(*Handle).MergePartitionStats2GlobalStatsByTableID\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/statistics/handle/handle.go:405\
github.com/pingcap/tidb/executor.(*AnalyzeExec).Next\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/analyze.go:210\
github.com/pingcap/tidb/executor.Next\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:319\
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:665\
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:516\
github.com/pingcap/tidb/executor.(*ExecStmt).Exec\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:465\
github.com/pingcap/tidb/session.runStmt\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:2069\
github.com/pingcap/tidb/session.(*session).ExecuteStmt\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/session/session.go:1938\
github.com/pingcap/tidb/server.(*TiDBContext).ExecuteStmt\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:230\
github.com/pingcap/tidb/server.(*clientConn).handleStmt\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2022\
github.com/pingcap/tidb/server.(*clientConn).handleQuery\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1876\
github.com/pingcap/tidb/server.(*clientConn).dispatch\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1371\
github.com/pingcap/tidb/server.(*clientConn).Run\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1121\
github.com/pingcap/tidb/server.(*Server).onConn\
\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:559"]
[2022/06/28 16:38:51.055 +08:00] [INFO] [domain.go:1749] ["refreshServerIDTTL succeed"] [serverID=1091904] ["lease id"=6eca81715eb7a38f]

translator_bot · June 23, 2024, 12:46pm

| username: yilong | Original post link

I tested and did not reproduce the issue. Created the table in v5.4.0, upgraded to v6.1.0, then modified to dynamic pruning mode, and collected information again.

image1046×142 38 KB
Could you please collect the statistics of this table? Based on your statistics, I will try again.
常规统计信息 | PingCAP 文档中心

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

Sure, please take a look:
Link: 百度网盘-链接不存在
Extraction code: ivj5

translator_bot · June 23, 2024, 12:46pm

| username: cheng | Original post link

I also couldn’t reproduce it. This empty table has no data at all, yet it takes so long.

translator_bot · June 23, 2024, 12:46pm

| username: yilong | Original post link

Thank you. After trying to import statistics, upgrading, setting dynamic, and then analyzing, it was successful.
I checked your statistics, and each partition has almost 70,000 data entries. There are a total of 8.9 million data entries across 127 partitions.
This issue seems similar to analyze table report error when usingd dynamic partition_prun_mode · Issue #34228 · pingcap/tidb · GitHub. We might need data to reproduce the problem. Could you please help export the data from this table? If you can provide a few rows of data, it would be best to reproduce the issue.
Additionally, we are continuously analyzing from the code perspective.

translator_bot · June 23, 2024, 12:46pm

| username: qizheng | Original post link

Did you encounter any issues with version 5.4.1 before the upgrade?

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

Tried again, created a new table, and tested with 7k, 12k, 50k, 88k, and 150k data. No errors were reported for smaller data volumes, but analyze reported errors for 88k and 150k data.

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

Tested in version 5.4.1, static is normal, dynamic reports the same error.

translator_bot · June 23, 2024, 12:46pm

| username: yilong | Original post link

Could you modify these 88,000 records to ensure they do not affect the security of your normal business data? After that, export and send them to us so we can reproduce the issue. Thank you.

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

Sorry, I asked the leader, and even if modified, we still can’t provide it. Can you generate some data yourself?

translator_bot · June 23, 2024, 12:46pm

| username: Hacker_ojLJ8Ndr | Original post link

Hello, I have confirmed the issue on my end, and it is caused by the float type. You can try it out; I have prepared the data, and it is reproducible.

CREATE TABLE `t1` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `num` float(9,8) DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin
PARTITION BY HASH (`id`) PARTITIONS 128;

t1_202207011138.sql (33.2 KB)

Every time this issue occurs, the connection will be disconnected.

translator_bot · June 23, 2024, 12:46pm

| username: yilong | Original post link

Reproduced, thank you very much.
Issue analyze partition table fail with dynamic prune mode · Issue #35910 · pingcap/tidb · GitHub