Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.
Original topic: ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.7.1
【TiDB Usage Environment】Test Environment
- PD: 192.168.3.221/222/223:2379
- TiDB: 192.168.3.221/222/223:4000
- TiKV: 192.168.3.224/225/226
- Haproxy: 192.168.3.221:13390
- Spark: Master (192.168.3.227:7077), Worker (192.168.3.228/229)
- ${SPARK_HOME}/conf/spark-defaults.conf content as follows
【TiDB Version】TiDB v5.4.2, Spark v3.1.3, TISpark tispark-assembly-3.1_2.12-3.0.1.jar
【Encountered Issue】
【Reproduction Path】What operations were performed to cause the issue
Executing SQL statements in spark-sql prompts ANTLR version mismatch.
【Issue Phenomenon and Impact】
spark-sql> show database;
ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.8 used for parser compilation does not match the current runtime version 4.7.122/07/27 22:53:34 WARN HttpClientUtil: Failed to send HTTP get request.
spark.tispark.tidb.port 13390
What port is 13390?
–
I see, Haproxy: 192.168.3.221:13390
The load balancing port of Haproxy
Are you using this package spark-3.1.3-bin-hadoop3.2?
Try switching to this one spark-3.1.3-bin-hadoop2.7.
We can display “show databases” but an error occurs related to Hive metadata when actually querying data.
Can the issue be reproduced after the replacement? Is your TiDB deployed independently or on Kubernetes? Do you have a Spark 3.0.x environment? Could you please help test whether the error can be reproduced under Spark 3.0.x?
I looked through the source code of TiSpark and made a preliminary judgment that Spark 3.1.x references org.antlr:antlr4-runtime:4.8-1 during compilation, while tispark-assembly-3.1_2.12-3.0.1.jar most likely references org.antlr:antlr4-runtime:4.7 during its compilation, hence the warning. TiSpark uses tikv-client, and tikv-client uses org.antlr:antlr4-runtime:4.7, so I guess tispark-assembly-3.1_2.12-3.0.1.jar most likely references org.antlr:antlr4-runtime:4.7.
I’ve seen this error in several posts:
Between your two tests, the problematic one and the one without issues, did you make any changes to the PD address? Or was it newly copied?
I set up a Spark 3.1.3 cluster in my own test environment. If needed, I can try a Spark 3.0.x environment tomorrow during the day.
I deployed TiDB independently, and the OS is CentOS 7.9. After replacing tispark-assembly-3.1_2.12-3.0.1.jar with tispark-assembly-3.1-2.5.1.jar, the ANTLR warning still exists. However, the error “WARN HttpClientUtil: Failed to send HTTP get request.” has disappeared. SQL statements can also be executed normally.
I also noticed this error.
During the first test, the PD configuration in spark-defaults.conf was set to:
spark.tispark.pd.addresses 192.168.3.221:2379,192.168.3.222:2379,192.168.3.223:2379
After discovering this error, I thought it was a problem with my configuration, so I kept only one PD address:
spark.tispark.pd.addresses 192.168.3.221:2379
Then, the error seemed to disappear (I don’t remember clearly).
During the second test, I switched to spark-3.1.3-bin-hadoop2.7 and didn’t notice the PD error.
During the third test, after changing the TiSpark package to tispark-assembly-3.1-2.5.1.jar, I reverted the PD configuration to:
spark.tispark.pd.addresses 192.168.3.221:2379,192.168.3.222:2379,192.168.3.223:2379.
The error did not reappear.
After replacing Spark 3.1.3 with Spark 3.0.3, the warning about ANTLR version mismatch disappeared, and SQL queries can be executed normally.
The verified combinations that do not produce the ANTLR version does not match error are as follows:
-
TiDB v5.4.2 - Spark v3.0.3 - TiSpark v2.5.1/v3.0.1
-
TiDB v6.1 - Spark v3.0.3 - tispark-assembly-3.0_2.12-3.0.1.jar
spark-sql> use tidb_catalog.mysql;
22/07/29 03:38:08 WARN HttpClientUtil: Failed to send HTTP get request.
22/07/29 03:38:08 WARN TiSparkTeleInfo$: Failed to get PD version For input string: "2379,192.168.3.222:2379,192.168.3.223:2379"
22/07/29 03:38:08 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Time taken: 2.741 seconds
spark-sql> use tidb_catalog.mysql;
Time taken: 0.025 seconds
spark-sql> select * from user;
22/07/29 03:38:19 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
% root *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B mysql_native_password Y Y Y Y Y Y Y Y Y Y YY Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y
% admin *4ACFE3202A5FF5CF467898FC58AAB1D615029441 mysql_native_password Y Y Y Y Y Y Y Y Y Y YY Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y
Time taken: 2.552 seconds, Fetched 2 row(s)
- TiDB v6.1 - Spark v3.0.3 - tispark-assembly-3.0-2.5.1.jar
spark-sql> use tidb_catalog.mysql;
22/07/29 03:32:35 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Time taken: 2.641 seconds
spark-sql> use tidb_catalog.mysql;
Time taken: 0.024 seconds
spark-sql> select * from user;
22/07/29 03:32:42 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
% root *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B mysql_native_password Y Y Y Y Y Y Y Y Y Y YY Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y
% admin *4ACFE3202A5FF5CF467898FC58AAB1D615029441 mysql_native_password Y Y Y Y Y Y Y Y Y Y YY Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y
Time taken: 2.78 seconds, Fetched 2 row(s)
The combinations that produce the ANTLR version does not match error are:
- TiDB v6.1 - Spark v3.1.3 - tispark-assembly-3.0-2.5.1.jar
ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.8 used for parser compilation does not match the current runtime version 4.7.1ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.8 used for parser compilation does not match the current runtime version 4.7.1default
-
TiDB v5.4.2 - Spark v3.1.3 - tispark-assembly-3.0-2.5.1.jar
-
TiDB v5.4.2 - Spark v3.1.3 - tispark-assembly-3.1_2.12-3.0.1.jar
Based on the above results, it is preliminarily determined that ANTLR 4.8 is introduced in Spark v3.1.3, while ANTLR 4.7 is introduced in Spark v3.0.3, and the ANTLR version introduced in TiDB 5.x/6.x and TiSpark 2.5.x/3.x is 4.7.1.
Environment:
TiDB: v5.4.2
TiSpark: 3.1_2.12-3.0.1
Spark: 3.1.3
I did not reproduce your issue. There is indeed a problem with the PD address resolution.
I tried changing the half-width colon to a full-width colon, and the error message is as follows:
22/07/29 07:51:42 WARN PDClient: failed to get member from pd server.
2022/7/29 15:51:42 com.pingcap.tikv.exception.TiKVException: Failed to build NettyChannelBuilder
Error with an added space:
java.lang.IllegalArgumentException: Illegal character in authority at index 7: http://basic-pd.tidb-cluster: 2379
It seems that the issue is not caused by half-width/full-width or space:
22/07/27 23:15:56 WARN TiSparkTeleInfo$: Failed to get PD version For input string: “2379,192.168.3.222:23379,192.168.3.223:2379”
Are you saying that even with the error “Failed to get PD version For input string: ‘2379,192.168.3.222:23379,192.168.3.223:2379’”, it is still accessible?
I looked through previous Q&A, and this post also has the same error, but it shouldn’t be the reason in the end.
There is an issue with the URL used to get the PD status, which causes an exception to be thrown. However, this does not affect the overall operation, and the value will be set to UNKNOWN. This is a bug.
This issue has been fixed.
|
username: OnTheRoad
|
username: 
|
username: 
|
username: 
|
username: 
|
username: