Failed to start when using TiDB as the metastore database for Hive

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 使用 TIDB作为 hive metastore 的元库,启动失败

| username: TiDBer_HPBq6Q7y

Hive 3.1 uses the metastore as the meta-database, and it seems that the initialization of the meta-database failed, causing it to fail to start successfully.
Initializing the schema to: 3.1.1000
Metastore connection URL: jdbc:hive2://?hive.conf.restricted.list=;hive.security.authorization.sqlstd.confwhitelist=.;hive.security.authorization.sqlstd.confwhitelist.append=.;hive.security.authorization.enabled=false;hive.metastore.uris=;hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdConfOnlyAuthorizerFactory;hive.support.concurrency=false;hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager;hive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
Metastore Connection Driver: org.apache.hive.jdbc.HiveDriver
Metastore connection User: hive
Hive Session ID = 00f19dbe-095d-474c-a273-1a1e23d7a66c
Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class is com.mysql.cj.jdbc.Driver’. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
Exception in thread “main” java.lang.RuntimeException: Error applying authorization policy on hive configuration: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hive.service.cli.CLIService.init(CLIService.java:121)
at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63)
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:293)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.getConnectionToMetastore(MetastoreSchemaTool.java:250)
at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.testConnectionToMetastore(MetastoreSchemaTool.java:333)
at org.apache.hadoop.hive.metastore.tools.SchemaToolTaskInit.execute(SchemaToolTaskInit.java:53)
at org.apache.hadoop.hive.metastore.tools.MetastoreSchemaTool.run(MetastoreSchemaTool.java:448)
at org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:932)
at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1893)
at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:134)
at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
… 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:964)
at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:924)
… 20 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:304)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:497)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:398)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:385)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:356)
at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:960)
… 21 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:95)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4867)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4935)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4915)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:5211)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:316)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:299)
… 26 more

| username: TiDBer_yangxi | Original post link

We are using TiDB, and there are no issues. Are you encountering an initialization error? You can check the initialization script to see if there are any unsupported syntax issues. You can try initializing with MySQL first and then migrate over. I remember there is also a parameter that needs to be changed.

| username: TiDBer_HPBq6Q7y | Original post link

Yes, it seems that the error occurred while executing the initialization command ‘schematool -info -dbType mysql -userName hive -passWord [PROTECTED] -verbose’.

| username: 有猫万事足 | Original post link

This is because versions above 2 need to initialize the source database first. You can use the following command to initialize:

./hive/bin/schematool -dbType mysql -initSchema

Could this be the reason?

| username: TiDBer_HPBq6Q7y | Original post link

No, it isn’t.

| username: yytest | Original post link

More and more companies are using TiDB now. It is recommended that the official team update the collection and organization of error documentation.