Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.Original topic: tispark 写入大宽表报错
|
username: TiDBer_AcAczQ2E
Error when writing using TiSpark
df.write.
format("tidb").
option("tidb.addr", "xxx").
option("tidb.port", "xxx").
option("tidb.user", "xxx").
option("tidb.password", "xxxxx@xxx").
option("database", "bdata").
option("table", tblName).
option("replace", true).
option("isolationLevel", "NONE").
option(JDBCOptions.JDBC_BATCH_INSERT_SIZE, 1000).
mode("append").
save()
Error message:
Exception in thread "main" com.pingcap.tikv.exception.GrpcException: retry is exhausted.
[INFO] 2022-08-11 01:14:08.812 - [taskAppId=TASK-39-42636-62375]:[127] - -> at com.pingcap.tikv.util.ConcreteBackOffer.doBackOffWithMaxSleep(ConcreteBackOffer.java:148)
at com.pingcap.tikv.util.ConcreteBackOffer.doBackOff(ConcreteBackOffer.java:119)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:215)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
...
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
...
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
...
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:221)
...
at com.pingcap.tikv.TwoPhaseCommitter.commitPrimaryKey(TwoPhaseCommitter.java:197)
at com.pingcap.tispark.write.TiBatchWrite.commitPrimaryKey(TiBatchWrite.scala:435)
at com.pingcap.tispark.write.TiBatchWrite.commitPrimaryKeyWithRetry(TiBatchWrite.scala:389)
at com.pingcap.tispark.write.TiBatchWrite.doWrite(TiBatchWrite.scala:322)
at com.pingcap.tispark.write.TiBatchWrite.com$pingcap$tispark$write$TiBatchWrite$$write(TiBatchWrite.scala:88)
at com.pingcap.tispark.write.TiBatchWrite$.write(TiBatchWrite.scala:45)
at com.pingcap.tispark.write.TiDBWriter$.write(TiDBWriter.scala:40)
at com.pingcap.tispark.TiDBDataSource.createRelation(TiDBDataSource.scala:57)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:83)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:81)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:80)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:127)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:75)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:285)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:271)
at erp.HiveToTiDB$$anonfun$main$1.apply(HiveToTiDB.scala:60)
at erp.HiveToTiDB$$anonfun$main$1.apply(HiveToTiDB.scala:37)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at erp.HiveToTiDB$.main(HiveToTiDB.scala:37)
at erp.HiveToTiDB.main(HiveToTiDB.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: com.pingcap.tikv.exception.GrpcException: Txn commit primary key failed, regionId=254055
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:219)
... 107 more
Caused by: com.pingcap.tikv.exception.KeyException: Key exception occurred and the reason is retryable: "Error(Txn(Error(Mvcc(Error(TxnLockNotFound { start_ts: TimeStamp(435198582254206977), commit_ts: TimeStamp(435198786186510337), key: [116, 128, 0, 0, 0, 0, 0, 105, 111, 95, 114, 128, 0, 0, 0, 0, 167, 225, 38] })))))"
at com.pingcap.tikv.region.RegionStoreClient.handleCommitResponse(RegionStoreClient.java:629)
at com.pingcap.tikv.region.RegionStoreClient.commit(RegionStoreClient.java:606)
at com.pingcap.tikv.txn.TxnKVClient.commit(TxnKVClient.java:148)
at com.pingcap.tikv.TwoPhaseCommitter.doCommitPrimaryKeyWithRetry(TwoPhaseCommitter.java:209)
... 107 more
[TiDB Usage Environment] Production, Testing, Research
[TiDB Version]
[Problem Encountered]
[Reproduction Steps] What operations were performed to cause the issue
[Problem Phenomenon and Impact]
[Attachments]
Please provide version information for each component, such as cdc/tikv, which can be obtained by executing
cdc version/tikv-server --version.
|
username: 
|
username: 
|
username: