TiSpark Reports Cop Task Error

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tispark报cop task错误

| username: h5n1

tidb: 5.2.3, tispark 2.4.3

When executing a count in spark-sql, the following error occurs:

spark-sql> 
         >  select count(*) from ti_o_credit_stop_open_wp_4;
22/12/28 10:23:57 WARN RegionStoreClient: Other error occurred, message: Coprocessor task terminated due to exceeding the deadline
22/12/28 10:23:57 ERROR DAGIterator: Process region tasks failed, remain 0 tasks not executed due to
com.pingcap.tikv.exception.GrpcException: Coprocessor task terminated due to exceeding the deadline
        at com.pingcap.tikv.region.RegionStoreClient.handleCopResponse(RegionStoreClient.java:717)
        at com.pingcap.tikv.region.RegionStoreClient.coprocess(RegionStoreClient.java:664)
        at com.pingcap.tikv.operation.iterator.DAGIterator.process(DAGIterator.java:219)
        at com.pingcap.tikv.operation.iterator.DAGIterator.lambda$submitTasks$1(DAGIterator.java:90)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

The error does not affect the result output. The cluster configuration is as follows:

tispark_masters:
- host: xxx.146
  ssh_port: 22
  port: 7077
  web_port: 8080
  deploy_dir: /data/tispark/master
  spark_config:
    spark.driver.memory: 8g
  spark_env:
    SPARK_EXECUTOR_CORES: "12"
    SPARK_EXECUTOR_MEMORY: 12g
    SPARK_WORKER_CORES: "12"
    SPARK_WORKER_MEMORY: 12g
  arch: arm64
  os: linux
tispark_workers:
- host: xx.146
  ssh_port: 22
  port: 7078
  web_port: 8081
  deploy_dir: /data/tispark/worker/deploy
  arch: arm64
  os: linux
- host: xxx.151
  ssh_port: 22
  port: 7078
  web_port: 8081
  deploy_dir: /data/tispark/worker/deploy
  arch: arm64
  os: linux
- host: xx.152
  ssh_port: 22
  port: 7078
  web_port: 8081
  deploy_dir: /data/tispark/worker/deploy
  arch: arm64
  os: linux
| username: WalterWj | Original post link

You can consider configuring the timeout. spark.tispark.grpc.timeout_in_sec

| username: h5n1 | Original post link

Both the master and worker’s spark-defaults.conf have been updated, and the timeout has been adjusted to 300 seconds, but the error still occurs.

| username: h5n1 | Original post link

This parameter works well. The reason it didn’t take effect earlier was that the new parameter was deleted when using the tiup reload method to restart. Using the restart method works.

| username: h5n1 | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.