How to Specify the Use of TiKV/TiFlash in TiSpark

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tispark 如何指定使用 tikv/tiflash

| username: ShawnYan

In TiDB, you can specify reading data from TiKV/TiFlash using hints.

However, Spark SQL does not recognize this. How can I use it? Please advise.

spark-sql> select /*+ read_from_storage(tikv[test.t1]) */ * from test.t1;
23/07/20 11:11:23 WARN HintErrorLogger: Unrecognized hint: read_from_storage('tikv['test.t1])
| username: shiyuhang0 | Original post link

spark-sql does not yet support the corresponding hint, nor can it perform statement-level configuration in spark-sql.
Currently, you can only:

  1. Configure in the configuration file, but this is not statement-level.
  2. Use programmable methods like spark-shell/pyspark, which can perform statement-level configuration spark.tispark.isolation_read_engines tispark/docs/userguide_3.0.md at master · pingcap/tispark · GitHub
| username: ShawnYan | Original post link

Based on actual tests, it seems that statement-level configuration is not very effective. Although the parameters are modified, it still automatically chooses to use TiKV or TiFlash. Only after modifying the configuration file and restarting the worker does it truly take effect.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.