How to completely destroy all intermediate data associated with TiDB Lightning?

FAQ
1

  1. Delete the checkpoint file.

    tidb-lightning-ctl --config conf/tidb-lightning.toml --checkpoint-remove=all

    If, for some reason, you cannot run this command, try manually deleting the file /tmp/tidb_lightning_checkpoint.pb.

  2. If you are using Local-backend, delete the sorted-kv-dir directory in the configuration. If you are using Importer-backend, delete the entire import directory on the machine hosting tikv-importer.

  3. Delete all tables and databases created on the TiDB cluster, if needed.

  4. Clean up the residual metadata. You need to clean up the metadata schema manually if either of the following conditions exist.

    • For TiDB Lightning v5.1.x and v5.2.x versions, the tidb-lightning-ctl command does not clean up the metadata schema in the target cluster. You need to clean it up manually.
    • If you have deleted the checkpoint files manually, you need to clean up the downstream metadata schema manually; otherwise, the correctness of subsequent imports might be affected.

    Use the following command to clean up the metadata:

    DROP DATABASE IF EXISTS `lightning_metadata`;