TiFlash cannot start

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tiflash无法启动

| username: heeewei

[TiDB Usage Environment] Production Environment
[TiDB Version] 6.5.0
[Reproduction Path] Operations performed that led to the issue
[Encountered Issue: Problem Phenomenon and Impact] tiflash server cannot start, reporting “No such file or directory: while stat a file for size.” What could be the reason? Detailed logs are as follows:
[2023/07/20 20:53:40.988 +08:00] [FATAL] [lib.rs:495] [“[region 574303310] 574303314 ingest uuid: 1DFED8A614F745618D6387463118FB4E range { start: 748000000000006EFF055F728000000000F F0000010000000000FA end: 748000000000006EFF055F728000000000FF0028800000000000FA } cf_name: "default" region_id: 574303310 region_epoch { conf_ver: 59234 version: 58269 }: EngineTr aits(Engine(Status { code: IoError, sub_code: None, sev: NoError, state: "IO error: No such file or directory: while stat a file for size: /u01/tidb/tidb-data/tiflash-19000/flash/i mport/1dfed8a6-14f7-4561-8d63-87463118fb4e_574303310_59234_58269_default.sst: No such file or directory" }))”] [backtrace=" 0: tikv_util::set_panic_hook::{{closure}}\n 1: <allo c::boxed::Box<F,A> as core::ops::function::Fn>::call\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/alloc/src/boxed.rs:2032:9\n std::panicking:: rust_panic_with_hook\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/std/src/panicking.rs:692:13\n 2: std::panicking::begin_panic_handler::{{closure}}\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/std/src/panicking.rs:579:13\n 3: std::sys_common::backtrace::__rust_end_short_backtrace\n at /rus tc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/std/src/sys_common/backtrace.rs:137:18\n 4: rust_begin_unwind\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/ library/std/src/panicking.rs:575:5\n 5: core::panicking::panic_fmt\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/core/src/panicking.rs:65:14\n 6: raft store::store::fsm::apply::ApplyDelegate::apply_raft_cmd\n 7: raftstore::store::fsm::apply::ApplyDelegate::process_raft_cmd\n 8: raftstore::store::fsm::apply::ApplyDelega te::handle_raft_committed_entries\n 9: <raftstore::store::fsm::apply::ApplyPoller as batch_system::batch::PollHandler<raftstore::store::fsm::apply::ApplyFsm,raftstore: :store::fsm::apply::ControlFsm>>::handle_normal\n 10: batch_system::batch::Poller<N,C,Handler>::poll\n 11: std::sys_common::backtrace::__rust_begin_short_backtrace\n 12: core::op s::function::FnOnce::call_once{{vtable.shim}}\n 13: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b171 0a8cebeef/library/alloc/src/boxed.rs:2000:9\n <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a 8cebeef/library/alloc/src/boxed.rs:2000:9\n std::sys::unix::thread::thread::new::thread_start\n at /rustc/96ddd32c4bfb1d78f0cd03eb068b1710a8cebeef/library/std/src/s ys/unix/thread.rs:108:17\n 14: \n 15: __clone\n"] [location=/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tics/contrib/tiflash-proxy/components/raf tstore/src/store/fsm/apply.rs:1921] [thread_name=apply-low-0]

[Resource Configuration] Go to TiDB Dashboard - Cluster Info - Hosts and take a screenshot of this page
[Attachments: Screenshots/Logs/Monitoring]

| username: ShawnYan | Original post link

Does this file not exist? Or is it corrupted?

| username: tidb菜鸟一只 | Original post link

Why do I see a space in the middle of “import”? Is it a display issue with the forum, or is there a problem with your path?

| username: redgame | Original post link

Check if /u01/tidb/tidb-data/tiflash-19000/flash/import/1dfed8a6-14f7-4561-8d63-87463118fb4e_574303310_59234_58269_default.sst exists.

| username: heeewei | Original post link

It doesn’t exist, but I don’t know what this file is for or why it doesn’t exist.

| username: heeewei | Original post link

Copy issue, no spaces

| username: heeewei | Original post link

There is no such file, but I don’t know why there isn’t one.

| username: ShawnYan | Original post link

The line breaks when copied from the screen.

| username: tidb菜鸟一只 | Original post link

Is your TiFlash newly deployed or did it suddenly report an error for some reason?

| username: heeewei | Original post link

It has been running normally, possibly related to writing data through TiSpark. The table being written to has TiFlash replicas enabled. After this error occurred, I don’t know how to fix it and can only redeploy TiFlash.

| username: tidb菜鸟一只 | Original post link

If only a specific table has an issue, you can directly use ALTER TABLE table_name TIFLASH REPLICA 0, then restart TiFlash and try ALTER TABLE table_name TIFLASH REPLICA 1.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.