Error Creating TiFlash Replica After Playground Startup: Access to File Denied

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: playground启动后,创建tiflash副本报错:Access to file denied

| username: TiDBevis

[TiDB Usage Environment] ubuntu1804 test
[TiDB Version] 7.6.0
[Reproduction Path]
Root installation and startup
After starting with tiup playground -T 20240311-tidb-test, generate sample data: tiup bench tpch --sf=1 prepare, then specify creating tiflash replicas:

ALTER TABLE test.customer SET TIFLASH REPLICA 1;
ALTER TABLE test.orders SET TIFLASH REPLICA 1;
ALTER TABLE test.lineitem SET TIFLASH REPLICA 1;

[Encountered Problem: Phenomenon and Impact]
Error:

tiflash quit: signal: segmentation fault (core dumped)
[2024/03/13 10:30:13.044 +08:00] [INFO] [SegmentReader.cpp:86] ["Pop fail, stop=true"] [thread_id=4]
[2024/03/13 10:30:13.045 +08:00] [INFO] [SegmentReader.cpp:86] ["Pop fail, stop=true"] [thread_id=5]
[2024/03/13 10:30:13.045 +08:00] [INFO] [SegmentReader.cpp:86] ["Pop fail, stop=true"] [thread_id=6]
[2024/03/13 10:30:13.045 +08:00] [INFO] [SegmentReader.cpp:86] ["Pop fail, stop=true"] [thread_id=2]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
[2024/03/13 10:30:13.045 +08:00] [DEBUG] [SegmentReader.cpp:45] [Stopped] [thread_id=58]
...
check detail log from: /root/.tiup/data/20240311-test/tiflash-0/tiflash.log

After the error, tiflash crashes.

[Resource Configuration]

[Attachments: Screenshots/Logs/Monitoring]
Log: tiflash_error.log

[2024/03/13 10:30:13.042 +08:00] [FATAL] [Exception.cpp:106] ["Poco::Exception. Code: 1000, e.code() = 13, e.displayText() = Access to file denied: /root/.tiup/data/20240311-test/tiflash-0/data/data/t_118/stable/.tmp.dmf_1, e.what() = Access to file denied"] [source="DB::RawCppPtr DB::PreHandleSnapshot(DB::EngineStoreServerWrap *, DB::BaseBuffView, uint64_t, DB::SSTViewVec, uint64_t, uint64_t)"] [thread_id=58]

I recursively modified the permissions of the .tmp.dmf_1 folder and found that the permissions were reset after restarting.

| username: 小龙虾爱大龙虾 | Original post link

Did you install and start it with root? Are there still permission issues? :thinking:

| username: TiDBevis | Original post link

Yes, that’s why I’m very puzzled by this issue.
So I wanted to see if anyone else has encountered it.

| username: zhanggame1 | Original post link

Don’t bother with this, deploying a single machine is also very simple.
TiDB Quick Start Guide | PingCAP Documentation Center

| username: TiDBevis | Original post link

That’s true, just thought that since we encountered it, we should investigate it.

| username: zhanggame1 | Original post link

The issues encountered with the playground are different from those in actual deployment, so studying the problems here is meaningless.

| username: TiDBevis | Original post link

Okay, got it. :+1:

| username: redgame | Original post link

Permission issue

| username: TiDBevis | Original post link

I just modified the file permissions, but after restarting, the file gets reset. I suspect that the file is being regenerated.

| username: TiDBer_aaO4sU46 | Original post link

Yes, it will reset. I change it every time to test if it still works.

| username: TiDBevis | Original post link

Hi there, after making changes, I’m not sure how to restart TiFlash. Could you please advise? :pray:

| username: ffeenn | Original post link

Try configuring sudo permissions.

| username: Kongdom | Original post link

What about restarting after adding --tag persistence?

| username: TiDBevis | Original post link

It will report an error immediately.

| username: Kongdom | Original post link

There may be two reasons for this. First, the TiDB cluster does not support deployment on Red Hat 6.

Second, it depends on how many columns the CPU line in the /proc/stat file has. As long as there are >=11 columns, there will be no problem. In the D version of Red Hat, if the number of columns is less, this error will occur. We will fix this issue in version 4.0.10. However, we still do not recommend deploying the TiDB cluster on Red Hat 6.

Upgrading to 7, you can first check by running cat /proc/stat.