Tikv_gc_safe_point is not advancing

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tikv_gc_safe_point 一直不推进

| username: wfxxh

How to troubleshoot this issue?

| username: h5n1 | Original post link

Check the pd-ctl service-gc-safepoint.

| username: wfxxh | Original post link

I checked, one page is not enough to capture it all. There are a few more that were added when tispark specified stale_read.

| username: h5n1 | Original post link

This caused it.

| username: wfxxh | Original post link

Can it be deleted? I did not find a delete command in pd-ctl service-gc-safepoint.

| username: h5n1 | Original post link

Take a look at any TiSpark tasks that are running continuously or not functioning properly. These cannot be deleted directly, as they are set with a safepoint to ensure that the necessary data is not garbage collected during task execution.

| username: wfxxh | Original post link

No, it has already been stopped.

| username: h5n1 | Original post link

I don’t quite understand how to handle Spark tasks? It seems that when you stopped the task, you didn’t clean up the corresponding safepoint information in PD. If the task is not needed, try deleting it.

| username: wfxxh | Original post link

Could you please advise on how to delete? I couldn’t find the relevant command.

| username: h5n1 | Original post link

Handling TiSpark tasks, I’m not very familiar with this either.

| username: h5n1 | Original post link

Are there still TiSpark nodes now?

| username: wfxxh | Original post link

We are using the TiSpark jar package and have not deployed TiSpark.

| username: h5n1 | Original post link

I don’t know how to clean up Spark, but when encountering this issue with CDC, we use the command tiup cdc:v5.1.0 cli --pd=<PD_ADDRESS> unsafe reset to clean it up. I think you can give it a try.

| username: wfxxh | Original post link

Okay, thank you.

| username: 有猫万事足 | Original post link

I would like to know more about this. Is the basis for this judgment that the final gc_safe_point is equal to the safe_point of this TiSpark?

| username: h5n1 | Original post link

Yes.

| username: 有猫万事足 | Original post link

Okay, thank you very much.

| username: 数据小黑 | Original post link

I remember there is a listener that will clear the connection after the Spark task fails. Show the process list to see if there are any suspicious connections.

| username: wfxxh | Original post link

Not really.

| username: ShawnYan | Original post link

What is the spark/tispark version?

What output is there in the tispark log?