Can TiDB logs be printed asynchronously?

This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: tidb日志能异步打印嘛

| username: TiDBer_jTV6VWod

【TiDB Usage Environment】Production Environment
【TiDB Version】6.6.0
【Encountered Problem: Problem Description and Impact】Is there an asynchronous logging configuration for TiDB? I see that version 6.6 logs are all printed synchronously without involving other threads. Now I want to do some performance tuning by putting the logs into a cache and handling them with a separate thread. I think TiDB, being so comprehensive, should have considered this.

| username: zhaokede | Original post link

I haven’t studied it, mainly using the database.

| username: Kongdom | Original post link

Do you think there are too many logs? You can adjust the log level to only record error-level logs.

| username: Inkjade | Original post link

Why log asynchronously? If the system crashes at this time, won’t the logs be lost? This is obviously unreasonable.

| username: Jack-li | Original post link

It seems that it can only be achieved through some middleware.

| username: TiDBer_jTV6VWod | Original post link

Indeed, but when the boss gives an order, we have to work tirelessly to make it happen…

| username: Kongdom | Original post link

Is the printing you mentioned related to log files or raft logs?

| username: dba远航 | Original post link

I think what you mean is the log replay feature, right?!

| username: yytest | Original post link

  • Custom Logger: You can create a custom Logger that starts a new goroutine to handle log output. This way, the main business logic can continue executing without waiting for the log writing to complete.
  • Buffered Queue: Use a buffered queue to temporarily store log entries to be processed. The main thread can return immediately after placing log entries into the queue, while another goroutine is responsible for retrieving and processing the log entries from the queue.
| username: hey-hoho | Original post link

The logging framework used by TiDB is zap, which I understand is inherently asynchronous. Could you describe in more detail what you mean by synchronous printing in version 6.6?

| username: 小龙虾爱大龙虾 | Original post link

+1 I feel like this framework can’t be this bad, right? :joy_cat:

| username: TiDBer_jTV6VWod | Original post link

Log file

| username: Kongdom | Original post link

:thinking: Log entries seem to be asynchronous, right? Why do you say it’s synchronous logging?

| username: FutureDB | Original post link

I understand as well. If it were synchronous printing, executing an SQL would have to wait for the log to finish printing. That would be too cumbersome.

| username: YuchongXU | Original post link

No longer supported.