Unable to Establish Connection

translator_bot · June 22, 2024, 11:57am

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: 拿不到连接

To improve efficiency, please provide the following information. A clear problem description will help resolve the issue faster:

[Overview] I am conducting a stress test. There are five stress testing machines continuously inserting data. The cluster has three TiDB instances, with load balancing done through tidb-loadbalance. Each stress testing machine initiates 300 threads and 300 database connection tests. After a period of stress testing, an exception occurs indicating that a connection cannot be obtained. This issue started appearing after generating approximately 3 billion records, and it was functioning normally before that.
java.util.concurrent.CompletionException: org.springframework.transaction.CannotCreateTransactionException: Could not open JDBC Connection for transaction
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not open JDBC Connection for transaction
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(DataSourceTransactionManager.java:309)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:601)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:702)
at com.example.demo.service.MakeDataService$$SpringCGLIB$$0.makeData()
at com.example.demo.service.MyService.lambda$start$10(MyService.java:146)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
… 3 common frames omitted
Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:181)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:146)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(DataSourceTransactionManager.java:265)
… 14 common frames omitted
Caused by: java.sql.SQLException: can not get connection
at com.tidb.jdbc.LoadBalancingDriver.connect(LoadBalancingDriver.java:276)
at com.tidb.jdbc.LoadBalancingDriver.connect(LoadBalancingDriver.java:286)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:733)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
… 3 common frames omitted
After the exception occurs, I can only restart the stress testing program. What is causing this issue and how can it be resolved? Thank you.