Spring Boot app (Hikari + Quartz + MySQL) randomly loses DB connections, floods logs until restart

I have a Spring Boot app using HikariCP (MySQL on AWS RDS) and Quartz with a JDBC job store. It runs fine for days, then suddenly stops getting DB connections. Logs fill up, and it won’t recover until I restart. After restart, it runs fine again. --- Example logs
{"log":"The last packet sent successfully... driver has not received any packets from the server."}
{"log":"Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure"}
{"log":"at com.zaxxer.hikari.HikariDataSource.getConnection"}
{"log":"Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available..."}
{"log":"WARN HikariPool-1 - Failed to validate connection... Possibly consider using a shorter maxLifetime value."}
{"log":"The last packet sent successfully... driver has not received any packets from the server."}
{"log":"Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure"}
{"log":"at com.zaxxer.hikari.HikariDataSource.getConnection"}
{"log":"Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available..."}
{"log":"WARN HikariPool-1 - Failed to validate connection... Possibly consider using a shorter maxLifetime value."}
--- Hikari config
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.idle-timeout=600000
spring.datasource.hikari.max-lifetime=1800000
spring.datasource.hikari.keepalive-time=60000
spring.datasource.hikari.validation-timeout=5000
spring.datasource.hikari.connection-test-query=SELECT 1
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.idle-timeout=600000
spring.datasource.hikari.max-lifetime=1800000
spring.datasource.hikari.keepalive-time=60000
spring.datasource.hikari.validation-timeout=5000
spring.datasource.hikari.connection-test-query=SELECT 1
--- Question Why does this keep happening? Why doesn’t Hikari just rebuild connections instead of flooding errors? Should I tune maxLifetime, keepaliveTime, or something else to stop it from hanging until restart? Any best practices to fix this are appreciated. Thanks.
1 Reply
JavaBot
JavaBot5mo ago
This post has been reserved for your question.
Hey @ysmr! Please use /close or the Close Post button above when your problem is solved. Please remember to follow the help guidelines. This post will be automatically marked as dormant after 300 minutes of inactivity.
TIP: Narrow down your issue to simple and precise questions to maximize the chance that others will reply in here. 💤 Post marked as dormant
This post has been inactive for over 300 minutes, thus, it has been archived. If your question was not answered yet, feel free to re-open this post or create a new one. In case your post is not getting any attention, you can try to use /help ping. Warning: abusing this will result in moderative actions taken against you.

Did you find this page helpful?