I’ll translate the article body directly, preserving all Markdown, code blocks, and internal links.
Only during the low-traffic hours late at night, or right after a DB restart, your logs fill up with HikariPool-1 - Failed to validate connection ... (Possibly consider using a shorter maxLifetime value). Nothing shows up during the day, so it is tempting to ignore. But when marked as broken or Connection is closed appear in the same time window and business processing fails, you can no longer look away.
This article sorts out which stage of the pool each message comes from, explains how to pick maxLifetime and keepaliveTime based on the values in your own environment, covers how to isolate cases where the pool does not recover after a DB restart, and provides steps to reproduce the problem with Docker Compose.
The target is Spring Boot 3.x (HikariCP 5.x to 6.x; keepaliveTime is available from HikariCP 4.0.1 onward), covering both MySQL Connector/J and the PostgreSQL JDBC driver. Note that Connection is not available, request timed out is a separate issue: pool exhaustion. For how to size maximumPoolSize and how to detect connection leaks, see the HikariCP tuning article.
Which Stage of the Pool Each Log Message Comes From
Let’s start by looking at real logs.
WARN com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@1a2b3c4d (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
WARN com.zaxxer.hikari.pool.ProxyConnection : HikariPool-1 - Connection com.mysql.cj.jdbc.ConnectionImpl@5e6f7a8b marked as broken because of SQLSTATE(08S01), ErrorCode(0)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction
Caused by: java.sql.SQLException: Connection is closed| Message | Source and when it occurs | Actual impact |
|---|---|---|
| Failed to validate connection | HikariCP. Exception during the isValid() check at borrow time (or keepalive). WARN | Essentially none. Only the latency of recreating the connection |
| Communications link failure, No operations allowed after connection closed. | Connector/J. A dead connection that slipped past validation surfaces as a SQLException while in use | That operation fails |
| This connection has been closed., An I/O error occurred while sending to the backend. | pgjdbc. Same as above | That operation fails |
| marked as broken | HikariCP. Logs a WARN right after detecting the exception above, and evicts the connection from the pool on return | A record of the same event |
| Connection is closed | HikariCP. An attempt to use a connection that was already evicted or returned | Use the preceding log line to tell whether it was caused by a disconnect or by use-after-close |
If you see Failed to validate connection on its own, it means HikariCP is correctly rejecting a dead connection. However, if many dead connections remain and validationTimeout is long, the validate-and-evict loop can eat up the entire connectionTimeout, turning into the pool-exhaustion side Connection is not available.
java.sql.SQLException: Connection is closed is split into two cases based on the preceding log line. If marked as broken or a driver disconnect exception appears immediately before on the same thread, the cause is a disconnect. At the moment of marked as broken, HikariCP swaps the underlying connection for a closed state, so subsequent SQL within the same @Transactional and Spring’s rollback processing fail with Connection is closed. The root cause is the DB or network disconnect itself while in use, so the maxLifetime and keepaliveTime settings in this article can reduce how often it happens.
On the other hand, if it appears alone without any disconnect log, it is use-after-close: the application keeps using a connection that has already been returned. Typical patterns include reading a Stream or ResultSet after leaving @Transactional, or self-invocation where @Transactional is not applied.
// NG: readOnly トランザクションを抜けた後で Stream を消費している
Stream<Order> stream = orderService.streamAll(); // 中で @Transactional(readOnly = true)
stream.forEach(this::export); // ここで Connection is closed
// OK: 消費までを @Transactional の範囲に収める
@Transactional(readOnly = true)
public void exportAll() {
try (Stream<Order> stream = orderRepository.streamAll()) {
stream.forEach(this::export);
}
}Configuration will not fix this case, so review your transaction boundaries instead.
HikariCP’s Validation Flow When Borrowing a Connection
When getConnection() takes a connection from the pool, HikariCP validates only connections that have been idle for 500ms (aliveBypassWindowMs) or more since last use, using JDBC’s isValid(). This check must complete within validationTimeout (default 5 seconds), and connections that fail are discarded and recreated. The keepalive described later uses the same validation routine, so it produces the same WARN.
Retirement via maxLifetime is scheduled as an individual task per connection, executed at “creation time + maxLifetime” shifted earlier by up to 2.5%. The housekeeper, running on a 30-second cycle, handles eviction of connections exceeding idleTimeout and replenishment up to minimumIdle.
In other words, the warning appears because something external is cutting the connection before retirement. Disconnects by the DB or network equipment are invisible to HikariCP, which is why the warnings cluster in the late-night hours after long idle periods.
Root Cause 1: The DB-Side Timeout Is Shorter Than maxLifetime
This is the most common pattern. The default maxLifetime is 30 minutes (1800000ms), but if the DB side cuts idle connections sooner than that, dead connections remain in the pool.
First, check the values in your own environment.
-- MySQL(JDBC は非対話型なので wait_timeout が効く)
SHOW VARIABLES LIKE 'wait_timeout';
-- PostgreSQL
SHOW idle_session_timeout; -- PG14 以降。0 なら無効
SHOW idle_in_transaction_session_timeout; -- トランザクション中のアイドルのみ対象MySQL’s wait_timeout defaults to 8 hours, but it is often shortened to anywhere from a few minutes to a few tens of minutes by cloud DB services or operational policies. PostgreSQL does not cut idle connections out of the box, but the same thing happens if either of the two settings above is configured. Note that statement_timeout cancels running queries and is unrelated here. The tcp_keepalives_* settings make the server send keepalives; they are not a cause of idle disconnects and actually work to prevent disconnects by NAT or firewalls.
The official HikariCP documentation also states clearly that maxLifetime “should be several seconds shorter than any database or infrastructure imposed connection time limit” and strongly recommends this.
Root Cause 2: Idle Disconnects by Firewalls, NAT, and Load Balancers
If the DB-side value is long enough but the warning still appears, suspect the network equipment along the path. Firewalls, NAT, and load balancers silently drop idle TCP sessions after roughly 5 to 15 minutes. AWS NLB uses 350 seconds, and Azure Load Balancer defaults to 4 minutes. The same applies when going through a cloud DB proxy.
The tricky part is that this disconnect is invisible to both the DB and the application. The DB side still appears to have the connection, while the application side only discovers the failure at the next borrow-time validation. If “wait_timeout is still 8 hours but the warning appears after 10 minutes idle”, this is almost certainly the cause.
How to Decide maxLifetime and keepaliveTime
The rule is simple: set maxLifetime 30 seconds to a few minutes shorter than the shortest value among the DB-side timeout and the network equipment idle timeouts.
For example, with wait_timeout=600 seconds and an NLB (350 seconds) in between, the shortest is 350 seconds. Leaving some margin, you would set maxLifetime to 300 seconds (300000ms). However, making it too short increases connection recreation, raising latency and DB load, so if you go below a few minutes, combine it with keepaliveTime described next.
keepaliveTime periodically sends an isValid() ping on idle connections to tell the network equipment and DB “this is still in use”. It must be at least 30 seconds and shorter than maxLifetime; a good guideline is about half the idle timeout. For an NLB at 350 seconds, that comes to around 120 to 180 seconds.
maxLifetime is an approach that retires connections, while keepaliveTime is an approach that extends their life. With both set, keepalive prevents disconnects while connections that reach their lifetime are replaced in a planned manner.
application.yml Example and Consistency Conditions for Each Value
Here is an example for MySQL.
spring:
datasource:
url: jdbc:mysql://db:3306/app
hikari:
maximum-pool-size: 10
minimum-idle: 2 # 省略すると maximum-pool-size と同値の固定サイズプール
connection-timeout: 3000 # ms
validation-timeout: 1000 # connection-timeout より短く
max-lifetime: 300000 # 5分。最短タイムアウトより短く
keepalive-time: 120000 # 2分。max-lifetime より短く
idle-timeout: 240000 # 4分。max-lifetime より短くFor PostgreSQL, everything under hikari is the same, and the URL becomes jdbc:postgresql://db:5432/app?tcpKeepAlive=true. pgjdbc’s tcpKeepAlive defaults to false, so it must be set explicitly; this lets OS-level TCP keepalive work as a supplementary measure. Connector/J’s tcpKeepAlive defaults to true.
Each value has consistency conditions. If you violate idleTimeout < maxLifetime, keepaliveTime < maxLifetime, or maxLifetime >= 30000, HikariCP logs a warning at startup and disables or corrects the value. Also, in a fixed-size pool where minimumIdle is omitted or equal to maximumPoolSize, idleTimeout has no effect, and setting it produces the third WARN below.
WARN com.zaxxer.hikari.HikariConfig : HikariPool-1 - idleTimeout is close to or more than maxLifetime, disabling it.
WARN com.zaxxer.hikari.HikariConfig : HikariPool-1 - keepaliveTime is greater than or equal to maxLifetime, disabling it.
WARN com.zaxxer.hikari.HikariConfig : HikariPool-1 - idleTimeout has been set but has no effect because the pool is operating as a fixed size pool.On the other hand, validationTimeout < connectionTimeout is not subject to automatic correction (only the lower bound below 250ms is corrected), so you have to uphold it yourself. MySQL URL parameters in general are covered in the MySQL connection configuration article.
Why You Should Not Casually Set connectionTestQuery
Searching the web often turns up outdated advice saying “set connection-test-query: SELECT 1 and it will be fixed”, but with Spring Boot 3.x this is generally unnecessary.
With JDBC4-compliant drivers such as Connector/J 8.x and later or pgjdbc 42.x, isValid() uses a lightweight protocol-level ping. Setting connectionTestQuery replaces that with a SQL round trip, making every borrow slightly heavier. HikariCP officially states that it strongly recommends not setting it if your driver supports JDBC4.
Fundamentally, this setting only changes the “method” of validation, not the “timing” at which a dropped connection is detected. It does nothing to address Failed to validate connection, so deal with it via maxLifetime and keepaliveTime.
How the Pool Recovers Naturally After a DB Restart or Failover
When the DB restarts, every connection in the pool is dropped. After that, each borrow triggers “validation failure, eviction, recreation”, and connections are replaced one by one. During this time, Failed to validate connection appears consecutively once per connection, but this is a normal recovery process.
Recovery time is roughly the recreation of all connections plus connectionTimeout, usually a few seconds to a few tens of seconds. Replenishment runs asynchronously through borrow requests and the housekeeper every 30 seconds, so restarting the application is not required. However, connections that were in use at the moment of the restart become driver exceptions and marked as broken, and those requests fail. If you want to bring this close to zero, retrying with Spring Retry is an option.
Isolating Cases That Do Not Recover
If the warnings do not stop after waiting a few minutes, or no connections can be created at all, check the following in order.
Start with the DNS cache. Even if the DB endpoint’s IP changes during failover, if the old IP is cached somewhere, the application keeps connecting to the old server. The JVM’s networkaddress.cache.ttl defaults to 30 seconds in a normal configuration without a security manager, so it is not a problem out of the box. The pitfall is environments where this value has been explicitly set to -1 (indefinite) or a long number of seconds (in older setups with the security manager enabled, the default is -1).
# $JAVA_HOME/conf/security/java.security や -Dsun.net.inetaddr.ttl を確認
# こうなっていたら 30 秒以下に戻す
networkaddress.cache.ttl=-1If the JVM side is fine, look at the container or OS resolver cache and the DNS TTL on the DB endpoint side (5 seconds for RDS). It is quickest to resolve the current IP with dig and compare it against where the application is actually connected using ss.
There are also cases where validationTimeout is too short. A DB that has just started up responds slowly, and if validation keeps timing out, connections are discarded as soon as they are created. If DEBUG logs show that the validation failure reason is a timeout, try extending validationTimeout a little.
connectionInitSql failures are also easy to overlook. If the initialization SQL executed on every recreation fails on the failover target due to insufficient privileges or similar, not a single connection can be created.
Note that if the DB is down when the application starts, startup fails due to initializationFailTimeout (default 1ms). If you are using Spring Data JPA (Hibernate), it connects during startup for metadata retrieval regardless of the ddl-auto value (the default of hibernate.boot.allow_jdbc_metadata_access is true), so this applies just as it does for Flyway, Liquibase, schema.sql, and a CommandLineRunner that uses the DB. For isolating that, see the startup failure article.
Reproducing With Docker Compose and Confirming the Fix
Once you understand the theory, reproduce it locally and confirm the fix works. Start MySQL with wait_timeout set to 60 seconds.
services:
db:
image: mysql:8.4
command: --wait_timeout=60
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: app
ports:
- "3306:3306"The application runs on the host side, so the connection target is localhost. The only things to toggle are the two commented lines.
spring:
datasource:
url: jdbc:mysql://localhost:3306/app
username: root
password: root
hikari:
maximum-pool-size: 3
# max-lifetime: 50000 # パターン2で有効化(wait_timeout=60 より短く)
# keepalive-time: 30000 # パターン3で有効化
logging:
level:
com.zaxxer.hikari: DEBUGAn endpoint that simply issues SELECT 1 is sufficient.
@RestController
public class PingController {
private final JdbcTemplate jdbcTemplate;
public PingController(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
@GetMapping("/ping")
public Integer ping() {
// 借り出し → SELECT 1 → 返却
return jdbcTemplate.queryForObject("SELECT 1", Integer.class);
}
}Hit /ping once, leave it for about 70 seconds, then hit it again. Extracting the key lines for the three patterns, the logs look like this.
# パターン1: max-lifetime デフォルト。放置後の1回目のリクエスト
WARN HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@6d2a1f (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
DEBUG HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@6d2a1f: (connection is dead)
DEBUG HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@7c31b0
# パターン2: max-lifetime: 50000。50秒弱ごとに接続が入れ替わり、警告は出ない
DEBUG HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@6d2a1f: (connection has passed maxLifetime)
DEBUG HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@0e8f42
# パターン3: keepalive-time: 30000。30秒弱ごとに ping が飛び、警告は出ない
DEBUG HikariPool-1 - keepalive: connection com.mysql.cj.jdbc.ConnectionImpl@6d2a1f is aliveIn pattern 2, the per-connection retirement task replaces the connection before wait_timeout, so it disappears before it dies. In pattern 3, the ping resets MySQL’s idle counter, so the connection is never cut in the first place. Either way the warning goes away, but keep in mind the semantic difference between retirement and life extension.
To reproduce with PostgreSQL, specify command: -c idle_session_timeout=60000 and you can confirm with the same steps.
How to Observe in Production
After changing the settings, check with numbers whether they are working in production. Actuator’s hikaricp.connections.creation is an indicator of connection recreation; if there is a time window where it spikes, something is still cutting connections somewhere. Meanwhile, hikaricp.connections.timeout, active, and pending are pool-exhaustion indicators, so viewing them by role makes isolation easier.
management:
endpoints:
web:
exposure:
include: health, metrics, prometheus
logging:
level:
com.zaxxer.hikari: DEBUG # 調査中だけ有効にするSetting com.zaxxer.hikari to DEBUG makes the housekeeper emit statistics every 30 seconds. If idleTimeout is active, you get Before cleanup stats and After cleanup stats; for a fixed-size pool or idleTimeout=0, the wording is Pool stats (total=10, active=0, idle=10, waiting=0). The log volume is large, so limit it to a specific period in production. For setting up Actuator, see Getting Started with Actuator, and for caveats when incorporating DB health checks into monitoring, see the custom HealthIndicator article.
Summary
Failed to validate connection is a validation failure at borrow time and is proof that HikariCP correctly rejected a dead connection. Actual harm occurs when a connection slips past validation and is cut while in use, which shows up as a driver exception, marked as broken, and the subsequent Connection is closed. Only a Connection is closed without any accompanying disconnect log is use-after-close, which requires a fix on the application side.
There are two axes for the remedy: set maxLifetime shorter than the shortest timeout among the DB and network equipment, and set keepaliveTime shorter still to keep idle connections alive. connectionTestQuery is unnecessary, and the pool normally recovers on its own after a DB restart. If it does not recover, suspect the DNS cache and validationTimeout.
Connection is not available, which involves pool size and connection leaks, is covered in the HikariCP tuning article.
