Jonathan Lalou's blog

Scenario

A Spring Boot application using ActiveMQ with @JmsListener suddenly stops receiving messages after running for a while. No errors in logs, and the queue keeps growing, but the consumers seem idle.

Setup

• ActiveMQConnectionFactory was used.

• The queue (myQueue) was filling up.

• Restarting the app temporarily fixed the issue.

Investigation

1. Checked ActiveMQ Monitoring (Web Console)

   • Messages were enqueued but not dequeued.

   • Consumers were still active, but not processing.

2. Thread Dump Analysis

   • Found that listener threads were stuck in a waiting state.

   • The problem only occurred under high load.

3. Checked JMS Acknowledgment Mode

   • Default AUTO_ACKNOWLEDGE was used.

   • Suspected an issue with message acknowledgment.

4. Enabled Debug Logging

   • Added:

     logging.level.org.springframework.jms=DEBUG

   • Found repeated logs like:

     JmsListenerEndpointContainer#0-1 received message, but no further processing

   • This hinted at connection issues.

5. Tested with a Different Message Broker

   • Using Artemis JMS instead of ActiveMQ resolved the issue.

   • Indicated that it was broker-specific.

Root Cause

ActiveMQ’s TCP connection was silently dropped, but the JMS client did not detect it.

• When the connection is lost, DefaultMessageListenerContainer doesn’t always recover properly.

• ActiveMQ does not always notify clients of broken connections.

• No exceptions were thrown because the connection was technically “alive” but non-functional.

Fix

1. Enabled keepAlive in ActiveMQ connection

   ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(); factory.setUseKeepAlive(true); factory.setOptimizeAcknowledge(true); return factory;

2. Forced Reconnection with Exception Listener

   • Implemented:

     factory.setExceptionListener(exception -> { log.error("JMS Exception occurred, reconnecting...", exception); restartJmsListener(); });

   • This ensured that if a connection was dropped, the listener restarted.

3. Switched to DefaultJmsListenerContainerFactory with DMLC

   • SimpleMessageListenerContainer was less reliable in handling reconnections.

   • New Configuration:

     @Bean public DefaultJmsListenerContainerFactory jmsListenerContainerFactory( ConnectionFactory connectionFactory) { DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory(); factory.setConnectionFactory(connectionFactory); factory.setSessionTransacted(true); factory.setErrorHandler(t -> log.error("JMS Listener error", t)); return factory; }

Final Outcome

✅ After applying these fixes, the issue never reoccurred.
🚀 The app remained stable even under high load.

Key Takeaways

• Silent disconnections in ActiveMQ can cause message listeners to hang.

• Enable keepAlive and optimizeAcknowledge for reliable connections.

• Use DefaultJmsListenerContainerFactory with DMLC instead of SMLC.

• Implement an ExceptionListener to restart the JMS connection if necessary.