Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

glassfishrobot · 2013-08-17T12:58:06Z

When we are killing our glassfish with kill -9 because of the following bug: https://java.net/jira/browse/MQ-340

We are not able anymore to directly restart the server. The following message appears in the server.log:

[#|2013-08-08T08:42:36.216+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|Exception in thread "Recovery Helper Thread" |#]

[#|2013-08-08T08:42:36.218+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|java.lang.NoClassDefFoundError: com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl
	at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:97)
	at com.sun.messaging.jms.ra.ManagedConnection.<init>(ManagedConnection.java:196)
	at com.sun.messaging.jms.ra.ManagedConnectionFactory.createManagedConnection(ManagedConnectionFactory.java:226)
	at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:292)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:210)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:234)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:331)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:106)
	at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
	at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
	at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
	at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
	at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
	at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
	at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1050)
	at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:358)
|#]

We are running JMS in embedded mode and it is doing recovery on restart.

The problem here is that the JMS system is started for automatic recovery and the stomp bridge is started too. Unfortunately the classloader for the recovery thread seams to have problems to load UnifiedConnectionImpl.

I also added some details in the following thread:
https://www.java.net/forum/topic/glassfish/glassfish/noclassdeffounderror-when-using-stomp-bridge-openmq-one-cluster-node

The following work around works:

Undeployed application
Restarted both nodes
Redeployed application

Environment

Glassfish 3.1.2.2, Linux, Java 7

Affected Versions

[4.5.2]

The text was updated successfully, but these errors were encountered:

glassfishrobot · 2013-08-17T12:58:06Z

@glassfishrobot Commented
Reported by manuel_b

glassfishrobot · 2013-08-18T01:01:57Z

@glassfishrobot Commented
@amykang2020 said:
Could you please provide a reproducible test case (example application, steps) ? Thanks

glassfishrobot · 2013-08-19T14:06:59Z

@glassfishrobot Commented
manuel_b said:

Install Glassfish 3.1.2.2 into /opt/glassfish3
start glassfish DAS```
/opt/glassfish3/bin/asadmin start-domain
Create a cluster```
/opt/glassfish3/bin/asadmin create-cluster local-cluster
Create a cluster node with SSH config via admin console```
/opt/glassfish3/bin/asadmin create-instance --node --cluster local-cluster

Authentication failed with password from login store: /Users/manuel/.asadminpass

Enter admin password for user "admin">

Enter the value for the instance_name operand> instance1
Start instance1```
/opt/glassfish3/bin/asadmin start-local-instance
Get test war with one mdb and deploy it```
wget --no-check-certificate https://dev.apaxo.de:8181/nexus/content/repositories/thirdparty/org/openmq/stomp/bugs/MQ-341/0.0.1/MQ-341-0.0.1.war /opt/glassfish3/bin/asadmin deploy --target local-cluster MQ-341-0.0.1.war
This will generate imq configuration. imq won't work because of a bug that local cluster gets the ip 127.0.0.1 (https://java.net/jira/browse/MQ-118)
Stop local instance

/opt/glassfish3/bin/asadmin stop-local-instance

Add the following to: /opt/glassfish3/glassfish/nodes/localhost-domain1/instance1/imq/instances/localclusterinstance1/props/config.properties

imq.bridge.enabled=true
imq.bridge.activelist=stomp
imq.instanceconfig.version=300
imq.bridge.admin.password=admin
imq.jmsra.managed=true
imq.bridge.admin.user=admin

/opt/glassfish3/bin/asadmin start-local-instance

Kill local instance with kill -9

Restart local instance

/opt/glassfish3/bin/asadmin start-local-instance

Observe error:

[#|2013-08-08T08:42:36.216+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|Exception in thread "Recovery Helper Thread" |#]

[#|2013-08-08T08:42:36.218+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|java.lang.NoClassDefFoundError: com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl
	at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:97)
	at com.sun.messaging.jms.ra.ManagedConnection.<init>(ManagedConnection.java:196)
	at com.sun.messaging.jms.ra.ManagedConnectionFactory.createManagedConnection(ManagedConnectionFactory.java:226)
	at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:292)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:210)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:234)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:331)
	at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:106)
	at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
	at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
	at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
	at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
	at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
	at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
	at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1050)
	at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:358)
|#]

glassfishrobot · 2013-08-19T14:07:52Z

@glassfishrobot Commented
manuel_b said:
@amyk unfortunately I was not able to get the local cluster working therefore this is only a theoretical explanation.

glassfishrobot · 2013-08-21T07:49:44Z

@glassfishrobot Commented
manuel_b said:
This ticket might be connected to:

https://java.net/jira/browse/GLASSFISH-17971

glassfishrobot · 2013-08-21T15:51:13Z

@glassfishrobot Commented
manuel_b said:
I debugged a little further and figured out that the OSGI Bundle Classloader is used for trying for finding this class. This does not work.

glassfishrobot · 2013-08-21T15:54:43Z

@glassfishrobot Commented
manuel_b said:
I now copied the following files into /opt/glassfish3/glassfish/lib and it works:

cd /opt/glassfish3/glassfish/lib
cp /opt/glassfish3/glassfish/lib/install/applications/jmsra/*.jar .

glassfishrobot · 2013-08-21T16:04:49Z

@glassfishrobot Commented
manuel_b said:
To be concrete I copied the following files:

-rw-rw-r--  1 ec2-user ec2-user   99926 21\. Aug 15:59 fscontext.jar
-rw-rw-r--  1 ec2-user ec2-user 2110762 21\. Aug 15:59 imqbroker.jar
-rw-rw-r--  1 ec2-user ec2-user  521942 21\. Aug 15:42 imq.jar
-rw-rw-r--  1 ec2-user ec2-user  195921 21\. Aug 15:59 imqjmsbridge.jar
-rw-rw-r--  1 ec2-user ec2-user 1525978 21\. Aug 15:59 imqjmsra.jar
-rw-rw-r--  1 ec2-user ec2-user   53723 21\. Aug 15:59 imqjmx.jar
-rw-rw-r--  1 ec2-user ec2-user   98191 21\. Aug 15:59 imqstomp.jar
-rw-rw-r--  1 ec2-user ec2-user  357149 21\. Aug 15:48 imqutil.jar

glassfishrobot · 2013-08-21T16:15:17Z

@glassfishrobot Commented
@nigeldeakin said:
Thanks for the report, the clear detail, your investigations and analysis, and the workaround.
As you suggest the problem appears to be that the JMSRA resource adapter does not appear
to be available to the class loader that is used during this recovery operation.
Obviously this should be investigated further.

Your report reminded me of this question on the forum
https://www.java.net/forum/topic/glassfish/glassfish/noclassdeffounderror-when-using-stomp-bridge-openmq-one-cluster-node
which I now see was from you.

glassfishrobot · 2013-08-21T16:22:25Z

@glassfishrobot Commented
@nigeldeakin said:
I agree that https://java.net/jira/browse/GLASSFISH-17971 looks similar.
That issue was apparently fixed in GlassFish 3.1.2_b20
Which version of GlassFish were you using?

(Updated) I see the version used was 3.1.2.2 (July 2012).
That should contain the fix to GLASSFISH-17971. Will follow up.

glassfishrobot · 2017-05-09T18:17:23Z

@glassfishrobot Commented
This issue was imported from java.net JIRA MQ-341

glassfishrobot · 2018-02-10T16:52:54Z

Issue Imported From: https://github.com/javaee/openmq/issues/341
Original Issue Raised By:@glassfishrobot
Original Issue Assigned To: @edbratt

glassfishrobot added Priority: Major Type: Bug labels Feb 10, 2018

glassfishrobot assigned edbratt Feb 10, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

glassfishrobot commented Aug 17, 2013

glassfishrobot commented Aug 17, 2013

glassfishrobot commented Aug 18, 2013

glassfishrobot commented Aug 19, 2013

Authentication failed with password from login store: /Users/manuel/.asadminpass

Enter admin password for user "admin">

Enter the value for the instance_name operand> instance1

glassfishrobot commented Aug 19, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented May 9, 2017

glassfishrobot commented Feb 10, 2018

Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

Comments

glassfishrobot commented Aug 17, 2013

Environment

Affected Versions

glassfishrobot commented Aug 17, 2013

glassfishrobot commented Aug 18, 2013

glassfishrobot commented Aug 19, 2013

Authentication failed with password from login store: /Users/manuel/.asadminpass

Enter admin password for user "admin">

Enter the value for the instance_name operand> instance1

glassfishrobot commented Aug 19, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented Aug 21, 2013

glassfishrobot commented May 9, 2017

glassfishrobot commented Feb 10, 2018