When Bamboo tries an ssh operation, it will fail with a NoSuchMethodError or NoClassDefFoundError
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
When Bamboo tries an ssh operation (normally while interacting with repositories in some way), it will fail with a NoSuchMethodError or NoClassDefFoundError.
Diagnosis
When either Bamboo server or agent fails an ssh operation, we'll see in the logs a NoSuchMethodError or NoClassDefFoundError. Examples of the errors to be found could be (yet not restricted to):
atlassian-bamboo.log or atlassian-bamboo-agent.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
021-02-17 19:50:55,113 WARN [http-nio-8086-exec-13] [AuthorizationLoggerListener] Credentials not found: org.acegisecurity.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext; secure object: com.atlassian.bamboo.ww2.actions.build.admin.config.repository.CreateRepository@4ec3648f; configuration attributes: [WW_EDIT, WW_NEWOBJECT, GLOBAL_READ]
2021-02-17 19:50:55,113 ERROR [http-nio-8086-exec-13] [ExceptionMappingInterceptor] An Authentication object was not found in the SecurityContext
org.acegisecurity.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext
at org.acegisecurity.intercept.AbstractSecurityInterceptor.credentialsNotFound(AbstractSecurityInterceptor.java:339)
[... truncated]
2021-02-17 19:50:55,139 ERROR [http-nio-8086-exec-13] [FiveOhOh] 500 Exception was thrown.
javax.servlet.ServletException: Filter execution threw an exception
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
[... truncated]
Caused by: java.lang.NoSuchMethodError: org.bouncycastle.asn1.ASN1ObjectIdentifier.equals(Lorg/bouncycastle/asn1/ASN1Primitive;)Z
at org.bouncycastle.openssl.MiscPEMGenerator.createPemObject(Unknown Source)
at org.bouncycastle.openssl.MiscPEMGenerator.generate(Unknown Source)
at org.bouncycastle.util.io.pem.PemWriter.writeObject(Unknown Source)
at org.bouncycastle.openssl.PEMWriter.writeObject(Unknown Source)
at org.bouncycastle.openssl.PEMWriter.writeObject(Unknown Source)
at com.atlassian.bamboo.utils.SshKeysProvider.getObjectAsPEM(SshKeysProvider.java:79)
at com.atlassian.bamboo.utils.SshKeysProvider.getRsaOpenSshKeyPair(SshKeysProvider.java:64)
at com.atlassian.bamboo.plugins.stash.v2.configurator.BitbucketServerSshKeyHandler.generateKeyPair(BitbucketServerSshKeyHandler.java:46)
at com.atlassian.bamboo.plugins.stash.v2.configurator.BitbucketServerServerConfigurator.negotiateStashSshKey(BitbucketServerServerConfigurator.java:259)
at com.atlassian.bamboo.plugins.stash.v2.configurator.BitbucketServerServerConfigurator.validate(BitbucketServerServerConfigurator.java:224)
[... truncated]and
atlassian-bamboo.log or atlassian-bamboo-agent.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2021-02-18 07:30:03,061 DEBUG [sshd-SshServer[4c0d9643]-nio2-thread-9] [ServerSessionImpl] exceptionCaught(ServerSessionImpl[null@/127.0.0.1:58798])[state=Opened] details
org.apache.sshd.common.RuntimeSshException
at org.apache.sshd.common.session.helpers.AbstractSession.handleMessage(AbstractSession.java:564)
at org.apache.sshd.common.session.helpers.AbstractSession.decode(AbstractSession.java:1498)
at org.apache.sshd.common.session.helpers.AbstractSession.messageReceived(AbstractSession.java:508)
at org.apache.sshd.common.session.helpers.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:66)
at org.apache.sshd.common.io.nio2.Nio2Session.handleReadCycleCompletion(Nio2Session.java:301)
at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:281)
at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:278)
at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
[... truncated]
Caused by: java.lang.NoClassDefFoundError: javax/crypto/ExemptionMechanism
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethod(Class.java:2128)
at org.apache.sshd.common.util.security.SecurityEntityFactory.toNamedProviderFactory(SecurityEntityFactory.java:117)
at org.apache.sshd.common.util.security.SecurityEntityFactory.toFactory(SecurityEntityFactory.java:69)
at org.apache.sshd.common.util.security.SecurityUtils.createSecurityEntityFactory(SecurityUtils.java:698)
at org.apache.sshd.common.util.security.SecurityUtils.lambda$resolveSecurityEntityFactory$1(SecurityUtils.java:680)
[... truncated]For this last error, the package
1
org.apache.sshdmust be enabled at DEBUG level for it to appear in the logs. For more information on how to do this, please visit Logging in Bamboo
Cause
Bamboo, when installed, provides a library for Bouncy Castle that includes all classes and methods needed for these operations. If errors like these arise, it might be because the jvm is loading this particular library from somewhere else. There are JDKs that deliver this library, and if there is a version mismatch between both libraries, and the jvm loads their own library, then this problem will occur.
Solution
The library that includes these classes is called bcprov-jdk15on-X.XX.jar. First, it's important to identify which library is being loaded. To identify it, please follow the following steps:
For Bamboo server:
Stop Bamboo.
Add the jvm argument following the instructions in Configuring your system properties
1
-verbose:classStart Bamboo again.
For the agent:
Stop the agent.
Open the file <Bamboo agent home directory>/conf/wrapper.conf file.
Add the parameter in the wrapper.java.additional.X section
1
-verbose:classX should be the next available integer in the list. Example:
For Bamboo versions prior 9
1
2
3
4
5
wrapper.java.additional.1=-Dbamboo.home=/Users/user/Documents/Agents/bamboo-7.2.1/home
wrapper.java.additional.2=-Dbamboo.agent.ignoreServerCertName=false
wrapper.java.additional.3=-Dbamboo.allow.empty.artifacts=false
wrapper.java.additional.4=-Dlog4j.configuration=/Users/user/Documents/Agents/bamboo-7.2.1/home/log4j.properties
wrapper.java.additional.5=-verbose:classFor Bamboo version 9 and above
1
2
3
4
5
wrapper.java.additional.1=-Dbamboo.home=/Users/user/Documents/Agents/bamboo-7.2.1/home
wrapper.java.additional.2=-Dbamboo.agent.ignoreServerCertName=false
wrapper.java.additional.3=-Dbamboo.allow.empty.artifacts=false
wrapper.java.additional.4=-Dlog4j2.configurationFile=/Users/user/Documents/Agents/bamboo-7.2.1/home/log4j2.properties
wrapper.java.additional.5=-verbose:classStart the agent.
After the process is started, Bamboo will log the source of each class. This is an example of what we look for:
atlassian-bamboo.log or atlassian-bamboo-agent.log or catalina.out
1
2
3
4
(...)
[Loaded org.bouncycastle.asn1.ASN1ObjectIdentifier from file:/home/user/atlassian-bamboo-7.2.1/atlassian-bamboo/WEB-INF/lib/bcprov-jdk15on-1.67.jar]
[Loaded org.bouncycastle.asn1.ASN1ObjectIdentifier$OidHandle from file:/home/user/atlassian-bamboo-7.2.1/atlassian-bamboo/WEB-INF/lib/bcprov-jdk15on-1.67.jar]
(...)In this example, we can see that the Bouncy Castle library comes from within the installation directory of Bamboo.
If instead we see
atlassian-bamboo.log or atlassian-bamboo-agent.log or catalina.out
1
2
3
4
(...)
[Loaded org.bouncycastle.asn1.ASN1ObjectIdentifier from file:/usr/lib/jvm/adoptopenjdk-8-hotspot/jre/lib/ext/bcprov-ext-jdk15on-158.jar]
[Loaded org.bouncycastle.asn1.ASN1ObjectIdentifier$OidHandle from file:/usr/lib/jvm/adoptopenjdk-8-hotspot/jre/lib/ext/bcprov-ext-jdk15on-158.jar]
(...)
then we can conclude that the origin of the library is external.
An option to fix this issue is by removing the library from that external source. This will force the jvm to use Bamboo as source.
Was this helpful?