Jira Data Center stalls due to StackOverflowError
Platform Notice: Data Center Only - This article only applies to Atlassian apps on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Problem
Jira Data Center can stall when a java.lang.StackOverflowError exception occurs due to corruption of internal memory structures.
Symptoms include:
Operations are interrupted with the error "java.lang.StackOverflowError"
Jira appears to be stalled, with a large amount of threads wait for non-existing Java lock:
Large Tomcat HTTP exec pool usage
Large amount of thread wait for DB pool, see related KB: JIRA applications stalls due to lost lock in dbcp pool caused by StackOverflowError
Diagnosis
After the most recent application restart, you observe the following in atlassian-jira.log:
java.lang.StackOverflowErrorThere will be additional details associated with the error, including the thread where the error occurred, along with the stack trace up until the stack was overflowed. These details will vary depending on the unique situation and are important in diagnosing it's root cause
Impact
The thread that caused the StackOverflowError will be terminated abruptly. This means that any processing by that thread will be interrupted, which could result in data loss. Furthermore, if a thread was holding a Java lock, that lock may not be released, causing other threads to infinitely wait. This would surface as a stalled application. From http://openjdk.java.net/jeps/270:
For instance, when a StackOverflowError is thrown in a critical section of the java.util.concurrent.locks.ReentrantLock class, the lock status can be left in an inconsistent state, leading to potential deadlocks. The ReentrantLock class uses an instance of AbstractSynchronizerQueue to implement its critical section.
Cause
A StackOverflowError in Java is a runtime error that occurs when a thread's call stack memory space is exhausted. This typically happens due to excessive deep or infinite recursion, where a method continually calls itself without a proper termination condition, thus piling up method calls on the stack. Each method invocation in Java consumes a portion of the stack, and if the stack reaches its maximum capacity, the Java Virtual Machine (JVM) throws a StackOverflowError.
It's important to note that a stack overflow is a generic Java feature, so it could be caused by Atlassian code, Third-Party plugin code, or even user code within a third party app (such as Groovy in Power Scripts or ScriptRunner)
To determine what is causing the error, check the stack trace for entries that might explain high stack usage, such as repeating lines, or blocks of lines that form a repeating pattern. For example:
2024-04-09 02:39:22,538-0400 http-nio-8080-exec-91 ERROR someuser 159x3518430x7 gwsdkn 192.168.0.1 /rest/issueNav/1/issueTable [c.a.p.r.c.error.jersey.ThrowableExceptionMapper] Uncaught exception thrown by REST service: null
java.lang.StackOverflowError
at java.base/com.sun.crypto.provider.GaloisCounterMode.doLastBlock(GaloisCounterMode.java:418)
at java.base/com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:604)
at java.base/com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1122)
at java.base/com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1059)
at java.base/com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:945)
at java.base/com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:491)
at java.base/javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:779)
at java.base/javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:730)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2497)
at java.base/sun.security.ssl.SSLCipher$T12GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1655)
at java.base/sun.security.ssl.SSLSocketInputRecord.decodeInputRecord(SSLSocketInputRecord.java:264)
at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:181)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1514)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1481)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1070)
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161)
(...)
at com.atlassian.jira.issue.IssueImpl.getCustomFieldValue(IssueImpl.java:945)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getCustomParentIssue(PortfolioParentsOfJQLFunction.java:69)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:122)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)
(...)
at com.keplerrominfo.jira.plugins.jjupin.jql.jiranative.PortfolioParentsOfJQLFunction.getParentsOf(PortfolioParentsOfJQLFunction.java:126)It's clear that the method getParentsOf() is executing in a recursive nature, which caused the stack to overflow. By searching the internet for com.keplerrominfo.jira.plugins.jjupin, it becomes clear that the plugin that is responsible is Power Scripts for JIra.
If it's difficult to locate the source, please contact Atlassian Support for assistance.
Solution
Workaround
Restart the application as soon as possible.
The underlying cause should then be addressed by the steps in the resolution heading below
Resolution
Check the stack trace included in the error to determine the problem's source - Atlassian, Third Party plugin, or user code (Groovy/ScriptRunner/PowerScripts, etc) within a third party plugin:
If the problem is generated from user code, track down that code and disable/update it
If the problem is generated from Atlassian, contact Atlassian Support
If the probelm is generated from a third-party plugin, contact that vendor's support
If in doubt, please contact Atlassian Support for assistance.
Was this helpful?