Bitbucket Data Center freezes after upgrading from 7.x.x to 8.x.x
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Bitbucket application freezes after upgrading from version 7.x.x to version 8.9.x very frequently. Users are unable to access Bitbucket UI and GIT operations start failing.
Environment
Tested on Bitbucket Server and Data Center 8.9.x (Recently upgraded from 7.x.x)
Applies to Bitbucket Server and Data Center 8.x.x
Diagnosis
Check if third-party plugins are on the latest version supported by Bitbucket Server or Data Center.
A lot of HTTP threads with similar stack traces can be seen in the
atlassian-bitbucket.log
file.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
2023-06-01 10:31:52,968 ERROR [http-nio-7990-exec-3145] admin @P0QVQUx638x34136x38 123f9aa x.x.x.x,127.0.0.1 "POST /rest/tagdetails/1.0/projects/xyz/repos/testrepo/notes HTTP/1.1" c.a.p.r.c.e.j.ThrowableExceptionMapper Uncaught exception thrown by REST service: TODO: Handle interrupts better java.lang.RuntimeException: TODO: Handle interrupts better at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:167) at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:139) at com.atlassian.stash.internal.scm.git.mesh.AbstractFragmentResponseObserver.asResult(AbstractFragmentResponseObserver.java:147) at com.atlassian.stash.internal.scm.git.mesh.BidirectionalFragmentResponseObserver.asResult(BidirectionalFragmentResponseObserver.java:66) at com.atlassian.stash.internal.scm.git.mesh.GrpcPlumbingClient.call(GrpcPlumbingClient.java:68) at jdk.internal.reflect.GeneratedMethodAccessor1228.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at jdk.internal.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at com.atlassian.stash.internal.scm.git.mesh.MeshGitFreeFormCommandBuilder$1.call(MeshGitFreeFormCommandBuilder.java:102) at jh.atlassian.bitbucket.tagdetails.RestService.getNotesByCommitIds(RestService.java:78) at jdk.internal.reflect.GeneratedMethodAccessor1811.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) at com.atlassian.bitbucket.internal.ratelimit.servlet.filter.RateLimitFilter.doFilter(RateLimitFilter.java:75) at com.atlassian.plugin.connect.plugin.auth.scope.ApiScopingFilter.doFilter(ApiScopingFilter.java:81) at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46) Caused by: java.lang.InterruptedException: null at java.base/java.util.concurrent.CompletableFuture.reportGet(Unknown Source) at java.base/java.util.concurrent.CompletableFuture.get(Unknown Source) at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.get(AbstractFutureResponseObserver.java:233) at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:151) ... 47 common frames omitted
From the above stack trace notice that
jh.atlassian.bitbucket.tagdetails.RestService.getNotesByCommitIds
is seen and causing the similar HTTP threads to wait forever and eventually saturating all HTTP threads which causes Bitbucket application to freeze.
Cause
The third-party plugins
may not be compatible or supported with the latest Bitbucket version. For example, Tag Details And Git Notes plugin version 5.0.x is not supported on Bitbucket 8.x and causes application to freeze.
Solution
Disable the offending plugin and monitor the application.
Update the third-party plugin such as Tag Details And Git Notes before upgrading Bitbucket Server/ Data Center from old version to 8.x. Once plugins are compatible, proceed with the upgrade.
If the issue still persist, contact Atlassian Support.
Was this helpful?