Bitbucket fails to start up with Hazelcast instance is not active error
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Problem
When starting Bitbucket Server, the application fails to start and the following appears in the atlassian-bitbucket.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
2016-05-01 11:05:22,082 ERROR [localhost-startStop-2] c.a.e.i.AsynchronousAbleEventDispatcher There was an exception thrown trying to dispatch event [com.atlassian.config.lifecycle.events.ApplicationStoppingEvent@74c57907] from the invoker [SingleParameterMethodListenerInvoker{method=public void com.atlassian.analytics.client.listener.BitbucketEventListener.onEvent(java.lang.Object), listener=com.atlassian.analytics.client.listener.BitbucketEventListener@605b0ae1}]
java.lang.RuntimeException: Hazelcast instance is not active!
at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:54) ~[atlassian-event-3.0.0.jar:na]
at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$1$1.run(AsynchronousAbleEventDispatcher.java:46) ~[atlassian-event-3.0.0.jar:na]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) [guava-18.0.jar:na]
at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:105) [atlassian-event-3.0.0.jar:na]
at com.atlassian.event.internal.LockFreeEventPublisher$Publisher.dispatch(LockFreeEventPublisher.java:260) [atlassian-event-3.0.0.jar:na]
at com.atlassian.event.internal.LockFreeEventPublisher.publish(LockFreeEventPublisher.java:102) [atlassian-event-3.0.0.jar:na]
at com.atlassian.stash.internal.event.TransactionAwareEventPublisher.publish(TransactionAwareEventPublisher.java:84) [bitbucket-platform-4.3.2.jar:na]
at com.atlassian.stash.internal.config.ConfigLifecycle.stop(ConfigLifecycle.java:45) [bitbucket-service-impl-4.3.2.jar:na]
at com.atlassian.stash.internal.spring.AbstractSmartLifecycle.stop(AbstractSmartLifecycle.java:36) [bitbucket-platform-4.3.2.jar:na]
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:229) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.DefaultLifecycleProcessor.access$300(DefaultLifecycleProcessor.java:51) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:363) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:202) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.DefaultLifecycleProcessor.onClose(DefaultLifecycleProcessor.java:118) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:969) [spring-context-4.2.2.RELEASE.jar:na]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:928) [spring-context-4.2.2.RELEASE.jar:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
... 11 frames trimmed
Caused by: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
at com.hazelcast.spi.AbstractDistributedObject.getService(AbstractDistributedObject.java:93) ~[hazelcast-3.5.2-atlassian-24.jar:na]
at com.hazelcast.map.impl.proxy.MapProxySupport.toData(MapProxySupport.java:1143) ~[hazelcast-3.5.2-atlassian-24.jar:3.5.2-atlassian-24]
at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:81) ~[hazelcast-3.5.2-atlassian-24.jar:3.5.2-atlassian-24]
at com.atlassian.hazelcast.quartz2.HazelcastJobStore.retrieveTrigger(HazelcastJobStore.java:498) ~[atlassian-hazelcast-extras-quartz2-1.3.3.jar:na]
at org.quartz.core.QuartzScheduler.getTrigger(QuartzScheduler.java:1530) ~[quartz-2.2.1.jar:na]
at org.quartz.impl.StdScheduler.getTrigger(StdScheduler.java:508) ~[quartz-2.2.1.jar:na]
at com.atlassian.scheduler.quartz2.Quartz2SchedulerFacade.getTrigger(Quartz2SchedulerFacade.java:191) ~[atlassian-scheduler-quartz2-1.6.0.jar:na]
at com.atlassian.scheduler.quartz2.Quartz2SchedulerFacade.unscheduleJob(Quartz2SchedulerFacade.java:313) ~[atlassian-scheduler-quartz2-1.6.0.jar:na]
at com.atlassian.scheduler.quartz2.Quartz2SchedulerService.unscheduleJob(Quartz2SchedulerService.java:122) ~[atlassian-scheduler-quartz2-1.6.0.jar:na]
at com.atlassian.stash.internal.sal.scheduling.PluginSchedulerAdapter.unscheduleJob(PluginSchedulerAdapter.java:75) ~[bitbucket-platform-4.3.2.jar:na]
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:30) ~[atlassian-plugins-core-4.0.4.jar:na]
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) ~[na:na]
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) ~[na:na]
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) ~[na:na]
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) ~[na:na]
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) ~[na:na]
at com.atlassian.scheduler.compat.local.LocalOnlyCompatibilityPluginScheduler.unscheduleClusteredJobImpl(LocalOnlyCompatibilityPluginScheduler.java:126) ~[na:na]
at com.atlassian.scheduler.compat.local.LocalOnlyCompatibilityPluginScheduler.unscheduleClusteredJob(LocalOnlyCompatibilityPluginScheduler.java:118) ~[na:na]
at com.atlassian.analytics.client.upload.PeriodicEventUploaderScheduler.unscheduleJob(PeriodicEventUploaderScheduler.java:113) ~[na:na]
at com.atlassian.analytics.client.upload.PeriodicEventUploaderScheduler.unscheduleJobs(PeriodicEventUploaderScheduler.java:156) ~[na:na]
at com.atlassian.analytics.client.listener.BitbucketEventListener.destroy(BitbucketEventListener.java:64) ~[na:na]
at com.atlassian.analytics.client.listener.BitbucketEventListener.onEvent(BitbucketEventListener.java:47) ~[na:na]
at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:36) ~[atlassian-event-3.0.0.jar:na]
... 19 common frames omittedDiagnosis
Environment
This issue may have been preceded by a restart, OS or Java upgrade.
Cause
JAVA_HOME is not set correctly, and points to an old or incorrect JRE location.
Note: if you are seeing this error in a different scenario, where Bitbucket does start up but later crashes with an OutOfMemory error, you may be running into a different issue: Bitbucket Server become unresponsive with "Hazelcast instance is not active"
Solution
Resolution
Set JAVA_HOME to the correct value.
JAVA_HOME may be set as an environment variable for the user that runs Bitbucket (e.g. atlbitbucket) or it may be set inside the $BITBUCKET_INSTALL/bin/setenv.sh file. Check both places and ensure that this is pointing to a valid JRE location.
ℹ️setenv and environment variable changes in Bitbucket Server 5.0+
Starting with Bitbucket Server 5.0, setenv.sh and setenv.bat have been removed. The options that were set in this file can now be set via environment variables. Where to set the environment variable depends on which Operating System you're running on.
Linux
When using the atlbitbucket service on Linux, the environment variables are ignored. You must set the parameters in _start-webapp.sh (or start-bitbucket.sh). These values will be read when the service starts.
As an example, to set JVM_SUPPORT_RECOMMENDED_ARGS, you would add this line to the file:
Example
1
JVM_SUPPORT_RECOMMENDED_ARGS=-XX:+HeapDumpOnOutOfMemoryErrorWindows
Set the parameter as an environment variable for the user running Bitbucket Server. For example, if you want to set JVM_SUPPORT_RECOMMENDED_ARGS, create it as an environment variable and assign the appropriate value to it. When Bitbucket Server starts using the startup scripts or service, it will pick up and apply this value.
Was this helpful?