Bamboo fails to start ands throws No Server Key Configured exception
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
When starting Bamboo, the application fails to start up with bootstrap errors.
The following appears in the atlassian-bamboo.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
2016-06-27 17:21:26,799 INFO [localhost-startStop-1] [AbstractUpgradeManager] --------------------------------------------------------------
2016-06-27 17:21:26,799 INFO [localhost-startStop-1] [AbstractUpgradeManager] 51102 : Make sure server key has been initialized and is valid
2016-06-27 17:21:26,799 INFO [localhost-startStop-1] [AbstractUpgradeManager] --------------------------------------------------------------
2016-06-27 17:21:26,799 ERROR [localhost-startStop-1] [ServerKeyIsValid] Server key hasn't been initialized or is invalid
java.lang.IllegalStateException: No server key configured
at com.google.common.base.Preconditions.checkState(Preconditions.java:173)
at com.atlassian.bamboo.setup.DefaultBootstrapManager.getServerKey(DefaultBootstrapManager.java:429)
at com.atlassian.bamboo.upgrade.tasks.validation.ServerKeyIsValid.doUpgrade(ServerKeyIsValid.java:20)
at com.atlassian.bamboo.upgrade.AbstractUpgradeManager.runUpgradeTask(AbstractUpgradeManager.java:206)
at com.atlassian.bamboo.upgrade.BootstrapUpgradeManagerImpl.runValidationTests(BootstrapUpgradeManagerImpl.java:84)
at com.atlassian.bamboo.setup.DefaultBootstrapManager.performPersistenceUpgrade(DefaultBootstrapManager.java:345)
at com.atlassian.config.bootstrap.DefaultAtlassianBootstrapManager.init(DefaultAtlassianBootstrapManager.java:77)
at com.atlassian.bamboo.setup.BootstrapLoaderListener.contextInitialized(BootstrapLoaderListener.java:117)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4812)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5255)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)Cause
TheserverKey property is missing from$BAMBOO_HOME/bamboo.cfg.xml or is set to an invalid value. A valid serverKey value looks like this:
1
<property name="serverKey">0</property>Diagnosis
We had customers unexpectedly failing with this error. Here is a atlassian-bamboo.log do demonstrate it:
atlassian-bamboo.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
2016-11-04 14:24:30,772 INFO [localhost-startStop-1] [lifecycle] *******************************
2016-11-04 14:24:30,773 INFO [localhost-startStop-1] [lifecycle] * Bamboo is starting up *
2016-11-04 14:24:30,773 INFO [localhost-startStop-1] [lifecycle] *******************************
2016-11-04 14:24:30,774 INFO [localhost-startStop-1] [ServletContextHolder] Setting servlet context: Bamboo
2016-11-04 14:24:30,826 INFO [localhost-startStop-1] [lifecycle] atlassian.org.osgi.framework.bootdelegation set to javax.servlet,javax.servlet.*,sun.*,com.sun.*,org.w3c.dom.*,org.apache.xerces.*
2016-11-04 14:24:32,199 INFO [localhost-startStop-1] [lifecycle] Starting Bamboo 5.10.3 (build #51020 Tue Mar 15 01:26:34 AEDT 2016) using Java 1.8.0_101 from Oracle Corporation
...
...
...
2016-11-04 14:34:17,554 INFO [localhost-startStop-1] [lifecycle] *******************************
2016-11-04 14:34:17,554 INFO [localhost-startStop-1] [lifecycle] * Bamboo is starting up *
2016-11-04 14:34:17,554 INFO [localhost-startStop-1] [lifecycle] *******************************
2016-11-04 14:34:17,555 INFO [localhost-startStop-1] [ServletContextHolder] Setting servlet context: Bamboo
2016-11-04 14:34:17,616 INFO [localhost-startStop-1] [lifecycle] atlassian.org.osgi.framework.bootdelegation set to javax.servlet,javax.servlet.*,sun.*,com.sun.*,org.w3c.dom.*,org.apache.xerces.*
2016-11-04 14:34:17,894 INFO [localhost-startStop-1] [BootstrapLoaderListener] App classloader is the same as web app classloader
2016-11-04 14:34:19,298 INFO [localhost-startStop-1] [lifecycle] Starting Bamboo 5.14.0.2 (build #51412 Mon Oct 31 18:46:51 AEDT 2016) using Java 1.8.0_101 from Oracle Corporation
2016-11-04 14:34:19,298 INFO [localhost-startStop-1] [lifecycle] Real path of servlet context: /u01/bamboo/atlassian-bamboo-5.14.0.2/atlassian-bamboo/
2016-11-04 14:34:19,614 INFO [localhost-startStop-1] [DefaultBootstrapManager] Running pre-bootstrap validation tasks
2016-11-04 14:34:19,740 INFO [localhost-startStop-1] [AbstractUpgradeManager] -----------------------------------------------------------------
2016-11-04 14:34:19,740 INFO [localhost-startStop-1] [AbstractUpgradeManager] 4300 : Make sure there's single row in HIBERNATE_UNIQUE_KEY table
2016-11-04 14:34:19,740 INFO [localhost-startStop-1] [AbstractUpgradeManager] -----------------------------------------------------------------
2016-11-04 14:34:20,235 INFO [MLog-Init-Reporter] [MLog] MLog clients using slf4j logging.
2016-11-04 14:34:20,379 INFO [localhost-startStop-1] [C3P0Registry] Initializing c3p0-0.9.5.2 [built 08-December-2015 22:06:04 -0800; debug? true; trace: 10]
2016-11-04 14:34:20,516 INFO [localhost-startStop-1] [AbstractPoolBackedDataSource] Initializing c3p0 pool... com.mchange.v2.c3p0.PoolBackedDataSource@a777bdf2 [ connectionPoolDataSource -> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@29f7e2ba [ acquireIncrement -> 1, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, identityToken -> 2v0tg59kcli3kqzw7203|29a55c95, idleConnectionTestPeriod -> 100, initialPoolSize -> 0, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 30, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 300, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 0, nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@9eee0482 [ description -> null, driverClass -> null, factoryClassLocation -> null, forceUseNamedDriverClass -> false, identityToken -> 2v0tg59kcli3kqzw7203|1a0be38e, jdbcUrl -> jdbc:mysql://bamboo-db.auspost.aws:3306/bamboo?autoReconnect=true, properties -> {user=******, password=******} ], preferredTestQuery -> null, privilegeSpawnedThreads -> false, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> false; userOverrides: {} ], dataSourceName -> null, extensions -> {}, factoryClassLocation -> null, identityToken -> 2v0tg59kcli3kqzw7203|5ae69fcf, numHelperThreads -> 3 ]
2016-11-04 14:34:21,583 INFO [localhost-startStop-1] [AbstractUpgradeManager] Completed task 4300 successfully.
2016-11-04 14:34:21,583 INFO [localhost-startStop-1] [AbstractUpgradeManager] -------------------------------------------------------------------------
2016-11-04 14:34:21,583 INFO [localhost-startStop-1] [AbstractUpgradeManager] 4410 : Make sure that all branch keys start with their master's chain key
2016-11-04 14:34:21,583 INFO [localhost-startStop-1] [AbstractUpgradeManager] -------------------------------------------------------------------------
2016-11-04 14:34:21,599 INFO [localhost-startStop-1] [AbstractUpgradeManager] Completed task 4410 successfully.
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] ---------------------------------------------
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] 51010 : Check if lucene index root dir exists
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] ---------------------------------------------
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] Completed task 51010 successfully.
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] -----------------------------------------------------------------
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] 51012 : Make sure that MySQL server uses InnoDB engine for tables
2016-11-04 14:34:21,600 INFO [localhost-startStop-1] [AbstractUpgradeManager] -----------------------------------------------------------------
2016-11-04 14:34:21,608 INFO [localhost-startStop-1] [AbstractUpgradeManager] Completed task 51012 successfully.
2016-11-04 14:34:21,609 INFO [localhost-startStop-1] [AbstractUpgradeManager] --------------------------------------------------------------
2016-11-04 14:34:21,609 INFO [localhost-startStop-1] [AbstractUpgradeManager] 51102 : Make sure server key has been initialized and is valid
2016-11-04 14:34:21,609 INFO [localhost-startStop-1] [AbstractUpgradeManager] --------------------------------------------------------------
2016-11-04 14:34:21,610 ERROR [localhost-startStop-1] [ServerKeyIsValid] Server key hasn't been initialized or is invalid
java.lang.IllegalStateException: No server key configuredIn this example, notice that Bamboo was upgraded from 5.10.3 to 5.14.2.
The part that is wrong with this upgrade is that it is is failing on the validation tasks. However, the concept of ServerKey was only introduced as of 5.11 and, therefore, the code shouldn't be running a validation on that but creating a ServerKey as part of the upgrade. Here is a very clear explanation of how the validation tasks are run (snippet from our code base):
1
2
3
4
5
6
7
8
9
10
11
12
13
<!--
Validation tasks are run before bootstrap upgrade tasks. If any of them will report an error application will not run.
Additional attributes:
- build-min when specified disables built-in version logic and makes this task run on existing installations that have build number
equal or greater than specified
- build-max when specified disables built-in version logic and makes this task run on existing installations that have build number
equal or lesser than specified
<validation>
...
<upgrade build="51102" build-min="51102" build-max="999999"
class="com.atlassian.bamboo.upgrade.tasks.validation.ServerKeyIsValid"/>
...
</validation>The logic we use on upgrade tasks basically reads the content of this tag that is in your your $BAMBOO_HOME/bamboo.cfg.xml:
<buildNumber>...</buildNumber>
As you can see from the above code snippet, this "Upgrade Validation task" will only run if the current buildNumber in your $BAMBOO_HOME/bamboo.cfg.xml as minimum 51102. The <buildNumber> that gets written in $BAMBOO_HOME/bamboo.cfg.xml is of the latest executed build task when you upgraded your instance last. So, in this case, the customer had the wrong number in $BAMBOO_HOME/bamboo.cfg.xmlthere for whatever reason (i.e. 51100+), which triggered this validation to be (incorrectly) run during their upgrade. It is incorrect because they were running 5.10.3 according to the logs immediately before the upgrade so any build task written in that file should be something like 51000+.
So the question is: why was their buildNumber wrong there? It was because of their upgrade methodology: they were using an automated method and got this file copied from elsewhere (i.e. that say had already been upgraded to 5.11) during the execution of your automated scripts before running the upgrade. In order for the tasks to be run properly and the ServerKey to be created automatically, they needed to restore the correct BAMBOO_HOME/bamboo.cfg.xml updated with a buildNumber to a value compatible to an instance that had been upgraded to 5.10.3.
Resolution
Retrieve property from backup
If you've encountered this issue during an upgrade or migration, you should be able to find this value in any previous or backup copies of your $BAMBOO_HOME/bamboo.cfg.xml file. Recreate this property with the same value as was set previously and restart Bamboo.
Recalculate property from database
If you cannot determine the previous value of this property from an old or backup copy of bamboo.cfg.xml, you can recalculate the correct value from your database. The value calculation is a bit wise operation of OID >> 53 where the OID value can be taken from any row from the project table or build table, as well as a number of other tables.
PostgreSQL and MySQL
The following query will calculate the correct serverKey value using the built in right bitshift operator
1
SELECT OID >> 53 FROM PROJECT LIMIT 1;SQL Server and Oracle
SQL Server/T-SQL and Oracle do not support native bitwise operations, so these need to be performed manually.
Select the first OID value from the PROJECT table, e.g. for SQL Server:
1SELECT TOP 1 OID FROM PROJECT;Perform the bitshift operation separately, using the OID value from the previous query. For example, if the OID value was 1234567890:
Python
11234567890 >> 53Bash/shell
1echo $((1234567890 >> 53))
Add the serverKey property using the output from the above:
1
<property name="serverKey">OUTPUT-FROM-ABOVE</property>Was this helpful?