Unable to start new Spot Instance in Elastic Bamboo due to validUntil time already being expired
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Unable to start a new Spot instance in Elastic Bamboo as the request is being abandoned. The application logs show that the 'validUntil' time for this request has already expired.
Environment
AWS
Elastic Bamboo
Diagnosis
The below error can be seen within the AWS logs within the UI
1
2
Apr 12, 2021 1:20:43 PM Requested that new elastic instance be created for configuration: $CONFIG / ami-AMIID
Apr 12, 2021 1:20:43 PM Detected that a pending instance request for image [$CONFIG], ami: ami-AMIID has been abandoned.Examining the $BAMBOO_HOME/logs/atlassian-bamboo.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
2021-04-12 13:14:52,937 ERROR [elastic-pool-2-thread-2] [RemoteEC2InstanceImpl] EC2 instance order for image ami-AMIID failed.
com.amazonaws.services.ec2.model.AmazonEC2Exception: Value (2021-04-12T03:16:52 UTC) for parameter validUntil is invalid. The 'validUntil' time for this request has already expired. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: $REQUEST_ID)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:20435)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:20402)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:20391)
at com.amazonaws.services.ec2.AmazonEC2Client.executeRequestSpotInstances(AmazonEC2Client.java:18916)
at com.amazonaws.services.ec2.AmazonEC2Client.requestSpotInstances(AmazonEC2Client.java:18888)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.atlassian.aws.CallTimingProxy$1.invoke(CallTimingProxy.java:34)
at com.sun.proxy.$Proxy1615.requestSpotInstances(Unknown Source)
at com.atlassian.aws.ec2.awssdk.launch.AwsSpotInstanceLauncher.call(AwsSpotInstanceLauncher.java:77)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.launchInstance(RemoteEC2InstanceImpl.java:363)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.backgroundStart(RemoteEC2InstanceImpl.java:346)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.access$100(RemoteEC2InstanceImpl.java:39)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$1.run(RemoteEC2InstanceImpl.java:95)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$CatchingRunnableDecorator.run(RemoteEC2InstanceImpl.java:79)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)Cause
The system clock on the Bamboo Server is not NTP synced so the validUntil parameter is not accurate and AWS thinks it to be expired.
Solution
NTP sync the system clock on the Bamboo Server. For example, on Linux:
Was this helpful?