Fail to start Bamboo Elastic agent in Custom Windows Image: Network is unreachable or Connection timed out
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Problem
After customizing a Windows Elastic AMI, the following appears in the C:\Users\Bamboo\bamboo-elastic-agent.out log file which suggests a connection issue when connecting to the EC2 Metadata URL: http://169.254.169.254/latest/meta-data/
1
2
3
4
5
6
7
8
9
10
11
12
Exception in thread "main" java.net.SocketException: Network is unreachable: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
...
at com.atlassian.aws.utils.URLFetcherImpl.fetchData(URLFetcherImpl.java:26)
at com.atlassian.aws.utils.URLFetcherUtils.fetchData(URLFetcherUtils.java:48)
at com.atlassian.aws.utils.URLFetcherUtils.fetchString(URLFetcherUtils.java:40)
at com.atlassian.aws.ec2.Ec2UtilsThin.getUserData(Ec2UtilsThin.java:49)
at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.<init>(ElasticAgentInstaller.java:47)
at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:166)Or:
1
2
3
4
5
6
7
8
9
10
11
12
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
...
at com.atlassian.aws.utils.URLFetcherImpl.fetchData(URLFetcherImpl.java:26)
at com.atlassian.aws.utils.URLFetcherUtils.fetchData(URLFetcherUtils.java:48)
at com.atlassian.aws.utils.URLFetcherUtils.fetchString(URLFetcherUtils.java:40)
at com.atlassian.aws.ec2.Ec2UtilsThin.getUserData(Ec2UtilsThin.java:49)
at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.<init>(ElasticAgentInstaller.java:47)
at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:166)Diagnosis
RDP into the instance and confirm if the URL http://169.254.169.254/latest/meta-data/ can be accessed.
Cause
The elastic agent fail to connect to AWS Instance Metadata due to incorrect route tables. Routes from the original subnet this AMI was launched into must have been snapshot when creating the new AMI and those routes are not applicable for the new VPC/AZ/Subnet. This can occur when the custom AMI is not prepared with (or without for Windows Server 2016) Sysprep via EC2 Launch.
Solution
Resolution
Windows Server 2016:
Make sure the EC2 instance stops instead of terminates upon shutdown so AMI can be created after instance shutdown: EC2 Console > Right click EC2 instance > Instance Settings > Change Shutdown Behaviour: Stop
Run the application EC2LaunchSettings
Click "Shutdown without Sysprep"
Once the instance is in the Stopped state, create a new AMI from it.
Windows Server 2012 & 2014:
Run the script file C:\ProgramData\Amazon\EC2-Windows\Launch\Module\Scripts\Add-Routes.ps1 or InitializeInstance.ps1 to fix the routing and test or run Sysprep to initialize the AMI correctly:
Was this helpful?