Java reporting less CPUs than what is actually available while running in a container service such as ECS
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
In Java, the reported number of CPUs may be lower than the allocated number when running in containerized environments like Amazon ECS, Docker, or Kubernetes. This discrepancy can impact JVM processes, such as garbage collection, and the application's internal workings, as thread pools are often sized based on concurrency scaling factors like CPU core count. Consequently, underreporting the available CPU count can lead to performance issues.
In Bamboo, the JVM-reported CPU cores determine the number of concurrent threads launched. As a result, critical threads such as the PlanExec thread and other essential application components may experience low performance due to the presentation of a single CPU core.
Environment
Bamboo Server hosted on Amazon ECS or other containerized environments/services like those backed by Docker & Kubernetes.
Java
Diagnosis
The CPU core count can be checked at:
Bamboo Administration >> Overview >> System Information >> (System Properties) Available processors
Alternatively, when generating a Support Zip, the application-properties/application.xml also contains the available processors:
1
<available-processors>1</available-processors>You can create the following Java showProc.java program inside your Bamboo container, compile it, and run it to output the number of processors seen by Java:
1
2
3
4
5
6
7
8
docker exec -i bamboo_container /bin/bash -c 'cd /tmp; cat > showProc.java; javac showProc.java; java showProc; rm showProc.java showProc.class' <<EOF
public class showProc {
public static void main(String[] args) {
System.out.print("Number of available processors: ");
System.out.println( Runtime.getRuntime().availableProcessors());
}
}
EOF1
Number of available processors: 1Cause
Amazon ECS:
The ECS agent forces the Docker's CpuShares setting always to be set for every container it runs, even if it's empty on the task definition. This negatively affects the performance of containers that use this information to configure their threading.
This can also be caused by leaving the cpu / CpuShares unset on the Task Definition.
More information:
Solution
Amazon ECS:
Set cpu / CpuShares on the Task Definition rather than leaving the default:
That is also fixed on a recent release of AWS CDK.
Workaround
Add specific properties to Java
If the actual processor count is known, you can override the active processor count by adding JVM arguments. The following would set the processor count to 10.
Java 8:
Replace "10" with the available cores number
1
-XX:+UnlockExperimentalVMOptions -XX:ActiveProcessorCount=10Java 11 and later:
1
-XX:ActiveProcessorCount=10See the below documentation on how to configure these depending on your Bamboo installation environment:
Was this helpful?