Pipeline build failed with Container “Build” exceeded memory limit error

Summary

There are 2 possible scenarios for this case:

• Generic Scenario

• Builds using Jest Test Framework are slow or frequently hang (based on the Pipeline build minutes consumption) or fail with Container “Build” exceeded memory limit error

See below for possible causes and troubleshooting steps

Solution

Scenario 1: Generic scenario

Possible causes:

• The build requires more than the allowed build container memory

Troubleshooting Steps:

Debug the build locally using the instructions on Debug pipelines locally with the Docker page. Note, memory limit issues can’t be debugged locally on macOS because you can’t configure swap memory below 100Mb on macOS. Does the build fail locally with a similar memory error?

YES

• Run the below commands in the Pipeline and observe which process is consuming more memory. This can help to narrow down the issue, identify the culprit, and take action.

  1 2 - while true; do echo "Memory usage in bytes:" && cat /sys/fs/cgroup/memory.current; echo "Swap memory usage in bytes:" && cat /sys/fs/cgroup/memory.swap.current; sleep 2; done & - while true; do date && ps aux && echo "" && sleep 2; done &

• Increase the allocated memory (using size attribute) for the Pipeline build and check if you are still getting the memory-related issue. Refer to Databases and service containers page for more information on memory allocation in Pipelines.

  • Note: Opting for larger step sizes may impact billing

NO

• Use the below sections to troubleshoot the issue further. Otherwise, raise a Support Ticket and share the Pipeline build URL from http://bitbucket.org

Scenario 2: Builds using Jest Test Framework are slow or frequently hang (based on the Pipeline build minutes consumption) or fail with Container “Build” exceeded memory limit error

Possible causes:

• A large number of workers running concurrently

Troubleshooting Steps:

Print Process list and Memory Usage in the background just before the test command starts in the Pipeline.

The output of the process list will show how many workers are running at the time when the build is slow or when it gets OOM(Out of Memory) and you can see the CPU and Memory Usage along with the worker process.

Does the output show around 4-7 worker processes with high CPU / Memory usage?

YES

• JEST by default creates 7 child workers, and the daemon process that calls the Jest Test assigns the memory to each of these Jest workers. If you have up to 2 GB per worker (as an example) configured, and all 7 of these workers are running concurrently, it could easily reach OOM(Out of Memory). This can result in the build running slowly or appearing stalled as the build container doesn’t have enough memory allocated. To fix the issue, take the following steps.

  • Add this command --maxWorkers=2 to your build command to limit the number of workers to 2.

  • Alternatively, you should be able to define it in package.json or in jest config as mentioned on Configuring Jest page.

NO

• Raise a Support Ticket and share the Pipeline build URL from bitbucket.org