Bitbucket Cloud Pipelines parallel steps do not run concurrently
Platform Notice: Cloud Only - This article only applies to Atlassian products on the cloud platform.
Summary
If you use parallel steps where a step has multiple caches and service configurations, it may affect the step's execution time.
Diagnosis
Example scenario:
The build has three parallel steps; Step-1,Step-2, and Step-3. Step-1 and Step-2 run concurrently while Step-3 is still starting, as gif example below:
Example YML:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
pipelines:
default:
- parallel:
- step:
name: 'Step-1'
script:
- echo "Your first step..."
- step:
name: 'Step-2'
script:
- echo "Your second step..."
- step:
name: 'Step-3'
caches:
- node
- maven
services:
- mysql
- docker
script:
- echo "Your third step"
definitions:
services:
mysql:
image: mysql:5.7
variables:
MYSQL_DATABASE: 'pipelines'
MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
MYSQL_USER: 'test_user'
MYSQL_PASSWORD: 'test_user_password'If we look at the YML config above, Step-3 has additional caches and services compared to Step-1 and Step-2.
The reason whyStep-3did not run concurrently with the other steps is that it has to prepare the step container to set up services by pulling each Docker image and downloading existing caches. While the UI shows the step as still in progress, it is run in parallel, but the step took longer because of the additional task it needed to start.
Workaround
There are two workarounds available to speed up the step:
Reduce the cache size as mentioned in the cache best practices to reduce the time to download the cache.
Skip artifacts downloads if the step does not require any artifacts.
Was this helpful?