Parallel step options

Parallel steps allow you to group pipeline steps that can run at the same time (concurrently) to reduce build time.

Overview

Set up or run parallel steps

Parallel steps enable you to build and test faster, by running a set of self-contained steps at the same time. The total number of build minutes used by a pipeline will not change if you make the steps parallel. You'll be able to see results sooner but the total build minutes used will calculate based on the total time taken on each step.

There are several ways in which you can use this feature, however, we recommend following these guidelines:

Set an initial step to build the software.
Add a set of parallel steps to test the software.
If you are doing a single build step leading to multiple parallel testing steps, cache external build dependencies as much as possible in the first step, so you don't waste time doing it in each of your testing steps. Then, run all your tests concurrently, they can use the cache.
Deploying to multiple similar environments at the same time.

For example:

pipelines:
  default:
    - step:          # non-parallel step
        script:
          - ./build.sh
    - parallel:      # these 2 steps will run in parallel
        steps: 
          - step:
              script:
                - ./integration-tests.sh --batch 1
          - step:
              script:
                - ./integration-tests.sh --batch 2
    - step:          # non-parallel step
        script:
          - ./deploy.sh

Default variables for parallel steps

In addition to the standard pipelines variables, parallel step groups also have the following default variables:

BITBUCKET_PARALLEL_STEP - zero-based index of the current step in the group (such as 0, 1, 2, …).
BITBUCKET_PARALLEL_STEP_COUNT - total number of steps in the group.

Limitations for parallel steps

The 100-step limit on pipelines includes each step in a parallel group.
All environments in a parallel step set must be of the same type. Don’t try to mix production and test environment types in the same set.
Parallel steps can produce and consume artifacts, however, keep in mind the following:
- Parallel steps can only use artifacts produced by previous steps, not by steps in the same parallel set.
- Steps after the parallel set will get a combination of all files produced.
- If parallel steps produce artifacts containing a file at the same location, conflicts are resolved on a per-file basis, and the files generated by the latest step in the YAML file will win.
The number of steps your Bitbucket workspace can run at once - across all running pipelines - varies according to your plan.
- Standard and premium workspaces can support 600 steps running concurrently across multiple pipelines.
- Free workspaces can support 10 steps runnings concurrently across multiple pipelines.
- Once this limit is reached, excess steps will fail.

Parallel step options

The following options can be used to configure parallel step groups:

Parallel
- Steps
- Fail-fast

Parallel

The parallel option allows you to to build and test your code faster by running a list of steps at the same time. The total number of build minutes used by a pipeline will not change if you make the steps parallel, but you'll be able to see the results sooner. The total number of steps you can have in a Pipeline definition is limited to 100, regardless of whether they are running in parallel or serial.

For details on using parallel steps, see Set up or run parallel steps.

Property — parallel

Required — No

Data type — Block of new-line separated key-value pairs (YAML spec - Block Mapping)

Allowed parent properties — default, branches, pull-requests, tags, and custom

Allowed child properties — steps (required) and fail-fast

Example — using the parallel option to run two steps at the same time

pipelines:
  default:
    - step: # sequential step
        name: Build
        script:
          - ./build.sh
    - step: # sequential step
        name: Build
        script:
          - ./build.sh
    - parallel: # these 2 steps will run in parallel
        steps:
          - step:
              name: Integration 1
              script:
                - ./integration-tests.sh --batch 1
          - step:
              name: Integration 2
              script:
                - ./integration-tests.sh --batch 2
    - step:  # non-parallel step
        script:
          - ./deploy.sh

Steps

The steps property contains the lists of steps in a stage or a parallel. At least 1 step is required within the steps list.

Property — steps

Required — Yes

Data type — Block of new-line separated key-value pairs (YAML spec - Block Mapping)

Allowed parent properties — parallel and stage

Allowed child properties — step (required)

Example — using the steps property to list the steps in a parallel group

pipelines:
  default:
    - step: # sequential step
        name: Build
        script:
          - ./build.sh
    - step: # sequential step
        name: Build
        script:
          - ./build.sh
    - parallel: # these 2 steps will run in parallel
        steps:
          - step:
              name: Integration 1
              script:
                - ./integration-tests.sh --batch 1
          - step:
              name: Integration 2
              script:
                - ./integration-tests.sh --batch 2
    - step:  # non-parallel step
        script:
          - ./deploy.sh

Example — using the steps property to list the steps in a stage

pipelines:
   default:
    - stage:
        name: Build and test
        steps:
          - step:
              name: Build app
              script:
                - sh ./build-app.sh
          - step:
              name: Run unit tests
              script:
                - sh ./run-tests.sh

Fail-fast

fail-fast can be applied to all parallel steps or to a specific step in a parallel group:

If a step has fail-fast: false, then the step can fail without the whole parallel group stopping.
If a step has fail-fast: true, then the whole parallel group will stop if the step fails.

Property — fail-fast

Required — No

Data type — Boolean

Allowed values — true or false

Default value — false

Allowed parent properties — step and parallel

Example — using fail-fast to stop parallel steps when one fails

pipelines:
  default:
    - step:
        name: Build
        script:
          - ./build.sh
    - parallel:
        # this option allows a force stop on all running steps if any step fails
        fail-fast: true
        steps:
          - step:
              name: Integration 1
              script:
                - ./integration-tests.sh --batch 1
          - step:
              name: Integration 2
              script:
                - ./integration-tests.sh --batch 2

Example — exclude a parallel from fail-fast

pipelines:
  default:
    - step:
        name: Build
        script:
          - ./build.sh
    - parallel:
        # this option allows a force stop on all running steps if any step fails
        fail-fast: true
        steps:
          - step:
              name: Integration 1
              script:
                - ./integration-tests.sh --batch 1
          - step:
              name: Integration 2
              script:
                - ./integration-tests.sh --batch 2
          - step:
              # option can be disabled for a step
              # and its failure won't stop other steps in a group
              fail-fast: false
              name: Upload metadata
              script:
                - ./upload-metadata.sh

Was this helpful?

It wasn't accurate

It wasn't clear

It wasn't relevant

Still need help?

The Atlassian Community is here for you.

Ask the Community