Cache, service container, and export pipelines definitions

The definitions option allows you to define custom dependency caches and service containers (including database services) for Bitbucket Pipelines.

Caches and Services for pipelines

The following options can be used for defining caches and service containers:

Definitions
- Caches
  - Key
    - Key files
  - Path
- Services

Definitions

The definitions property is used to define resources used elsewhere in your pipeline configuration. Resources can include:

Services that run in separate Docker containers – see Databases and service containers.
Caches – see Caches.
YAML anchors - a way to define a chunk of your YAML for easy re-use - see YAML anchors.

Property — definitions

Required — No

Data type — Block (YAML spec - Block Mapping)

Allowed parent properties — The YAML root (definitions can only be a top-level property)

Allowed child properties — Requires one or more of the caches and services properties.

Example — using definitions to add a custom cache and a database service to a pipeline step

definitions:
  caches:
    bundler-packages: vendor/bundle   
  services:
    my-mariadb:
      image: mariadb:latest
      variables:
        MARIADB_USER: $MY_MARIADB_USER
        MARIADB_PASSWORD: $MY_MARIADB_PASSWORD
        MARIADB_ROOT_PASSWORD: $MARIADB_ADMIN_PASSWORD

pipelines:
  default:
    - step:
        name: Hello world example
        services:
          - my-mariadb
        caches:
          - bundler-packages
        script:
          - ruby -e 'print "Hello, World\n"'

Example — using YAML anchors to create reusable steps

definitions: 
  steps:
    - step: &build-test
        name: Build and test
        script:
          - mvn package
        artifacts:
          - target/**

pipelines:
  branches:
    develop:
      - step: *build-test
    main:
      - step: *build-test

Caches

Bitbucket Pipelines supports caching build dependencies and directories, enabling faster builds and reducing the number of consumed build minutes.

There are several predefined caches available. For a complete list of predefined caches, see Caches — Predefined caches.

For information on using the caches option, see Caches.

Property — caches

Required — No

Data type — Block of new-line separated name-value pairs (YAML spec - Block Mapping), with the cache name as the name and values being either of:

A string representing the file or directory path (allows glob patterns)
A node containing the cache key and path options

Allowed parent properties — definitions

Allowed child properties — Either:

If you are providing a string representing a file or directory path: None
If you are using the key and path options: A cache name (nested under caches)

Example — using the caches option to create a custom dependency cache for a Ruby project

definitions:
  caches:
    my-bundler-cache: vendor/bundle

pipelines:
  default:
    - step:
        caches:
          - my-bundler-cache # Cache is defined above in the definitions section
        script:
          - bundle install --path vendor/bundle
          - ruby -e 'print "Hello, World\n"'

Example — using the caches option to create a custom dependency cache for a Ruby project with file checksum-based hashing

The key files option is used to specify files to monitor for changes. The cache specified by the path will be versioned based on changes to the key files.

definitions:
  caches:
    my-bundler-cache:
      key:
        files:
          - Gemfile.lock
          - "**/*.gemspec" # glob patterns are supported for cache key files
      path: vendor/bundle

pipelines:
  default:
    - step:
        caches:
          - my-bundler-cache # Cache is defined above in the definitions section
        script:
          - bundle install --path vendor/bundle
          - ruby -e 'print "Hello, World\n"'

Key

The caches key option defines the criteria for determining when to create a new version of the cache. The cache key used for versioning is based on the hashes of the files defined.

Property — key

Required — No

Data type — Block of files

Allowed parent properties — A cache name (nested under caches)

Allowed child properties — files

Example — using the caches key option to create a custom dependency cache for a Ruby project with file checksum-based hashing

definitions:
  caches:
    my-bundler-cache:
      key:
        files:
          - Gemfile.lock
          - "**/*.gemspec" # glob patterns are supported for cache key files
      path: vendor/bundle

pipelines:
  default:
    - step:
        caches:
          - my-bundler-cache # Cache is defined above in the definitions section
        script:
          - bundle install --path vendor/bundle
          - ruby -e 'print "Hello, World\n"'

Key files

The caches key files property lists the files in the repository to monitor for changes. A new version of the cache will be created when the hashes of one or more of the files change.

Property — files

Required — Yes

Data type — List of Strings matching file names (allows glob patterns)

Allowed parent properties — key

Example — using the caches key files option to create a custom dependency cache for a Ruby project with file checksum-based hashing

definitions:
  caches:
    my-bundler-cache:
      key:
        files:
          - Gemfile.lock
          - "**/*.gemspec" # glob patterns are supported for cache key files
      path: vendor/bundle

pipelines:
  default:
    - step:
        caches:
          - my-bundler-cache # Cache is defined above in the definitions section
        script:
          - bundle install --path vendor/bundle
          - ruby -e 'print "Hello, World\n"'

Path

The caches path option defines the path to be cached. This can be outside the source repository.

Property — path

Required — No

Data type — User-defined file path as String (allows glob patterns)

Allowed parent properties — A cache name (nested under caches)

Example — using the caches path option to create a custom dependency cache for a Ruby project with file checksum-based hashing

The key files option is used to specify files to monitor for changes. The cache specified by the path will be versioned based on changes to the key files.

definitions:
  caches:
    my-bundler-cache:
      key:
        files:
          - Gemfile.lock
          - "**/*.gemspec" # glob patterns are supported for cache key files
      path: vendor/bundle

pipelines:
  default:
    - step:
        caches:
          - my-bundler-cache # Cache is defined above in the definitions section
        script:
          - bundle install --path vendor/bundle
          - ruby -e 'print "Hello, World\n"'

Services

Bitbucket Pipelines can create separate Docker containers for services, which results in faster builds, and easy service editing. For details on creating services see Databases and service containers. This services option is used to define the service, allowing it to be used in a pipeline step.

Secrets and login credentials should be stored as user-defined pipeline variables to avoid being leaked. For details, see Variables and secrets — User-defined variables.

Property — services

Required — No

Data type — Block (YAML spec - Block Mapping)

Allowed parent properties — definitions

Allowed child properties — A user-defined string. This string will be the name of the service.

Example — using a service definition to add a database service to a build step

definitions:
  services:
    my-service-name:
      image: mariadb:latest
      variables:
        MARIADB_USER: $MY_MARIADB_USER
        MARIADB_PASSWORD: $MY_MARIADB_PASSWORD
        MARIADB_ROOT_PASSWORD: $MARIADB_ADMIN_PASSWORD

pipelines:
  default:
    - step:
        name: Hello world example
        services:
          - my-service-name
        script:
          - echo "Hello, World"

Image

For details on the image options, including using the image options for services, see Docker image options.

Memory

The memory option is used to limit the memory (in megabytes) available to a service container. 4096MB (or 8192 MB for size: 2x builds) is the total amount of memory that can be allocated to a step, including scripts and pipeline services. For example: if a service requires 3072MB, then only 1024MB is available to run steps using that service. For details on service container memory limits see Databases and Service Containers — Service memory limits.

Property — memory

Required — No

Data type — Integer

Allowed values — From 128 to 4096MB (or 256 to 8192 for size: 2x builds).

Default value — 1024

Allowed parent properties — A service name (nested under services)

Example — using the memory option to restrict a service container to 512MB of memory

definitions:
  services:
    my-postgresql-db:
      image: postgres:latest
      memory: 512
      variables:
        POSTGRES_PASSWORD: $MY_POSTGRES_PASSWORD

pipelines:
  default:
    - step:
        name: Hello world example
        services:
          - my-postgresql-db
        script:
          - echo "Hello, World"

Type

Only available for self-hosted pipeline runners.

The type option is used to specify Docker services containers (running Docker-in-Docker) for:

Docker-in-Docker service containers with a custom name.
Multiple Docker-in-Docker service containers.

For details on:

Adding custom Docker-in-Docker services, see Configure your runner in bitbucket-pipelines.yml — Custom docker-in-docker image.
Using a Docker-in-Docker service, see Run Docker commands in Bitbucket Pipelines.

Property — type

Required — No

Data type — String

Allowed values — docker

Allowed parent properties — A service name (nested under services)

Example — using the type option to add multiple Docker-in-Docker services

definitions:
  services:
    docker: # Default Docker-in-Docker service
      image: docker:dind
    my-docker-in-docker-name:
      image: docker:latest
      type: docker
    docker-custom:
      type: docker 
      image: docker:dind

pipelines:
  default:
    - step:
        script:
          - docker version
          - docker run hello-world
        services:
          - docker
    - step:
        script:
          - docker version
          - docker run hello-world
        services:
          - my-docker-in-docker-name
    - step:
        script:
          - docker version
          - docker run hello-world
        services:
          - docker-custom

Variables

The services variables option is used to pass environmental variables to service containers, typically used to configure the service. For details on creating services see Databases and service containers.

Secrets and login credentials should be stored as user-defined pipeline variables to avoid being leaked. For details, see Variables and secrets — User-defined variables.

Property — variables

Required — No

Data type — Block of new-line separated name-value pairs (YAML spec - Block Mapping)

Allowed parent properties — A service name (nested under services)

Example — using variables to set up a MariaDB service container

definitions:
  services:
    my-service-name:
      image: mariadb:latest  # Official MariaDB image from DockerHub: https://hub.docker.com/_/mariadb/
      variables:
        MARIADB_USER: $MY_MARIADB_USER
        MARIADB_PASSWORD: $MY_MARIADB_PASSWORD
        MARIADB_ROOT_PASSWORD: $MARIADB_ADMIN_PASSWORD

pipelines:
  default:
    - step:
        name: Hello world example
        services:
          - my-service-name
        script:
          - echo "Hello, World"

Example — using variables to set up a PostgreSQL service container

definitions:
  services:
    my-postgresql-db:
      image: postgres:latest  # Official PostgreSQL image from DockerHub: https://hub.docker.com/_/postgres
      variables:
        POSTGRES_PASSWORD: $MY_POSTGRES_PASSWORD

pipelines:
  default:
    - step:
        name: Hello world example
        services:
          - my-postgresql-db
        script:
          - echo "Hello, World"

Pipelines - Bitbucket Premium only

All pipelines defined under the pipelines variable will be exported and can be imported by other repositories in the same workspace.

Property — pipelines

Required — No

Data type — Block of new-line separated name-value pairs (YAML spec - Block Mapping), with the pipeline name as the name and values being pipeline

Allowed parent properties — definitions

Allowed child properties — Requires one or more of the step, stage, or parallel properties.

Example — exported pipeline with name export-pipeline and have one step defined:

export: true

definitions:
  caches:
  services:
  pipelines:
    export-pipeline:
      - step:
        script:
          - echo hello

Was this helpful?

It wasn't accurate

It wasn't clear

It wasn't relevant

Still need help?

The Atlassian Community is here for you.

Ask the Community

Cache, service container, and export pipelines definitions

Caches and Services for pipelines

Definitions

Example — using definitions to add a custom cache and a database service to a pipeline step

Example — using YAML anchors to create reusable steps

Caches

Example — using the caches option to create a custom dependency cache for a Ruby project

Example — using the caches option to create a custom dependency cache for a Ruby project with file checksum-based hashing

Key

Example — using the caches key option to create a custom dependency cache for a Ruby project with file checksum-based hashing

Key files

Example — using the caches key files option to create a custom dependency cache for a Ruby project with file checksum-based hashing

Path

Example — using the caches path option to create a custom dependency cache for a Ruby project with file checksum-based hashing

Services

Example — using a service definition to add a database service to a build step

Image

Memory

Example — using the memory option to restrict a service container to 512MB of memory

Type

Example — using the type option to add multiple Docker-in-Docker services

Variables

Example — using variables to set up a MariaDB service container

Example — using variables to set up a PostgreSQL service container

Pipelines - Bitbucket Premium only

Example — exported pipeline with name export-pipeline and have one step defined:

Still need help?

Path