Run Docker commands in Bitbucket Pipelines

Bitbucket Pipelines allows you to build a Docker image from a Dockerfile in your repository and to push that to a Docker registry, by running Docker commands within your build pipeline. Dive straight in – the pipeline environment is provided by default and you don't need to customize it!

Enable access to Docker

To enable access to Docker daemon, you can either add docker as a service on the step (recommended), or add the global option in your bitbucket-pipelines.yml.

1 2 3 4 5 6 7 pipelines: default: - step: script: - ... services: - docker

Note that Docker does not need to be declared as a service in the definitions section. It is a default service that is provided by Pipelines without a definition.

Add Docker to all build steps in your repository

1 2 options: docker: true

Note that even if you declare Docker here, it still counts as a service for Pipelines, has a limit of 1 GB memory, and can only be run with two other services in your build step. This setting is provided for legacy support, and we recommend setting it on a step level so there's no confusion about how many services you can run in your pipeline.

How it works

Configuring Docker as a service will:

  • mount the Docker CLI executable in your build container

  • run and provide your build access to a Docker daemon

You can verify this by running docker version:

1 2 3 4 5 6 7 pipelines: default: - step: script: - docker version services: - docker

You can check your bitbucket-pipelines.yml file with our online validator.

Enable Docker BuildKit

To use Docker BuildKit in a Bitbucket Pipeline, set the DOCKER_BUILDKIT=1 environment variable in the pipeline configuration (bitbucket-pipelines.yml).

For example:

1 2 3 4 5 6 7 8 pipelines: default: - step: script: - export DOCKER_BUILDKIT=1 - docker build . services: - docker

For information on Docker BuildKit, visit: Docker Docs — Build images with BuildKit.

Troubleshooting Docker BuildKit

If you are experiencing issues Docker build issues due to Docker BuildKit, you can disable BuildKit by setting DOCKER_BUILDKIT=0 before the docker command.

Such as:

1 2 3 4 5 6 7 8 pipelines: default: - step: script: - export DOCKER_BUILDKIT=0 - docker build . services: - docker

Running Docker commands

Inside your Pipelines script you can run most Docker commands. See the Docker command line reference for information on how to use these commands.

We've had to restrict a few for security reasons, including Docker swarm-related commands, docker run --privileged, docker run --mount, and mapping volumes with a source outside $BITBUCKET_CLONE_DIR.

Full list of restricted commands

The security of your data is really important to us, especially when you are trusting it to the cloud. To keep everybody safe we've restricted the following:

For docker container run/docker run we don't allow:

  • privileged

  • device

  • mount

  • volume (other than /opt/atlassian/bitbucketci/agent/build/.* or /opt/atlassian/pipelines/agent/build/.*)

  • pid

  • ipc

  • uts

  • userns

  • security-opt

  • cap-add

  • network

For docker container update/docker update we don't allow:

  • devices

For docker container exec/docker exec we don't allow:

  • privileged

For docker image build / docker build we don't allow:

  • security-opt

  • network

Using Docker Compose

If you'd like to use Docker Compose in your container, you''ll need to install a binary that is compatible with your specified build container.

Using an external Docker daemon

If you have configured your build to run commands against your own Docker daemon hosted elsewhere, you can continue to do so. In this case, you should provide your own CLI executable as part of your build image (rather than enabling Docker in Pipelines), so the CLI version is compatible with the daemon version you are running.

Docker layer caching

If you have added Docker as a service, you can also add a Docker cache to your steps. Adding the cache can speed up your build by reusing previously built layers and only creating new dynamic layers as required in the step.

1 2 3 4 5 6 7 8 9 pipelines: default: - step: script: - docker build ... services: - docker caches: - docker # adds docker layer caching

A common use case for Docker cache is when you are building images. However, if you find that performance slows with the cache enabled, check you are not invalidating the layers in your dockerfile.

Docker layer caches have the same limitations and behaviors as regular caches as described on Caching Dependencies.

Docker memory limits

By default, the Docker daemon in Pipelines has a total memory limit of 1024 MB. This allocation includes all containers run via docker run commands, as well as the memory needed to execute docker build commands.

To increase the memory available to Docker you can change the memory limit for the built-in docker service. The memory parameter is a whole number of megabytes greater than 128 and not larger than the available memory for the step.

Below is a working example of how you can set memory limits to multiple Docker services and use the appropriate service depending on the step requirements.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 definitions: services: docker: memory: 512 docker-with-more-memory: memory: 2048 type: docker docker-with-large-memory: memory: 5120 type: docker pipelines: custom: pipeline1: - step: services: [docker] script: - echo "Docker service with 512 MB memory" pipeline2: - step: services: [docker-with-more-memory] script: - echo "Docker service with 2048 MB memory" pipeline3: - step: services: [docker-with-large-memory] size: 2x script: - echo "Docker service with 5120 MB memory"

In the example below, we are are giving the docker service twice the default allocation of1024 MB (2048). Depending on your other services and whether you have configured large builds for extra memory, you can increase this even further (learn more about memory limits).

1 2 3 4 5 6 7 8 9 10 11 pipelines: default: - step: script: - docker version services: - docker definitions: services: docker: memory: 2048

Authenticate when pushing to a registry

To push images to a registry, you need to use docker login to authenticate prior to calling docker push. You should set your username and password using variables.

For example, add this to your pipeline script:

1 docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD

Reserved ports

Port restrictions

There are some reserved ports which can't be used:

  • 29418

Additional Help