Set up runners for Linux Docker

Runners allow you to run builds in Pipelines on your own infrastructure, and you won’t be charged for the build minutes used by your self-hosted runners.

Prerequisites

A 64-Bit Linux instance with at least 8GB of RAM as a host for the runner.
- More RAM may be required for builds with 2x-Steps or build services.
Allocate at least minimum 512MB memory for the runner container.
Docker v19.03 and above - Install Docker

Best practice

We strongly recommend disabling swap and configuring vm.swappiness. Having swap enabled can lead to non-deterministic build results in regards to memory and OOMing, meaning that sometimes enough swap is available and a build may pass while other times not enough swap is available and the same build will OOM.

Disable swap in a Linux environment

Below are the steps to disable swap for most Linux distributions. If the following commands aren’t installed, you will have to install them. Consult your distributions documentation to configure swap.

Use the following command to check if swap is enabled:
1sudo swapon -sv
If swap is enabled, you should see output similar to the following:
1 2NAME TYPE SIZE USED PRIO /dev/sda3 partition 2G 655.2M -1
If swap is enabled, you will have to disable it using the following processes:
1. Disable any swap using the following command:
  1sudo swapoff -av
2. Open /etc/fstab and remove any swap partitions or files that are configured.
3. Reboot your machine.
4. Run the following command again ensuring there is no output:
  1sudo swapoff -av
5. If there is output, repeat Step 2, ensuring all swap files are removed from /etc/fstab.

Configure vm.swappiness in a Linux environment

Below are the steps to configure vm.swappiness for most Linux distributions. If the following commands aren’t installed, you will have to install them. Consult your distributions documentation to configure swap.

Use the following command to check the value of vm.swappiness:
1sudo sysctl -n vm.swappiness
If the value is anything other than 1, it means you have some swap behavior still enabled.
If the swappiness value is anything except 1, configure it using the following process:
1. Open /etc/sysctl.conf and add vm.swappiness = 1 to the file on its own line.
2. Reboot your machine.
3. Run the following command ensuring that the output is now 1.
  1sudo sysctl -n vm.swappiness
4. If there is an output other than 1, repeat Step 2 and ensure that /etc/sysctl.conf is configured correctly.

Schedule docker images clean up

We recommend setting up a process to automatically remove docker images to avoid running out of disk space. You can create a cron job using the command docker system prune -af to remove all unused images. The schedule depends on the size of images you use, disk space available, and how often you run builds on a runner.

For example, do the following to clean up images once a week on Sunday at midnight:

Use the crontab -e command to open your user account’s crontab file.
Append the following entry 0 0 * * 0 docker system prune -af
Save and close the file.

For more details, refer to the crontab documentation.

Starting your runner

Navigate to the Runners page:
- For Workspace runners, visit Workspace settings > Workspace runners.
- For Repository runners, visit Repository settings > Runners.
Select Add runner.
From the Runner installation dialog, under System and architecture, select Linux Docker (x86_64) or Linux Docker (arm64).
Use the pre-configured Docker command provided in Run step on the Runner installation dialog to run the runner.
- If this is the first time running the runner, it will pull the image.
- If you are starting the runner again or to update the runner, always pull the runner manually before using the following command to ensure you are always running the most up-to-date runner.
  1docker image pull docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1
- If you encounter the following error, run the ‘docker container rm -f runner’ command below to remove the runner.
  Error
  1 2docker: Error response from daemon: docker: Error response from daemon: Conflict. The container name "/runner-76b247e7-b925-5e7b-9da2-1cda14c4ff2c" is already in use by container "c3403236e3af5962ed3a9b8771561bd2021974941cc8a89a40c6c66cecb18f53". You have to remove (or rename) that container to be able to reuse that name. See 'docker run --help'.
  docker container rm -f runner command
  1docker container rm -f runner-76b247e7-b925-5e7b-9da2-1cda14c4ff2c

Changing the working directory of your runner

If you want to change the working directory that the runner uses on your host machine, add the following two flags to the docker run command when you start the runner:

docker run [all existing parameters] -v /mydir:/mydir -e WORKING_DIRECTORY=/mydir

In this command, the first value in -v parameter will be the local directory on your machine that will serve as the working directory. The second value will be the directory inside the runner. It can be anything you like, it just needs to match the value specified in the WORKING_DIRECTORY environment variable.

The working directory stores the runner’s logs which are persistent, but it is also used to store temporary files while you are executing a step.

Access files on the host device

It's not possible to access the local files with the docker runtime. By default, the runner limits access to the host, This is to avoid pipeline failures due to dependencies on files that may not be on all machines the runners are on.

As a workaround we can suggest accessing the files on the host with an SFTP client from the Runner's build by running the following command:

sftp {user}@{host}:{remoteFileName} {localFileName}

For information on using the OpenSSH sftp command, visit the OpenBSD manual page server — sftp.

Using an image from an insecure Docker registry

To use an image from an insecure Docker registry on the Linux Docker runner, use a custom docker-in-docker service. For example:
dockerfile

# my-custom-dind-image
FROM docker:dind
ENTRYPOINT [ "sh", "-c", "dockerd-entrypoint.sh $DOCKER_OPTS" ]

bitbucket-pipelines.yml

definitions:
  services:
    docker:
      image: my-custom-dind-image
      variables:
         DOCKER_OPTS: "--insecure-registry=my.docker.registry"

pipelines:
  default:
    - step:
        runs-on: self.hosted
        services:
          - docker
        script:
          - docker build -t my.docker.registry/$IMAGE_NAME .
          - docker push my.docker.registry/$IMAGE_NAME

Was this helpful?

It wasn't accurate

It wasn't clear

It wasn't relevant

Still need help?

The Atlassian Community is here for you.

Ask the Community