How to cache a pipe in Bitbucket Pipelines
Platform Notice: Cloud Only - This article only applies to Atlassian products on the cloud platform.
Summary
Problem
You use Bitbucket Pipelines on Atlassian's infrastructure or with a self-hosted Linux Docker runner and you use a pipe in your build.
Downloading the pipe's image during the build takes a long time and you would like to cache it in order to speed up the build.
Solution
A pipe starts a new Docker container, and the image that is downloaded before the pipe's execution is a Docker image. You can use the predefined Docker cache to cache the pipe's Docker image (see here for more info on Pipelines caches). Please keep in mind the following:
Any cache which is older than 1 week will be cleared automatically and repopulated during the next build.
There is a bug with Docker caches at the moment, and as a result of this bug, Docker caches may not always be used: BCLOUD-23193
If this occurs in your builds, you may want to remove the Docker cache definitions from your bitbucket-pipelines.yml file until the bug is fixed, so that your build won't consume build minutes for creating and downloading a cache that is not used.
Use the predefined Docker cache for a pipe's image
Let's assume you have a pipelines step in your bitbucket-pipelines.yml file that looks like this:
1
2
3
4
5
6
7
- step:
name: 'Upload file to Downloads'
script:
- pipe: atlassian/bitbucket-upload-file:0.7.3
variables:
BITBUCKET_ACCESS_TOKEN: $BITBUCKET_ACCESS_TOKEN
FILENAME: "app/somefile.apk"You can adjust this step as follows so that the pipe's Docker image is cached:
1
2
3
4
5
6
7
8
9
10
11
- step:
name: 'Upload file to Downloads'
script:
- pipe: atlassian/bitbucket-upload-file:0.7.3
variables:
BITBUCKET_ACCESS_TOKEN: $BITBUCKET_ACCESS_TOKEN
FILENAME: "app/somefile.apk"
services:
- docker
caches:
- dockerImportant Note:
If you are building a Docker image either in this or in any other pipeline of the same repository and you want to use cache for that as well, then you won't be able to use the predefined Docker cache both for the pipe and for the image you build. In this case, you can use the above solution for caching the pipe's Docker image, and then use a custom Docker cache for the image you build. Steps on how to use a custom Docker cache for the image your pipeline builds are provided below.
Use a custom Docker cache for a Docker image built in Pipelines
Add the following definition in your bitbucket-pipelines.yml file for a custom cache:
1
2
3
4
5
6
7
definitions:
caches:
my-docker-cache:
key:
files:
- Dockerfile
path: docker-cacheThis definition assumes that the Dockerfile is located at the root of your repo. If it is not, provide the relative path to the Dockerfile in the files option, e.g. - docker/Dockerfile instead of - Dockerfile.
Please note that this is a definition for a custom cache with file-based cache keys, which means that a new cache will be created when you change the Dockerfile.
If you don't want a cache that gets updated every time you change the Dockerfile, you can use the following definition:
1
2
3
definitions:
caches:
my-docker-cache: docker-cacheThen, adjust the step that builds the Docker image as follows:
1
2
3
4
5
6
7
8
9
10
11
- step:
name: Build Docker image
script:
- docker load -i docker-cache/* || echo "No cache"
- docker build <other parameters> .
- mkdir -p docker-cache && docker save $(docker images -aq) -o docker-cache/cache.tar
- docker image ls -aq
caches:
- my-docker-cache
services:
- dockerWas this helpful?