Bitbucket Pipelines supports caching build dependencies and directories, enabling faster builds and reducing the number of consumed build minutes.

What is dependency caching?

Most builds start by running commands that download dependencies from the internet, which can take a lot of time for each build. As the majority of dependencies stay the same, rather than download them every time, we recommend downloading them once into a cache which you can reuse for later builds.

Set up caching

To enable caching, add a caches section to your step .

Here's an example of how to cache your node_modules directory for a Node.js project using a pre-defined cache.

1 2 3 4 5 6 7 8 9 10 bitbucket-pipelines.yml pipelines: default: - step: caches: - node  script: - npm install - npm test

The first time this pipeline runs it won't find the node cache and so the npm commands will download dependencies from the internet. For future builds, our service will have them cached, so loading them into your build will be quicker.

Pre-defined caches

Pipelines provides pre-defined cache locations for commonly used language tools:

Cache name

In your step



1 2 caches: - docker

n/a -  we create the cache from the layers generated by your build


1 2 caches: - composer



1 2 caches: - dotnetcore



1 2 caches: - gradle



1 2 caches: - ivy2



1 2 caches: - maven



1 2 caches: - node



1 2 caches: - pip



Note: When you use the sbt tool you need to enable both the sbt and ivy2 caches.

1 2 3 caches: - sbt - ivy2



Pre-defined caches don't support file-based cache keys. To better manage cached content, define a custom cache with file-based cache keys.

Custom caches for other build tools and directories

If your build tool isn't listed above, you can still define a custom cache for your repository in your bitbucket-pipelines.yml file. First, in the definitions section of the yml, define the cache name and the directory to be cached, then you can refer to that cache in any step:

1 2 3 4 5 6 7 8 9 10 11 12 image: ruby:2.4.0 pipelines: default: - step: caches: - bundler #cache is defined below in the definitions section script: - bundle install --path vendor/bundle definitions: caches: bundler: vendor/bundle

The defined cache will be saved after the first successful build.

Cache directory paths can be absolute, or relative to the clone directory, for example:

  • $HOME/.npm

  • ~/.gradle/wrapper

  • /usr/local/lib/node_modules

  • vendor/bundle

Note that, for Ruby, you should install gems in a separate location from the system's gem repo, like this:

1 $ bundle install --path vendor/bundle

Caching with file-based cache keys

Custom caches can support file-based cache keys as an alternative to the basic `cache-name: /path` configuration. File-based cache keys allow for the generation and restoration of caches based on a set of files. Any changes to those files would result in a new cache.

A typical use case might involve defining a cache key based on files that define a project's dependencies. Pipelines can restore cached dependencies from the cache to avoid retrieving them from external sources and save time during builds. When dependencies are updated, the hash of the cache key files changes, and Pipelines creates a new version of the cache.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 definitions: caches: my-bundler-cache: key: files: - Gemfile.lock - "**/*.gemspec" # glob patterns are supported for cache key files path: vendor/bundle pipelines: default: - step: caches: - my-bundler-cache # Cache is defined above in the definitions section script: - bundle install --path vendor/bundle - ruby -e 'print "Hello, World\n"'

Caching multiple directories

Some builds might benefit from caching multiple directories. Simply reference multiple caches in your step like this:

1 2 3 4 5 6 7 8 9 10 11 12 image: openjdk:8 pipelines: default: - step: caches: - gradle # pre-defined cache - gradlewrapper # custom cache that must be defined below script: - ./gradlew build definitions: caches: gradlewrapper: ~/.gradle/wrapper

How does caching work?

When does a cache get saved?

Caches are saved on successful builds when the cache is empty. Only caches under 1GB once compressed are saved.

For the cache to compress to under 1GB, the size of the original images in the docker daemon must be < 2GB.
You can check the size by adding this command to the script in your bitbucket-pipelines.yml:

1 docker image inspect $(docker image ls -aq) --format {{.Size}} | awk '{totalSizeInBytes += $0} END {print totalSizeInBytes}'

When does a cache get restored?

At the beginning of each build, saved caches are downloaded if available and placed in their directories.

When does a cache get cleared?

Any cache which is older than 1 week will be cleared automatically and repopulated during the next build. Additionally, caches can be cleared manually in the Bitbucket UI. Just choose Caches at the top right of the Pipelines page, then delete a cache from the list using the trash icon:

Best practices

Dependency caching is all about reducing build times by avoiding downloading and building the same files repeatedly.

That means you should cache things like:

  • Language-specific dependencies.

  • Binaries of dependencies that take a while to compile.

You shouldn't cache:

  • Sensitive data such as passwords and credentials

Note: We do not cache symlinks, so if your dependencies rely heavily on them caching may not be efficient.

Remember that caches are temporary, so your builds should be configured to work whether or not the cache is present.

Additional Help