Bitbucket Cloud pipelines cache not used during build

Summary

You have defined a cache for your Bitbucket Pipelines builds. However, this cache is not used during the build.

Cause

There are different possible causes:

• No successful step ran since you added the caches definition in your bitbucket-pipelines.yml file. Caches are generated after a successful step.

• The cache may have been larger than 1 GB after compression. We only support caches under 1 GB once compressed, so caches larger than that will not be saved.

• The cache may be empty. Your dependencies may be installed in a different directory than the one you have specified for the cache in your bitbucket-pipelines.yml.

• The cache may have expired. Caches are saved for one week and deleted after that. They should be repopulated during the next successful step.

• You use a static cache, and your dependencies have changed since the cache was generated. In case of a static Docker cache, your Dockerfile has changed since the cache was generated. If the Dockerfile changes after the cache is saved, the signature of that intermediate layer will change and the existing saved Docker cache will not be utilized while building the image.

• [Docker cache only] There is a bug with the pre-defined Docker cache at the moment, BCLOUD-23193. Docker caches are not consistently used in Pipelines builds.

• [Docker cache only] You use Runtime v3 in Pipelines and the predefined Docker cache. The Bitbucket Pipelines Docker cache feature has been deprecated in Runtime v3.

Solution

Diagnosis

1. Check if the cache is present. Open the Pipelines page of the repository and select the button Caches (near the top right corner, between the buttons Schedules and Usage). If you don't see a cache listed there, open the last build that should have generated this cache. Select the step that should have generated the cache, and then expand the Build teardown section in the build log.

   • Check if the step failed. Caches are only saved on a successful step. If your step has failed, you will see a message in the Build teardown section indicating that the cache will not be uploaded for a failed step. 

   • If the cache is larger than 1 GB once compressed, you will see a message in the Build teardown section indicating this.

   • If the cache is empty, you will see a message in the Build teardown section indicating this.

   • Add the command ls -lah <cache-directory> (where <cache-directory> in the command is replaced with the directory of your cache) in your bitbucket-pipelines.yml, in the script of the step that generates the cache, after the command that should generate the cache. This is to confirm that the cache directory is empty.

   • If the Build teardown section shows the message "Docker cache upload is not supported in this environment. WARNING: Pipelines docker cache has been deprecated.", like below, this means that your step is using Pipelines Runtime v3. The pre-defined Docker cache is not supported for Runtime v3.

   • Check the date that this step ran. You can find this in the column next to the build log. If this step ran more than a week ago, then any cache it generated will have expired.

2. Check your source code and verify if your dependencies have changed since the cache was generated. If you use a Docker cache, check if the Dockerfile has changed since your cache was generated. If you use a static cache definition, like below, or one of our pre-defined caches, and a dependency has changed, then the cache will not be used or updated:

Example of static cache

Solutions

The solution depends on the cause.

• If no successful step has run since you added the caches definition in your bitbucket-pipelines.yml file, you will first need to have a successful step and the cache should be generated after this step runs.

• If your cache is larger than 1 GB after compression and you are caching a directory, check if it's possible to split the cache directory into two or more smaller ones. Bitbucket Pipelines do not support caches larger than 1 GB once compressed.

• If the cache directory is empty, check the commands in your bitbucket-pipelines.yml file and the configuration of your source code. Your build may be configured to store your dependencies in a directory different than the cache directory. You will need to either change the directory in the caches definition to match the directory where your dependencies are stored, or adjust your configuration so that dependencies are stored in the directory of the caches definition.

• If the last successful step that generated a cache ran more than a week ago, the cache will have expired. It should be repopulated after the next successful step runs.

• If you use a static cache and your dependencies or your Dockerfile have changed, you can remove the existing cache from the Pipelines page of the repo, button Caches (near the top roght corner of the screen). The cache should then be repoulated after the next successful step. You may also want to consider using caching with file-based cache keys instead. File-based cache keys allow for the generation and restoration of caches based on a set of files. With this type of cache, when dependencies are updated, the hash of the cache key files changes, and Pipelines creates a new version of the cache.

• Pipelines Docker cache has been deprecated for Runtime v3. If you use Docker in a Pipelines step that runs with Runtime v3, consider using Docker buildx caches instead. See https://docs.docker.com/build/cache/backends/inline/

• If you use a Docker cache, you may be affected by the bug BCLOUD-23193 and your cache may not be used consistently. Until this bug is fixed, consider using Runtime v3 for the step and utilizing Docker buildx caches instead. See https://docs.docker.com/build/cache/backends/inline/

If you cannot figure out what is causing the issue, raise a Support Ticket and share the Pipeline build URL from bitbucket.org.