robotsnoindex

Bitbucket is great for keeping your source files and managing changes, especially when working on a team, but it's not the best place to keep large binary files, such as executables and other build artifacts, or videos and other media files.

We recommend repositories be kept under 1.0 GB to help ensure that our servers are fast and downloads are quick for our users. Bitbucket Cloud repositories have a 2.0 GB limit. When that limit is reached, you will only be able to push changes that undo the latest commits. There is also a hard limit of 5.0 GB. When that limit is reached, you will not be able to push any changes, and your repository becomes read-only.

If you need to keep large files in Bitbucket, consider introducing Git Large File Storage (Git LFS) as part of your workflow, and Use BFG to migrate a repo to Git LFS.

Repository size limits

If the size is over these limits you'll see a warning in the Repository details panel:

Over 1.0 GB limit

You’ll get a warning on the command line if you push commits while over the 1.0 GB limit:

$ git push 
Enumerating objects: 5, done. 
Counting objects: 100% (5/5), done. 
Delta compression using up to 12 threads 
Compressing objects: 100% (2/2), done. 
Writing objects: 100% (3/3), 332 bytes | 332.00 KiB/s, done. 
Total 3 (delta 0), reused 0 (delta 0) 
remote: This repository is currently 1.0 GB in size. If it exceeds 2 GB it will be put into read-only mode. 
remote: Learn how to reduce your repository size: https://confluence.atlassian.com/x/xgMvEw 
To https://bitbucket.org/example/monster.git 
	3db4505..0393b72 master -> master 
$ _

If you see this warning you should perform maintenance on the repository to remove large files; see Removing large files below.

Over 2.0 GB limit

If the size exceeds the 2.0 GB limit you will not be able to push any more commits:

$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 406 bytes | 406.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: b'Repository is over the size limit (2 GB) and will not accept further additions.
remote: 
remote: Learn how to reduce your repository size: https://confluence.atlassian.com/x/xgMvEw.
remote: '
To https://bitbucket.org/example/monster.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://user@bitbucket.org/example/monster.git'
$ _

To continue making changes you’ll need to undo the last commit; see Undoing the last push below. This will bring the size below the 2.0 GB limit and remove the push restriction, allowing you to perform maintenance on the repository.

Over 5.0 GB limit

Above 5.0 GB all changes are rejected. You will need to raise a support request to get help reducing the size of your repository.

Undoing the last push

To remove large files you need to rewrite history; otherwise, Git just keeps the large files in the history.

tip/resting Created with Sketch.

Before doing these steps, you can enable the Delete dangling commits when over size limit feature in Labs to trigger garbage collection automatically.

This will delete the large files you remove from history when you rewind your branch head, avoiding the need to submit a Support request.

Rewind history to undo large commits

Rewind the branch containing the bad commit to just before that commit, and then prune the last commit to bring the repository size under the repository size limit. This process is assuming that the bad commit is only on one branch and hasn’t been merged to other branches.

It is essential to inform anyone else using the affected branch that you are undoing the last commit, and not to merge that commit into any other branches.

Remote history in Bitbucket looks similar to the following example:

			branch
            |
o---o---o---o <= last (bad) commit in Bitbucket

Local history might look similar to the following example, assuming you only need to undo one commit:

  			branch
            |
o---o---o---o---o <= last local commit failed to push
        |
        reset to here and push

To rewind history on the branch containing the bad commit:

  1. Create a temporary branch to keep any local commits.

  2. Reset the original branch to the commit just before the bad commit containing the large files.

  3. Push the new head to Bitbucket (rewriting history).

  4. Restore local changes – you won’t be able to push these yet, you’ll need to remove any large files first.

git branch <keeper>
git reset --soft @{u}^
git push --force
git merge --ff-only <keeper>
git branch -d <keeper>

Using the --soft option will keep any commits you haven't pushed yet and any changes you haven’t committed, so you can push them later once you’ve removed any large files. If you don’t have changes you want to keep, you can use the --hard option and skip Steps 1, 4, and 5 in the command line example above.

Run garbage collection to delete dangling commits

If you enabled the Labs feature (see tip above), the bad commits will be deleted by automatic garbage collection and your repo size should be reduced in a few minutes.

Otherwise you'll need to raise a Support request in order to run garbage collection to delete the bad commits that are no longer in the history and reduce the repository’s size.

Remote history in Bitbucket should now look similar to the following example:

		branch
        |
o---o---o-/-o
            |
            dangling commit deleted

Once that’s done pushes will work again, so you can go ahead and reduce the repository size to below the 1.0 GB limit by removing large files; see Removing large files below.

Removing large files

Once pushes are unblocked, you can go ahead and remove large files from the repository. Below are some resources to help you maintain your Git repository and provide more information about using Git LFS.

If you want to keep a lot of large files without paying for extra LFS storage you’ll need to put them elsewhere; see Options for storing large files below for a few of the available options.

Once large files have been removed, it is a best practice for everyone using the repository to make a new clone; otherwise, if someone does a force push, they will push the large files again and you’ll be back to where you started.

Avoiding large commits

There are a couple of things you can do to avoid accidentally adding large files to your repository:

  • Tell Git to ignore the kinds of files you don’t want to include.

  • Install automation to prevent large commits from being created.

Ignore large files

You can tell Git to exclude files from commits by adding pathname patterns to a file called .gitignore in the directory containing your local repository. 

For example:

# File types to ignore
*.exe
*.bin
*.jar
*.war
*.mp3
*.mp4
# Directories to ignore
target/
.build/
.env/

In general, you may want to tell Git to ignore:

  • build artifacts – best to put all these in a directory, for example Maven puts them in a target directory.

  • IDE settings – you don’t usually want these in the repository, for example ignore the .idea directory.

  • dependencies – exclude caches of dependencies, for example Python’s virtualenv or node’s local packages.

  • media files – a git repository is not the best place to keep large audio or video files.

Block large commits

To prevent any large files getting included in commits, you can install a local hook that checks the size of files in every commit and will reject the commit if it is too large.

Start with the check_added_large_files hook, by copying it into the repository so anyone can install it by adding it as the pre-commit hook for their local repository:

ln -s check_added_large_files.py .git/hooks/pre-commit.py

You can change the copy of the script in your repository to work however you want, and everyone will get the latest logic when they pull.

Options for storing large files

Your Bitbucket repository is the best place to keep source files. There are better places to keep other files generated from that source. Here we give some examples but encourage you to explore all the available options.

Use an artifact repository

There are many services that store build artifacts, two popular examples being:

Configure your build process to upload build artifacts to these repositories so they can be shared. Also, make sure .gitignore is configured to exclude build artifacts from commits.

Use a Docker repository

If your Bitbucket repository is used to build executables, consider building them as Docker images.

Your build process can push the images to a Docker repository hosted on Docker Hub, or a local Docker repository.

Use AWS S3

Rather than storing large media files in your Bitbucket repository, upload them to an S3 bucket where they can be easily downloaded.

Use Git LFS

If the files really need to be part of the Bitbucket repository, use the Large File Storage available on your plan. You can buy more storage if required.

Tell Git to use LFS for specific types of file using a wildcard pattern:

git lfs track "<pattern>"

For example, to use LFS for MP4 movies:

git lfs track "*.mp4"

*The quotes in the example above are important.

tip/resting Created with Sketch.

Use BFG to migrate a repo to Git LFS by moving existing large files into more efficient Large File Storage, reducing your repository size and giving a better Bitbucket experience.