Remove a file from the repository history when there are multiple files with the same name in Bitbucket Cloud

Summary

This article covers how to remove specific files from a repository's history using git filter-repo in Bitbucket Cloud.

Using BFG Repo-Cleaner

You can use BFG to remove files from the entire repository's history. However, that KB does not work if:

• you have multiple files with the same name in different folders, and you want to delete only one set of files

• you want to delete an entire directory (You must enter individual filenames, but this becomes cumbersome with a large number of files)

Prerequisite

• You must have git filter-repo installed. If you have not installed it, please install it as per the instructions here. You will need to install Python 3 if you don't already have it for git filter-repo to function.

Solution

To resolve this problem, we can use the third-party tool git filter-repo. You can find a cheatsheet for similar BFG commands and general documentation for the git filter-repo command. However, this KB will highlight the commands to remove a specific file.

Create a mirror clone

1. Take a mirror clone of the repo with the command:

   git clone --mirror https://<username>@bitbucket.org/<workspace-ID>/<repository-name>.git

2. Replace <username>, <workspace-ID>, and <repository-name> in the clone URL with the respective values of your Bitbucket username, your workspace's ID, and your repo's slug.

Removing a file from branches

We want to remove a file from all branches of the repository, so we need a mirror clone as it will have all branches of the remote repo.

1. Navigate to the directory of the mirror clone.

2. The command below will remove the file remove.txt from the repository. (It will only remove remove.txt if it is in the repository root folder.)

   git filter-repo --invert-paths --path remove.txt

3. To remove the file dir1/dir2/remove.txt.

   git filter-repo --invert-paths --path dir1/dir2/remove.txt

4. To remove the file remove.txt from multiple folders.

   git filter-repo --invert-paths --path-glob '*/remove.txt'

Wildcards

The * includes multiple folders at any level. It will remove dir1/dir2/remove.txt as well as dir1/remove.txt. However, it will not remove remove.txt from the root folder. Whenever we use wildcards (*), we must use the argument --path-glob instead of --path in the git filter-repo command and enclose the path in quotes.

1. To remove an entire directory called dir1/dir2 in the repository's root.

   git filter-repo --invert-paths --path-glob 'dir1/dir2/*'

2. To remove all files with the extension .asset.

   git filter-repo --invert-paths --path-glob '*.asset' --use-base-name

This behaves like BFG. When we use the use-base-name parameter, it does not find files by absolute path. Instead, it considers the filename. Hence, every file name that ends in '.asset' is deleted.

Pushing changes to an empty repo

Once you have removed the files, you'll want to push your changes back to the remote repo.

Remember that git filter-repo will remove the remote origin from the mirror clone once you run it. For an explanation, see the git-filter-repo manual page, section "Why is my origin removed?".

It is recommended that you create a new empty Bitbucket Cloud repo and push your changes there first, so that you can inspect if the repo is in good shape after the changes you made. You can push from the mirror clone to the newly created empty repo with the command:

Replace <username>, <workspace-ID>, and <EMPTY-repository-name> in the clone URL with the respective values of your Bitbucket username, your workspace's ID, and the slug of your newly created empty Bitbucket repo.

Pushing changes to the original repo

After you inspect the empty repo you pushed to above and find it in good shape, you can push to your original Bitbucket repo.

It is essential to communicate to your team members that you intend to push a history rewrite to the repo, and ask all your team members to take a new clone after you push your changes. Otherwise, your team members may push back the old commits with the files/folders you removed from history.

You can push from the mirror clone to your original Bitbucket repo with the command:

Replace <username>, <workspace-ID>, and repository-name> in the clone URL with the respective values of your Bitbucket username, your workspace's ID, and the slug of your Bitbucket repo.

This push is an implicit force push.