Git operations on Bitbucket Data Center fail due to issues with file locking
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Git operations on Bitbucket Data Center fail due to issues with file locking.
You may see various git operations intermittently fail, for example:
A client's git push request fails, but if the client repeats the same operation, the next attempt may work as expected.
A Pull Request may intermittently not get updated after a change is pushed to a related branch.
Webhooks may not always be triggered as expected.
Environment
This issue has been observed in Bitbucket DC version 7.7.1, but the same issue may also occur with other versions.
Diagnosis
Log messages
The following may appear in atlassian-bitbucket.log
1
2
3
4
5
6
7
2021-08-23 09:55:12,286 INFO [pull-request-rescoping:thread-1] c.a.s.i.p.r.PullRequestRescopeBucketProcessor PROJ/repo[123]: Some pull requests were not rescoped. (0 failed, 1 locked). Scheduling attempt 2 in 20000ms
2021-08-23 12:25:48,580 INFO [pull-request-rescoping:thread-1] c.a.s.i.p.r.PullRequestRescopeBucketProcessor PROJ/repo[123]: Some pull requests were not rescoped. (0 failed, 1 locked). Scheduling attempt 2 in 20000ms
2021-08-23 14:49:24,227 INFO [pull-request-rescoping:thread-1] c.a.s.i.p.r.PullRequestRescopeBucketProcessor PROJ/repo[123]: Some pull requests were not rescoped. (0 failed, 1 locked). Scheduling attempt 2 in 20000ms
2021-08-23 14:49:25,783 INFO [pull-request-rescoping:thread-1] c.a.s.i.p.r.PullRequestRescopeBucketProcessor PROJ/repo[123]: Some pull requests were not rescoped. (0 failed, 1 locked). Scheduling attempt 3 in 40000ms
2021-08-24 15:30:26,760 ERROR [hz.hazelcast.partition-operation.thread-4] c.h.c.l.operations.UnlockOperation [10.234.23.123]:5701 [prod] [3.12.6] Current thread is not owner of the lock! -> <not-locked>
2021-08-24 15:30:27,411 ERROR [hz.hazelcast.partition-operation.thread-10] c.h.c.l.operations.UnlockOperation [10.234.23.123]:5701 [prod] [3.12.6] Current thread is not owner of the lock! -> <not-locked>
Environment
Bitbucket Data Center with more than 1 node,
Bitbucket nodes running as Docker containers, started via podman,
A volume under control of podman is mounted in
$BITBUCKET_HOME/shared
Diagnostic Steps
Running Bitbucket instance with only one node seems to resolve the problem.
Cause
$BITBUCKET_HOME/shared does not handle file locking correctly.
Solution
Mount the volume for
$BITBUCKET_HOME/shareddirectly on the host and pass it through to Bitbucket's Docker container.Double-check and confirm that the mount options are exactly as recommended in the documentation:
Install Bitbucket Data Center - Step 2. Provision your shared file system(expand the section: Recommended mount options)
Was this helpful?