Active Directory incremental synchronization is intermittently failing in Bitbucket Datacenter
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
When AD external directory is configured for incremental sync, it may intermittently fail with the following error and revert to a full sync
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
2024-09-18 10:45:31,641 ERROR [Caesium-1-4] c.a.c.d.DbCachingRemoteDirectory Incremental synchronisation for directory [ 45526243 ] was unexpectedly interrupted, falling back to a full synchronisation
com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresherIncSyncException: Cache returned different number of guids and non-local groups (possible reason is overlapping guids in cache, most likely null/empty values).
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.getAndValidateGroupGuidsFromCache(UsnChangedCacheRefresher.java:445)
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.synchroniseGroupChanges(UsnChangedCacheRefresher.java:395)
at com.atlassian.crowd.directory.synchronisation.cache.UsnChangedCacheRefresher.synchroniseChanges(UsnChangedCacheRefresher.java:129)
at com.atlassian.stash.internal.crowd.CustomizedUsnChangedCacheRefresher.synchroniseChanges(CustomizedUsnChangedCacheRefresher.java:66)
at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:1080)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.lambda$synchronise$0(DirectorySynchroniserImpl.java:87)
at com.atlassian.crowd.audit.NoOpAuditLogContext.withAuditLogSource(NoOpAuditLogContext.java:17)
at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:85)
at jdk.internal.reflect.GeneratedMethodAccessor5766.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at jdk.proxy3/jdk.proxy3.$Proxy276.synchronise(Unknown Source)
at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:48)
at com.atlassian.crowd.manager.directory.monitor.poller.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:92)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:518)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:513)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:537)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:433)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
at java.base/java.lang.Thread.run(Thread.java:840)
... 9 frames trimmed
2024-09-18 10:45:31,649 INFO [Caesium-1-4] c.a.c.d.DbCachingRemoteDirectory FULL synchronisation for directory [ 45526243 ] startingEnvironment
7.x,8.x
Cause
The following error messages in the logs indicate that groups are getting modified or renamed in the Active Directory, which is internally causing an issue with the incremental sync and Bitbucket is switching back to full sync.
1
com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresherIncSyncException: Cache returned different number of guids and non-local groups (the possible reason is overlapping guids in the cache, most likely null/empty values) Additionally, the following error can be seen shows that groups are being added and deleted, indicating changes within the groups
1
2
3
4
2024-07-18 10:45:32,891 DEBUG [Caesium-1-4] c.a.c.d.DbCachingRemoteChangeOperations group [ testGlobalGrp ] not found, deleting
2024-07-18 10:45:32,896 DEBUG [Caesium-1-4] c.a.c.d.DbCachingRemoteChangeOperations group [ DEVTools ] not found, deleting
2024-07-18 10:45:33,447 DEBUG [Caesium-1-4] c.a.c.d.s.c.DefaultGroupActionStrategy group 'ExcelLabsgroup' not found, adding
2024-07-18 10:45:33,449 DEBUG [Caesium-1-4] c.a.c.d.s.c.DefaultGroupActionStrategy group 'citrix-group' not found, adding Solution
Please confirm whether changes are being applied to the existing group names at the Active Directory level. If so, please be advised that groups cannot be renamed. We have a known issue, CWD-3606, where incremental synchronization fails to correctly recognize LDAP group renaming. This necessitates reverting to full synchronization, which resolves the issue of group renaming.
Was this helpful?