Re-indexing progress stuck on other nodes with message xx% complete. x days, xx hours, xx minutes, xx seconds elapsed
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
After successful re-index operation on one node, the index progress on other nodes still displays stuck message with "92% complete. 2 days, 11 hours, 23 minutes, 42 seconds elapsed"
Environment
All JIRA versions
DiagnosisBullet list
Check the index progress on other nodes, some of them might be working and shows correct status
Check the logs on the Node performing re-indexing operation
Cause
The node performing re-indexing operation was having intermittent N/W issues due to which it could not replicate the TaskManagerImpl cache via-copy and it removed it from the localQ and thus the consecutive cache messages were lost and cache on the other nodes (problematic nodes) displayed the last % perform operation completed.
1
2023-05-26 09:20:11,046-0700 localq-reader-56 INFO [c.a.j.c.distribution.localq.LocalQCacheOpReader] [LOCALQ] [VIA-COPY] Checked exception: RecoverableFailure occurred when processing: LocalQCacheOp{cacheName='com.atlassian.jira.task.TaskManagerImpl.taskMap', action=PUT, key=14409, value == null ? false, replicatePutsViaCopy=true, creationTimeInMillis=1685118001033} from cache replication queue: [queueId=queue_node3_6_1315e07dc5ecfdcec39f54ec16f564b7_put, queuePath=/atlassian/application-data/jira/localq/queue_node3_6_1315e07dc5ecfdcec39f54ec16f564b7_put], failuresCount: 1. Will not retry as this is a cache replicated by value. Removing from queue.Solution
Check for the cause on the Cache replication failure in the logs where the indexing is performed and see if internode issue and rectify it accordingly based on the caused by in the exception Stacktrace.
Restarting of the nodes should rectify the issue in most of the cases where Issue is intermittent.
Was this helpful?