InvalidDocumentFieldException error when Re-indexing in Jira Data Center
Platform Notice: Data Center Only - This article only applies to Atlassian apps on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Re-indexing in Jira 10.4+ fails with an InvalidDocumentFieldException when custom fields are missing from the index schema. The application logs shows 'Error indexing issue <ISSUEKEY>: Dropping 'customfield_xxxxx'com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields'error while indexing.
<Date> <Timestamp> JiraTaskExecutionThread-1 WARN <username> [c.a.j.issue.index.DefaultIssueDocumentFactory] Error indexing issue <ISSUEKEY>: Dropping 'customfield_xxxxx'
com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [string_customfield_xxxxx]
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.lambda$validate$0(SchemaValidatedFieldValueCollector.java:35)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.validate(SchemaValidatedFieldValueCollector.java:35)
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.add(SchemaValidatedFieldValueCollector.java:28)
at com.atlassian.jira.search.field.FieldValueCollector.add(FieldValueCollector.java:18)
at com.atlassian.jira.search.field.FieldValueCollector.add(FieldValueCollector.java:22)Diagnosis
Install a third-party app that provides additional custom fields or scripted/calculated custom fields(ex. Scriptrunner).
Run a re-index.
The re-index may take longer depending on the volume of issues that use the custom field. The logs contain the error with a similar stack below:
<Date> <Timestamp> JiraTaskExecutionThread-1 WARN <username> [c.a.j.issue.index.DefaultIssueDocumentFactory] Error indexing issue <ISSUEKEY>: Dropping 'customfield_xxxxx'
com.atlassian.jira.search.exception.InvalidDocumentFieldException: Document contains invalid fields [string_customfield_xxxxx]
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.lambda$validate$0(SchemaValidatedFieldValueCollector.java:35)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.validate(SchemaValidatedFieldValueCollector.java:35)
at com.atlassian.jira.search.field.SchemaValidatedFieldValueCollector.add(SchemaValidatedFieldValueCollector.java:28)
at com.atlassian.jira.search.field.FieldValueCollector.add(FieldValueCollector.java:18)
at com.atlassian.jira.search.field.FieldValueCollector.add(FieldValueCollector.java:22)
at com.atlassian.jira.search.issue.index.indexers.impl.NumberCustomFieldIndexer.indexFields(NumberCustomFieldIndexer.java:113)
at com.atlassian.jira.search.issue.index.indexers.impl.FeatureFlagGuardedFieldIndexer.indexFields(FeatureFlagGuardedFieldIndexer.java:48)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:419)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAllWithPrefetchedData(DefaultIssueDocumentFactory.java:376)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.getDocument(DefaultIssueDocumentFactory.java:165)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.lambda$createDocuments$1(DefaultIssueDocumentFactory.java:142)
at java.base/java.util.Collections$SingletonList.forEach(Collections.java:4966)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.createDocuments(DefaultIssueDocumentFactory.java:141)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.createDocument(DefaultIssueDocumentFactory.java:110)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:1455)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$IssuesOperation.createDocument(DefaultIssueIndexer.java:1073)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:906)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$perform$8(DefaultIssueIndexer.java:592)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:7)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:5)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$perform$9(DefaultIssueIndexer.java:590)
at com.atlassian.jira.issue.util.DatabaseIssuesIterable.foreach(DatabaseIssuesIterable.java:57)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.perform(DefaultIssueIndexer.java:559)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.reindexIssues(DefaultIssueIndexer.java:409)
at com.atlassian.jira.issue.index.IndexManagerBase.doBackgroundReindex(IndexManagerBase.java:1080)
at com.atlassian.jira.issue.index.IndexManagerBase.reIndexAll(IndexManagerBase.java:364)
at com.atlassian.jira.issue.index.IndexManagerBase.reIndexIssuesInBackground(IndexManagerBase.java:464)
at com.atlassian.jira.util.index.IndexLifecycleManager$$FastClassByCGLIB$$a948afa0.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at com.atlassian.jira.config.component.AbstractDelegatedMethodInterceptor.intercept(AbstractDelegatedMethodInterceptor.java:48)
at com.atlassian.jira.issue.index.JiraIndexManager$$EnhancerByCGLIB$$3162e03.reIndexIssuesInBackground(<generated>)
at com.atlassian.jira.util.index.DefaultCompositeIndexLifecycleManager.reIndexIssuesInBackground(DefaultCompositeIndexLifecycleManager.java:114)
at com.atlassian.jira.web.action.admin.index.ReIndexBackgroundIndexerCommand.doReindex(ReIndexBackgroundIndexerCommand.java:33)
at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:63)
at com.atlassian.jira.web.action.admin.index.ReIndexBackgroundIndexerCommand.call(ReIndexBackgroundIndexerCommand.java:16)
at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:26)
at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:539)
at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:497)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.atlassian.jira.task.ForkedThreadExecutor$ForkedRunnableDecorator.run(ForkedThreadExecutor.java:216)
at java.base/java.lang.Thread.run(Thread.java:842)Cause
The Index Schema is introduced in Jira 10.4 as documented in FieldIndexer migration details | Administering Jira applications. The error happens when a custom field being indexed is not included in the index schema. An index schema refresh will load custom fields that are not yet included in the index schema.
Solution
Below are the steps to trigger an index schema refresh.
Navigate to Jira Administration > Issues > Custom Fields.
Choose Edit Details for any custom field.
Click Update (no changes are required).
After the index schema refresh, the custom fields should be included in the index schema, and the re-index can be re-run again to resolve the InvalidDocumentFieldException errors during re-indexing.
If the steps above do not work, restarting Jira will also trigger an index schema refresh.
Was this helpful?