JIRA Indexing Performance and Lucene maxrambuffermb
プラットフォームについて: Data Center のみ。 - This article only applies to Atlassian apps on the Data Center プラットフォーム。
この KB は Data Center バージョンの製品用に作成されています。Data Center 固有ではない機能の Data Center KB は、製品のサーバー バージョンでも動作する可能性はありますが、テストは行われていません。 Server* 製品のサポートは 2024 年 2 月 15 日に終了しました。Server 製品を実行している場合は、 アトラシアン Server サポート終了 のお知らせにアクセスして、移行オプションを確認してください。
*Fisheye および Crucible は除く
コンテキスト
Jira 8.0 has a new version of the Lucene library. In this version, we have periodic Lucene data commits done every 30 sec by default. On top of that, Lucene can also trigger flushes according to memory usage: maxrambuffermb– RAM usage of documents added to the IndexWriter. We have also noticed that Lucene flushes can consume a lot of memory (exceeding maxrambuffermb), especially when flushing a low number of documents with a large number of Lucene Doc Values of type SortedDocValues and SortedSetDocValues (for example in multiple custom fields).
To find out the version of Lucene Jira Software is using, go to/Installation-directory/atlassian-jira/WEB-INF/lib and locate the Lucene jar files. The Lucene version number will be part of the filename.
Performance tests
Example test results for different maxrambuffermb sizes on Jira 8.0 (1M issues, 5000 custom fields, 100 projects, 6CF per issue, 16GB heap):
| Flushes interval (from average in 5-min period) average / 95th percentile | Full reindex time |
|---|---|---|
256MB | n/a | 32 hours |
512MB | 0.7 sec / 0.19 sec | 17.5 hours |
768MB | n/a | 3 hours |
1024MB | 4.2 sec / 1.8 sec | 2.3 hours |
1536MB | 5 sec / 2.7 sec | 2.2 hours |
2048MB | n/a | 2.2 hours |
変更
Jira 8.0 comes with a larger maxrambuffermb size (Jira Config property: jira.index.batch.maxrambuffermb, default 1024) to handle issues with a large number of CF by default. As per Jira Software 8.0.x upgrade notes:
The maxrambuffermbproperties define the maximum size of a memory write buffer for Lucene documents queued to be saved into the index files. We’ve increased it to better handle issues with a large number of custom fields. Because of this change, we’ve also increased the default maximum heap size (xmx).
There are also new logs and warnings in the application logs (atlassian-jira.log) to help diagnose the problem:
New Lucene stats about flushes (logged every 5 minuntes for every indexer):
INFO [lucene-stats] flush stats: snapshotCount=59, totalCount=1649, periodSec=308, flushIntervalMillis=5229, indexDirectory=/opt/atlassian/jira/jira_home/caches/indexesV1/issuessnapshotCount– the number of flushes in this period (~5 minutes by default).totalCount– the number of flushes since this index writer was created.periodSec– this period in seconds.flushIntervalMillis– the average flush interval in milliseconds.indexDirectory– the index directory path
The most interesting part of this log is the flushIntervalMillis value. With very small flushIntervalMillis (less than 1 second), we can see young GC start having issues with cleaning the garbage created by the flush. As a result, GC pauses start to affect the indexing time and performance.
There is a new warning logged when
flushIntervalMillisis below 1 second (by default). If this warning appears, you should review and, probably, increase themaxrambuffermbvalue to avoid any potential performance problems:WARN - Detected frequent flushes (every 600 millis) of lucene index which is below warning limit: jira.index.warn.flush.min.interval.millis=1000 millis. This may affect indexing performance. Please visit https://ja.confluence.atlassian.com/x/w0VwOQ for more information.
False alarm
The log statistics and warnings only apply to "real indexes" stored on the disk. Jira also uses in-memory indexes (RAMDirectory) for temporary indexing. Statistics and warnings for those indexes can be ignored: JRASERVER-69389 - Frequent lucene-stats in logs triggered when closing temporary index (RAMDirectory)
この内容はお役に立ちましたか?