How to find out which table data is slowing down the XML import
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
The Python script on this page is provided on best-effort basis to aid with the diagnosis of an import. Consequently, Atlassian Support cannot guarantee the provision of any support for the steps described on this page as these steps are not covered under Atlassian Support Offerings. Please be aware that this material is provided for your information only and that you use it at your own risk.
XML imports in Jira may take long, way too long sometimes. In some extreme case, they completely stall.
It is useful to know which table is making the import slow or stall. If it happens, it's most likely an AO table from activeobjects.xml.
The import screen will show which entity is being imported.
Example: Entity 10866453 of 15777975
Solution
The following Python script will output the list of tables and the number of records from activeobjects.xml.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import xml.etree.ElementTree as ET
ET.register_namespace('', "http://www.atlassian.com/ao")
ET.register_namespace('xsi', "http://www.w3.org/2001/XMLSchema-instance")
tree = ET.parse('activeobjects.xml')
root = tree.getroot()
aoStats = []
# take second element for sort
def takeSecond(elem):
return elem[1]
for child in root.findall('{http://www.atlassian.com/ao}data'):
tableName = child.attrib.get('tableName')
childFile = tableName + '.xml'
rowCount = len(child)
aoStats.append((tableName, rowCount))
'''
# The following splits every AO table in its own file
childData = ET.tostring(child)
with open(childFile, 'w') as f:
f.write(childData)
f.close()
'''
# aoStats.sort(key=takeSecond, reverse=True)
for item in aoStats:
print("Table " + item[0] + " has " + str(item[1]) + " rows.")
How to run the script
The results of the script will redirected to xml_stats.txt
1
python ao_stats_and_split.py > xml_stats.txtSample lines from xml_stats.txt:
1
2
3
4
5
6
7
8
9
10
Table AO_8B1069_TEST_RUN_STATUS has 1015573 rows.
Table AO_8B1069_ACTIVITY_ENTRY has 5657977 rows.
Table AO_8B1069_TEST_MEMENTO has 1917998 rows.
Table AO_733371_EVENT_PARAMETER has 4235763 rows.
Table AO_575BF5_DEV_SUMMARY has 3135030 rows.
Table AO_60DB71_LEXORANK has 2782083 rows.
Table AO_8B1069_ACTIVITY_ITEM has 5631474 rows.
Table AO_C77861_AUDIT_ENTITY has 2723793 rows.
Table AO_8B1069_TEST_RUN has 2294542 rows.
Table AO_8B1069_REQ_HSTATUS_ENTITY has 1027885 rows.How to find out which table is problematic
Some maths has to be done at this step to find out where the "bad" record fits.
For example, if the import stalls with the message "Entity 10866453 of 15777975" on the screen. We will need to find out which table has entity 10866453.
If we look at the row count for the example, entity 10866453 falls within the table AO_733371_EVENT_PARAMETER. This table should be inspected at the source or in the XML file.
Optional: How to sort tables by number of records in the XML
Just comment out the following line from the Python script and run it again:
1
aoStats.sort(key=takeSecond, reverse=True)Was this helpful?