Assets CSV import job does not import all assets from the file
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Assets CSV import job does not import all assets from the file. For example, only 2000 assets are imported from a CSV file containing more than 10000 assets.
Environment
Assets 8.10.15 or newer
Diagnosis
There are no errors in insight_import.log files. However, after enabling debugging for package com.riadalabs.jira.plugins.insight.services.imports.schema.log.ImportLogger, the following appears in the import log while the import task is running:
1
2023-10-16 13:49:08,777 [insight-InsightImportThreadGroup-worker-thread-1] | Row not read on line number (heading row included) 4622 due to The number of columns to be processed (22) must match the number of CellProcessors (21): check that the number of CellProcessors you have defined matches the expected number of columns being read/writtenFurthermore, analyzing the file, we can see there are many data rows parsed as a single row for a data locator:
1
2
3
4
5
6
2023-10-16 13:49:13,462 [insight-InsightImportThreadGroup-worker-thread-6] | dataLocatorsValue: value1;value2;value3;value4;;
000000000000001;12/10/2023;01:15:16;0003000000;1000;60;10;X;;*test1*;;;;;;;;;;;;;
000000000000002;12/10/2023;01:15:16;0003000001;1001;20;10;X;;*test2*;;;;;;;;;;;;;
...
...
000000000009999;12/10/2023;01:37:19;00030000002;1002;50;10;;;*test9999*;;;;;;;;;;;;;; for dataLocator [DataLocator [locator=test-locator]]]]><![CDATA[Cause
This problem could occur when a CSV column contains a single quotation (") char. A CSV file should be escaped properly with the quotes and if not, the application would consider it as a single parameter until it finds the escape character.
Debug lines show many rows are being treated as a single data locator value for a data locator, effectively treating number of assets as a single one; this is a consequence of the single / double quote not being escaped.
Solution
Check the CSV file and see where are quotes that don't have its closing match, or if there are semicolon ";" characters that are being part of the value, instead of being a delimiter. After fixing the cause in the CSV file, run the Assets import task again.
Was this helpful?