How to convert all attachments in a CSV file to use FILE protocol for an External System Import in Jira Server and Data Center

Summary

Problem

When migrating from one instance to another using CSV Import (External System Import), a user may run into the error An exception occurred dealing with attachment when importing issues with attachments:

Diagnosis

Using External System Import for the CSV import

Cause

When the CSV file is exported, the attachments path will be in a URL format. The error message appears because the attachments location is not accessible by the target Jira instance.

Solution

We may configure the two instances (source and target) to access each other, but it is not feasible to do this, just for the CSV import. We can use FILE protocol to include the attachments in the CSV import. To programmatically append the attachments URL to FILE protocol, we can use sed:

1. Copy the attachment directory from source Jira: $JIRA_HOME/data/attachments/ABC to the import directory in the target Jira: $JIRA_HOME/import/attachments/ABC

2. Move all attachments in the directory $JIRA_HOME/import/attachments/ABC AND its sub-directories, to the main directory $JIRA_HOME/import/attachment/ABC.

   1. At the target Jira, cd into project directory in the import/attachments directory (usually project key e.g ABC)

      1 cd ABC

   2. Copy all attachment files and place them under the main project directory ABC.

      1 find -type f -print0 | xargs -0 cp -t .

      1. The command is going to copy all the files that it finds in a current directory or any subdirectory back to the current directory. For example, let's see we have the following path:

         /jira_home/data/attachments/ABC/10000/ABC-123/10100

         Directory structure will look like this:

         1 2 3 4 ABC └── 10000 └── ABC-123 └── 10100

         By running the command, we copy the 10100 attachment directly under directory ABC

   3. At this point, the attachment files will all be here. They are named with 5 numerical digits which are their respective attachment IDs from the source Jira.

      1 2 3 4 5 6 7 8 9 10 $JIRA_HOME/import/attachments/ABC$ ls -lp | grep -v / total 1616 -rw-r----- 1 akmal akmal 52792 Nov 29 07:31 10201 -rw-r----- 1 akmal akmal 37611 Nov 29 07:31 10202 -rw-r----- 1 akmal akmal 1008590 Nov 29 07:31 10203 -rw-r----- 1 akmal akmal 477637 Nov 29 07:31 10204 -rw-r----- 1 akmal akmal 11162 Nov 29 07:31 _thumb_10201.png -rw-r----- 1 akmal akmal 5585 Nov 29 07:31 _thumb_10202.png -rw-r----- 1 akmal akmal 31374 Nov 29 07:31 _thumb_10203.png -rw-r----- 1 akmal akmal 10582 Nov 29 07:31 _thumb_10204.png

3. What we want to do is to change the format of the URL in the CSV file to use FILE protocol, for example:

   http://<jira-base-url>/secure/attachment/10204/gc-log.jpg

   to

   file://ABC/10204

   To append the attachments URL in the CSV file, to FILE protocol using sed. Carefully replace <jira-base-url> in the command below, with the actual base URL of the source Jira, and <project-key>. To append the CSV file, for example, a CSV file named import.csv

   1 sed -E 's@http://<jira-base-url>/secure/attachment/([^/"]*)/([^",]+|\\")@file://<project-key>/\1@g' import.csv > new_import.csv

   ⚠️ Mac users may need to include -i.bak to get sed to work. Windows might have different syntax.

4. The sed command above will save the appended CSV file as new_import.csv. Use the new_import.csv for the CSV Import.