Jira DC Mail Handler: Why Comments Are Added Instead of New Tickets

Summary

The Jira or the Jira Service Management (JSM) mail handler added a comment to an existing Jira ticket instead of creating a new ticket from the incoming email:

• the email did not have any Jira issue key in the subject

• despite not having any issue key in the subject, the email was linked to an existing ticket and a comment was added to it, using the body of the email

  

The purpose of this knowledge base article is to explain why a mail handler might decide to add a comment to an existing Jira issue instead of creating a new Jira issue.

Note about this knowledge base article

It is important to note that Jira comes with 2 types of mail handlers (provided that Jira Service Management is installed on top of it).

Both types of mail handlers are actually 2 different functionalities (they are using different source codes), and therefore work differently for the most part:

• the Jira Mail Handler (configured in ⚙ > System > Incoming email)

• the Jira Service Management (JSM) Mail Handler (Configured in Project Settings > Email Requests within the Service Desk project configuration page)

However, both types of mail handlers use the same logic to decide whether they should create a new ticket or add a comment to an existing ticket from an incoming email. For this reason, this KB article is valid for these 2 types of mail handlers.

Solution

Environment

• In case of an issue with the Jira Mail Handler:

  • Jira Server/Data Center 7.0.0 and any higher version

• In case of an issue with the JSM Mail Handler:

  • Jira Service Management (JSM) Server/Data Center 3.0.0 and any higher version

Diagnosis

• When checking the Inbox folder of the mailbox used by the Mail Handler, we can see that the email that was used to add a comment is a reply (or a forward) of an older email:

  

• Looking at the Jira issue where the email was added as a comment, we can see that the Jira issue summary and description were populated using the data from the older email, while the comment was added using the data from the new email

  

• Checking the mail source of the new email, we can see that it contains references to an older email (since it was sent as a reply/forward of an older email)

  • We can verify it by checking the References and/or the In-reply-to headers which contain the ID of the email that the user replied to (<CAGf7kN8i2mKMSEv9M6Fmm6nuN4fYwLBnmCZiUupxKyKWNFiGsA@mail.gmail.com> in the example below)

    Delivered-To: incomingmailhandler@gmail.com Received: by 2002:a92:d112:0:0:0:0:0 with SMTP id a18csp322311ilb; Wed, 13 Apr 2022 06:50:09 -0700 (PDT) ... MIME-Version: 1.0 References: <CAGf7kN8i2mKMSEv9M6Fmm6nuN4fYwLBnmCZiUupxKyKWNFiGsA@mail.gmail.com> In-Reply-To: <CAGf7kN8i2mKMSEv9M6Fmm6nuN4fYwLBnmCZiUupxKyKWNFiGsA@mail.gmail.com> From: Julien Rey <senderoftheemail@gmail.com> Date: Wed, 13 Apr 2022 15:49:57 +0200 Message-ID: <CAGf7kN-Cyea-Huj7qWHgPgy8geuikzZaRVh2z2reYYSU=W_LkA@mail.gmail.com> Subject: Re: Some task to do To: Jira Incoming Mail <incomingmailhandler@gmail.com> Content-Type: multipart/alternative; boundary="000000000000999fe105dc89743b" --000000000000999fe105dc89743b Content-Type: text/plain; charset="UTF-8" Will this email create a new ticket, even though there is no issue key in the subject line? On Wed, Apr 13, 2022 at 3:47 PM Julien Rey <senderoftheemail@gmail.com> wrote: > Hi, here is some details > --000000000000999fe105dc89743b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Will this email create a new ticket, even though ther= e is no issue key in the subject line?</div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 13, 2022 at 3:47 PM Julie= n Rey <<a href=3D"mailto:senderoftheemail@gmail.com">senderoftheemail@gmail.com</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0= px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir= =3D"ltr">Hi, here is some details</div> </blockquote></div></div> --000000000000999fe105dc89743b--

• When checking the source of the old email, we can see that the ID found in the References and/or the In-reply-to headers of the new email matches the ID found in the Message-ID header of the old email:

  Delivered-To: incomingmailhandler@gmail.com Received: by 2002:a92:d112:0:0:0:0:0 with SMTP id a18csp320989ilb; Wed, 13 Apr 2022 06:48:07 -0700 (PDT) ... MIME-Version: 1.0 From: Julien Rey <senderoftheemail@gmail.com> Date: Wed, 13 Apr 2022 15:47:56 +0200 Message-ID: <CAGf7kN8i2mKMSEv9M6Fmm6nuN4fYwLBnmCZiUupxKyKWNFiGsA@mail.gmail.com> Subject: Some task to do To: Jira Incoming Mail <incomingmailhandler@gmail.com> Content-Type: multipart/alternative; boundary="00000000000062643d05dc896dfe" --00000000000062643d05dc896dfe Content-Type: text/plain; charset="UTF-8" Hi, here is some details --00000000000062643d05dc896dfe Content-Type: text/html; charset="UTF-8" <div dir="ltr">Hi, here is some details</div> --00000000000062643d05dc896dfe--

• When running the following SQL query (after replacing ABC with the project key and 123 with the issue number), we can see that the Jira issue where the comment was added is mapped to the Message-ID of the old email (and also the one from the new email):

  select * from notificationinstance where source in ( select id from jiraissue where project = ( select id from project where pkey = 'ABC' ) and issuenum in (123) );

  • Example

    select * from notificationinstance where source in ( select id from jiraissue where project = ( select id from project where pkey = 'BUS' ) and issuenum in (81) ); id notificationtype source emailaddress messageid 15603 "ISSUE_COMMENTED_FROM_EMAIL" 19304 "senderoftheemail@gmail.com" "<CAGf7kN-Cyea-Huj7qWHgPgy8geuikzZaRVh2z2reYYSU=W_LkA@mail.gmail.com>" 15602 "ISSUE_CREATED_FROM_EMAIL" 19304 "senderoftheemail@gmail.com" "<CAGf7kN8i2mKMSEv9M6Fmm6nuN4fYwLBnmCZiUupxKyKWNFiGsA@mail.gmail.com>"

Cause

As explained in the section Issue/comment creation of the Jira Mail Handler documentation, the mail handler examines various data from any incoming email to decide whether it should create a new Jira issue, or add a comment to an existing issue. If the Mail Handler is able to detect any trace of an existing issue in an incoming email (via some of the headers of the email source), a new comment will be added to the existing ticket. Otherwise, a new issue will be created.

In other words, if an email was sent by either clicking the Reply or the Forward button on an older email, then the Mail Handler will do the following:

• it will loop over the ID of email from the mail chain

• it will check if there is any record of that Email ID in the Jira DB

• if it finds that one of the old emails was used to create a Jira ticket, it will add a comment to the existing ticket instead of creating a new ticket

⚠️ Note that the same logic is used by both the Jira Mail Handler and the JSM Mail Handler.

Solution

Typical scenario where this issue happens

• Customer A sends an fresh new email (using the new email or compose button from any mail client) to the Jira (or JSM) Mail Handler, and adds Customer B as a CC of this email

  • The Mail Handler created a new ticket with key ABC-123 from this email

  • Customer B receives the email (since they were added in CC)

• Customer B decides to Forward this email to the Mail Handler, after changing its content and its subject, in the hope that the email will be treated as a new email

  • The Mail Handler is able to detect the original email the new email was created from

  • It manages to map the old email to the existing Jira ticket

  • And as a result, it adds a comment to the old ticket instead of creating a new ticket

Basically, it does not matter if the subject or the content of the email was changed. What matters if the fact that the email is "not entirely fresh" (meaning that it was sent as a reply or forward of an old email). In such case, the Mail Handler will not treat the email as a "new email", and will try to map it to an older ticket

Technical explanation describing how the the mail handler looks for any trace of an existing Jira issue in an email

The mail handler checks 3 places/headers in the source of the email to determine whether it should create a new issue or add a comment to an existing issue:

• the mail Subject line (Subject header):

  • the mail handler looks for an issue key in the subject, and if there is a matching issue in Jira, a comment is added to this existing issue. If no issue key is found, it moves on to the next step.

• the In-Reply-To and References headers:

  • the mail handler tries to detect if there is any message-ID that is linked to an existing Jira ticket. If any message-ID is found in the 2 headers above, the mail handler will analyze it and try to map it to an existing Jira ticket

  • if the message-ID has the format "JIRA.12700.1562154694000.1.1562154780223@Atlassian.JIRA"

    • the ID after "JIRA" (in this case 12700) is compared to the id column of the database table jiraissue

    • if an existing row is found in the jiraissue table, then a comment is added to the existing ticket

    • Note that the message-ID will have this format if a user replies to a notifications email received from a Jira issue

  • if the message-ID has any other format

    • this message-ID will be compared to the ones stored in the database table notificationinstance.

      • This table contains all the email message-IDs that have been associated with each Jira ticket (whether the email was used to create the issue or add a comment to that issue)

    • if there is a match, then a comment will be added to the corresponding issue

    • Note the message-ID will have a random format in case the user replied to an email that was originally used to create a new Jira issue

Workaround

Please note that the behavior described in this knowledge base article is expected by design, therefore this is not a bug that requires a solution.

Having said that, there are a few options that can be considered to ensure that new emails (without an issue key in the subject) will always create a new ticket instead of a comment on an existing ticket:

Option 1

Instead of using the reply or forward button in the mail client, the user needs to create a fresh new email, so that the email will not contain any trace of an old email.

⚠️ Please note that simply clearing the subject or the body of the original email is not enough, because the reply will still contain some traces of the original email in the source. This is the reason why it is necessary to create a completely new email

Option 2

This option consists in using the database native tool or the script of your choice to frequently empty the content of the notificationinstance table in the Jira database. By doing this, the mail handler will not be able to map any older email to existing Jira issues, and will always create new issues from incoming emails, provided that they don't have an issue key in the subject.

⚠️ Some important notes about this workaround:

• This workaround is unsupported by Atlassian and risky, as it involves database manipulation

• When implementing this workaround, the Mail Handler will no longer be able to detect if a new email was a reply/forward to an older email that was used in the past to create a Jira issue

• There is an existing feature request to automatically meant to empty the content of the notificationinstance table, which is tracked in https://jira.atlassian.com/browse/JRASERVER-44418. However, this feature has not been added to the product roadmap.