Use conversation data to improve your virtual service agent's performance

The Conversations page in the virtual service agent provides you with a log of virtual service agent conversations. You can use this data to make improvements to your virtual service agent’s performance.

Today, the Conversations page only shows conversations from your portal, your help center, and Slack. We’re working on showing conversations from other channels – check back here for updates.

See your virtual service agent’s conversation data

To see your virtual service agent’s conversations:

1. From your service project, select Project settings, then Channels & self service, then Virtual service agent..

2. Select Conversations.

3. Choose a Start date and an End date to narrow down the conversations you want to see.

4. Use the Filters to find conversations with specific attributes.

5. If a conversation was matched to an intent, you can jump straight to that intent from the Intent column.

6. To see the details of a conversation, select the arrow in the Link column.

Filters available

The following data is visible in the table and can be used to filter your conversations.

In the Action column, you can see if a conversation was:

• Matched: The virtual service agent detected an intent with high confidence, asked the customer if the detected intent was correct, and the customer confirmed that it was. Read more about the match intent standard flow.

• AI answered: The virtual service agent answered the customer question using Atlassian Intelligence answers.

• Unassisted: No intents were matched, and no Atlassian Intelligence answers were provided.

In the Resolution column, you can see if a conversation was:

• Escalated: The virtual service agent created an issue in Jira Service Management.

• Resolved: The conversation was marked as resolved by the customer after an intent was matched or answered using Atlassian Intelligence answers.

  • Resolved conversations are sometimes escalated after resolution (for example, when someone sends a message in a Slack thread after the virtual service agent resolved the conversation). When this happens, the conversation will be shown as Resolved in performance metrics, but as Escalated on the Conversations page.

• Closed: No intents were matched and no Atlassian Intelligence answers were provided during the conversation, and then the customer abandoned it for 5 minutes. After being nudged by the virtual service agent, the customer indicated they no longer needed help. Read more about the auto-close standard flow.

In the CSAT column, you can see the customer satisfaction (CSAT) score that was provided by the customer for each conversation.

Use conversation data to improve virtual service agent performance

There are endless ways to use conversation data to improve the performance of your virtual service agent. Below are some examples to get you started.

Reduce escalations

Let’s say your main goal for using the virtual service agent is to reduce your team’s workload – specifically, you want less issues created in your Jira Service Management project.

In that case, you might want to filter your conversations by Escalated. If a lot of these conversations were also Matched to an intent, you might start by reading back over some specific conversations and seeing where you could make improvements.

If things went wrong during the conversation, you might want to look at improving your intent’s conversation flow. If the wrong intent is being matched, you might want to go and refine your training phrases.

Improve the quality of Atlassian Intelligence answers

Let’s say you filter your conversations by AI answered, and discover that many of these conversations are also being Escalated – or they have low CSAT scores. This might indicate that your knowledge base is out of date, and providing wrong (or not enough) information to your customers.

To check, you could open some individual conversations, and review the source articles being used to generate Atlassian Intelligence answers. Are the articles missing key information, or is the information out of date? By making sure your connected knowledge base is accurate and contains the information that your customers are asking for, you can quickly improve answer quality, which is likely to reduce escalations and improve your virtual service agent’s CSAT score.

Use the virtual service agent to more effectively triage issues for your human agents

Unlike the above scenario where you want to avoid creating issues in Jira Service Management, let’s say you’re using the virtual service agent to gather information from customers, create issues on their behalf, and route them to the correct request types – so that your human agents can resolve them faster.

In this case, you might filter your conversations by Unassisted and discover customer queries that may need a new intent to cover them. When building that new intent’s conversation flow, you could use the Change request type and fields step to make sure issues are being created using the correct request type, and then use Ask for information steps to gather information that might help your human agents resolve those issues faster. Read more about step types in the conversation flow builder.