How to coach AI Agent and give feedback

Just like you would with human agents on your team, coaching and providing feedback is a critical part of improving AI Agent’s effectiveness.

You can give feedback directly on the messages that AI Agent provides to customers in your tickets.

When reviewing a ticket handled by AI Agent, you can:

Mark AI Agent’s message or any of the resources it used as correct or incorrect
See which resources AI Agent used to write the message, including Shopify order info, Help Center articles, webpages, Macros and Guidance
Suggest that AI Agent use a different resource instead, if a better or more correct piece of knowledge exists
Report an issue with AI Agent to the Gorgias Product team

Watch this short video and follow the steps below to get started.

A short video about using Gorgias's AI Agent, specifically about how to observe and coach AI Agent to improve its responses to customers

3:12

1:18

Requirements

You must have an Active Automate subscription
You must have AI Agent set up and enabled for your store

Giving AI Agent feedback

You can give AI Agent direct feedback on the responses it provides to customers in your tickets.

Go to Tickets
Scroll down to your Views for AI Agent
Select a ticket that AI Agent handled (typically in the Handover or Closed views)
Select Review Response below AI Agent’s message in the thread
- Alternatively, you can select ✨ AI Feedback in the ticket sidebar

Within the ✨ AI Feedback ticket sidebar, you’ll find: a summary of the response AI Agent provided, including why it responded the way it did. Also, the resources AI Agent used to form its response (Help Center article, Guidance, Macros and Actions).

Opening a closed ticket handled by AI Agent, then opening the feedback tab within the sidebar to give feedback on AI Agent's message to the customer

Mark a response as correct or incorrect

To give feedback, use the 👍 or 👎 icons to mark AI Agent’s response as correct or incorrect. This type of feedback is used to inform AI Agent’s model and improve responses over time.

You can also mark the individual resources that AI Agent used as correct or incorrect, like Help Center articles or Guidance.

Using the thumbs up and thumbs down buttons within the feedback tab of a ticket to mark a message that AI Agent sent a customer as correct or incorrect

Suggest a different resource

You must have Lead or Admin permissions to perform this action

You can suggest a different resource to AI Agent if you think there’s a better Action, Guidance, Help Center article or Macro that it should have used to create its message.

Open the dropdown menu to select the alternate resource.

Alternatively, if the resource you have in mind doesn’t exist, select Create Resource to start a new Help Center article, Action or Guidance. The better the knowledge you create for AI Agent the more it can respond to tickets with relevant information and automate more of your team’s support interactions.

Using the thumbs up and thumbs down buttons within the feedback tab of a ticket to mark a message that AI Agent sent a customer as correct or incorrect

Report an issue with AI Agent

You must have Lead or Admin permissions to perform this action

If you have a problem with AI Agent’s message (typos, wrong tone of voice), or how it processed a ticket (it responded to a handover topic), you can report the issue.

Use the Report another issue to select a problem from the list. This type of feedback goes directly to Gorgias’s Product team to continually inform our work and help make AI Agent better.

Opening the feedback tab in the ticket sidebar to report an issue to Gorgias's Product team about how AI Agent responded to or handled a ticket

Who can give AI Agent feedback?

Any agent on Gorgias (Basic user or above) can mark a message sent by AI Agent as correct or incorrect using the 👍 or 👎 icons.

Only Lead or Admin users can provide more detailed feedback, including:

Suggesting a different resource that AI Agent should have used
Creating a new resource directly from the AI Agent tab in the ticket sidebar
Reporting an issue with AI Agent to the Gorgias’ Product team

Best practices for giving AI Agent feedback

Regularly review tickets → we recommend setting aside at least one hour per week to review AI Agent’s responses and give feedback on tickets — or more time if your team has bandwidth. Assign 1-2 team members as an “AI Coach”. The more you can improve AI Agent’s accuracy, the more time you can get back for other (revenue-generating) activities.
Identify knowledge gaps→ giving feedback to AI Agent is the perfect opportunity to find gaps in its knowledge. Empower your team to identify topics where AI Agent could benefit from new or updated information. Your Lead and Admin agents can create Help Center articles, Guidance and Actions directly from in-ticket feedback.
- Lead or Admin agents can also use the Test environment to see how AI Agent responds to specific questions from different types of customers.
Stay organized → it’s important to keep track of what tickets from AI Agent your team has or has not reviewed yet to give feedback. Consider creating a tag called reviewed that you can add to tickets once feedback has been given.
- You can also create a new View for AI Agent called To Review to identify tickets that have not yet been reviewed (the View should contain tickets with the tag ai_closed AND that do not contain the tag reviewed).

Understanding why AI Agent didn’t respond

There are a number of scenarios where AI Agent may choose not to respond to a ticket. In these cases, AI Agent adds an internal note to the ticket (visible only to you) that explains why it did not respond, and then either closes or automatically hands over the ticket to your team.

Here’s a list of common scenarios where AI Agent won’t send a response:

Reason	Description	Change in ticket status
Provider marked as spam	The message was marked as spam by your email provider	Closes ticket
Message is empty	The message received was empty and contained no text	Closes ticket
Too many messages	AI Agent receives a long thread of messages where it connect easily identify the questions or main needs of sender	Hands over ticket
Processing error	AI Agent encountered an error while processing the ticket	Hands over ticket
`ai_ignore` tag found	AI Agent didn’t respond because a Rule added the `ai_ignore` tag to the ticket	Hands over ticket
Angry customer or handover topics	AI Agent detected an angry sentiment from the customer or they mentioned a handover topic	Hands over ticket
No response required	The message did not require an answer from AI Agent (for example, customer writing to say “thank you”	Closes ticket

Messages that fail QA

Every message that AI Agent writes in response to a ticket goes through an internal quality assurance (QA) process before being delivered.

When the message AI Agent drafted fails the QA process, AI Agent adds a helpful tip to the ticket (visible only to you) that explains why it did not respond, and then automatically hands over the ticket to your team.

AI Agent’s internal note includes:

Why AI Agent didn’t send the message
A draft of the original message
The resources used to draft the message
Tags added to the ticket by AI Agent or Rules (editable by you)
The change in ticket status AI Agent would have set (editable by you)

You or a team member can review the message, then choose to select Copy to Editor to proceed with sending the AI Agent's response.

Reviewing a ticket in Gorgias where AI Agent drafted a respond to a customer, but didn't send the message because it wasn't confident it was correct. Selecting the Copy to Editor button to use AI Agent's drafted message.

Note: when AI Agent drafts a message that does not pass QA, you are not billed because the ticket was not fully resolved and does not count as an automated interaction.

How to coach AI Agent and give feedbackUpdated 15 days ago