Coach AI Agent and give feedback to improve its responsesUpdated a day ago

You can coach AI Agent by giving feedback on the tickets it answers. Your feedback helps AI Agent select the right skills and knowledge to answer customer inquiries, and improves its ability to respond to similar questions in the future.

Giving feedback also helps you troubleshoot AI Agent’s responses. By reviewing AI Agent’s reasoning, you can better understand why it responded or acted the way it did, and add or improve your skills or knowledge when needed.

Requirements

• You must have an active AI Agent subscription.
• You must have Lead or Admin permissions to give AI Agent feedback or ask Gaia about a ticket. Any role (Basic and above) can read AI Agent’s reasoning.

Giving feedback on tickets

You can give feedback on any closed ticket that AI Agent has worked on. Go to To review in your AI Agent ticket view to see tickets that are ready for feedback.

1. Click the dropdown menu in the top-left corner, then select  Inbox.
2. Under your shared views, select AI Agent, then click To review.
3. Click on a ticket to review.
4. Click  AI Feedback in the ticket sidebar.
5. Give feedback on the ticket by rating the conversation and reviewing the sources AI Agent used. Your feedback saves automatically.
6. When finished, select Review next ticket to go to the next ticket in the view.

Optional: rate conversation quality

Rating the quality of an AI Agent ticket provides Gorgias with valuable data to monitor conversation quality and inform product improvements.

1. Open a ticket handled by AI Agent, then select  AI Feedback in the ticket sidebar.
2. Under How was this conversation?, rate the conversation as Bad, Okay, or Good.
3. If you select Bad or Okay, select one or more reasons from the What went wrong? dropdown menu.
4. Under Share additional feedback, add more detail. For example, you could say “should not have handed over” or “should not have offered discount”.

Rate the skills and knowledge AI Agent used

Under Sources used in  AI Feedback, you can rate each skill and knowledge source that AI Agent used in its response. Skills appear first, because they hold the main instructions that AI Agent follows. If a conversation covered more than one request, you might see more than one skill.

Select the thumbs up 👍 or thumbs down 👎 buttons beside a source to rate how relevant it was to the ticket.

When you rate a skill, Gorgias uses your feedback to monitor and improve AI Agent quality over time. When you rate knowledge, you are coaching AI Agent to prioritize or not deprioritize a source for similar tickets in the future.

Sources you’ll find here include:

• Skills
• Guidance
• Actions
• Help Center articles
• Store website
• Products from your Shopify store
• Orders
• URLs
• Documents

Suggest a different source

If there is a source that AI Agent should have used to respond to the ticket but didn’t, you can add it under Was something missing? Doing this coaches AI Agent to retrieve the source for similar questions in the future.

1. From  AI Feedback, select Add source under Was something missing?
2. Select a category, such as Skills, Guidance, or Help Center articles, then select the source. You can also search by name.
3. To remove a source you added, select the x on its label.

If you notice that you don’t already have a source that answers the customer’s question, select Add source > Create new. This will start a chat with Gaia where you can draft and work on a new skill, guidance, or Help Center article based on the ticket.

Edit existing sources

To help AI Agent respond more successfully in future conversations, you can edit any of the sources that it used directly from the ticket, including skills, guidance, and Help Center articles.

1. Open a ticket handled by AI Agent.
2. Select  AI Feedback in the ticket sidebar, or select Show reasoning   beneath one of AI Agent’s messages in the ticket thread.
3. Select the name of a source under Sources used, or select a source icon in the reasoning summary. This will open the source in an editor next to the ticket.
4. Select  Edit to make changes to the source, then select Publish.

Troubleshoot AI Agent messages

With each message that AI Agent sends, you can review a summary of its reasoning to better understand why it responded in a particular way and why it took specific actions, like closing a ticket or handing the conversation over to your team.

By reviewing reasoning, you can investigate what happened in a ticket and identify the specific skills, knowledge, or actions that determined its behavior. You can then confidently make changes to your AI Agent settings or create and edit sources to help it respond more consistently.

To find AI Agent’s reasoning:

1. Open a ticket handled by AI Agent.
2. Select Show reasoning   beneath any of AI Agent’s messages to see reasoning details.
3. Hover over a source icon to see the type of source AI Agent used.
4. Select a source to open it from the ticket. For skills, guidance, and Help Center articles, you can edit it without leaving the page.

Rate AI Agent’s reasoning

When you select Show reasoning   beneath an AI Agent message, you can rate the quality of the summary given to you about AI Agent’s decision-making steps.

Your rating should focus on how well the reasoning summary explains AI Agent’s response. If you want to give feedback about AI Agent’s handling of the ticket, you can rate the conversation quality in  AI Feedback.

Use the thumbs up 👍 or thumbs down 👎 buttons within reasoning to rate the summary. If AI Agent’s response was not well explained, select one or more options under What’s wrong with this explanation? to say why.

Investigate a ticket with Gaia

If reasoning doesn’t fully explain what happened, you can ask Gaia about the ticket. Gaia reads the conversation and AI Agent’s reasoning, explains what happened, and recommends ways you can improve your setup.

1. Open a ticket handled by AI Agent.
2. Select Show reasoning   beneath any of AI Agent’s messages to see reasoning details.
3. Select Ask Gaia. A chat with Gaia will open with the current ticket attached.
4. Ask Gaia about the ticket (for example, “explain the handover,” “help AI Agent resolve tickets like this,” or “review response accuracy”).