Coach AI Agent and give feedback to improve its responsesUpdated a day ago

You can coach AI Agent by giving feedback on the tickets it answers. Your feedback helps AI Agent select the right knowledge sources to answer customer inquiries, and improves its ability to respond to similar questions in the future.

Giving feedback also helps you troubleshoot AI Agent’s responses. By reviewing AI Agent’s reasoning for the answers it gave in a ticket, you can better understand its responses and add or correct knowledge as needed.

Requirements

• You must have an active AI Agent subscription.
• You must have Lead or Admin permissions to give AI Agent feedback. Any role (Basic and above) can read AI Agent’s reasoning.

Giving feedback on tickets

You can give AI Agent feedback on any conversation it has responded to. Click To Review in the AI Agent section of your ticket views to see tickets that are ready for feedback.

1. From the main menu, go to Tickets.
2. Scroll down to your shared views for AI Agent, then select To Review.
3. Select a ticket to review.
4. Click on Give Feedback next to a message sent by AI Agent in the conversation. This will open the AI Feedback tab in the ticket sidebar.
5. After giving feedback, select Review next ticket to move onto the next ticket within the current view.

Optional: rate conversation quality

You can rate AI Agent’s conversation to provide Gorgias with valuable data to monitor conversation quality and inform product improvements.

1. Open a ticket handled by AI Agent, then click on AI Feedback
2. Under How was this conversation, rate the conversation as Bad, Okay or Good.
   
   
   • If you select Bad or Okay, select an option from the What went wrong dropdown menu
3. Add your reason for the rating under Additional feedback. For example, you could say “should not have handed over” or “should not have offered discount”.

Review knowledge sources used

Under Review sources used in the AI Feedback tab, you can rate each knowledge source that AI Agent used to create its response.

Rate each source based on its relevance to the customer’s question or inquiry. AI Agent will be more likely to use a source with a thumbs up 👍 for similar questions in the future, and less likely to use a source with a thumbs down 👎.

Knowledge sources you’ll find here include:

• Guidance
• Help Center articles
• Store website
• Products from your Shopify store
• URLs
• Documents
• Actions

Suggest a different knowledge source

If there is a knowledge source that AI Agent should have used to respond to the ticket but didn’t, you can select the source from the Was relevant knowledge missing? dropdown menu. Doing this coaches AI Agent to retrieve the source for similar questions in the future.

If you notice that you don’t already have a knowledge source that answers the customer’s question, you can select Create Knowledge to address the missing topic for AI Agent.

1. From the AI Feedback tab, select Create Knowledge.
2. Choose either Create Guidance or Create Help Center article.
3. Use the pop-out text editor to write Guidance or a Help Center article.
4. Before publishing, select the Use in similar requests checkbox to coach AI Agent to use your new knowledge for similar tickets in the future.

Editing existing knowledge sources

You can select and edit knowledge sources from the AI Feedback tab or from AI Agent’s reasoning summary to address gaps in its knowledge. Editing knowledge to add or correct information helps AI Agent respond to customer requests more successfully.

1. Open a ticket handled by AI Agent
2. Click on AI Feedback, then select the name of a knowledge source to open it in a new tab.
   
   
   • Alternatively, select Show reasoning beneath a message from AI Agent in the conversation thread, then click on the icon of a knowledge source to open it in a new tab.
3. For knowledge sources like Guidance and Help Center articles, you can view and edit knowledge directly from the ticket. Select the pencil icon to make and save changes.

Using AI Reasoning to troubleshoot responses

To troubleshoot how AI Agent responded to a ticket, start by reviewing its reasoning. Reasoning is a detailed summary of AI Agent’s decision-making process. It explains how AI Agent came to its conclusions and shows the specific knowledge sources that it used to create its response.

By reviewing reasoning, you can better understand AI Agent’s responses and investigate why it took specific actions, like handing over a ticket to a human agent. You can then adjust AI Agent’s settings or create and edit knowledge to change its behavior.

To find AI Agent’s reasoning:

1. Open a ticket handled by AI Agent.
2. Click on Show reasoning beneath any of AI Agent’s messages to see reasoning details.
3. Hover over a knowledge icon within the reasoning summary to see a preview of the knowledge sources AI Agent used.
4. Click on a knowledge icon to open the knowledge source. Edit the knowledge to correct information, or create new knowledge to address missing topics as needed.

Rate AI Agent’s reasoning

When you select Show reasoning beneath a response from AI Agent, you can rate the quality of the summary given to you about AI Agent’s decision-making process.

Your rating should focus on how well the reasoning summary explains AI Agent’s response. If you want to give feedback about AI Agent’s handling of the ticket, you can rate the conversation quality in the AI Feedback tab.

1. Click on Show reasoning beneath any of AI Agent’s messages to see a summary of its decision-making
2. Select either the thumbs up 👍 or 👎 down icons to say how well the summary explains AI Agent’s response. If AI Agent’s response was not well explained, choose one or more options under What’s wrong with this explanation to say why.