Coach AI Agent and give feedbackUpdated 18 days ago

You can coach AI Agent by giving it feedback on the tickets it answers. Your feedback tells AI Agent which knowledge sources it correctly (or incorrectly) used to respond to a customer’s request. By rating its use of knowledge, AI Agent learns from previous tickets and can better identify which sources it should use to address similar topics in the future.

Reviewing tickets regularly also allows you to notice where AI Agent’s knowledge should be strengthened. You can then create new knowledge to increase AI Agent’s coverage and its ability to tackle common ticket topics going forward.

Requirements

• You must have an active AI Agent subscription
• Any role (Observer, Lite, Basic, Lead, Admin or Account Owner) can give AI Agent feedback. Only Leads, Admins and Account Owners can create new knowledge from AI Feedback.

Giving AI Agent feedback

You can give AI Agent feedback on any conversation where it has provided a response. Typically this means any ticket that AI Agent has closed or handed over to a human agent.

Alternatively, you can find tickets ready for review in the AI Agent section, under the view named To Review. This view is filtered to include tickets that were either handed over or closed by AI Agent and haven’t received feedback.

1. Go to Tickets from the main menu.
2. Scroll down to your shared views for AI Agent, then select To Review.
3. Select a ticket to review.
4. Click on Give Feedback next to a message sent by AI Agent in the conversation. This will open the AI Feedback tab in the ticket sidebar.

Rate the quality of AI Agent’s conversation

Under “How was this conversation?” in the AI Feedback tab, you can rate AI Agent’s conversation as Bad, Okay or Good. 

Information from this survey is used by Gorgias to monitor conversation quality and inform improvements. Your quality rating does not directly affect AI Agent’s behavior in real time.

• If you select Bad or Okay, open and select an option from the What went wrong? dropdown menu
• Optionally, no matter which rating you choose, enter more details about your choice in the Additional feedback textbox — for example, “should not have handed over” or “should not have offered discount code”.

Review knowledge sources used

Under “Review sources used” in the AI Feedback tab, you can rate the knowledge sources that AI Agent used to generate its response.

Your input in this section directly influences which knowledge sources that AI Agent uses to respond to similar questions in the future. Therefore, you should rate each source based on its relevance to the question that was asked, rather than the quality of the response (use “How was this conversation?” to give feedback on conversation quality).

AI Agent will be more likely to use a source with a thumbs up 👍 for similar questions in the future, and less likely to use a source with a thumbs down 👎.

Knowledge sources you’ll find here include:

• Guidance
• Store website
• Products from your Shopify store
• Help Center articles
• URLs
• Documents

Suggest a different knowledge source

If there is a knowledge source that AI Agent should have used to respond to the ticket but didn’t, you can select the source from the Was relevant knowledge missing? dropdown menu. This way, AI Agent knows to retrieve the source for similar questions in the future.

Alternatively, you can select Create Knowledge to create new Guidance or a Help Center article directly from the ticket. This is useful if you notice AI Agent was unable to respond to a ticket because it lacked knowledge to address the customer’s question.

1. From the AI Feedback tab, select Create Knowledge
2. Choose either Create Guidance or Create Help Center article
3. Use the pop-out text editor to write Guidance or a Help Center article
4. Before publishing, select the Use in similar requests checkbox to tell AI Agent to use your new knowledge for similar tickets in the future

Reviewing multiple AI Agent tickets

To give feedback on multiple, consecutive AI Agent tickets, you can use the Review next ticket button at the bottom of the AI Feedback tab.

Review next ticket takes you directly to the next chronological ticket within the same ticket view.