Gorgias logo
Gorgias logo
All articles

Test how AI Agent responds to customer questionsUpdated 9 days ago

Once you’ve set up AI Agent with knowledge from your Help Center or website, created Guidance and enabled Actions, you can use the test environment to try out AI Agent.

The test environment is a controlled space where you can simulate real interactions — including asking questions about orders and imitating Actions — to build trust with AI Agent, refine your settings, and address any gaps in AI Agent’s knowledge.

Requirements

  • You must have set up AI Agent
  • You must have Lead, Admin or Account Owner permissions

What should I test for?

Use the test environment to gain trust with AI Agent. Testing allows you to identify any configuration settings you want to change, or address gaps in AI Agent’s knowledge.

If you’re not sure what to ask, use the common question templates in the test environment to get started.

When you ask AI Agent a test question, consider the following to evaluate its response:

  • Did AI Agent use the right Help Center articles?
  • Did AI Agent follow the correct Guidance and excluded topics?
  • Did AI Agent use the correct tone of voice?
  • Does the answer AI Agent provided make sense? If not, check that its knowledge sources, like Help Center articles and Guidance, are up to date.

Ask AI Agent a test question

Go to your settings for AI Agent to open the test environment. When you use the test environment no actual messages are sent outbound, no data is changed, and AI Agent does not perform any real Actions in connected 3rd-party apps.

  1. Go to AI Agent from the main menu, then select Test
    • If you have more than 1 Shopify store connected to Gorgias, use the dropdown menu to select a store
  2. Enter a question your customers may ask AI Agent
    • Use the Email or Chat options to toggle between channels
    • Use the dropdown menu to test AI Agent's response using an Existing Customer, New Customer or an Existing Ticket
  3. Select Submit to get a response from AI Agent
  4. Use the Reset button to start a new test conversation

In your AI Agent settings, selecting the Test option from the menu. Then, on the Test page entering a question in the text box to evaluate how AI Agent responds

Review AI Agent's response

With each response you receive from AI Agent, evaluate whether the answer met your expectations, and if not, identify opportunities to improve AI Agent's settings or its sources of knowledge, like your Help Center.

  • With each response, AI Agent includes an internal note that explains why it gave the answer it did, plus a link to the source of knowledge that it used
  • Didn’t get the response you expected? Try to identify gaps or improvements in the sources of knowledge that AI Agent uses. For example, you may need to create a new Help Center article, add a new handover topic or adjust your current Guidance
  • Give us feedback. Use the How did it do? survey to provide feedback to the Gorgias team. We use this information to make AI Agent's better

In the AI Agent test environment, looking at the response that AI Agent provided to your test question. Using the answer to evaluate what improvements you can make to AI Agent's settings or knowledge.

Was this article helpful?
Yes
No