Test how AI Agent responds to customer questionsUpdated 4 months ago
Once you’ve set up AI Agent with knowledge from your Help Center or website, created Guidance and enabled Actions, you can use the test environment to try out AI Agent.
The test environment is a controlled space where you can simulate real interactions — including asking questions about orders and imitating Actions — to build trust with AI Agent, refine your settings, and address any gaps in AI Agent’s knowledge.
Requirements
- You must have set up AI Agent
- You must have Lead or Admin permissions
What should I test for?
Use the test environment to gain trust with AI Agent. Testing allows you to identify any configuration settings you want to change, or address gaps in AI Agent’s knowledge.
If you’re not sure what to ask, use the common question templates in the test environment to get started.
When you ask AI Agent a test question, consider the following to evaluate its response:
- Did AI Agent use the right Help Center articles?
- Did AI Agent follow the correct Guidance and excluded topics?
- Did AI Agent use the correct tone of voice?
- Does the answer AI Agent provided make sense? If not, check that its knowledge sources, like Help Center articles and Guidance, are up to date.
Ask AI Agent a test question
Go to your settings for AI Agent to open the test environment. When you use the test environment no actual messages are sent outbound, no data is changed, and AI Agent does not perform any real Actions in connected 3rd-party apps.
- Go to Automate
- Select your store from the sidebar, then click on AI Agent
- Select the Test tab
- Use the dropdown menu to choose who AI Agent is responding to:
- Existing customer: use this option to select the email address of a customer who has previously interacted with your store and may have prior order information
- New customer: who this test an inquiry from customer who has no prior order information
- Use the toggle to switch between Email or Chat
- Enter a question into the text box, then select Submit to get a response from AI Agent
Review AI Agent's response
With each response you receive from AI Agent, evaluate whether the answer met your expectations, and if not, identify opportunities to improve AI Agent's settings or its sources of knowledge, like your Help Center.
- With each response, AI Agent includes an internal note that explains why it gave the answer it did, plus a link to the source of knowledge that it used
- Didn’t get the response you expected? Try to identify gaps or improvements in the sources of knowledge that AI Agent uses. For example, you may need to create a new Help Center article, add a new handover topic or adjust your current Guidance
- Give us feedback. Use the How did it do? survey to provide feedback to the Gorgias team. We use this information to make AI Agent's better
Reset the test environment
When you’re ready to ask AI Agent a completely new question, select New Conversation to reset the test environment. You can then change your audience (Existing customer or New customer) or your channel (Chat or Email), then enter a new question text box.