Preview AI Agent responses with test conversationsUpdated 2 hours ago

Who can use this feature?

Owner, Admin and Lead roles

All plans with the AI Agent

Can be used with Shopify stores connected to Gorgias

Test conversations let you preview how AI Agent would respond to real customer questions and scenarios. You can use test conversations to ask AI Agent questions, evaluate its reasoning, and confirm that your content, actions, tone of voice and other settings are being used as expected.

Test conversations are free to use (they don't contribute to your ticket usage), and won't impact your reporting or analytics.

Start a test conversation with AI Agent

You can start a test conversation from either the Test page or from the Test panel on any page where you’re making changes to AI Agent.

In both places, you can simulate end-to-end customer conversations. The Test page is a focused, full-page conversation with AI Agent. The Test panel lets you start a conversation alongside your settings for AI Agent so you can see how changes to your setup affect its responses, without leaving the current page.

From the main menu, click AI Agent.
Click Test to start a conversation with AI Agent.
Enter a message to ask AI Agent a question, then click Send to get a response.
When finished, click Reset to end the current test conversation and start a new one.

Tip: Start a test conversation whenever you want to confirm that AI Agent is working correctly or to see how it responds, like before going live on a new channel or to test a draft of your guidance or help center article before you publish it.

Configure your test channel and audience

You can customize your test conversation to preview how AI Agent responds in specific scenarios, like whether the conversation happens on chat or email, with a new customer or with someone who has an existing customer profile and order history.

From the main menu, go to AI Agent.
Click Test to start a new conversation.
Click Configure.
Open the dropdown menus to change the channel and target audience of your test.

The table below provides a breakdown of the different configuration options for a test conversation:

Setting	Options
Channel	Choose a channel for your test conversation with AI Agent, including Email, Chat or SMS. If the channel for your test conversation is Chat, choose whether the Availability is set to Online or Offline. This lets you test settings like handover instructions for when your agents are available or unavailable to respond to customers.
Target	Choose a target audience. This option lets you test how AI Agent responds to: A new customer. Best for testing a conversation as someone who AI Agent doesn’t recognize as having an existing customer profile in Shopify or Gorgias. An existing customer. Best for testing a conversation with a real customer profile and Shopify order data. An existing ticket. Best for re-simulating a conversation where AI Agent did not initially respond as expected, but after you’ve made changes to its knowledge or settings.

Setting

Options

Channel

Choose a channel for your test conversation with AI Agent, including Email, Chat or SMS.

If the channel for your test conversation is Chat, choose whether the Availability is set to Online or Offline. This lets you test settings like handover instructions for when your agents are available or unavailable to respond to customers.

Target

Choose a target audience. This option lets you test how AI Agent responds to:

A new customer. Best for testing a conversation as someone who AI Agent doesn’t recognize as having an existing customer profile in Shopify or Gorgias.
An existing customer. Best for testing a conversation with a real customer profile and Shopify order data.
An existing ticket. Best for re-simulating a conversation where AI Agent did not initially respond as expected, but after you’ve made changes to its knowledge or settings.

Allow support actions in test conversations

By default, AI Agent does not use actions as part of your test conversations.

If you want to start a test conversation where AI Agent uses actions (for example, to test the cancellation of an order or change your shipping address), you can configure the test to use actions.

From the main menu, go to AI Agent, then click Test.
Click Configure.
Click the toggle to enable Actions.

Warning: When AI Agent uses an action in a test conversation, it can affect real customer and order data (like cancelling an order). To be safe, configure the conversation to target a fake customer profile.

Test drafts before publishing

Whenever you create new guidance, create a new help center article, or make edits to existing knowledge without publishing the changes, the content will be saved as a draft.

A draft is created whenever you create new guidance or a help center article, or make edits to an existing one, without publishing your changes.

You can test drafts to see how your changes affect AI Agent's responses before you publish them. When you test a draft, AI Agent continues to use all live settings and enabled knowledge to form its responses, but substitutes the published version of your guidance or article for the current draft.

From the main menu, go to AI Agent, then click Knowledge.
Select an existing guidance or help center article with a draft version to open it. You can also click Create content to start a new draft.
While the draft guidance or article is open, click Test in the top right-hand corner.
Enter a message to ask AI Agent a question and preview its response.

FAQs

Do test conversations affect real customers?

No. Test conversations are sandboxed and never reach customers. They don’t create tickets or send real messages. Your test conversation may only affect real customer or order data if you enable actions as part of the test.

What knowledge does AI Agent use in a test conversation?

By default, AI Agent uses all current live settings and enabled knowledge to form its responses in test conversations. If you start a test conversation from a draft guidance or help center article, AI Agent will use the draft instead of the published version.

Can I test knowledge that isn’t published or enabled yet?

Yes. You can test draft help center articles and guidance before publishing them. You can test only one draft at a time.

You can also test any knowledge that is published, but isn’t currently enabled for AI Agent to help make decisions about what content performs best.

Can I see why AI Agent responded a certain way?

Yes. Test conversations show AI Agent’s reasoning for each response, the same way you would see the AI’s reasoning in real conversation. Reasoning helps you understand how AI Agent made its decisions, and it shows the knowledge AI Agent used to create its response so you can review and modify your content, if needed.

Was this article helpful?

Yes