Gorgias logo
Gorgias logo
All articles

How AI Agent improves over timeUpdated 8 hours ago

Who can use this feature?

 
All plans with AI Agent
 
Can be used with Shopify stores connected to Gorgias

AI Agent isn't something you set up once and walk away from. It's a system you work with over time, and the more deliberately you work with it, the better it gets at handling your shoppers' conversations.

The cycle that makes this possible has four steps: train AI Agent to handle the conversations you want, test the changes, deploy them, and analyze how they performed. Then you start the next loop.

Each loop in this cycle focuses on a specific behavior: usually one type of conversation AI Agent handles, like returns or order status. That makes improvements concrete: instead of trying to improve AI Agent all at once, you start with a few skills covering the conversations that matter most and expand AI Agent's coverage one loop at a time.

In this article, you'll learn:

  • Train — teach AI Agent how to handle conversations
  • Test — preview changes before they go live
  • Deploy — bring changes live to your shoppers
  • Analyze — see what's working and where to improve next

Train

Training is where you teach AI Agent how to handle conversations. When setting up for the first time, this means giving AI Agent knowledge about your business, your tone of voice, and a starting set of skills for your highest-volume, most predictable conversations — like order tracking, store hours, or return eligibility. See set up and go live with AI Agent for the full first-time workflow.

Once AI Agent is live, training shifts to sharpening existing knowledge and skills or building new ones, based on what you find in Analyze.

What ongoing training looks like in practice:

Some training changes only affect a specific type of conversation. Editing a Returns and exchanges skill to cover an edge case, for example, only changes how AI Agent handles returns and exchanges. Other changes shape every response AI Agent generates, like updates to your tone of voice.

Test

Before publishing a change, test how AI Agent responds. Test conversations let you send AI Agent any message and preview its response, so you can confirm a change lands the way you intend before it reaches your shoppers.

What testing looks like:

  • Open a test conversation from anywhere you're making changes, whether it's a skill, your knowledge, or another AI Agent setting.
  • Send a message that should trigger the scenario you want to test.
  • Review AI Agent's response. Confirm that it follows your instructions, uses the right knowledge, and sounds on-brand.
  • Use Show reasoning to see why AI Agent responded the way it did.

Testing isn't a one-time check before going live. Any meaningful change to AI Agent's setup warrants a test before you publish it.

For more, see preview AI Agent responses with test conversations.

Deploy

Deploy is the moment a change reaches your shoppers. It takes two forms. The first is turning AI Agent on for a channel (email, chat, SMS), typically a one-time event per channel. The second is deploying a change you've prepared in Train. Both make new behavior live for shoppers.

Common deployments include:

  • Publish a skill draft to replace its live version. AI Agent uses the new version for matching conversations going forward
  • Turn a skill on or off without affecting others
  • Restore a previous version of a skill, guidance, or help center article (each has its own version history)
  • Publish updates to knowledge or settings, which go live immediately

You deploy one improvement at a time, whether that's a sharper version of an existing skill or expanded coverage to a new conversation type. Watch how each change lands in Analyze before deciding what to improve next.

For first-time channel setup, see set up and go live with AI Agent.

Analyze

Analyze is where you find what to improve next, whether that's an existing skill that's underperforming or a conversation type AI Agent isn't covering yet. Performance signals tell you which conversation types are working and which need attention, both up close for a single skill or intent and across AI Agent as a whole.

Three places to look:

  • The Skills page. Each skill has its own performance signals: ticket volume, handover rate, and average CSAT. A high handover rate on a specific skill is a strong signal that something in that skill needs attention.
  • The Intents panel (in Skills > View intents). Shows every intent AI Agent is currently responding to, its ticket volume, handover rate, and the skill (if any) that AI Agent is currently using to handle it. Useful for spotting high-volume conversation types that aren't yet covered by a skill, and for finding which specific intent within a skill is causing most of its handovers.
  • The Overview and AI Agent reports. Show broader patterns: automation rate, CSAT, ticket coverage by topic, and trends over time.

For example, from the Skills page you might notice that your Returns and exchanges skill has a 30% handover rate. From there, you can look at the recent tickets that were handed over to see the specific cases your instructions don't currently cover. Your next loop has a clear goal: update the skill's instructions to cover those cases, test the change, and publish it.

For more on reading the reports, see review your AI Agent performance.

What you find in Analyze becomes the input for your next loop in Train. Each loop either sharpens how AI Agent handles a conversation type it already covers, or expands its coverage to one it doesn't. Over time, these small, focused changes add up to an AI Agent that handles more of your conversations, and handles them better.

Next steps

Was this article helpful?
Yes
No