Automate Reports 101Updated 25 days ago
- Automate Overview - focusing on high-level impact and performance
- Automate Performance by feature - delving into specific feature performance
Here's the logic of how data is displayed and the main terms used when explaining how the metrics are computed - Customer interaction and Automated interaction.
How data is displayed
All metrics in our report take into consideration the interaction's starting event, as opposed to the confirmation event - if you're looking at your January 1st metrics, you're looking at interactions that started on January 1st. However, if not more than 72 hours have passed, the data won't be shown.
A customer interaction is any exchange of messages that has occurred between you and your customer. The messages sent to your customers can come from a human (ticket handled by an agent) or a machine (Flow).
In other words, customer interactions reflect every touchpoint between your customer and your brand, no matter how it was responded to.
There are 2 main ways in which we determine that an interaction is automated:
1. A customer creates a ticket, it triggers a message from an autoresponder, and there's no message from the customer after 72 hours.
2. A customer uses one of our Automate features (Flows, Order Management, Article Recommendation) and it doesn't result in a ticket being handed over to an agent - if a customer starts a multi-step Flow, but doesn't reach the end (drop-off), this won't be considered an automated interaction.
Performance by feature
This report is currently based on our legacy data source which is why summing up data from the Performance by feature section won't match the metrics in the Overview report.
How data is displayed
There's no 72-hour hold-up with this report - as soon as the appropriate interaction takes place, it'll be reflected in relevant metrics.
The automation rate for each feature takes into account only interactions started through that feature. For example, the automation rate of a “What size should I pick” Flow will be 50% if half of the customers that started the Flow didn't ask for more help, regardless of the total number of overall customer interactions.
The number of interactions is not de-duplicated as it is in the Overview report. This means that if the same shopper starts the same Flow multiple times within the same day, each of those interactions will be counted in metrics that reflect it.
Do tickets resolved by a Rule count as automated interactions?
What's the difference between the Overview report and the Performance by feature report?
If I add in the totals of the Performance by feature report, they don't match the numbers in the Overview report. Why is that?
Why is my automation rate lower compared to the previous Automate report?
Here’s how the calculation has changed:
- Previous rate: (# of automated interactions / # of Automate interactions) * 100
- New rate: (# of automated interactions / # of total customer interactions) * 100
As a result of this change, it’s likely your automation rate is now lower compared to the previous report.