This article will walk you through the various sections and capabilities of the Analytics Dashboard in the Intelligems App. After reading this, you should feel comfortable reading your test's results and making a data-driven decision!
In this article:
Getting Started
There are a few capabilities that remain consistent between each tab, including:
- Exports: You have the ability to export test data on each page using the button in the top right corner - check out this article for more information on that.
- Filters: Each page has a 'Filters' option in the top right corner where you can adjust the:
- Date Range: Select a start and/or end time. We typically recommend filtering out the first day or so of traffic after starting a test as those users may have been on the site more recently and impact results.
- Analytics View Type: Set whether you'd like to include only orders that contain test products (Test Orders Only), or include all orders placed by visitors exposed to the test, regardless of whether the order contained test products (Sitewide Orders). Note this setting is only visible for Price Tests as this is not relevant to Shipping Tests.
- Z-Score: This is important for removing outliers from the analytics - it is set to 3 by default and you can learn more here.
- Test you are Viewing: You can toggle between all tests with analytics using the dropdown under 'Analytics Dashboard'.
- Stop Test or Edit Groups: For any live test, you can stop the test or edit the test groups directly from the analytics dashboard using the buttons near the top left.
Now let's get into what we're all here for - reading the analytics on each page and understanding what they mean for your store!
✨ We are always working to improve our analytics - please reach out to support@intelligems.io if there are specific metrics you are looking for that we do not currently have in the app, and we will do our best to work with you on those!
Overview
The Overview tab is where you will land by default when you click into Analytics, and is a great place to start when analyzing the results of a test - it will give you a great high-level view of some of the most important data to consider. This page includes bar charts for the following metrics, where you will see a bar for each test group:
Conversion Rate: Percent of unique site visitors that placed an order through the online store. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in conversion rate) x (total number of visitors in the test) x (30 / days the test has run). For example, in the image on the right, the '10% Lower' group results in a 49.3% lift in conversion rate when compared to the 'Control' group and an estimated additional 29 orders monthly.
Revenue per Site Visitor: Total revenue divided by number of unique site visitors. Revenue includes product and shipping revenue, net of discounts. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in revenue per site visitor) x (total number of visitors in the test) x (30 / days the test has run). For example, in the image on the left, the '$50 Thresh' group results in a 42.4% lift in revenue per site visitor when compared to the '$25 Thresh' group and an estimated additional $2,350 in revenue monthly.
Profit per Site Visitor: Total profit divided by number of unique site visitors. Profit includes product and shipping revenue, net of discounts, COGS and cost of shipping. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in profit per site visitor) x (total number of visitors in the test) x (30 / days the test has run). In certain cases, this impact could also result in 'No Change', as seen in the image on the right.
❗We will need a complete list of your store's Cost of Goods Sold to provide Profit per Site Visitor and other Profit analytics - please see more on how to enable this here!
Visitors: Number of unique site visitors included in the test in each test group. Note this is visitors to the site as a whole, not visitors to test-specific PDPs. Reminder: Visitors are randomly assigned based on the percentages you set while setting up your test. So, if you set your two groups to 50% and 50%, we would expect to see a relatively even number of visitors in each group.
Orders: Number of orders placed on the site from visitors in each test group.
Finally, this page includes a table breaking down the visitors, orders, CVR (conversion rate), total revenue, and revenue per visitor for each test group.
Revenue
This tab focuses entirely on revenue based metrics. If top-line growth is your store's top goal, this may be where you want to spend the most time.
Revenue per Site Visitor: Total revenue divided by number of unique site visitors. Revenue includes product and shipping revenue, net of discounts. Note this is also visible in the 'Overview' tab.
AOV Breakdown: Average order value, split by revenue generated from product sales (Product Revenue Per Order) and shipping charges (Shipping Revenue Per Order). Scroll over each bar to see these breakdowns.
Average Units per Order: Average quantity of product units contained in each order.
Average Product Revenue per Unit: Average selling price for a single unit. Equals average product revenue per order divided by average units per order.
Profit
❗This tab and these reports will only show up in your app if you have provided a complete list of your store's Cost of Goods Sold - please see more on how to enable this here!
Similar to the above, this tab focuses entirely on profit based metrics. You will want to focus on this tab if bottom-line growth is most important to your store.
Profit Analysis: Summary of revenue, costs and profit, including both product and shipping contribution. Metrics are shown in total and normalized per order and per visitor at the bottom of the table.
Gross Margin: Total profit divided by total revenue.
Profit per Site Visitor: Total profit divided by number of unique site visitors. Profit includes product and shipping revenue, net of discounts, COGS and cost of shipping. Note this is also visible in the 'Overview' tab.
Shipping
✨ As the name suggests, this tab is largely targeted at Shipping Tests! That said, a Pricing Test can still impact your shipping revenue (i.e. higher price point may result in higher AOV which may result in a larger percent of orders having free shipping) so we recommend still checking the results in this tab out for a Pricing Test.
Shipping Revenue per Order: Average revenue per order generated by shipping charges.
Percent of Orders with Free Shipping: Percentage of all orders that had no associated shipping revenue.
Orders by Shipping Method: Total orders for each shipping method. Use the dropdown in the top right corner of this graph to adjust which shipping method you are viewing the data for.
Average Shipping Revenue by Method: Average revenue per order generated by shipping charge for each shipping method. Use the dropdown in the top right corner of this graph to adjust which shipping method you are viewing the data for.
Visitors
View over Time: Number of visitors included in the test, by day by test group group. If a user returns to the site over multiple days, only their first visit is reflected in this graph.
Conversion Funnel: Percentage of visitors that Add to Cart, Begin Checkout and Convert for each group.
Visitors: Number of unique site visitors included in the test in each test group. Note this is also visible in the 'Overview' tab.
Orders Detail
In this tab, you will be able to see the results on a number of orders basis, rather than on a dollar basis.
Orders over Time: Number of orders placed on the site, by day by test group.
Order Value Histogram: Distribution of orders by order value (in increments of $10). You can filter this to show all orders, or view by test group, and it is a great visual on whether there are any major outliers.
% Orders by Unit Quantity: Breakdown of orders by the quantity of units contained in each order.
% Net Revenue by Order Unity Quantity: Breakdown of net revenue by the quantity of units contained in each order.
Unit Quantity Histogram: Distribution of orders by quantity of units contained. You can filter this to show all orders, or view by test group.
New vs. Returning Customers: Number of orders, percent of orders and average order value (including both product sales and shipping charges) broken down by new versus returning customers for each test group.
Discounts
Orders/Percent Total Orders: Number of orders with discount applied and percentage of all orders in test group, subject to other filters.
Average Discount per Order: Average discount per order (includes only orders with applicable discount).
% of Gross Revenue (Pre-Discount): Discount total as a percentage of total gross revenue (pre-discount) for orders containing discounts (‘Discounted Orders’) and all orders (‘All Orders’), subject to other filters.
% of Gross Profit (Pre-Discount): Discount total as a percentage of total gross profit (pre-discount) for orders containing discounts (‘Discounted Orders’) and all orders (‘All Orders’), subject to other filters.
Discount Histogram: Distribution of orders by total discount, in increments of $10. You can filter this to show all orders, or view by test group.
Product Groups
Product Groups are a great way to gain an understanding of how different product categories performed in a test in comparison to each other. Because we set up your Product Group information, you can group products in any way that you desire.
❗Please reach out to support@intelligems.io to set up your Product Group information! If we have not received Product Group information from you, the only Product Group that will show up is 'None'.
Revenue by Product Group: Breakdown of product revenue by product group for each test group. Revenue is net of discounts and excludes shipping.
Units by Product Group: Breakdown of units sold by product group for each test group.
Data by Product Group: The bottom eight charts allow you to view revenue per visitor, profit per visitor, units sold, percentage of total units, revenue per unit, profit per unit, average units ordered and average units per visitor by product group using the dropdown in the top right corner.