This article will walk you through the various sections and capabilities of the Analytics Dashboard in the Intelligems App. After reading this, you should feel comfortable reading your test's results and making a data-driven decision!
In this article:
Getting Started
Access analytics by first finding the test you want to analyze and clicking the analytics icon or navigate to the 'Analytics' page of sidebar menu. Use the dropdown under "Analytics Dashboard" to toggle between tests.
Now let's get into what we're all here for - reading the analytics on each page and understanding what they mean for your store!
✨ We are always working to improve our analytics - please reach out to support@intelligems.io if there are specific metrics you are looking for that we do not currently have in the app, and we will do our best to work with you on those!
Filters
Each page has a 'Filters' menu in the top right corner where you can adjust the:
- Date Range: Select a start and/or end time. We typically recommend filtering out the first day or so of traffic after starting a test as those users may have been on the site more recently and impact results.
- Analytics View Type: Set whether you'd like to include only orders that contain test products (Test Orders Only), or include all orders placed by visitors exposed to the test, regardless of whether the order contained test products (Sitewide Orders). Note this setting is only visible for Price Tests.
- Z-Score: This is important for removing outliers from the analytics - it is set to 3 by default. Learn more here.
Exports
Want to run additional analyses or see the individual orders / products that feed the dashboard? Use our exports buttons to download the following information:
- Order Detail: Information about each order, including order value, profit, customer ID, shipping method and many more metrics.
- Order Line Item Detail: Information about each product ordered during the test, including the produt / variant IDs, associated order, revenue and profit, among others
See here for more information on our order exports.
Analytics by Tab
Analytics in the dashboard are broken down into ten tabs. See below for a breakdown of which metrics are included in each section.
Overview
The Overview tab is where you will land by default when you click into Analytics, and is a great place to start when analyzing the results of a test - it will give you a high-level view of some of the most important data to consider. This page includes bar charts for the following metrics, where you will see a bar for each test group:
| Conversion Rate: Percent of unique site visitors that placed an order through the online store. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in conversion rate) x (total number of visitors in the test) x (30 / days the test has run). For example, in the image on the right, the '10% Lower' group results in a 49.3% lift in conversion rate when compared to the 'Control' group and an estimated additional 29 orders monthly. |  |
 |
Revenue per Site Visitor: Total revenue divided by number of unique site visitors. Revenue includes product and shipping revenue, net of discounts. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in revenue per site visitor) x (total number of visitors in the test) x (30 / days the test has run). For example, in the image on the left, the '$50 Thresh' group results in a 42.4% lift in revenue per site visitor when compared to the '$25 Thresh' group and an estimated additional $2,350 in revenue monthly. |
| Profit per Site Visitor: Total profit divided by number of unique site visitors. Profit includes product and shipping revenue, net of discounts, COGS and cost of shipping. In addition to the bar graph, this metric includes the positive or negative impact that the winning or losing test group could have, and an estimate of the monthly impact. The estimate here is calculated by (change in profit per site visitor) x (total number of visitors in the test) x (30 / days the test has run). In certain cases, this impact could also result in 'No Change', as seen in the image on the right. |  |
❗We will need a complete list of your store's Cost of Goods Sold to provide Profit per Site Visitor and other Profit analytics - please see more on how to enable this here!
AOV Breakdown: Average order value, split by revenue generated from product sales (Product Revenue per Order) and shipping charges (Shipping Revenue per Order).
Visitors: Number of unique site visitors included in the test in each test group. Note this is visitors to the site as a whole, not visitors to test-specific PDPs. Reminder: Visitors are randomly assigned based on the percentages you set while setting up your test. So, if you set your two groups to 50% and 50%, we would expect to see a relatively even number of visitors in each group.
Orders: Number of orders placed on the site from visitors in each test group.
Finally, this page includes a table breaking down the visitors, orders, CVR (conversion rate), total revenue, and revenue per visitor for each test group.
Revenue
This tab focuses entirely on revenue-based metrics. If top-line growth is your store's top goal, this may be where you want to spend the most time.
Revenue per Site Visitor: Total revenue divided by number of unique site visitors. Revenue includes product and shipping revenue, net of discounts. Note this is also visible in the 'Overview' tab.
AOV Breakdown: Average order value, split by revenue generated from product sales (Product Revenue Per Order) and shipping charges (Shipping Revenue Per Order). Scroll over each bar to see these breakdowns.
Average Units per Order: Average quantity of product units contained in each order.
Average Product Revenue per Unit: Average selling price for a single unit. Equals average product revenue per order divided by average units per order.
Profit
❗This tab and these reports will only show up in your app if you have provided a complete list of your store's Cost of Goods Sold - please see more on how to enable this here!
This tab focuses entirely on profit-based metrics. You will want to focus on this tab if bottom-line growth is most important to your store.
Profit Analysis: Summary of revenue, costs and profit, including both product and shipping contribution. Metrics are shown in total and normalized per order and per visitor at the bottom of the table.
Gross Margin: Total profit divided by total revenue.
Profit per Site Visitor: Total profit divided by number of unique site visitors. Profit includes product and shipping revenue, net of discounts, COGS and cost of shipping. Note this is also visible in the 'Overview' tab.
Shipping
✨ As the name suggests, this tab is largely targeted at Shipping Tests! That said, a Pricing Test can still impact your shipping revenue (i.e. higher price point may result in higher AOV which may result in a larger percent of orders having free shipping) so we recommend still checking the results in this tab out for a Pricing Test.
Shipping Revenue per Order: Average revenue per order generated by shipping charges.
Percent of Orders with Free Shipping: Percentage of all orders that had no associated shipping revenue.
Orders by Shipping Method: Total orders for each shipping method. Use the dropdown in the top right corner of this graph to adjust which shipping method you are viewing the data for.
Average Shipping Revenue by Method: Average revenue per order generated by shipping charge for each shipping method. Use the dropdown in the top right corner of this graph to adjust which shipping method you are viewing the data for.
Visitors
View over Time: Number of visitors included in the test, by day by test group group. If a user returns to the site over multiple days, only their first visit is reflected in this graph.
Conversion Funnel: Percentage of visitors that Add to Cart, Begin Checkout and Convert for each group.
Visitors: Number of unique site visitors included in the test in each test group. Note this is also visible in the 'Overview' tab.
Orders Detail
Dive into additional order-level detail, including:
Orders over Time: Number of orders placed on the site, by day by test group.
Order Value Histogram: Distribution of orders by order value (in increments of $10). You can filter this to show all orders, or view by test group, and it is a great visual on whether there are any major outliers.
% Orders by Unit Quantity: Breakdown of orders by the quantity of units contained in each order.
% Net Revenue by Order Unity Quantity: Breakdown of net revenue by the quantity of units contained in each order.
Unit Quantity Histogram: Distribution of orders by quantity of units contained. You can filter this to show all orders, or view by test group.
New vs. Returning Customers: Number of orders, percent of orders and average order value (including both product sales and shipping charges) broken down by new versus returning customers for each test group.
Discounts
Orders/Percent Total Orders: Number of orders with discount applied and percentage of all orders in test group, subject to other filters.
Average Discount per Order: Average discount per order (includes only orders with applicable discount).
% of Gross Revenue (Pre-Discount): Discount total as a percentage of total gross revenue (pre-discount) for orders containing discounts (‘Discounted Orders’) and all orders (‘All Orders’), subject to other filters.
% of Gross Profit (Pre-Discount): Discount total as a percentage of total gross profit (pre-discount) for orders containing discounts (‘Discounted Orders’) and all orders (‘All Orders’), subject to other filters.
Discount Histogram: Distribution of orders by total discount, in increments of $10. You can filter this to show all orders, or view by test group.
Product Groups
Product Groups are a great way to gain an understanding of how different product categories performed in a test in comparison to each other. Because we set up your Product Group information, you can group products in any way that you desire.
❗You can follow this guide to set up your Product Group information! If you have not uploaded Product Group information, the only Product Group that will show up is 'None'.
Revenue by Product Group: Breakdown of product revenue by product group for each test group. Revenue is net of discounts and excludes shipping.
Units by Product Group: Breakdown of units sold by product group for each test group.
Data by Product Group: The bottom eight charts allow you to view revenue per visitor, profit per visitor, units sold, percentage of total units, revenue per unit, profit per unit, average units ordered and average units per visitor by product group using the dropdown in the top right corner.
Trend
The Trend tab will show you how conversion rate, average order value, revenue per visitor and profit per visitor have changed for each test group as the test has progressed.
Stat Sig
The first step towards meaningful, actionable results is understanding statistical significance and having confidence in test results. This tab provides several different charts to help you understand which group is a 'winner' and how confident you can be in that. To learn more on stat sig, checkout this article.
Revenue per Visitor Probability to be Best: The percent probability that each group is the best performing group on a revenue per visitor basis.
Profit per Visitor Probability to be Best: The percent probability that each group is the best performing group on a profit per visitor basis.
CVR Probability to be Best: The percent probability that each group is the best performing group on a conversion rate basis.
AOV Probability to be Best: The percent probability that each group is the best performing group on an average order value basis.
Revenue Opportunity Cost to be Best: On the x-axis is '% Revenue per Visitor Opportunity Cost' and the y-axis '% Probability to be Best' — so the plot shows the probability that the winning group is truly the winner from a revenue perspective, or, if it’s not better, then at least not worse by more than [x]%. You can switch test groups using the drop down in the top right.
Profit Opportunity Cost to be Best: On the x-axis is '% Profit per Visitor Opportunity Cost' and the y-axis '% Probability to be Best' — so the plot shows the probability that the winning group is truly the winner from a profit perspective, or, if it’s not better, then at least not worse by more than [x]%. You can switch test groups using the drop down in the top right.