--- title: "A/B testing" slug: "ab-testing" tags: ["AB testing", "Shopping App"] updated: 2025-03-14T09:40:17Z published: 2025-03-14T09:40:17Z canonical: "docs.newstore.com/ab-testing" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.newstore.com/llms.txt > Use this file to discover all available pages before exploring further. # A/B testing A/B testing is a experimentation process in which two or more versions of a variable (page, page element, etc.) are shown to different segments of users. The process collects information on the performance of each variant. This helps brands decide which changes they want to move forward with. In NewStore, you can test variants of Shopping App feature configurations using [Firebase](https://firebase.google.com/docs/ab-testing). This article describes how you can set up A/B tests for the Shopping App. It also contains a list of the supported A/B test scenarios. ## Prerequisites To set up and run A/B tests in the Shopping App, the following are required: - Consumers using iOS core version 2.124.0 or higher. - The person setting up the test needs access to the Firebase Console account provided by NewStore during onboarding. ## Setting up an A/B test To set up an A/B test, perform the following steps: 1. In the Firebase Console, navigate to the **A/B Testing** option on the left, and select **Create experiment** - **Remote Config**. 2. Follow the steps to set up the experiment: 1. Type an **Experiment name** for your A/B test. 2. In the **Targeting** section, define the percentage of users that should be a part of this A/B test. 3. In the **Goals** section, select the **Primary metric to track** (e.g. Purchase Revenue, Retention, etc.). You can define additional metrics as well. 4. In the **Variants** section, define the baseline and the variants that you want to test. For both baseline and variants, add the following: 1. In the **Parameter** field, add the key that corresponds to the feature you want to test. See the [available tests and the corresponding keys](/v1/docs/ab-testing#list-of-available-ab-tests) later in this article. 2. In the **Value** field, select which behavior you want to show in this variant. For example, you might want to test different layouts on the product list. In that case, add the `productListLayout` parameter to the baseline, and in the value field add which behavior you want to test in the Baseline group (inspirational or two). Then in the other variant(s), add the same parameter with another value. 3. Under **Adjust variant weights**, add participation percentages for baseline and variants groups. 5. After reviewing the draft experiment, select **Start experiment** to activate the variants and start collecting data. 3. During and after the A/B test, draw conclusions from the results shown in the **Experiment results** section. When there is a clear winner or change required, this can be managed from Firebase directly. ## Example As an example, let's see an A/B test for showing the color swatches on the product list page. To see if hiding color swatches on the product list page works better for an imaginary brand's users than showing the swatches, the brand decides to set up and run an experiment for 100% of the users: - The `Variant A` user group is 80% of the users and receives `false` value for `showColorSwatchesOnProductList`. - The `Baseline` user group is thus 20% and receives a value of `true` for `showColorSwatchesOnProductList`. After running the test for some days, Firebase shows the following test results: ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/efccb9ba-b6d1-43ac-ac1b-475eedbd34dd.png) When analyzing the results of this example, for the tested users the baseline is a clear winner based on several metrics, but mainly due to the revenue per user, which is $55.05 compared to the $21.08 result of Variant A. According to the results, the brand can stop the experiment and make the baseline the default. ## List of available A/B tests The following table shows the keys you can use for A/B testing: | **Key** | **Controls** | **Type** | **Possible values and their meaning** | | --- | --- | --- | --- | | `productListLayout` | Changes the product list layout. | `String` | `"inspirational"`: inspirational layout `"two"` : two-column layout | | `showApplePayOnPDV` | Shows the Apple Pay express checkout on the product detail view in the main app. | `Boolean` | `true`: enable `false`: disable | | `showBottomPopoverSoldOutVariants` | Shows the unavailable variants in the configuration at the bottom. | `Boolean` | `true`: show unavailable options on the bottom `false`: sort options as they are sent from the e-commerce environment | | `showColorSwatchesOnProductList` | Shows color swatches at the bottom of the gallery view on the product listing page. This test needs previous configuration. If you want to use this key, reach out to your NewStore contact to ensure that the necessary feature is enabled. | `Boolean` | `true`: show `false`: hide | | `showExpandedInfoTextOnDetailView` | Changes the "More info" text on product detail view. | `Boolean` | `true`: "Expand details" `false`: "More details" | | `showMediaGalleryOnProductList` | Makes the gallery views horizontally scrolling and enables scrolling them on the product listing page. | `Boolean` | `true`: enable `false`: disable | ### What do the keys change The following images show which part of the user interface each key controls. The differences are marked on the side of the screenshots. ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/0d7f1f51-ecb3-432f-8153-2bd8a423ea08.png) ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/fa854b47-f463-4465-a9a7-f9e6843cfe88.png) ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/4dd233f3-a64f-428e-b9d1-c97bafbde285.png) ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/ae996b76-a6f7-4b1e-8c28-07461fd42d59.png) ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/7971bb36-ad5d-4a53-accc-40eabb2ee15b.png) ![](https://cdn.document360.io/c2d22fcd-a44b-4e1e-bf3a-84f0ffba0dbe/Images/Documentation/4999cb83-231b-431f-82b2-40c252eb2cc5.png)