Canonical URL: https://chargezen.com/docs/checkoutos/ab-testing/creating

# Creating Experiments

Step-by-step guide to setting up A/B tests in CheckoutOS

Last updated: 2025-01-16

Creating an A/B test in CheckoutOS is straightforward. This guide walks you through setting up experiments from hypothesis to launch.

## Before You Start

#### Pre-Test Checklist

*   Define your hypothesis — What do you expect to happen?
*   Choose one variable — Test a single change at a time
*   Set success metrics — What will you measure?
*   Estimate sample size — How long will the test need to run?
*   Get stakeholder buy-in — Ensure everyone agrees on the test

## Creating an Experiment

1

#### Navigate to A/B Testing

Go to CheckoutOS → A/B Testing → Create Experiment.

2

#### Name Your Experiment

Use a descriptive name like "Upsell Headline Test - Jan 2025".

3

#### Select What to Test

Choose the element: Upsell offer, checkout extension, bundle, etc.

4

#### Create Variants

Set up your control (current) and variant (new) experiences.

5

#### Configure Traffic Split

Decide how to divide visitors between variants.

6

#### Set Success Metric

Choose your primary metric (conversion, AOV, etc.).

7

#### Launch

Review settings and start the experiment.

## Experiment Types

### Upsell Offer Test

Test different upsell products, discounts, or messaging:

Upsell A/B Test

```json
{
  "experiment": {
    "name": "Premium Product Upsell Test",
    "type": "upsell_offer",
    "variants": [
      {
        "name": "Control",
        "weight": 50,
        "offer": {
          "product_id": "standard-upsell",
          "discount": "15%",
          "headline": "Add this to your order"
        }
      },
      {
        "name": "Premium Offer",
        "weight": 50,
        "offer": {
          "product_id": "premium-upsell",
          "discount": "20%",
          "headline": "Exclusive: Save 20% on this upgrade"
        }
      }
    ],
    "success_metric": "upsell_acceptance_rate"
  }
}
```

### Pricing Test

Test different discount amounts or pricing strategies:

Pricing A/B Test

```json
{
  "experiment": {
    "name": "Discount Format Test",
    "type": "pricing",
    "variants": [
      {
        "name": "Percentage",
        "weight": 50,
        "discount": {
          "type": "percentage",
          "value": 20,
          "display": "20% OFF"
        }
      },
      {
        "name": "Fixed Amount",
        "weight": 50,
        "discount": {
          "type": "fixed",
          "value": 10,
          "display": "$10 OFF"
        }
      }
    ],
    "success_metric": "revenue_per_visitor"
  }
}
```

### Placement Test

Test where offers appear in the checkout flow:

Placement A/B Test

```json
{
  "experiment": {
    "name": "Upsell Placement Test",
    "type": "placement",
    "variants": [
      {
        "name": "Post-Purchase",
        "weight": 50,
        "placement": "post_purchase_page"
      },
      {
        "name": "Thank You Page",
        "weight": 50,
        "placement": "thank_you_page"
      }
    ],
    "success_metric": "conversion_rate"
  }
}
```

## Configuring Variants

Each variant can customize:

#### Variant Options

*   Product or offer shown
*   Discount amount and type
*   Headline and description text
*   Button text and color
*   Image or media
*   Placement location
*   Timing (when offer appears)

##### Keep Variants Distinct

Make sure your variants are meaningfully different. Testing a 14% discount vs 15% discount won't give you actionable insights.

## Multivariate Testing

For advanced users, CheckoutOS supports multivariate testing where you test multiple variables simultaneously:

Multivariate Test

```json
{
  "experiment": {
    "name": "Headline + Discount MVT",
    "type": "multivariate",
    "factors": [
      {
        "name": "headline",
        "levels": ["Save Big!", "Exclusive Offer", "Don't Miss Out"]
      },
      {
        "name": "discount",
        "levels": ["10%", "15%", "20%"]
      }
    ],
    "success_metric": "upsell_acceptance_rate"
  }
}
```

##### Sample Size Warning

Multivariate tests require much larger sample sizes. The example above has 9 combinations (3 x 3), requiring 9x the traffic for statistical significance.

## Scheduling Experiments

Control when experiments run:

Scheduled Experiment

```json
{
  "schedule": {
    "start_date": "2025-02-01T00:00:00Z",
    "end_date": "2025-02-28T23:59:59Z",
    "auto_conclude": true,
    "minimum_runtime_days": 14,
    "minimum_conversions": 500
  }
}
```

## Best Practices

##### Experiment Setup Tips

*   **Write a hypothesis first** — "We believe X will improve Y by Z%"
*   **Use descriptive names** — Future you will thank you
*   **Start with 50/50 splits** — Equal traffic for fastest results
*   **Set minimum runtime** — Avoid peeking and stopping early
*   **Document everything** — Record your reasoning and results

## Related Topics

[

### Traffic Allocation

Advanced traffic splitting strategies

Learn more



](https://chargezen.com/docs/checkoutos/ab-testing/traffic)[

### Analyzing Results

How to interpret experiment data

Learn more



](https://chargezen.com/docs/checkoutos/ab-testing/results)

Was this page helpful?

Need more help? [Contact support](https://chargezen.com/docs/checkoutos/troubleshooting/support)