Skip to content

One campaign, evaluated before and after

You’re about to launch a product. First, we review your plan. After the launch, you bring back the numbers and we measure how it went. Here’s what you send and what you get at each step.
Constructed example
A made-up example that shows how the method works. Nothing here was measured.

Score

68 of 100 for how this idea stands today.

A judgment of the plan, not a chance of success.

Could move to 53 or 83 as what we don't know gets settled.

Range 53 to 83, centre 68

A useful next test

One afternoon: send 200 people on the waitlist a real checkout link at $72 and count purchases within 48 hours.

Judgment
Purpose80
A named audience, one action and a dated goal.
Support55
The waitlist is real, but no price test or past conversion supports 7.5%.
Feasibility75
Checkout and email already exist; the launch is a price change and two sends.
Value70
Annual plans bring cash forward and retention, if early-bird buyers renew.
Downside60
A deep early-bird discount may set the price people expect at renewal.

Where this idea is strongest

Purpose: A named audience, one action and a dated goal.

This is its strongest part relative to the others; it can still need work.

Biggest uncertainty

Whether people on the waitlist will pay $72 for a year up front rather than try a monthly plan first.

Why

A clear offer to people who already asked for the product, with a dated goal. The 7.5% conversion it needs has no price evidence behind it yet.

Ready to test? Yes, once success is agreed: paid annual sign-ups from the waitlist by day 30, refunds excluded.

Method, arithmetic and Coverage

Method project-campaign1.0experimental

methods.md 1.0: mean known, M=20*unknown, D reserve 5, F declared; clip range.

DimensionJudgmentWhy
Purpose
80
A named audience, one action and a dated goal.
Support
55
The waitlist is real, but no price test or past conversion supports 7.5%.
Feasibility
75
Checkout and email already exist; the launch is a price change and two sends.
Value
70
Annual plans bring cash forward and retention, if early-bird buyers renew.
Downside
60
A deep early-bird discount may set the price people expect at renewal.
Coverage
5 / 5. dimensions assessed
Band widths
missing information 0; disagreement 5 (disagreement not measured in full; reserve retained); future sensitivity 10.

Joining a waitlist is free; paying for a year up front is not.

The launch, score and results in this example are made up.

The Review scores the plan. The Result scores performance against a simulated group of comparable launches. Live Result benchmarking is not available yet; this example shows the defined method.

1. Before launch: find what needs testing

What you send

“Launch our annual plan to the 4,000 people on our waitlist at $96 a year, with a two-week early-bird price of $72. Goal: 300 paid annual sign-ups in the first 30 days.”

What we do

We show you how we understood the plan so you can correct it. Then we review the goal, the evidence behind it, whether it is practical, what you could gain and what could go wrong.

What you get

A Review with a Score, the reasons behind it and a useful next test. Here, the weak spot is whether people will pay. The suggested test is a small paid offer to 200 people before the full launch.

A made-up example that shows how the method works. Nothing here was measured.
Registered , before the launch
ObjectiveTargetmeasured by
Paid annual sign-ups by day 30
At least 300
Checkout records, refunds excluded
Refunds within 14 days
At most 15
Payment provider refunds

2. Before launch: set the targets and benchmark

You ask to measure the launch. NeoNumber proposes what to count, when to count it and which source to use. You confirm the targets and counting rules before launch and get a saved measurement plan.

The benchmark is fixed at the same time: half the Result score comes from waitlist conversion and half from the refund rate, compared with similar launches. Your sign-up target does not set the scoring scale.

A made-up example that shows how the method works. Nothing here was measured.

Result

49 of 100 for measured performance.

Illustrative Result score, compared with 100 simulated launches with similar subscriptions, prices and waitlists.

What was measuredOutcome
Waitlist placing eligible annual orders
6.13% (245 / 4,000, before refunds)
Eligible orders refunded
3.67% (9 / 245)
How this benchmark score is calculated

Each measure is compared with the same measure in the reference set. We combine those positions using fixed weights. Your target does not affect the score.

MeasurePosition out of 100Weight
Waitlist placing eligible annual orders
52
50%
Eligible orders refunded
46
50%

The weighted result is 49, displayed as 49. It is an index of the measures above, not the percentage of whole projects you beat.

These references are simulated to explain the method. Each range below contains 100 equally spaced values; the same numbered values belong to the same simulated record.

  • Waitlist placing eligible annual orders: 2%–9.92% in 0.08-point steps.
  • Eligible orders refunded: 1%–5.95% in 0.05-point steps.
History
Definitions registered before start
measured by
Your report from checkout and payment records
Measured
What was measuredTargetResult
Paid annual sign-ups by day 30
At least 300
236 sign-ups
Refunds within 14 days
At most 15
9 refunds

This launch sits near the reference middle, even though it missed its own sign-up goal. The benchmark uses conversion and refund rates, so setting a smaller target would not improve the score. The example assumes 245 eligible orders before refunds: 236 kept and 9 refunded, with every order’s 14-day refund window complete.

Corrections and extra measurements
Corrected: four purchases by our own team were removed from the sign-up count. The earlier Result, 240 sign-ups, stays in the record.

Monthly-plan sign-ups from the same emails: 410. This was added after the launch ended, with no target agreed beforehand.

added after completion

3. After launch: score the performance

What you send

After the 30-day launch and each order’s 14-day refund window, you enter 245 eligible orders: 236 kept and 9 refunded, from a waitlist of 4,000. You include the measurement date and explain any gaps in the data.

What we do

We calculate the conversion and refund rates, then compare each with the frozen reference group. The example gives both measures equal weight. In beta, submitted outcome figures are labeled customer-reported; the reference group here is simulated.

What you get

An illustrative Result score of 49 out of 100: near the middle of this reference. Separately, sign-ups fell 64 short of your goal and refunds stayed within your limit. Lowering the sign-up goal would not raise the benchmark score.

What the two parts tell you

The Review is 68 out of 100 for the plan; the Result is 49 out of 100 against comparable launches. A promising plan delivered a near-middle outcome in this reference. These scores answer different questions: the gap is not a 19-point decline on one scale.

As more comparable campaigns get results, we can check whether the plans we rated higher actually did better. That puts our scoring to the test too. We need real outcomes to do that; this example only shows how it works.

When you have an update

Open the Result, choose “Correct this report” and enter the revised figures and why they changed. NeoNumber saves an updated Result and keeps the earlier figures available, so you can see what changed.

In this example, the first report counted 240 annual sign-ups. Removing four team purchases corrected it to 236. The Review still shows the advice you had before launch.

Using the same benchmark, that correction changes the illustrative Result score from 50 to 49. The earlier Result remains available; neither the reference group nor the original Review is rewritten.

4. Use the lesson in your next campaign

The launch missed its signup goal, while refunds stayed low. That suggests a useful next question: can a clearer offer increase purchases without losing the customers who stay?

You send two versions of the next offer, with the same audience and budget. NeoNumber compares the plans. Your new campaign starts with the previous Review, Result and lesson one click away.

This is a hypothesis to test. The first campaign alone does not establish what caused the outcome.