Skip to content

How it works

The Review score answers: how strong is the idea? The Result score answers: how well did it perform compared with similar work? You keep both, with the reasons and measurements behind them.
A made-up example that shows how the method works. Nothing here was measured.

Score

68 of 100 for how this idea stands today.

A judgment of the plan, not a chance of success.

Could move to 53 or 83 as what we don't know gets settled.

Range 53 to 83, centre 68

A useful next test

One afternoon: send 200 people on the waitlist a real checkout link at $72 and count purchases within 48 hours.

Judgment
Purpose80
A named audience, one action and a dated goal.
Support55
The waitlist is real, but no price test or past conversion supports 7.5%.
Feasibility75
Checkout and email already exist; the launch is a price change and two sends.
Value70
Annual plans bring cash forward and retention, if early-bird buyers renew.
Downside60
A deep early-bird discount may set the price people expect at renewal.

Where this idea is strongest

Purpose: A named audience, one action and a dated goal.

This is its strongest part relative to the others; it can still need work.

Biggest uncertainty

Whether people on the waitlist will pay $72 for a year up front rather than try a monthly plan first.

Why

A clear offer to people who already asked for the product, with a dated goal. The 7.5% conversion it needs has no price evidence behind it yet.

Ready to test? Yes, once success is agreed: paid annual sign-ups from the waitlist by day 30, refunds excluded.

Method, arithmetic and Coverage

Method project-campaign1.0experimental

methods.md 1.0: mean known, M=20*unknown, D reserve 5, F declared; clip range.

DimensionJudgmentWhy
Purpose
80
A named audience, one action and a dated goal.
Support
55
The waitlist is real, but no price test or past conversion supports 7.5%.
Feasibility
75
Checkout and email already exist; the launch is a price change and two sends.
Value
70
Annual plans bring cash forward and retention, if early-bird buyers renew.
Downside
60
A deep early-bird discount may set the price people expect at renewal.
Coverage
5 / 5. dimensions assessed
Band widths
missing information 0; disagreement 5 (disagreement not measured in full; reserve retained); future sensitivity 10.

Joining a waitlist is free; paying for a year up front is not.

The beta starts with campaign teams: agencies and growth teams deciding which campaign to run next. Reviews and outcome reporting are available in beta. Reviews are experimental during the beta. The Result benchmark method is defined below; live benchmark scores still need qualified reference data and deployment qualification. The examples use simulated data to show how it will work.

Those examples are marked “Constructed for design — not observed performance”: we wrote them to show how the method works, and nothing in them was measured.

1. Describe your plan

Tell us what you want to do, who it is for and what you hope to achieve. Include any evidence you have. We’ll show you what we understood so you can correct it before the Review.

2. Get the first part: your Review

You get a Score out of 100, the reasons behind it and a practical test to try next. We also show the biggest uncertainty and a range for how much the score could change as we learn more.

A Score of 68 does not mean a 68% chance of success. It is a judgment of the plan based on the information available.

How the Score is calculated

Review Methods 1.0 assesses five areas: purpose, supporting evidence, feasibility, potential value and downside. Each area is scored against written criteria. The overall Score is the average of the areas that can be assessed.

The range allows for missing information, possible disagreement between evaluators and plausible changes ahead. When there is too little information, the Review says “Not enough to give one Score yet” and shows what could be assessed.

A made-up example that shows how the method works. Nothing here was measured.

Result

49 of 100 for measured performance.

Illustrative Result score, compared with 100 simulated launches with similar subscriptions, prices and waitlists.

What was measuredOutcome
Waitlist placing eligible annual orders
6.13% (245 / 4,000, before refunds)
Eligible orders refunded
3.67% (9 / 245)
How this benchmark score is calculated

Each measure is compared with the same measure in the reference set. We combine those positions using fixed weights. Your target does not affect the score.

MeasurePosition out of 100Weight
Waitlist placing eligible annual orders
52
50%
Eligible orders refunded
46
50%

The weighted result is 49, displayed as 49. It is an index of the measures above, not the percentage of whole projects you beat.

These references are simulated to explain the method. Each range below contains 100 equally spaced values; the same numbered values belong to the same simulated record.

  • Waitlist placing eligible annual orders: 2%–9.92% in 0.08-point steps.
  • Eligible orders refunded: 1%–5.95% in 0.05-point steps.
History
Definitions registered before start
measured by
Your report from checkout and payment records
Measured
What was measuredTargetResult
Paid annual sign-ups by day 30
At least 300
236 sign-ups
Refunds within 14 days
At most 15
9 refunds

This launch sits near the reference middle, even though it missed its own sign-up goal. The benchmark uses conversion and refund rates, so setting a smaller target would not improve the score. The example assumes 245 eligible orders before refunds: 236 kept and 9 refunded, with every order’s 14-day refund window complete.

Corrections and extra measurements
Corrected: four purchases by our own team were removed from the sign-up count. The earlier Result, 240 sign-ups, stays in the record.

Monthly-plan sign-ups from the same emails: 410. This was added after the launch ended, with no target agreed beforehand.

added after completion

3. Decide what success looks like

We help you decide what to measure, where the figures will come from and when to check. You confirm the targets before you start. For example: at least 300 paid sign-ups in 30 days, excluding refunds. The benchmark has its own fixed measures and reference group; choosing an easier personal target cannot change your Result score.

You get a saved measurement plan to use when reporting outcomes. If the work has already finished, you can still measure it; those targets are labelled “added after completion”.

You can also get a Review without running the idea, or measure work that you never had reviewed.

4. Get the second part: your Result

Bring back the figures for each goal, when you measured them and anything the data leaves out. During beta, you enter these yourself, so we label them as customer-reported.

The Result benchmark adds a score for performance against comparable work, with the raw measures and each part of the score underneath. Your targets stay visible as a separate comparison. A strong sales figure cannot hide poor retention or a missing required measure.

If a figure changes, submit a correction with the reason. The updated Result shows the new figures, and you can still read the earlier report.

Keep the lesson with both records. Start the next campaign from what you learned: what to repeat, what to change and what to test next.

A benchmark you cannot make easier by lowering your goal

A campaign can hit a modest target and still perform poorly beside comparable campaigns. It can also miss an ambitious target and perform well. Your goal and your benchmark score tell you different things.

Before you start, we fix the comparison group, the measures and how much each measure counts. For an acquisition campaign, the starting method gives equal weight to cost per new customer and the share of orders refunded. Lower is better on both.

Each measure gets a position out of 100 in the reference group. The Result score combines those positions using the published weights. A measure at 50 sits at the reference middle. An overall 80 is an average position across measures, not a claim that you beat 80% of whole campaigns.

The benchmark is tested too

Once enough outcomes are in, we can check whether the ideas we scored higher actually performed better. That tells us where our evaluation is useful and where it needs work.

This is why we keep both parts. You can learn from your track record, and we can test our scoring against what happened. Our goal is a standard people can rely on because they can see how it holds up.

When there is enough evidence to assess the method

It requires at least 30 comparable Review-and-outcome pairs for the same method version, domain and objective. Below that minimum, the record says “not enough results yet”. The counts are shown with the comparison.

What makes a reference group qualify

The starting minimum is 100 complete, comparable pieces of work from at least ten unrelated owners, with no owner contributing more than 10%. Outcomes must be available for at least 80% of eligible work, including misses. Meeting those minimums alone is not enough: the data, comparison conditions and uncertainty must also pass review.

The group must match the kind of work, audience, scale and measurement window. Reported data stays separate from independently checked data. Forecast series use the same events and deadlines for every forecaster. A single correct prediction is not a track record.

If there is no suitable reference group, or a required measure is missing, the Result says “Not enough comparable results yet” and shows the measurements we do have.