Glossary · Our vocabulary
Hindsight split tests
Our name for split tests ranked on Yield rather than completion rate — where the winner cannot be known at submit time, so the test waits.
A term we coined, not an industry standard. See the whole glossary.
Definition
A split test whose ranking metric is Yield rather than completion rate. Variants are compared on what their submissions turned out to be, which means the result is not available at the moment of conversion — it arrives when the verdicts do.
Why it matters
Because every existing form A/B tool declares a winner at the submit event — not out of carelessness but out of necessity, since the submit event is the last thing it can observe. The winner is announced before anybody has picked up a phone.
The research is also unusually clear about how rare form-level testing is in the first place. Marketers split-test ads, landing pages and creative. When forms come up, the test is a one-off before-and-after or nothing at all — and the workaround people are advised to use is their landing-page tool’s page-level test.
“I’d track completion rate by step and booked-visit rate, not just total form submissions. That’ll tell you whether the automation is helping or just moving the phone call friction onto the page.”
Practitioners keep describing the right experiment. Nobody in our whole research corpus was running it, because no tool ranks variants on anything but fills.
In practice
- Measure [time to disposition](/glossary/time-to-disposition) first. It sets the minimum length of every test you can run, and if it is very long you need an interim outcome instead.
- Pick the ranking metric before the test starts, and record how much of each cohort has resolved when you read it.
- Expect the two metrics to disagree. A test where Yield and completion agree taught you nothing you could not have learned for free.
- Keep the losing variant’s data. The interesting result is usually which field caused the divergence, not which variant won.
The common mistake
Running one when you cannot wait for the outcome.
The failure mode is not calling the test wrongly. It is calling it early — waiting three weeks, running out of patience, and reading a completion-rate result off a test that was designed to measure something else.
If you cannot wait, do not run a shortened version. Change the ranking metric to the earliest signal you can wait for — sales accepted, meeting held — and be explicit that this is what you are measuring.
Related terms
- YieldOurs
- Our name for the quality-adjusted metric: Yield rate is the share of submissions that reached a good verdict; Yield value is revenue per hundred submissions.
- VerdictOurs
- Our name for the outcome written back onto a submission: Won, Lost, Disqualified, or Awaiting verdict — plus a value.
- Time to disposition
- How long from submission until you know what the lead was worth — the number that decides whether outcome-based testing is possible for you at all.
- Multi-step form
- Splitting one form across several screens — the category’s most confident best practice, and one of its least evidenced.
- Form drop-off analysis
- Seeing exactly which question people stop at — the feature even competing vendors name as the one worth paying for.