Data & Methodology

The rules are fixed before the experiment starts.

If the standard can move after the numbers arrive, it is not a test. These principles, states and definitions are frozen in advance.

Ten principles

P1Evidence Before Narrative
Never create a conclusion first and search for evidence afterward.
P2Reality Decides
AI opinion is not market validation.
P3Do Not Scale Before Evidence
Do not recommend major investment before meaningful evidence appears.
P4Website Is a Validation Bridge
A website exists to test reality, not merely to generate pages.
P5Kill Experiments, Preserve Learning
Failure remains useful data.
P6First-Party Evidence Has Priority
Real analytics, rankings, inquiries, orders and observed user behaviour are stronger than AI speculation.
P7No Fabricated Metrics
Unknown values must remain unknown.
P8No Cross-Project Mutation
The control platform cannot silently modify experiment projects.
P9Public Claims Require Evidence
A public result must be traceable to evidence.
P10Low-Cost Validation First
Prefer small, measurable experiments before large builds.

Lifecycle — where the test is

Every experiment carries exactly one state.

Experiment lifecycle states
StateMeaning
IDEAA hypothesis exists but has not entered live-market testing.
TESTA real website or other live market asset is running and collecting evidence.
SIGNALMeaningful real-world evidence has appeared.
PROVENSignals are repeated, meaningful and commercially credible.
BUSINESSThe experiment has graduated into an operating business.
CLOSEDThe experiment has ended or has been archived.

Evidence that counts toward SIGNAL includes organic search visibility, ranking keywords, returning users, contact clicks, inquiries, quote requests, registrations, external references and relevant AI or search citations.

Decision — what we do about it

Decision is recorded separately from lifecycle. An experiment can be live and undecided, or closed and stopped.

Experiment decisions
DecisionMeaning
GOEvidence supports further investment.
HOLDEvidence is insufficient; maintain at low cost and continue observation.
KILLThe current hypothesis does not justify additional investment.
UNDECIDEDNo formal review has been completed yet.
Non-negotiable

Stopping an experiment never deletes its evidence. A stopped experiment remains part of the historical dataset. Kill experiments. Preserve learning.

Visibility — what may be published

Every piece of experiment information is classified before it can be published.

Publication visibility levels
LevelMeaning
PUBLICSafe to publish immediately.
DELAYED_PUBLICPotentially commercially valuable information that may be published later.
PRIVATENever publish automatically.

Never published: customer identities, private emails, phone numbers, contracts, unreleased high-value opportunities, proprietary algorithms, confidential pricing, credentials, access tokens and unpublished commercial strategies.

How evidence is ranked

  1. First-party observation. Our own analytics, rankings, inquiries, orders and observed user behaviour.
  2. Externally verifiable fact. Something a third party can independently check.
  3. Interpretation. Clearly labelled as reasoning, never presented as measurement.
  4. Opinion. Not evidence. An AI's opinion is not evidence either.

Anything that has not been observed stays marked unknown rather than estimated. A blank is more useful than a plausible-looking number, because a blank can still be filled correctly.