How we test, and what we will not claim
Every factual claim on this site carries exactly one of five labels. They are not decoration.
The five labels
We tested it We ran it ourselves and kept the run log or output file. If we cannot produce the artefact, we do not use this label.
We use it We run it in our own work on an ongoing basis, not just once for an article.
Documented Stated in the vendor's own current documentation. We link it and date it. This tells you what the vendor says, not whether it is true.
Maker's claim The vendor asserts it and we could not verify it. We never restate these in our own voice.
⚠ We could not verify this We looked for a primary source and did not find one. We publish the gap rather than the guess.
How we test a workflow
- Build it against a local test stack. We run stand-in services for the tools a small business uses, so a workflow can be executed end to end over real HTTP.
- Use synthetic data only. Every name, email and address in our tests is invented and points at
example.invalid. We never test against a real business, a real store, or real customer records. - Run it, and count. We record units of work consumed, run time, and how consumption scales.
- Break it on purpose. We induce failures — outages, malformed payloads, missing fields — and record what happens, especially whether anything tells you.
- State the boundary. Every test page says what the measurement does and does not cover.
The boundary we are careful about
Measuring a workflow is not the same as reviewing a platform
Our runner measures workflow design — what a design costs and how it fails. It is not Make, n8n or Pabbly, and it tells you nothing about their reliability, interfaces or billing behaviour. Where we have not run a platform, its page says so and we do not review it.
This distinction is easy to blur and we would rather be dull than misleading about it.
What we will not do
- Describe a workflow as working when we only built it. Built is not tested.
- Claim a time saving for a real business. We do not run one.
- Use “best” or “our pick” without first-hand testing of the realistic alternatives.
- Publish a price or commission rate without the date we checked it.
- Take a commission rate from an affiliate directory. We found fourteen such figures that were wrong or unsourceable.
The full rules are in our editorial standard.
The same discipline is built into the Automation Decision Sheet: it asks you to measure your own task before deciding, rather than estimating it.