growth diagnostic
press enter
Diagnostic · Site Health & Test Hygiene

Site Health — the floor, not the cliff

Core Web Vitals · prior 30d → recent 30d

The scorecard — bad, and stable

Every vital is over the line, and none of them moved. A chronic conversion tax on every visitor — worth fixing for its own sake — but it cannot explain a May–June dip.

Behavior friction

Friction signals — same story

Source: Cowork / Clarity dashboard, 30d windows — our direct Clarity API sees only 3 days, so the prior-vs-recent comparison reuses the Cowork pull (adjudicated on the funnel page).

For the CTO

One genuinely broken thing

Test hygiene

Rules for the restart

The kill-switch (zero tests, 7–10 days) comes first. When testing resumes, these rules prevent a repeat — each one maps to a specific failure found in this diagnostic.

The rules

  1. Stagger tests — never launch a batch in one week; you can’t attribute a CVR move to anything.
  2. Protect a clean control — a global holdout that no experiment can touch, so the baseline is always observable.
  3. Cap concurrency at 1–2 — June ran 4.6 concurrent tests on average; CVR was 2.51% on 0–1-test days vs 2.20% on 4+ days.
  4. Never route cold paid traffic to the homepage or a listicle — homepage-redirect variants exit the landing-page bucket entirely and land on a page not built for cold traffic.
  5. Every variant must reach a real PDP — the canonical Rise 2 collapse was traffic rerouted away from product pages, not lost.
  6. Read LP performance with variant-subtraction filters — exact-match URL filters return 0 sessions in Intelligems; only contains/doesNotContain cuts are honest.
The cadence problem

Tests launched per window

Bars = A/B test spans starting inside each two-week Intelligems window; amber bars = windows where Rise 2 URL tests (Rise 2 / Super LP / Switcher) were live. Same windows as the funnel evolution page.

The kill-switch — what would falsify all of this

Zero Intelligems tests for 7–10 days. The Rise 2 URL tests were paused the evening of Jun 11, so the experiment is already running. April — near-zero tests, CVR peak — is the control window.

If the test-program diagnosis is right, with tests staying off:

canonical LP→PDP recovers toward ~25% within days content-as-2nd-page falls back toward ~10% Rise 2 CVR climbs back toward ~2%+

If those numbers don’t move while tests stay off, the rerouting story is wrong — attention shifts fully to the media side (media page) and to the page’s own change history (Shopwatch log, still pending).