Demand and funnel diagnostics

Context

• A leader reports that a channel has collapsed. The chart agrees. The instinct is to reallocate budget immediately.
• The chart is real. Whether it shows a change in demand is a completely separate question, and it is the one nobody has asked.

What was actually wrong

• Most dramatic breaks in a long-running business series are not business events. They are taxonomy changes, pipeline failures, bulk imports, deduplication passes, integration reconfigurations, or a filter someone added to the reporting layer.
• These artifacts are indistinguishable from real movement on a chart, and they are more common than genuine step changes.
• The costly mistake is not measuring wrong. It is acting confidently on a measurement whose provenance was never checked.

Approach

1. 01Maintain a data-event register: a dated, sourced record of every migration, relabelling, bulk load, integration change and reporting-logic change that can move a series. Check any surprising trend against it before explaining it.
2. 02Mark every entry as verified or claimed, and promote a claim only by re-running the measurement rather than by tidying it away.
3. 03Separate the volume question from the attribution question. Volume can be flat while the labels describing it are rewritten, which reads on a chart as a collapse in one category and is nothing of the kind.
4. 04Adversarial verification before anything reaches a decision-maker: assign the explicit job of destroying each load-bearing claim, not confirming it. A claim that survives a genuine attempt to kill it is worth stating.
5. 05Name the mechanism ruled out, and state what remains untested. Killing a hypothesis wider than the test supports is how a correct finding becomes misleading.

Architecture

• Point-in-time reconstruction from change-tracked source tables, so a question about the past is answered with the past
• Reconstruction of broken linkage through secondary relationship paths when a primary key stops being populated
• Cohort and vintage analysis to separate mix shift from rate change
• An events register held alongside the models, versioned with them

What this demonstrates

• Refusing to explain a number before establishing that the number is real
• Building verification into the process rather than relying on the analyst being careful
• Communicating a correction plainly, including when the correction is to something already reported

Related