Data Quality

6 Signs Your Dashboard Is Lying to You (Data Quality Red Flags)

Dashboards rarely lie on purpose. They drift quietly, one unnoticed mismatch at a time, until the number everyone trusts is wrong.

ZE

Zach Edelstein

Founder, KPI Compass

Published 3 min read

1. The number that never moves

Real metrics have texture. Small week-to-week variance, even when nothing dramatic's happening. A KPI that's suspiciously flat, or that only ever changes in perfectly round increments, is often a sign the pipeline feeding it broke silently. The dashboard's probably showing a cached or default value instead of a live one.

Easy to miss, because a flat line doesn't look alarming. It looks stable. Get in the habit of treating suspicious stability with the same scrutiny you'd give a suspicious spike.

2. A definition that depends on who you ask

Ask two people on the same team to define "active user" and get two different answers? Your dashboard isn't measuring one thing. It's measuring whichever definition happened to get implemented in the query that built it. This is the single most common source of dashboards quietly disagreeing with each other while each one looks internally consistent.

A certified KPI definition in KPI Compass with its formula, owner, and review cadence.
One governed, certified definition per metric — so “active user” means the same thing to everyone.

3. No agreed source of truth

If your reporting dataset's never been checked against a direct platform export — actual Google Ads or Meta Ads numbers, not a copy someone forwarded — you don't actually know how far it's drifted. Teams that skip this check usually aren't wrong by a lot. But they have no evidence they're right, which is a worse position to be in the moment a client asks.

KPI Compass reconciling a reporting dataset against a source-of-truth export, flagging each metric as match, minor drift, or discrepancy.
Reconciling a reporting dataset against a direct platform export, column by column — match, minor drift, or real discrepancy.

4. Campaign names that don't follow the schema

A dashboard that rolls up performance "by channel" or "by region" is only as reliable as the naming convention behind those groupings. Campaign names drift — inconsistent capitalization, abbreviations that changed mid-quarter, a region tagged two different ways — and some spend silently falls into the wrong bucket. The rollup's wrong even though every individual campaign's raw data is totally fine. A free Taxonomy Checker catches these drifting names before they poison a rollup — upload a CSV and it flags the inconsistencies in your browser.

The KPI Compass Taxonomy Builder generating a consistent campaign naming string from fixed segment values.
A naming schema that generates consistent campaign strings, so rollups don’t silently misfile spend.

5. Team rollups that don't add up to the company total

Marketing's reported spend, sales' reported pipeline, finance's reported revenue — do they sum cleanly to the company-wide numbers leadership sees? If not, something in between is double-counting, under-counting, or using a different date boundary. Easy to miss because each team's own number looks perfectly reasonable on its own. The tell only shows up when you try to add them together.

6. Nobody remembers the last time it was checked

The most reliable red flag isn't in the data at all. It's in the team. If nobody can say when a KPI's definition, formula, or source was last reviewed, treat that as a signal on its own. Metrics that go unexamined for a year don't usually stay accurate by accident.

Worth asking out loud in a meeting, not just wondering about privately. "When did we last check this against a live platform export?" is simple enough that an honest "I don't know," said in front of the team, gets the check scheduled a lot faster than any written policy ever will.

A dashboard doesn't have to be wrong to be untrustworthy — it just has to be unverifiable.

Make your dashboards verifiable

Every red flag above is really the same problem: a number nobody can verify. KPI Compass is built to make them verifiable — a governed KPI dictionary with one certified definition and owner per metric, source-of-truth reconciliation that flags drift column by column, automated taxonomy checks, and a review cadence so "when did we last check this?" always has an answer.

If your dashboards can't answer "is this number right, and who signed off on it?", that's the gap we close. Start a free 30-day trial — or see pricing.

ZE

About the author

Zach Edelstein

Founder, KPI Compass

Zach has spent the last decade in data analytics, working both inside large media agencies and in-house at enterprise companies. He built KPI Compass because he kept hitting the same walls every BI team eventually hits — messy definitions, benchmarks nobody can verify, dashboards that quietly drift from reality. He's especially into where AI actually helps with this work, and he's still actively evolving KPI Compass to keep up with how fast the data landscape moves.

KPI Compass on LinkedIn

Ready to put this into practice?

KPI Compass ships with the KPI dictionary, taxonomy governance, and reconciliation checks this post covers, out of the box.