1. The number that never moves
Real metrics have texture. Small week-to-week variance, even when nothing dramatic's happening. A KPI that's suspiciously flat, or that only ever changes in perfectly round increments, is often a sign the pipeline feeding it broke silently. The dashboard's probably showing a cached or default value instead of a live one.
Easy to miss, because a flat line doesn't look alarming. It looks stable. Get in the habit of treating suspicious stability with the same scrutiny you'd give a suspicious spike.
2. A definition that depends on who you ask
Ask two people on the same team to define "active user" and get two different answers? Your dashboard isn't measuring one thing. It's measuring whichever definition happened to get implemented in the query that built it. This is the single most common source of dashboards quietly disagreeing with each other while each one looks internally consistent.

3. No agreed source of truth
If your reporting dataset's never been checked against a direct platform export — actual Google Ads or Meta Ads numbers, not a copy someone forwarded — you don't actually know how far it's drifted. Teams that skip this check usually aren't wrong by a lot. But they have no evidence they're right, which is a worse position to be in the moment a client asks.

4. Campaign names that don't follow the schema
A dashboard that rolls up performance "by channel" or "by region" is only as reliable as the naming convention behind those groupings. Campaign names drift — inconsistent capitalization, abbreviations that changed mid-quarter, a region tagged two different ways — and some spend silently falls into the wrong bucket. The rollup's wrong even though every individual campaign's raw data is totally fine. A free Taxonomy Checker catches these drifting names before they poison a rollup — upload a CSV and it flags the inconsistencies in your browser.

5. Team rollups that don't add up to the company total
Marketing's reported spend, sales' reported pipeline, finance's reported revenue — do they sum cleanly to the company-wide numbers leadership sees? If not, something in between is double-counting, under-counting, or using a different date boundary. Easy to miss because each team's own number looks perfectly reasonable on its own. The tell only shows up when you try to add them together.
6. Nobody remembers the last time it was checked
The most reliable red flag isn't in the data at all. It's in the team. If nobody can say when a KPI's definition, formula, or source was last reviewed, treat that as a signal on its own. Metrics that go unexamined for a year don't usually stay accurate by accident.
Worth asking out loud in a meeting, not just wondering about privately. "When did we last check this against a live platform export?" is simple enough that an honest "I don't know," said in front of the team, gets the check scheduled a lot faster than any written policy ever will.
A dashboard doesn't have to be wrong to be untrustworthy — it just has to be unverifiable.
Make your dashboards verifiable
Every red flag above is really the same problem: a number nobody can verify. KPI Compass is built to make them verifiable — a governed KPI dictionary with one certified definition and owner per metric, source-of-truth reconciliation that flags drift column by column, automated taxonomy checks, and a review cadence so "when did we last check this?" always has an answer.
If your dashboards can't answer "is this number right, and who signed off on it?", that's the gap we close. Start a free 30-day trial — or see pricing.
About the author
Zach Edelstein
Founder, KPI Compass
Zach has spent the last decade in data analytics, working both inside large media agencies and in-house at enterprise companies. He built KPI Compass because he kept hitting the same walls every BI team eventually hits — messy definitions, benchmarks nobody can verify, dashboards that quietly drift from reality. He's especially into where AI actually helps with this work, and he's still actively evolving KPI Compass to keep up with how fast the data landscape moves.
KPI Compass on LinkedIn