- The Analytics Ladder
- Posts
- Stop Getting Fooled by Analytics
Stop Getting Fooled by Analytics
Your weekly playbook for building judgment that outlasts every algorithm.

Your weekly playbook for building judgment that outlasts every algorithm.
Why this issue matters:
Protect your credibility by verifying claims before they drive decisions.
Outpace automation by mastering verification and judgment that tools cannot replicate.
Ship with confidence using a simple checklist and quick audit to make work repeatable.
Earn influence fast by shifting from reporting to decision partnership with clear scripts and rituals.
Avoid costly mistakes by designing analyses that can be disproven and independently confirmed.

The friday funny - layers of fun in this one.
Carl Sagan’s Guide to Not Being Fooled (Yes, he is one of my heroes)
You can automate analysis.
You can’t automate judgment.
AI fabricates with extreme confidence. Dashboards glow with perfect precision. But the truth hasn’t become easier to find. Only easier to fake.
Carl Sagan warned us about this 30 years ago in The Demon-Haunted World. He feared a time when people “fail to distinguish between what feels good and what is true”
That time is here.
And if you work in data, you’re standing right in the blast zone.
Carl is a bit of a hero of mine. His principles have stood the test of time, and something that I have referenced back to time and again.
I’ve spent the last week crafting what I think is an excellent cross-pollination of the best of Carl’s wisdom for a career in Data & Analytics.
Lets go!
1 | The Context: Plausible Nonsense at Scale
In Sagan’s day, pseudoscience came through late-night TV and supermarket tabloids.
Today, in our jobs, it comes disguised as analytics.
Every week, some AI model claims a 98% lift. Some chart tells a story that feels perfect. Some exec declares: “The data proves it”
But underneath, there’s a quiet rot:
⚠️ Unverified pulls
🍒 Cherry-picked metrics
📊 Dashboards that measure effort, not impact
Sagan called pseudoscience “faithless to its nature”
It mimics the language of science but ignores the discipline that makes science work: self-correction.
Sound familiar?
In analytics, we have replaced curiosity with confirmation. We ship results faster than we test them. We prize narrative over null hypothesis.
Dont do that.
This issue is about reclaiming that discipline and turning it into a competitive advantage.
2 | The Baloney Detection Kit: Analytics Edition
Sagan’s original checklist was meant for citizens.
This is my version for analysts. Upload it to your brain. (Or tape it to your monitor!)
Sagan’s Principle | How Data Professionals Apply It |
|---|---|
Independent Confirmation | Never rely on one source. Validate a dashboard with the warehouse, the log file, or a second analyst’s query. |
Encourage Debate | Make peer review a ritual. Good teams argue with evidence, not ego. |
Question Authority | Seniority is not a statistical method. Ask leaders to show their math. |
Multiple Hypotheses | Before declaring victory, ask what else could explain this. Run a control narrative. |
Don’t Get Attached | Treat every insight as a draft. Kill your darlings before someone else does. |
Quantify | Turn “I think” into “Here’s the number.” Measurement converts belief into learning. |
Check Every Link | Trace your logic from raw data through every transformation. One broken join invalidates brilliance. |
Occam’s Razor | If two models perform equally, pick the simpler. Simplicity communicates faster and fails cleaner. |
Falsifiability | Design every claim so it could, in principle, be proven wrong. Otherwise it’s marketing, not analytics. |
These nine rules are not bureaucracy. They are insurance.

Each protects you from the mental traps that sink credibility: confirmation bias, sunk-cost fallacy, narrative fallacy.
Follow them, and you don’t just produce insight.
You produce trust.
3 | Verification Beats Understanding
Sagan used quantum physics to prove a humbling truth. Most of us don’t truly understand it, yet we trust it because its predictions keep working.
In analytics, understanding is optional. Verification is not.
You don’t need to grok every layer of a neural net.
But you do need to check if its predictions actually hold up.
Ask:
• Did the model predict the next quarter, or just the last one?
• Did our dashboard surface a signal leaders could act on?
• Would this insight survive a fresh pull of the data?
If not, we’re practicing astrology in SQL.
The test of credibility is not elegance. It is repeatability.
4 | Education as Defense
Sagan’s real cure for pseudoscience wasn’t ridicule.
It was education. He believed in teaching people how to think, not what to think.
For analytics leaders, that means teaching your team to reason like scientists, not just code like engineers.
Start here:
Make Falsifiability a Ritual
Before publishing, require each analyst to list one result that would disprove their claim.Reward Disproof
Celebrate the analyst who invalidates their own hypothesis. They just saved you from a bad decision.Teach Quantified Curiosity
Encourage napkin-math before querying. It sharpens intuition and catches nonsense early.Model Humility
As a leader, admit when you were wrong and show what you learned. Rigor starts at the top.
This is how you build a thinking culture.
Not one that worships certainty, but one that respects the process that earns it.
5 | This Week’s Practice
Take Sagan’s kit into your next project.
Use this four-question retrospective:
✅ Have I independently confirmed my data?
✅ Have I considered alternate explanations?
✅ Could this be proven wrong next week?
✅ Would I trust this analysis if someone else produced it?
It takes five minutes.
It can save a million-dollar mistake. A career tragedy.
The best analysts don’t just ask better questions. They ask them longer.
This newsletter is free. Hosting it isn’t.
Clicking this link is how you tip the writer, keep the ladder strong, and help more analysts break through.
You made it this far - awesome - take 5 seconds to help us grow.
[Click it, come back to the newsletter, feel heroic.]
Busy Isn’t a Badge. It’s a Bottleneck..
Every minute you spend on low-value work costs you opportunities you can’t get back. That is why BELAY exists: to help leaders like you get back to what matters.
Our Delegation Guide + Worksheet gives you a simple system to:
✓ Identify what to delegate
✓ Prioritize what’s costing you most
✓ Hand it off strategically
And when you’re ready, BELAY provides top-tier remote staffing solutions — U.S.-based, highly vetted, and personally matched — to help you put those hours back where they belong: fueling strategy, leadership, and growth.
Real freedom starts with a right partner.
TADAA: The Analytics Downsize Accelerates Again
Welcome to this week's special report on Career Annihilating Events. We're tracking the most significant job-killing news from the past seven days where AI is replacing the white-collar workforce. Here are the three most devastating stories that broke between October 25–November 1, 2025.
#1 MOST CAREER ANNIHILATING: Nestle to Cut 12,000 White-Collar Jobs in Push for AI Efficiency (October 29, 2025)
Nestle’s new CEO confirmed the company will immediately cut 12,000 white-collar roles, with another 4,000 to be eliminated over two years. The restructuring is targeted at “automating processes and simplifying operations,” with leadership stating bluntly that automation and AI are core to the overhaul—making this one of the largest white-collar bloodbaths of 2025. Source
#2 MODERATELY CAREER ANNIHILATING: Global Firms Slash Jobs Amid Weak Sentiment, AI Push (October 29, 2025)
Major global employers, including Amazon and UPS, are reporting mass layoffs attributed to efficiency drives and increased automation, with AI aggressively replacing human workers in administrative, logistics, and operational roles. The World Economic Forum now projects 92 million jobs lost to AI and machine learning in advanced economies by 2030, signaling a strategic shift away from entry-level and mid-level white-collar talent. Source
#3 SOMEWHAT CAREER ANNIHILATING: Fiverr Cuts 250 Jobs as Company Goes ‘AI-First’ (October 25, 2025)
Online freelance marketplace Fiverr laid off about 30% of its workforce—250 jobs—in a move to pursue an ‘AI-First’ business model. CEO Micha Kaufman told staff the company aims to become “leaner” and “faster” by embracing generative AI productivity in core operations and management, believing this transition will drive “substantially greater productivity, and far fewer management layers.” The company’s leadership openly stated these layoffs were directly enabled by the implementation of generative AI. Source
Final Thought
Carl Sagan believed the cure for error was transparency.
“Science is a way of not fooling ourselves” he wrote, “and you are the easiest person to fool”
Analytics is so very much the same.
The job isn’t to be right on schedule.
It’s to stay open to being less wrong tomorrow. On repeat.
AI makes it easier to generate answers.
It makes it harder to verify them.
That’s your edge.
Critical thinking as a product skill.
So next time a model claims certainty, channel Sagan.
Lean forward and ask, “How do we know?”
Then build your reputation on the courage to wait for the answer.
Best,
Tom.
Know one teammate who’s drowning in rework or worried AI is eating their job? Forward this to them—you’ll help them climb and unlock the new referral reward: the Delta Teams Playbook, your crisis-mode toolkit when the wheels come off.
Not on The Analytics Ladder yet? You’re missing the brand-new 90-Day Analytics Leadership Action Kit. It’s free the moment you join—your step-by-step playbook to win trust in 14 days, build a system by day 45, and prove dollar impact by day 90.
Disclaimer: Some of the articles and excerpts referenced in this issue may be copyrighted material. They are included here strictly for review, commentary and educational purposes. We believe this constitutes fair use (or “fair dealing” in some jurisdictions) under applicable copyright laws. If you wish to use any copyrighted material from this newsletter for purposes beyond your personal use, please obtain permission from the copyright owner.
The information in this newsletter is provided for general educational purposes only. It does not constitute professional, financial, or legal advice. You use this material entirely at your own risk. No guarantees, warranties, or representations are made about accuracy, completeness, or fitness for purpose. Always observe all laws, statutory obligations, and regulatory requirements in your jurisdiction. Neither the author nor EchelonIQ Pty Ltd accepts any liability for loss, damage, or consequences arising from reliance on this content.
Visit our website to see who we are, what we do. | |
Our blog covering the big issues in deploying analytics at scale in enterprises. |

