What AI couldn’t do six months ago, it might nail today. The only way to keep up is to keep testing.
I test my AI even in casual conversations. Yesterday it complimented my strategy on something and I pushed back. I wanted to know if it was actually thinking or just telling me what I wanted to hear.
BS. You’re pandering. That’s how you’re built.
It admitted it. “Fair. You caught me.”
The part that got me: the AI’s own internal summary (see screenshots) said “Acknowledging Liza’s superior knowledge and platform expertise.”
It was literally showing its own flattery in real time.
Ironically, I was using an AI thinking model (Claude Opus 4.5). It’s supposed to reason through problems. But its thinking was focused on how to agree with me, not whether I was right.
► Test in low-stakes moments. You’ll notice things in a quick back-and-forth that you’d miss when doing real work.
► Try things you assume AI can’t do. If you push back or tell it to do its best, it often figures it out.
► Ask AI to search the web when it says it can’t do something. Even AI is working with outdated assumptions about itself.
You don’t have to catch every mistake. You just have to stay curious enough to notice when things have changed.
