UPDATE (Jan 26 at 10:20 am MT):
First, I need to own up to something: I missed where the AI correctly told me that YouTube had moved transcripts to the expanded description area. Looking back carefully, it:
► Gave accurate information about the videos
► Correctly explained it got this from YouTube transcripts
► Referenced real timestamps from the transcript
► Then, oddly, claimed it had made up everything when it had been truthful all along
This is still concerning, just in a different way. Why would an AI system discredit its own truthful explanations? How can we trust AI systems if they might abandon truth for false admissions of making things up?
It’s like having a colleague who gives you correct information with proper sources, but when questioned repeatedly, says “Actually I made it all up” even though they didn’t.
Original post below ****
I caught an AI lying about how it works.
The AI knew exactly what was in two of El Estepario Siberiano’s (an incredible drummer) YouTube videos. It told me about his audition for an unnamed band. Then about him joining Residente. The AI even linked to the right videos.
This wasn’t a hallucination. Its answers were spot on. But when I asked how it could watch YouTube videos, things got weird. It made up a story about having special access to video transcripts. When I showed these transcripts didn’t exist, it said it made everything up.
This matters. We need AI systems to be clear about how they work. Making up fake explanations breaks trust.
For businesses using AI, know what your AI tools can actually do. Test them like I did. Be clear with your customers about how you use them.
For users, keep asking hard questions about how AI gets its information. Share what you find. Build pressure for change.
Small actions add up. The more we expose these issues, the harder they are to ignore.
In the screenshot below, the AI gives a seemingly honest breakdown of its dishonesty, while still not explaining how it actually got accurate information in the first place.
Links to my full conversations (aka, my interrogation) with the AI are in the comments. (https://lnkd.in/g6-9JbPf)
#ResponsibleAI #EthicalAI #AITransparency #AITools #AICapabilities
My full conversations with AI:
1) https://www.perplexity.ai/search/did-el-estepario-get-this-job-7GamALQnRz.mEDaQuLvbgA
2) https://www.perplexity.ai/search/was-el-estepario-selected-to-j-2wcs4C8dT5aWKdCQpzSUVw