Get into the habit of asking AI its confidence level in its responses. You’ll be shocked how often it says “medium” or “low” including for some of the most compelling insights.
That’s the problem. I’ve been getting great research from AI, but I had no way to know how much I could actually rely on it.
So I now make AI check itself. Not just give answers, but audit them.
I ask AI to rate its own output on key dimensions. For each takeaway, I have it share:
➡︎ How confident it is (High/Medium/Low) and why
➡︎ What it’s assuming to be true
➡︎ What info would improve its rating
When AI says “Medium confidence because I’m assuming X, Y, and Z,” I know exactly what to check. I don’t waste time second-guessing everything. I zero in on where the risks are.
These insights turn AI into a thinking partner, not just a fast Q&A machine. I still verify the work, but now I know where to lean in instead of treating every output as equally trustworthy or equally risky.
I also ask it to rate things like risk assessment, evidence quality, implementation difficulty, and impact potential. Then explain the rationale and assumptions behind each.
Here’s a prompt I use often: “For each key takeaway, rate your confidence (high/medium/low) and explain why. List your assumptions, and note what would increase your confidence if it’s medium or low.”
Interestingly, AI has told me that it is not confident in its response because it can’t find on the web or in its knowledge any supporting data. Truthful and transparent. However, without the confidence level, I could’ve easy assumed that it’s a sound insight. (Sample output in the comments)
I put together a simple table below that shows what questions to ask for each dimension.
This helps me check what AI produces without having to be an expert in every topic. It’s a simple addition to the prompts that changed how I work with AI completely.
If you’re using AI for research or decision support, give this a try. Would love to hear what’s working for you.
If you found this useful, feel free to share it with others on your team.
