AI makes things up. It sounds just as sure when it does. Here is where it fakes, and how to catch it.
AI makes things up. It sounds just as sure when it does. There is no tell.
That matters because you are the one who sends its work out. Your name goes on the report or the email. Not the AI's.
By the end of this page you will know where to look first, and what to ask for so checking is quick.
AI can go look things up. But that is not its first instinct.
Its first instinct is to answer from memory. It learned from a huge amount of writing, and it answers by guessing which word should come next, over and over, based on everything it saw while learning.
Most of the time, the most likely word is also the true one. That is why AI is so good. The problem is the times it is not, because you cannot hear the difference. A guess sounds exactly like a fact.
So why is it so confident? AI is trained like a student taking a test. A guess might score points. A blank never does. So it learned to always guess, and never say "I do not know" (OpenAI's research).
The good news: it slips in patterns, and you can learn them.
Today's models are smart. They pass the old trick questions. But they still slip in the same few places.
The fix for recent stuff is one word: search. I once asked about a brand-new AI model, and it told me that model did not exist yet. I told it to search first and come back to me, and then it found it.
That is the real lesson: once you know where these tools are limited, you know how to work with them.
People say "check the output." Check it against what? You end up reading something you did not write, hoping a mistake jumps out. It usually does not.
So make the AI hand you the thing you will check against, before it starts.
Ask for a detailed plan: what it will make, what it will use, what it is unsure about. You read that in ten seconds, then approve it or fix it.
Here is the bonus most people miss: today's models are good at sticking to an approved plan. I scrutinize the work less, because I approved the plan and I know the direction it is taking. Review is actually quicker.
Use this on anything that matters. Approve the plan, let it work, then compare the result to the plan.
Before you do anything, give me your plan. I want the full plan so I know exactly what you are going to do. Wait for my approval. Once I approve the plan, you can go ahead and act on it. When you are done, mark anything you were unsure of and tell me where each fact came from.
AI can flag things it is unsure about when you ask. But do not trust it to catch every uncertainty. It can still sound sure when it is wrong.
So ask:
Then go look at what it flags. People skip this part. The flag is not the check. It just tells you where to check.
One warning. Asking for sources does not mean the sources are real. If it cannot search the web, it can make up a link as easily as it made up the fact. Click it. If your AI is connected to real files or search, you are on much safer ground.
It is hard to actually read and review everything an AI does. Generating work is easy. Checking it all gets tiring.
So have another AI check it. A different agent is more likely to find issues than the agent that did the work, because it has no stake in it. The first chat will defend its own answer. A new chat reads it the way a stranger would.
In a chat, it is simple. Open a brand new chat, tell it what you were trying to do, and paste in the first AI's work.
The research supports this too: AIs critiquing each other make fewer mistakes than one working alone.
It works across tools too. Have Claude check what ChatGPT wrote. They make different mistakes.
Use a brand new chat, not the one that made the work. Fill in both brackets.
I asked another AI to help me with this: [what I was trying to do] Here is what it gave me: [paste the work] Review its work. Does it actually do what I asked for? Point out anything wrong, missing, or made up. Do not redo the work. Just give me your honest review.
You can also make the same AI check its own work.
The trick is giving it something to compare against. If you just ask "is this good?", it will say yes. Instead, hand it back its own plan plus the finished work, and ask where the two do not match. Now it has something real to compare.
My AI set up a new client's billing from a signed contract. The last thing it did was pull the saved record back up and compare it, line by line, to the contract. That caught a misspelled name before the first bill went out.
Same idea, smaller: "Here is what I asked for. Here is what you gave me. What did you miss?"
Some questions need judgment. Is this email too blunt? Does this argument hold up? You have to read those yourself.
But a lot of checks are not like that at all.
Those have one right answer. They are just right or wrong, so you do not have to judge them yourself.
Ask for them by name and you get the same checks every time, instead of whatever the AI felt like looking at that day.
You can make this automatic in a normal chat. Add one line to your AI's instructions in settings: After you finish any task, check your own work before you show me. Now the check happens every time, without you asking.
Coders take this further with tests that run on their own after every change. Same principle, any kind of work: move the checking off your plate and onto something that never gets tired.
Pick one thing AI made for you recently. Have another AI review it, using the prompt in step 5. That takes about a minute and needs no setup.
Then, the next time you hand AI real work, ask for the plan first, using the prompt in step 3.
Those two habits cover most of this page.
Want better results from the start? Read our prompting guide.