Issue 127  September 12th 2025

AI can be wrong.

• I learned this the hard way with a simple survey.
• I trusted the results, but the numbers did not add up.
• I found myself double-checking every answer.

But here’s what most people miss:

• AI tools like ChatGPT and Copilot can process hundreds of survey responses in seconds.
• They can also skip steps, make up data, and give you answers that look right but are not.
• I saw this firsthand with a dataset of a thousand survey responses.

The first time I ran the analysis, the AI finished in under a minute. AI is fast, but not that fast, so I double checked. Turns out it looked at “representative answers,” not all.

It also made up company names that were never in the data.

It counted things that did not exist.

It missed things that did.

I had to spell out every step:

• Analyze all thousand rows, not just a sample.
• Use a semicolon to split answers, not a comma.
• List every answer that does not match the survey options.
• Show me the raw text, not just a summary.

I found mistakes by checking the data myself.

And then it even tried to make me feel good about catching its mistakes.

“Great question, good catch,” it said.

But I did not want praise.

I wanted the truth.

Here is what I learned:

1. Never trust the first answer from an AI.
2. Give step-by-step instructions, down to the smallest detail.
3. Always check the raw data yourself.
4. Ask for the full list of answers, not just a summary.
5. Watch for made-up data - AI will fill in blanks if you let it.
6. Ignore the flattery. Focus on the facts.

AI is fast, but it is not careful.

If you want real answers, you have to be.

This is Critical Thinking in motion. Curiosity drives the probing. Observation catches what automation skips.