Intro

As I continue to gain experience with AI, I quickly discovered that AI does make mistakes -- And quite often! As of this writing, I have created over 80+ blog posts using ChatGPT-4o. Out of those, I found that a handful of articles contained either incorrect or incomplete information. I resolved this issue by either manually rewriting sections or by rephrasing my prompts. Ultimatly, the more I work with AI, I am getting better at creating effective prompts to get better results.

Real Examples of ChatGPT-4o Errors

One example involved asking ChatGPT-4o to summarize a website that belongs to a researcher I personally know. The site contained his unique research, and while the summary itself was mostly correct, the AI mistakenly attributed his book to someone else entirely. He was a little upset when I showed it to him.

In another case, I was working on an article about AMD processors. ChatGPT-4o ended up mixing up some details between AMD and Intel, which could easily confuse readers who aren’t already familiar with computer hardware. After making a simple change to rephrase the prompt, I received the correct result.

Double-Check AI Results

The main lesson here is simple: you need to read AI-generated results carefully and confirm they are accurate. Don’t blindly accept what the AI gives you. (Especially, if your planning on sharing results with other people.) In some cases, I was able to slighly rephrase my question and get the right answer the second time.

I did have one case where ChatGPT never did give me what I wanted, regardless of how I phrased the prompt. I proably tried at least ten different times. I finally ended up heavily editing the AI generated articles and took the correct parts from each iteration to form a complete article that finally met my needs.

Based on my own experience, I estimate that at least 10% of the initial answers from ChatGPT-4o have some kind of mistake. Of course, this is just my small sample and not scientific. I asked Google for more accurate data, and it said ChatGPT-4o was 88.7% accurate in an official study. That's assuming the AI summary that provided the statistic was correctly interpeting the data.

AI Listing Mistakes on Google

I also saw an AI mistake on Google related to my very own website. Google’s AI generated summary had direct quotes taken from from my site, but it confused my business with another business of the same name located in Hawaii. My business is actually located in Illinois! (Please don't hold that against me!) How many more are on Google? This is a big deal as it could cause all sorts of problems when people don't realize they are being given incorrect information.

If you want to read more about this type of issue, there’s a good article about AI attribution errors here:
ChatGPT’s Major Attribution Errors – Tow Center

Be Clear with Your Questions

Another thing I’ve noticed is that when I ask vague or incomplete questions, ChatGPT-4o sometimes gives incomplete or off-topic results. To be fair, this is probably more my fault than the AI’s. The AI has to guess what I meant, and sometimes it gets it wrong.

The solution? Be as clear and specific as possible when asking your questions. Specify things you want included or excluded. Specify how you want the results formatted such as HTML, Python Code or a Blog Post etc. The better your prompt, the better the AI response will be. I may write another blog post in the future with tips on better writing prompts for ChatGPT.

Conclusion

AI tools like ChatGPT-4o are powerful and helpful, but they are not perfect. It’s important to review the results, correct mistakes, and refine your questions. When used carefully, AI can save time and boost productivity—but always stay alert for errors.