In 2023, I was coaching a client through her literature review. She had a pretty niche topic, and there just wasn’t much relevant literature published. We spent a week digging, trying different search terms and databases, but found nothing.
Then, the following week, she came to our meeting with about two dozen sources. I couldn’t believe it. “How did you find these?!” I asked. “They’re so closely related to your topic!” How had we missed this wellspring of literature?
And then she replied: ChatGPT.
I’d heard of ChatGPT but had never used it. This was 2023, when AI was still new. I also had no idea how drastically AI would change education as we knew it.
During the meeting, I tried searching for a couple of the titles she’d come up with… and couldn’t find anything. So I tried again.
Still nothing.
After our meeting, I checked each title she added and discovered that all but three were completely fabricated.
Fortunately, we caught this before she submitted and she was able to revise accordingly.
But other students haven’t been so lucky. In fact, discovering fake sources and misattributed information is alarmingly common these days. Just a couple of weeks ago, I worked on a final manuscript for a client preparing for a defense and discovered that over 50 sources were fabricated.
FIFTY.
For a manuscript slated to be defended in two weeks.
The client relied heavily on ChatGPT to find sources but never checked them.
AI has a penchant for taking creative liberties… that’s why the ChatGPT disclaimer exists at the bottom of every discussion: ChatGPT can make mistakes. Check important info.
Another recent client had her final manuscript kicked back because she had misattributed so much information to her sources. Catching misattributions made by AI can be far trickier than dealing with fake sources. Because AI may come up with a source that actually exists, but it may not say what AI tells you it says. AI can misrepresent findings, exaggerate conclusions, or even come up with statements that aren’t close to true.
Two particularly dangerous AI research pitfalls that are worth remembering:
- Making up sources. They may look legit, but when you try to find them… you realize they don’t exist.
- Misattribution. A source may exist, but it may not say what AI tells you it says.
How to Avoid an AI Reference Nightmare
- Verify the publication exists. Search Google Scholar, your university library, or the publisher’s website. Don’t rely on AI’s assertion that it’s real.
- Verify the bibliographic details. Confirm the authors, publication year, title, journal, and DOI.
- Read the original source. Never assume AI has accurately represented the findings, arguments, or conclusions.
- Check every citation against its claim. Just because an article is real, it may not support a claim you’ve made.
Friends, always check your sources. It’s not enough to just make sure a source exists – also be sure the source actually says what you’re claiming. The truth is, it’s hard to “prove” AI use based on how common AI writing patterns are in your text. But fake sources and misattributions can definitely create red flags and undermine the credibility of your entire dissertation.
The only thing worse than discovering that ChatGPT invented your references is having your dissertation committee discover it first.
Concerned about the accuracy of your dissertation’s references? Dissertation Angels offers manuscript reviews to help identify citation inconsistencies, questionable sources, and other issues before submission.