To detect hallucinated references before you submit, check each entry against the published record: open its DOI and confirm it leads to the same paper, compare title, authors, journal and year, and search for it in a scholarly database. Treat any reference you cannot match as unverified. RefGuard by Proofig AI flags hallucinated references across the whole list.
What does a hallucinated reference look like?
A hallucinated reference is a citation to a publication that cannot be matched to any real record. The term comes from AI writing tools, which can produce text shaped like a citation without looking anything up. Octym’s explainer on hallucinated references covers why; this post is about catching them.
The difficulty is that they rarely look wrong. In a 2026 audit of biomedical papers, published as a letter in The Lancet, the authors describe the fabricated references they found as on-topic, neatly formatted, credited to real researchers and carrying believable publication years.
In practice, they fall into four patterns:
- The invented paper. A plausible title, real-sounding authors and a genuine journal name, with no matching record anywhere.
- The borrowed identifier. A DOI or PubMed ID that works but belongs to a different paper. The Lancet audit gives the example of a reference on rheumatology biomarkers linked to the identifier of a study on nematode worms.
- The misattributed paper. A real researcher’s name attached to a title that person never published, sometimes on a topic squarely in their field.
- The damaged real reference. The paper exists, but the authors, year, volume or pages are wrong. In a Scientific Reports study, 24% of the real citations produced by the newer of two chatbot versions still contained substantive errors.
The last is not strictly a hallucination, but it fails the same checks.
How common are hallucinated references in published papers?
Common enough to reach print. The Lancet audit, by Maxim Topaz and colleagues, verified 97.1 million references in 2.5 million open-access biomedical papers and identified 4,046 fabricated references across 2,810 papers. By the authors’ estimate, about one paper in 2,828 had at least one fabricated reference in 2023; in the opening seven weeks of 2026, the figure was one in 277. At the time of the audit, 98.4% of the affected papers had received no publisher action. The audit counts fabricated references whatever produced them: its authors name paper mills, deliberate misconduct and uncritical use of AI writing tools as possible sources.
The AI route is well documented. When researchers writing in JMIR gave three AI chatbot models the inclusion criteria of published systematic reviews and asked for matching references, between 28.6% and 91.4% of the references each model returned were hallucinated. Newer models do better, but not well enough to skip checking: in the Scientific Reports study, 55% of the citations from an older chatbot version were fabricated, compared with 18% from its successor.
How do you check a reference by hand?
Work through the list one entry at a time and run four checks. Each one catches a different pattern.
- Open the identifier. Paste the DOI after https://doi.org/ (or look up the PubMed ID) and read what opens. It should be the same paper, with the same title and authors. A DOI that goes nowhere may mean an invented paper (or a typo); one that opens a different paper may mean a borrowed identifier.
- Compare the record field by field. Check title, first author, journal, year, volume and pages against the record. One wrong field is usually a typo. Several wrong fields are a warning sign: the JMIR study counted a reference as hallucinated when any two of title, first author and year were wrong.
- Search for the paper itself. For entries without an identifier, search the exact title in a bibliographic database such as PubMed, your library’s catalog or the journal’s own site. The ICMJE Recommendations tell authors to verify references against a bibliographic source or against the original sources.
- Confirm it says what you cite it for. Read at least the abstract and make sure it supports your sentence. ICMJE also expects authors to be able to attest that each reference supports the statement it is attached to.
When a reference fails, do not leave it in and hope. Find the source you meant and cite that, replace it with a source you have read, or remove the claim. ICMJE states plainly that AI-generated material is not acceptable as a primary source.
Why doesn’t manual checking scale?
The routine is simple, but it is slow, and it has to be repeated.
- Time. For a list of 60 references, even three minutes per entry adds up to three hours, and revisions bring new entries to check.
- Many hands. References arrive from several co-authors, sometimes late and sometimes pasted from tools the person doing the checking never saw.
- Sampling misses them. In the Lancet audit, 91% of affected papers contained one or two fabricated references. A spot check of ten entries can easily pass a list with one bad entry in it.
- Responsibility stays with you. The COPE position on authorship and AI tools holds authors fully responsible for the content of their manuscript, including the parts an AI tool produced.
The Lancet authors recommend that publishers build automated reference verification into submission, before peer review starts. Running that kind of check yourself, before you submit, means the problem is fixed before an editor or reviewer sees it.
What does RefGuard flag in your reference list?
RefGuard, Proofig AI’s reference integrity product, checks the reference list in one pass. You upload the manuscript as a PDF or Word file (text-based, up to 150 pages), and RefGuard compares each reference with the scholarly publishing record to confirm the article is real. In its report, RefGuard:
- flags references that cannot be matched to any real publication, which is where hallucinated and invented citations show up;
- points out incomplete or broken references, such as a wrong year or a missing author, so they are easy to fix.
The same report covers the rest of your reference integrity. RefGuard flags citations to retracted papers, with the reason, date and type of notice (Octym’s post on citing retracted papers explains why that matters). It checks claims against citations, flagging statements that need a source but lack one and cited papers that do not fit the claim or that contradict it. It also notes references 15 or more years old and paratext (items that are not primary scholarly content), and scores the bibliography for Quality, Impact, Recency and Diversity, with a sortable reference list and CSV export.
RefGuard’s flags are signals for you to review, not verdicts: you decide what to fix, and you stay responsible for your references. RefGuard is available on its own, inside PubShield for publishers and research institutions, and as the reference checks in Octym. Proofig AI covers image integrity and, with RefGuard, references and retractions; Octym reviews the entire paper with 26 checks. See what RefGuard checks in a manuscript, or read how Proofig AI supports researchers before submission.
Frequently asked questions
Are some kinds of paper more likely to contain hallucinated references?
Yes: in the Lancet audit, review articles had a fabrication rate 57% higher than other paper types. The rates were 16.7 and 10.6 per 10,000 papers. If you are writing a review, check the whole list rather than a sample, and look hardest at entries an AI tool helped find or format. RefGuard by Proofig AI checks the list for you and flags references that cannot be matched to a real publication.
Do newer AI models still invent references?
Newer AI models still invent references, though less often in published tests. Newer chatbot versions have fabricated fewer citations than older ones, but not none: in the 2023 Scientific Reports study, 18% of the newer version’s citations were still fabricated, and in the 2024 JMIR study the lowest hallucination rate among three chatbot models was 28.6%. Whichever AI tool helped draft your manuscript, treat each reference it supplied as unverified until you have matched it to the published record. RefGuard by Proofig AI flags the hallucinated ones.
Should I disclose that an AI tool helped with my references?
Authors should usually disclose that an AI tool helped with their references. COPE’s position is that authors must disclose which AI tool they used and how, in the Methods or a similar section. Journal policies vary, so check your target journal’s instructions too. Disclosure does not shift responsibility: COPE holds authors fully responsible for their manuscript, including the parts an AI tool produced. Before you submit, match each AI-supplied reference to the published record; RefGuard by Proofig AI flags references it cannot match to any real publication.
