Problem
Scientific writing now contains references that look real but do not exist, and real papers cited for things they never said. AI writing tools have made both more common, and reviewers have no fast way to check a reference list against the sources.
Who it is for
Authors before submission, reviewers and editors, and anyone reading AI-assisted writing.
How it works
Paste a manuscript or upload a PDF.
1. References. Each reference is looked up in Crossref, OpenAlex, arXiv, Semantic Scholar and doi.org. Title, authors, year and DOI are compared by deterministic code, not by a language model. The result is one of: verified, details differ, retracted, not found, could not check.
2. Claims. For each citing sentence ClaimTrace fetches the cited source (open full text where available, otherwise the abstract, and says which), retrieves the most relevant passages, and asks two models from different providers, a judge (Claude) and a skeptic (Kimi), whether the source supports the sentence.
3. Evidence. A verdict is shown only with a quote of at least five words found verbatim in the source. If the two models disagree, the claim is flagged for human review. A provider failure is reported as "could not check", never as a negative finding.
The report streams in as checks finish, puts problems first, and can be shared by link or exported.
What we measured
- References: 1,728 test cases built from 399 real references (unchanged, wrong year, negated title, swapped first author, fabricated, retracted). Accuracy 0.956, with 0 false "verified" and 0 real papers called non-existent.
- Claims: SciFact dev set, n = 100. Judge + skeptic reached 0.90 accuracy with a 0.10 false-supported rate (judge alone: 0.87 and 0.167).
- Prompt injection: 10 kinds of injected instructions planted in source texts, 50 live checks. No contradicted claim was shown as supported, and no displayed quote contained injected text.
- Speed on the Vultr instance: a 40-reference PDF in 191 s, first result after 19 s (one run).
Limitations
- A 40-reference paper takes about 3–5 minutes, depending on the LLM providers. Our 3-minute target was not reached reliably.
- Only open-access full text is used; otherwise the claim is checked against the abstract and labelled so.
- Citations of methods or datasets often come out "partly supported".
- If the skeptic's provider is unavailable, claims are checked by one model and labelled "judge only".
- In the 1,728-case run, 67 of 150 fabricated references came back "could not check" because an index failed under load. None came back "verified".
-- Sentences that cite a work as an example or for comparison, not as evidence, can come out "not supported" even though nothing is wrong with the citation.
Try it
Open https://66-135-11-202.sslip.io and press "Try a sample". A finished audit of "Attention Is All You Need" (40 references, 58 claims): https://66-135-11-202.sslip.io/r/Bmu5Xq9Q1H9aZ5TWWkzpLw
Built from scratch during HackGlobal 2026 by Izbassar Orynbassar (Syracuse University) and Madina Sissenbay (Atyrau Oil and Gas University).
Tried it on a published paper. The report with exact quotes from the sources is very clear.