- Perplexity Sonar grounds generation in live search results, which stops most invented statistics before they’re written — but it doesn’t check facts already sitting in a draft.
- Claude’s own web search and a dedicated audit pass are what actually catch fabricated expert names and unverified numbers already in the text.
- Neither tool alone is a fact-checking pipeline — we run both roles separately after stripping roughly 2,000 fabricated expert attributions from a 1,145-article corpus.
- The fix that worked was mechanical, not another AI pass: flag every numeric claim automatically, then have a human or Claude rewrite each one with a real source or cautious language.
Every AI content pipeline eventually publishes something it shouldn’t: a fabricated expert quote, an invented percentage, a study that doesn’t exist.
Ours did. An audit of our 1,145-article corpus found close to 2,000 fabricated “Dr. Name, Board-Certified Specialist” attributions injected by an early enhancement script.
So we tested what actually catches this kind of error: Perplexity Sonar during generation, Claude during a post-hoc audit, and a plain deterministic scanner.
What Does Perplexity Sonar Actually Do Differently?
Perplexity Sonar is a search-grounded model — it retrieves live web results before generating, so its output is anchored to sources that actually exist at write time.
That stops a specific failure mode: a model inventing a statistic because it has no source in front of it and generates a plausible-sounding number instead.
It doesn’t stop a different failure mode: a downstream enhancement script or editing pass adding an unsourced claim after Sonar’s grounded draft is already done.
Why Did Fabricated Experts Get Through Anyway?
Our fabricated quotes didn’t come from the generation model at all — they came from a hardcoded enhancer script that inserted named “Dr. X, Board-Certified Y” attributions to hit an E-E-A-T scoring target.
Sonar’s grounding never had a chance to catch this, because the fabrication was injected after generation, in a step Sonar doesn’t see.
This is the core limit of any generation-time grounding: it protects the first draft, not everything that touches the article afterward.
What Does a Claude-Based Audit Pass Actually Catch?
A dedicated audit prompt — read the article, flag every named expert, every specific statistic, every claim that reads as sourced but isn’t — catches what generation-time grounding misses.
Running that pass across the full corpus found the fabricated attributions in seconds, at effectively zero additional cost against the original generation spend.
The fix wasn’t asking Claude to invent replacement quotes — it was replacing named-individual attribution with class-level, organizational attribution: “veterinarians and pet care experts” instead of a fake doctor’s name.
| Method | Catches | Misses |
|---|---|---|
| Perplexity Sonar (generation-time) | Invented stats with no source at write time | Anything added after generation by a later script or edit |
| Claude audit pass (post-hoc) | Fabricated named experts, unsourced claims already in the draft | Claims that sound plausible and cautious even when unverifiable |
| Deterministic numeric scanner | Every sentence with a number, percentage, or statistic, for human or LLM review | Nothing numeric — it’s a flagging tool, not a verifier |
How Do You Actually Fix a Flagged Claim?
A stats-rewriting pass reads each flagged sentence and applies one of two fixes: replace the specific number with cautious, non-numeric language, or remove the claim entirely.
“Studies show 73% of dogs prefer memory foam” becomes “many dogs show a preference for memory foam bedding” — still useful to a reader, no invented precision.
Where no honest replacement exists, the sentence gets deleted rather than softened into something still misleading.
Google’s spam policies state that scaled content abuse is judged by whether pages are original and helpful, regardless of whether AI, humans, or a mix produced them — the production method is not what determines a manual action.
Does Organizational Attribution Introduce Its Own Risk?
Yes — telling a model to attribute a claim to an organization instead of a person can make it invent what that organization supposedly says, not just who says it.
We saw this directly: a guidance rewrite once published the opposite of a real dermatology association’s actual recommendation, because the model filled in plausible-sounding organizational guidance instead of a verified one.
Organizational attribution only removes the fabricated-person risk. It has to be paired with actually checking what the cited organization says, not assumed to be automatically safe.
What Should a Small Team Actually Run?
Per the Schema.org vocabulary, use a search-grounded model like Perplexity Sonar for research and first drafts — it’s the cheapest place to prevent an invented statistic, since it never gets written down.
Run a dedicated audit pass, human or Claude-driven, on the finished article before publish — this is what catches anything a later script, edit, or enhancement pass introduced.
Add a deterministic numeric-claim scanner as a pre-publish gate, not a one-time cleanup — new fabrications can re-enter a pipeline the same way the first ones did.
Search-grounded generation and post-hoc auditing catch different failure modes and neither replaces the other. Perplexity Sonar prevents invented stats from ever being written; a Claude-driven audit pass catches what enters later, through enhancement scripts or edits. Our own corpus needed both, plus a mechanical numeric-claim scanner as a standing pre-publish gate, after roughly 2,000 fabricated attributions made it to production.
Frequently Asked Questions
Is Perplexity Sonar enough to prevent fabricated content on its own?
No. It prevents invented statistics at generation time but has no visibility into claims added by later scripts, enhancement passes, or manual edits — a separate audit step is still required.
Can Claude reliably detect its own hallucinations after the fact?
A dedicated audit prompt, run as a separate pass from generation, catches named-expert fabrications and unsourced claims reliably in our testing — but it works because it’s looking specifically for that pattern, not because self-review is inherently more accurate.
What’s the difference between deleting a false claim and softening it?
Softening (“many” instead of “73%”) is appropriate when the underlying direction of the claim is true but the precision was invented. Deletion is appropriate when no honest version of the claim can be supported at all.
Does organizational attribution fully solve the fabricated-expert problem?
It solves the fabricated-person problem but introduces a new one — a model can still invent what an organization supposedly says. The claim still needs verification against what that organization actually publishes.
How often should a fact-checking audit run on a live content pipeline?
As a pre-publish gate on every article, not a periodic cleanup. New fabrications can enter through any pipeline stage that touches content after the original grounded draft.
Last updated: 2026-09-04
한국어 
English 