cite-holmes (Cite Holmes): deep research with citation verification

One line: a question goes in — a verified report comes out.

Three differences from a plain "search and summarize":

1. Calibrate before working — ask sharp questions first; the most expensive waste is researching the wrong question.
2. Conclusions carry evidence grades — 🟢 two independent sources agree / 🟡 single authority / 🔴 contested.
3. Every citation is checked — mechanical layer (reachability, domain authority, field completeness, dedup) plus semantic layer (does the source actually support the claim?). Unverified references never masquerade as real.

⛔ Iron rules (zero exceptions)

1. Never fabricate: citations must come from pages actually fetched this session. Re-search rather than write URLs from memory.
2. Never pretend: unchecked references are marked unverified; fetch failures are unreachable (≠ nonexistent — flagged for human review).
3. Don't hide conflicts: when sources disagree, present the disagreement, mark 🔴, show each side's evidence.
4. Budgeted search: QUICK ≤6 searches, FULL ≤15. Out of budget → state the gaps honestly instead of forcing conclusions.
5. Calibrate before searching (FULL mode): scope / timeframe / audience / output format must be locked first.

Step 0: mode selection

| Mode | Fits | Calibration | Budget | Output | |---|---|---|---|---| | QUICK | Single fact-check: "is this claim true", "when was X released" | skipped | ≤6 | short report | | FULL | Open research: "state of X", "A vs B", "do a survey" | mandatory | ≤15 | full report |

One verifiable fact → QUICK. Synthesis or trade-offs → FULL. "Quick check" forces QUICK; "thorough" forces FULL. Unsure → FULL. Pre-submission bibliography final check skips this choice: --preset submission (one flag = strict + GB·T/BibTeX/CSV exports).

Five-phase workflow

1. CALIBRATE (FULL only)

Ask 3–5 high-leverage questions at once: scope, timeframe, audience/depth, output format, decision context. Never re-ask what the user provided; if they decline, proceed with stated defaults. Three rules: order questions by dependency; each carries a recommended answer phrased so "yes" accepts it; facts you can look up yourself never become questions — decisions stay with the user.

2. PLAN

Show a short plan: 3–7 sub-questions, source priority (primary/official > major media > community/blog as leads only), budget.

3. SEARCH (iterative)

Read references/search-strategies.md first. Each round targets one sub-question; evolve queries with discovered terms; search both English and Chinese when the topic spans both; fetch full text of the 2–5 most valuable sources (never conclude from snippets); verify key numbers on the original page before quoting.

4. VERIFY (the heart of this skill)

Register every reference in research_refs.json (schema in references/report-template.md), then run:

python scripts/verify_refs.py --refs research_refs.json --out verify_report.md
# Zotero/EndNote .bib feeds in directly:
python scripts/verify_refs.py --refs bibliography.bib --out report.md
# Big batch? Self-check first (zero outbound by default):
python scripts/verify_refs.py --doctor --net

Five verdicts: verified / partial / unreachable (needs_human_check) / invalid / unverified. Every identifier (url/doi/pmid/arxiv) is cross-checked against its official registry, so fabricated IDs are judged invalid — never silently unreachable. Every report opens with a BLUF dual-reader header + CiteScore (0-100 + A-D grade) + a one-line pre-submission conclusion.

Mechanical layer (the script)

| Capability | Boundary | |---|---| | Registry consistency (title/year/journal/author) | 12 machine-readable error codes; thresholds 0.50/0.82 | | Clone-pair detection | same title under different DOIs / one DOI with different titles — paired leads, never verdicts | | Cross-lingual title bridging | six writing-system families (Latin/Cyrillic/Greek/Han/Kana/Hangul); hit→verified, miss→partial; language difference is never fabrication evidence; families outside the six (Arabic/Hebrew/Thai…) do NOT trigger the guard yet — attach an English/romanized title | | Bibliography profiling | list-level batch-fabrication fingerprints — review leads, never verdicts | | Preprint upgrade hints | arXiv-only refs since published get the formal DOI annotated | | Retraction checks | Crossref online, or offline via --retraction-cache; notice granularity: retraction / partial retraction / withdrawal → human hand-off, Expression of Concern → caps verified at partial, correction → note only; a failed check is flagged, never reported as clean | | Identifier-less references (Vancouver) | PubMed title reverse-lookup → verified with PMID anchor back-filled; works from 2 content words; a transport failure is marked network_error and never counted as a miss | | GB/T 7714 Chinese medical references | [1]-style bracket numbering and title-first entries parse; inline PMID:/DOI: anchors are extracted and verified; when no ID exists, journal+year+volume(issue):page uniquely located in PubMed upgrades to verified (Single-Citation-Matcher semantics, language-independent — a Chinese claimed title against an English registered title never counts as fabrication) | | Chinese scholarly registries | wanfangdata / CNKI / yiigle classified at journal tier | | Non-MEDLINE journals (identifier-less) | OpenAlex anchor search → DOI resolution: the aggregator is only an index, the verdict evidence comes entirely from the DOI registry (retraction checks still run on the anchored DOI) | | Retraction checks at scale | tools/build_retraction_cache.py builds a local Retraction Watch index from the official daily-updated dump — --retraction-cache then verifies offline with notice granularity | | Literature discovery | --discover "topic" searches PubMed + OpenAlex and verifies every hit at birth - a verified reference list out, GB/T 7714 / BibTeX export ready; --discover-since 2020 filters by year; --discover-sort cited|newest orders by citations or date; --demo runs the bundled 8-reference sample (3 planted fakes) for a zero-setup tour | | EB/OL references (guidelines, labels) | URLs in [EB/OL]-style entries are extracted (when no stronger identifier exists) and routed through reachability + source-tier checks - drug labels, guidelines and consensus documents stop being bare unverified | | Reference-list completeness | --check-document audits orphans (listed but never cited — a writing-integrity advisory, never a verdict) and coverage; one-file mode: pass the document alone and the reference section is auto-extracted | | Wayback archive links | attached to dead links automatically | | Completeness scoring | how many export fields the registry CSL can back-fill | | Speed | 20-ref mixed list ≈18s at --workers 4 (default), ≈7s cache-warm, ≈47s serial — measured table: references/verification-details.md | | Weak network | --proxy / --cn / --preflight; auto degraded-network mode after 3+ registry transport failures |

Full semantics: references/verification-details.md.

Semantic layer (the model)

Decompose each cited claim into atomic claim-triplets (subject–relation–object) and verify each against the source; register the verdict structurally ("semantic": {"claim", "support", "quote", "note"}) — it becomes auditable workpaper; not_in_source/contradicted cap the mechanical verdict at partial — a source that exists is not a source that agrees.

Task → flag

| Task | Flags | |---|---| | Medical topics | --easy · --profile medical | | Share / archive | --format html | | Export | --export bibtex,gbt7714,ris,csv,auditjson | | CI | --strict · --offline | | Polite pacing | --mailto you@lab.edu | | API keys | --openalex-key / --s2-key / --ncbi-key | | Weak network | --proxy · --cn · --preflight · --timeout | | Cache | --no-cache / --refresh-cache / --cache-ttl | | Self-check | --doctor (add --net) | | Pre-submission check | --preset submission | | Explain one verdict | --explain N | | Pre-submission grill | --grill report.json (adversarial review checklist) | | External judging | --judge-url / --nli-url / --fast-judge-url (notes only) | | MCP server | mcp/server.py — verify_references / explain_verdict / check_document |

5. SYNTHESIZE

Follow references/report-template.md: executive summary first; every key conclusion carries a confidence grade + citation ids; the reference table carries verdicts; unverified/unreachable live only in the "human review" section; finish with a finding ledger — revisit the plan's question list and mark each as answered / partially answered / unresolved (with the budget ledger: rounds used, remaining), then gaps, disagreements, follow-up questions.

Files

| File | When | |---|---| | scripts/verify_refs.py | VERIFY mechanical check (pure stdlib, cross-platform) | | references/GLOSSARY.md | canonical terms (five verdicts / signals / error codes) with Avoid lists | | references/out-of-scope.md | explicit non-goals + honest limits, each with a reason | | references/security.md | full security & behavior declaration | | docs/adr/ | architecture decision records (public layer) | | mcp/server.py | optional MCP server (same engine) | | references/search-strategies.md | read before SEARCH | | references/report-template.md | SYNTHESIZE skeleton; refs schema | | references/verification-details.md | full option semantics, thresholds | | references/anti-patterns.md | misuse catalog, each with an alternative | | references/faq.md | common questions | | references/medical-mode.md | read before medical/clinical work | | examples/demo_refs.json | 8 refs, 3 planted fabrications |

⚠️ Common mistakes

See references/anti-patterns.md (14 entries, each with the right way instead). The two that matter most: never treat verified as "the content is correct" — that is the semantic layer's job; never treat unreachable as "fake" — retry via --cn/--proxy first.

Security, honest limits, FAQ

• Security & behavior declaration: single-run; HTTPS-only to the official academic registries (doi.org / api.crossref.org / eutils.ncbi.nlm.nih.gov / export.arxiv.org / api.openalex.org / api.semanticscholar.org / archive.org / pubmed.ncbi.nlm.nih.gov); zero telemetry. Full text: references/security.md
• Honest limits & explicit non-goals: references/out-of-scope.md
• Common questions: references/faq.md

30-second quickstart

# 1. references as JSON (title/url/source/year) — or a Zotero .bib
# 2. verify:
python scripts/verify_refs.py --refs refs.json --out report.md
# 3. read report.md — CiteScore + pre-submission conclusion at the top.
# Paper? add --export bibtex.  Medical? add --profile medical.
# Already verified once? add --grill report.json for the review checklist.

Environment fallback

Without web tools: state honestly that only the "user-supplied material + mechanical verification" mode is possible; never pretend to search.