A Research Agent That Reads 100+ Long-Form Sources Per Run
Not a GPT wrapper — a research analyst. Citation-backed briefing documents from podcasts, transcripts, blogs and video, with factual consistency measured rather than assumed.
- Client
- Podcast and content producer
- Industry
- Media production
100+
Sources per run
~70%
Less research time
~95%
Factual consistency
~60%
Less manual editing
The problem
Preparing a single long-form episode meant working through dozens of podcasts, transcripts, articles and videos, extracting what mattered, and assembling a briefing document that a host could actually use on air.
It was days of work per episode, and the quality varied with how tired the person doing it was. Generic AI summarisation had been tried and rejected: it produced fluent paragraphs that were subtly wrong, and on air a subtly wrong fact is worse than no fact.
What made this different from a wrapper
A GPT wrapper takes a source, asks for a summary, and returns prose. That fails at this job for two reasons: it cannot hold a hundred sources, and it cannot tell you where anything came from.
We built a retrieval pipeline instead. Sources are ingested, transcribed where needed, chunked and embedded into a vector store per run. Retrieval is grounded and scoped, composition is constrained to retrieved passages, and every claim in the output carries a citation back to the source and position it came from.
Output formatting is strict rather than free-form. The briefing document has a schema — themes, claims, supporting quotes, open questions, contradictions between sources — so the host receives a research artefact, not an essay.
Measuring factual consistency
The ~95% figure comes from checking generated claims against their cited passages rather than from asking whether the output looked right. That check runs as part of the pipeline, and claims that fail it are dropped rather than shipped with a caveat.
The dropped-claim rate turned out to be a useful signal in itself: a spike usually meant a source had been transcribed badly, which is worth knowing before the episode rather than during it.
Results
Each run processes 100+ long-form sources into a structured, citation-backed briefing. Research time fell roughly 70% and manual editing of the output fell roughly 60% — the second number matters because a research tool that produces work needing heavy editing has moved the labour rather than removed it.
The client's assessment: "He's a true professional that is wicked good at what he does. His work with me is going to save HOURS and thousands of dollars down the line leveraging the bot that he helped me create."
What we'd flag
Transcription quality caps everything downstream. Where audio was poor, retrieval was working from a corrupted source and no amount of prompt engineering recovers that — budget for good transcription before anything else.
The strict output schema was the single highest-value decision and the one most tempting to relax. Every request to make the briefing "more natural" was a request to make it harder to verify.
