Lesson 32: Delta Summarization — Updating Summaries Without Reprocessing Everything
What We’re Building Today
A checkpoint tracker that records
last_summarized_atper topic, enabling incremental fetches of only new contentA two-phase LLM pipeline: delta summarizer compresses new tweets, then a merge call folds the delta into the existing summary
A token savings calculator and semantic equivalence test that verifies the merged result is substantively identical to a full re-summarization at a measured fraction of the cost
Why This Matters
In 2015, Slack’s post-mortem on their notification digest system revealed a pathological pattern: as channel volume grew, their summarization jobs were re-reading every message in a channel to regenerate digests. At modest scale this was invisible. Past a threshold, it became a feedback loop — digests delayed, workers backed up, users got stale summaries of conversations that had already resolved. The fix wasn’t faster hardware; it was checkpoint-based incremental processing. NEXUS faces the same problem. If every trending topic summary requires scanning the full tweet history every minute, you will burn LLM tokens proportional to
O(corpus × topics × frequency). At 1,000 topics refreshing every minute, that’s not a latency problem — it’s a cost cliff you fall off suddenly.
Core Concepts
