AI PIPELINE // PYTHON
GROUND_REPORT
A fully automated daily newsletter pipeline that researches, writes, edits, and delivers market intelligence across 7 independent industry verticals. Three AI agents — researcher, writer, and reader — collaborate to produce daily briefings with cognitive bias checking, cross-newsletter intelligence routing, and Polymarket prediction integration.
TECH_STACK
NEWSLETTER_VERTICALS
The Cattle Wire
Livestock Markets
The Load Board
Trucking & Freight
The Dose
Peptides
The Script
Pharmacy
The Brownstone
NYC Housing
The Lot Line
SF Housing
The Benchmark
Vancouver Housing
KEY_FEATURES
3-Agent AI Pipeline
Researcher searches via Exa.ai, Writer drafts with industry-specific voice, Reader grades with cognitive bias checking. Low grades trigger automatic rewrites (up to 2 revision passes).
Cross-Newsletter Intelligence
Signal routing between related industries — cattle corn data flows to trucking, pharmacy regulatory flags flow to peptides, housing rate signals flow to all markets.
Polymarket Integration
Paginates active Polymarket events, keyword-matches per newsletter, and injects prediction odds with industry-specific interpretation guides.
Cognitive Bias Detection
Reader agent checks for anchoring bias, signal vs noise confusion, narrative fallacy, false precision, and missing base rates before publishing.
SOURCE_CODE
async def run_single_newsletter(cfg, date, exa_session):
"""Full pipeline: research + signals -> write -> review -> publish"""
yesterday_summary = load_history(cfg.slug)
previous_urls = load_previous_urls(cfg.slug)
# Concurrent research + signal gathering
gather_tasks = [
research_industry(cfg, date, yesterday_summary,
exa_session=exa_session, previous_urls=previous_urls),
fetch_polymarket_signals(cfg, cached_events=cached_events),
]
if cfg.nadac_watchlist:
gather_tasks.append(fetch_nadac_prices(cfg.nadac_watchlist))
if cfg.clinical_trials_watchlist:
gather_tasks.append(fetch_peptide_trials(cfg.clinical_trials_watchlist))
results = await asyncio.gather(*gather_tasks)
# Write -> Review -> Rewrite loop
draft = await write_newsletter(cfg, research_data, date, ...)
feedback = await review_newsletter(cfg, draft, research_data)
if feedback["overall_grade"] != "publish_as_is":
newsletter = await rewrite_newsletter(cfg, draft, feedback)
# Second pass for low grades
if grade in ("significant_gaps", "needs_rewrite"):
feedback_v2 = await review_newsletter(cfg, newsletter)
newsletter = await rewrite_newsletter(cfg, newsletter, feedback_v2) SIGNAL_ROUTES = {
"cattle": {
"receives_from": ["trucking", "housing_ny", "housing_sf"],
"keywords": ["diesel", "transport cost", "consumer spending"],
},
"trucking": {
"receives_from": ["cattle", "housing_ny", "housing_sf"],
"keywords": ["corn", "feed hauling", "construction"],
},
"peptides": {
"receives_from": ["pharmacy"],
"keywords": ["FDA", "tariff", "compounding", "GLP-1"],
},
}
def get_cross_signals(slug):
"""Retrieve signals from other newsletters for this industry."""
routes = SIGNAL_ROUTES.get(slug, {})
for source_slug in routes.get("receives_from", []):
data = json.load(f"${CROSS_INTEL_DIR}/${source_slug}_latest.json")
for flag in data.get("delta_flags", []):
for kw in routes["keywords"]:
if kw.lower() in flag.lower():
relevant_signals.append(f"[from ${source_slug}] ${flag}") READER_FEEDBACK_TOOL = {
"name": "reader_feedback",
"input_schema": {
"properties": {
"overall_grade": {
"enum": ["publish_as_is", "minor_tweaks",
"significant_gaps", "needs_rewrite"],
},
"quality_score": {"type": "integer"},
"cognitive_check": {
"properties": {
"anchoring_appropriate": {"description": "Does it lead with the most decision-relevant number, or the most dramatic one?"},
"signal_vs_noise": {"description": "Does the writer distinguish signal from noise? Flag normal daily variation presented as signal."},
"narrative_fallacy": {"description": "Does the writer construct causal stories for every move? Flag invented explanations."},
"false_precision": {"description": "Any point estimates that should be ranges?"},
"base_rate_missing": {"description": "Any dramatic number presented without historical context?"},
},
},
"decision_test": {"description": "Can the reader make a BETTER decision today than without this newsletter?"},
},
},
} ARCHITECTURE
Research Phase
- Exa.ai web search per vertical
- Polymarket event pagination
- CMS NADAC drug pricing
- ClinicalTrials.gov + openFDA
- Source dedup via URL history
Writing Phase
- Industry-specific voice & persona
- Structured section templates
- Cross-newsletter signal injection
- Prediction extraction for scorecard
Review & Delivery
- Kahneman cognitive bias framework
- 4-tier grading with auto-rewrite
- Buttondown draft publishing
- GitHub Actions daily cron (5 AM EST)