Skip to content
BACK_TO_INDEX

AI PIPELINE // PYTHON

GROUND_REPORT

A fully automated daily newsletter pipeline that researches, writes, edits, and delivers market intelligence across 7 independent industry verticals. Three AI agents — researcher, writer, and reader — collaborate to produce daily briefings with cognitive bias checking, cross-newsletter intelligence routing, and Polymarket prediction integration.

PYTHON ANTHROPIC CLAUDE EXA.AI POLYMARKET NEWSLETTER MULTI-AGENT
Abstract data flow visualization with glowing lines

TECH_STACK

Python Anthropic Claude Exa.ai aiohttp asyncio Buttondown API Polymarket API CMS NADAC API ClinicalTrials.gov GitHub Actions tweepy

NEWSLETTER_VERTICALS

The Cattle Wire

Livestock Markets

The Load Board

Trucking & Freight

The Dose

Peptides

The Script

Pharmacy

The Brownstone

NYC Housing

The Lot Line

SF Housing

The Benchmark

Vancouver Housing

KEY_FEATURES

3-Agent AI Pipeline

Researcher searches via Exa.ai, Writer drafts with industry-specific voice, Reader grades with cognitive bias checking. Low grades trigger automatic rewrites (up to 2 revision passes).

Cross-Newsletter Intelligence

Signal routing between related industries — cattle corn data flows to trucking, pharmacy regulatory flags flow to peptides, housing rate signals flow to all markets.

Polymarket Integration

Paginates active Polymarket events, keyword-matches per newsletter, and injects prediction odds with industry-specific interpretation guides.

Cognitive Bias Detection

Reader agent checks for anchoring bias, signal vs noise confusion, narrative fallacy, false precision, and missing base rates before publishing.

SOURCE_CODE

main.py PIPELINE ORCHESTRATION
async def run_single_newsletter(cfg, date, exa_session):
    """Full pipeline: research + signals -> write -> review -> publish"""
    yesterday_summary = load_history(cfg.slug)
    previous_urls = load_previous_urls(cfg.slug)

    # Concurrent research + signal gathering
    gather_tasks = [
        research_industry(cfg, date, yesterday_summary,
            exa_session=exa_session, previous_urls=previous_urls),
        fetch_polymarket_signals(cfg, cached_events=cached_events),
    ]
    if cfg.nadac_watchlist:
        gather_tasks.append(fetch_nadac_prices(cfg.nadac_watchlist))
    if cfg.clinical_trials_watchlist:
        gather_tasks.append(fetch_peptide_trials(cfg.clinical_trials_watchlist))

    results = await asyncio.gather(*gather_tasks)

    # Write -> Review -> Rewrite loop
    draft = await write_newsletter(cfg, research_data, date, ...)
    feedback = await review_newsletter(cfg, draft, research_data)

    if feedback["overall_grade"] != "publish_as_is":
        newsletter = await rewrite_newsletter(cfg, draft, feedback)
        # Second pass for low grades
        if grade in ("significant_gaps", "needs_rewrite"):
            feedback_v2 = await review_newsletter(cfg, newsletter)
            newsletter = await rewrite_newsletter(cfg, newsletter, feedback_v2)
signals/cross_intelligence.py SIGNAL ROUTING
SIGNAL_ROUTES = {
    "cattle": {
        "receives_from": ["trucking", "housing_ny", "housing_sf"],
        "keywords": ["diesel", "transport cost", "consumer spending"],
    },
    "trucking": {
        "receives_from": ["cattle", "housing_ny", "housing_sf"],
        "keywords": ["corn", "feed hauling", "construction"],
    },
    "peptides": {
        "receives_from": ["pharmacy"],
        "keywords": ["FDA", "tariff", "compounding", "GLP-1"],
    },
}

def get_cross_signals(slug):
    """Retrieve signals from other newsletters for this industry."""
    routes = SIGNAL_ROUTES.get(slug, {})
    for source_slug in routes.get("receives_from", []):
        data = json.load(f"${CROSS_INTEL_DIR}/${source_slug}_latest.json")
        for flag in data.get("delta_flags", []):
            for kw in routes["keywords"]:
                if kw.lower() in flag.lower():
                    relevant_signals.append(f"[from ${source_slug}] ${flag}")
agents/reader.py COGNITIVE BIAS CHECKER
READER_FEEDBACK_TOOL = {
    "name": "reader_feedback",
    "input_schema": {
        "properties": {
            "overall_grade": {
                "enum": ["publish_as_is", "minor_tweaks",
                        "significant_gaps", "needs_rewrite"],
            },
            "quality_score": {"type": "integer"},
            "cognitive_check": {
                "properties": {
                    "anchoring_appropriate": {"description": "Does it lead with the most decision-relevant number, or the most dramatic one?"},
                    "signal_vs_noise":      {"description": "Does the writer distinguish signal from noise? Flag normal daily variation presented as signal."},
                    "narrative_fallacy":     {"description": "Does the writer construct causal stories for every move? Flag invented explanations."},
                    "false_precision":       {"description": "Any point estimates that should be ranges?"},
                    "base_rate_missing":     {"description": "Any dramatic number presented without historical context?"},
                },
            },
            "decision_test": {"description": "Can the reader make a BETTER decision today than without this newsletter?"},
        },
    },
}

ARCHITECTURE

Research Phase

  • Exa.ai web search per vertical
  • Polymarket event pagination
  • CMS NADAC drug pricing
  • ClinicalTrials.gov + openFDA
  • Source dedup via URL history

Writing Phase

  • Industry-specific voice & persona
  • Structured section templates
  • Cross-newsletter signal injection
  • Prediction extraction for scorecard

Review & Delivery

  • Kahneman cognitive bias framework
  • 4-tier grading with auto-rewrite
  • Buttondown draft publishing
  • GitHub Actions daily cron (5 AM EST)
BACK_TO_INDEX