Mark Hrytchak Submission Studio 1

studio/markhryt/code/README.md

@@ -0,0 +1,80 @@
+# AI News Digest — recurring ingest → summarize → output job
+
+Bounded, recurring job that watches three pages, summarizes what is new, and writes a digest:
+
+| Source | Requested page | What the job actually reads | Full text? |
+|---|---|---|---|
+| Anthropic News | https://www.anthropic.com/news | the listing HTML, then each new article page | yes |
+| Anthropic Engineering | https://www.anthropic.com/engineering | the listing HTML, then each new article page | yes |
+| OpenAI News | https://openai.com/news/ | https://openai.com/news/rss.xml | no — openai.com serves HTTP 403 to non-browser clients, so the feed's title + description is the input |
+
+## Pipeline
+
+```
+ingest  (newsdigest/ingest.py)   listing → Article records → dedupe against outputs/state.json → cap N newest/source → fetch bodies
+summary (newsdigest/summarize.py) ClaudeSummarizer (structured JSON output) or ExtractiveSummarizer (offline fallback)
+output  (newsdigest/output.py)    outputs/digests/<ts>.md + .json, outputs/latest.md, outputs/runs.jsonl (one trace line per run)
+```
+
+`newsdigest/job.py` orchestrates one run and the in-process loop; `run_job.py` is the CLI.
+
+## Run
+
+```bash
+cd studio/markhryt/code
+python3 -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt   # only needed for LLM summaries
+export ANTHROPIC_API_KEY=...        # or `ant auth login`; never commit a key
+
+python3 run_job.py --dry-run        # list what would be ingested; writes nothing
+python3 run_job.py --no-llm         # offline run: extractive summaries, stdlib only
+python3 run_job.py                  # real run: Claude summaries (claude-opus-5 by default)
+python3 run_job.py --loop --interval 6h            # keep running in the foreground
+python3 run_job.py --sources openai-news --max-per-source 5 --force
+```
+
+Flags: `--max-per-source N` (default 10), `--model`, `--force` (ignore seen-state), `--always-write`
+(write a digest even when nothing is new), `--outputs-dir` (default `../outputs`), `-v`.
+
+Exit code is 0 for `ok` / `ok-empty` / `dry-run`, 2 for `degraded` (a source failed, or no
+credentials so summaries are extractive) or `error` (every source failed).
+
+## Scheduling
+
+- In-process: `python3 run_job.py --loop --interval 6h`.
+- cron: see `schedule/crontab.example`.
+- macOS launchd: see `schedule/com.markhryt.newsdigest.plist`.
+
+Whichever you use, credentials must come from the environment of the scheduled process.
+
+## What a run records
+
+- `outputs/latest.md` — the most recent digest (overview, themes, per-source items with
+  headline, summary, "why it matters", tags).
+- `outputs/digests/` — every digest as Markdown and JSON.
+- `outputs/runs.jsonl` — one line per run: status, listed/new counts per source, source errors,
+  body-fetch failures, summarizer method and model, token usage, output paths, duration, notes.
+- `outputs/logs/job.log` — full log.
+- `outputs/state.json` — seen URLs. Everything listed in a run is marked seen (only the newest
+  N per source are summarized), so old backlog does not trickle into later digests.
+
+## Failure behavior (bounded by design)
+
+- A single source failing → the run continues with the others and is marked `degraded`.
+- An article body failing to fetch → its listing teaser is summarized and the digest says so.
+- No credentials / SDK → `degraded`, extractive summaries, clear note in the output and run log.
+- The model declining a request or returning malformed JSON → that batch falls back to extractive.
+- All sources failing → `error`, nothing written except the run-log line, state untouched.
+
+## Tests
+
+```bash
+cd studio/markhryt/code && python3 -m unittest -v test_newsdigest
+```
+
+Offline only: fixture HTML/RSS, a fake fetcher for end-to-end runs, no network or API key.
+
+## Cost note
+
+Summaries use `claude-opus-5` (adaptive thinking, effort `medium`, structured output). A full
+run of 30 new articles is roughly 100–150k input tokens across 4–5 requests; the system prompt
+is cache-marked. Lower `--max-per-source` or pass `--model claude-sonnet-5` to spend less.

studio/markhryt/code/newsdigest/__init__.py

@@ -0,0 +1,3 @@
+"""AI news digest: ingest -> summarize -> output for Anthropic and OpenAI news pages."""
+
+__all__ = ["config", "models", "ingest", "summarize", "output", "state", "job"]

studio/markhryt/code/newsdigest/config.py

@@ -0,0 +1,71 @@
+"""Static configuration: sources, paths, model."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+
+CODE_DIR = Path(__file__).resolve().parent.parent
+STUDENT_DIR = CODE_DIR.parent
+DEFAULT_OUTPUT_DIR = STUDENT_DIR / "outputs"
+
+# Default summarization model. Override with --model or NEWSDIGEST_MODEL.
+DEFAULT_MODEL = "claude-opus-5"
+
+# A browser-like UA: anthropic.com serves plain HTML to it; openai.com's HTML
+# pages sit behind a bot check (HTTP 403) regardless, so that source uses RSS.
+USER_AGENT = (
+    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 "
+    "(KHTML, like Gecko) Chrome/128.0 Safari/537.36"
+)
+FETCH_TIMEOUT_SECONDS = 25
+FETCH_RETRIES = 2
+
+# Per-article body cap sent to the model (characters). Article pages on
+# anthropic.com run 15-25k characters, so this rarely truncates.
+MAX_BODY_CHARS = 40_000
+# Articles per summarization request.
+SUMMARY_BATCH_SIZE = 8
+
+
+@dataclass(frozen=True)
+class Source:
+    key: str
+    name: str
+    url: str          # the page the job is about (what the user asked for)
+    kind: str         # "anthropic_listing" | "rss"
+    fetch_url: str    # what we actually download for the listing
+    path_prefix: str = ""   # article href prefix on anthropic_listing pages
+    fetch_bodies: bool = True  # whether individual article pages are fetchable
+
+
+SOURCES: tuple[Source, ...] = (
+    Source(
+        key="anthropic-news",
+        name="Anthropic News",
+        url="https://www.anthropic.com/news",
+        kind="anthropic_listing",
+        fetch_url="https://www.anthropic.com/news",
+        path_prefix="/news/",
+    ),
+    Source(
+        key="anthropic-engineering",
+        name="Anthropic Engineering",
+        url="https://www.anthropic.com/engineering",
+        kind="anthropic_listing",
+        fetch_url="https://www.anthropic.com/engineering",
+        path_prefix="/engineering/",
+    ),
+    Source(
+        key="openai-news",
+        name="OpenAI News",
+        url="https://openai.com/news/",
+        kind="rss",
+        fetch_url="https://openai.com/news/rss.xml",
+        # openai.com article pages return 403 to non-browser clients, so the
+        # feed's title + description is the ingested text for this source.
+        fetch_bodies=False,
+    ),
+)
+
+SOURCES_BY_KEY = {s.key: s for s in SOURCES}

studio/markhryt/code/newsdigest/http.py

@@ -0,0 +1,58 @@
+"""Minimal HTTP fetch with retries (stdlib only)."""
+
+from __future__ import annotations
+
+import gzip
+import logging
+import time
+import urllib.error
+import urllib.request
+from typing import Callable
+
+from .config import FETCH_RETRIES, FETCH_TIMEOUT_SECONDS, USER_AGENT
+
+log = logging.getLogger(__name__)
+
+Fetcher = Callable[[str], str]
+
+
+class FetchError(Exception):
+    def __init__(self, url: str, reason: str, status: int | None = None):
+        super().__init__(f"{url}: {reason}")
+        self.url = url
+        self.reason = reason
+        self.status = status
+
+
+def fetch(url: str, timeout: float = FETCH_TIMEOUT_SECONDS, retries: int = FETCH_RETRIES) -> str:
+    """GET `url` and return decoded text. Retries on 5xx / network errors, not on 4xx."""
+    req = urllib.request.Request(
+        url,
+        headers={
+            "User-Agent": USER_AGENT,
+            "Accept": "text/html,application/xhtml+xml,application/xml,text/xml;q=0.9,*/*;q=0.8",
+            "Accept-Language": "en-US,en;q=0.9",
+            "Accept-Encoding": "gzip",
+        },
+    )
+    last: Exception | None = None
+    for attempt in range(retries + 1):
+        try:
+            with urllib.request.urlopen(req, timeout=timeout) as resp:
+                raw = resp.read()
+                if resp.headers.get("Content-Encoding", "").lower() == "gzip":
+                    raw = gzip.decompress(raw)
+                charset = resp.headers.get_content_charset() or "utf-8"
+                return raw.decode(charset, errors="replace")
+        except urllib.error.HTTPError as e:
+            if 400 <= e.code < 500:
+                raise FetchError(url, f"HTTP {e.code}", e.code) from e
+            last = FetchError(url, f"HTTP {e.code}", e.code)
+        except (urllib.error.URLError, TimeoutError, OSError) as e:
+            last = FetchError(url, f"{type(e).__name__}: {e}")
+        if attempt < retries:
+            delay = 1.5 * (attempt + 1)
+            log.warning("fetch retry %d for %s after %s", attempt + 1, url, last)
+            time.sleep(delay)
+    assert last is not None
+    raise last