System Documentation

Daily AI Digest

Automated content curation from Product Hunt and Reddit, powered by Claude Code

1

System Overview

The digest system runs daily at 9 PM via systemd timer. An orchestrator script manages Claude Code execution with retry logic and completion detection.

21:00 Trigger
→
Orchestrator
→
Claude Code
→
index.php
→
Telegram
⏰
Scheduled Execution
systemd timer triggers at 21:00 daily. Persistent flag ensures missed runs execute on next boot.
↻
Retry Logic
Up to 4 attempts, 20 minutes each. Process killed and restarted on timeout.
✓
Completion Detection
Polls COMPLETED-DAYS.md for today's date. Strict freshness check (2 hour window).
2

Orchestrator Flow

The orchestrator manages Claude Code execution, handling failures gracefully with automatic retries.

Start
Already completed?
YES
Exit 0
NO
Start Claude
Poll for 20min
COMPLETE
Write Ack
Exit 0
TIMEOUT
Kill Claude
Retry < 4?
Start/End
Process
Decision
Output
3

Digest Generation Pipeline

Claude Code reads CLAUDE.md and executes a 9-step pipeline to generate the daily digest.

0
Initialize
VPN, checkpoint
→
1
Product Hunt
Top 20 products
→
2
AI/LLMs
Reddit source 1
→
3
GenAI
Reddit source 2
→
4
Apps
Reddit source 3
→
5
Select
Merge & rank
→
6
Generate
HTML output
→
7-9
Finalize
Notify & signal
4

Data Sources

Content is discovered from primary sources only. WebSearch/WebFetch are used for verification, never discovery.

Product Hunt
GraphQL API
r/openai
Reddit MCP
r/LocalLLaMA
Reddit MCP
r/singularity
Reddit MCP
r/StableDiffusion
Reddit MCP
r/comfyui
Reddit MCP
r/macapps
Reddit MCP
r/selfhosted
Reddit MCP
Primary source (discovery)
5

Checkpoint System

All state is persisted to checkpoint.md after every action. This enables crash recovery and ensures data isn't lost if context runs out.

Fetch Data
→
Write to checkpoint.md
→
Verify Write
→
Continue
Phase Resume Point Description
starting Step 0 Initialize and scan previous digests
source-0-complete Step 2 Product Hunt done, start AI/LLMs
source-1-complete Step 3 AI/LLMs done, start GenAI
source-2-complete Step 4 GenAI done, start Apps
source-3-complete Step 5 All sources done, final selection
selection-complete Step 6 Selection done, generate HTML
complete Done All steps finished
Checkpoint write
6

File Structure

Key files and their purposes in the digest system.

CLAUDE.md
Execution instructions for Claude Code. The complete 9-step pipeline specification.
run-digest-orchestrator.sh
Bash orchestrator with retry logic, timeout handling, and completion detection.
COMPLETED-DAYS.md
Append-only ledger of completed digests. Orchestrator polls this for completion signal.
digest-archive/{date}/checkpoint.md
All state for a single digest. Raw data, analysis, decisions, URLs.
digest-archive/{date}/index.php
Final output. HTML digest with PHP feedback form handler.
DIGESTS/{source}/INTERESTS.md
Filtering criteria for each source. What to look for, what to skip.
DIGESTS/{source}/MEMORY.md
Long-term context. Current tech stack, developing stories, preferences.
logs/past-logs/{date}.md
Structured Markdown report per day. Attempt summaries, timing, outcomes.
7

Completion Signal

The orchestrator uses a strict two-part check to verify completion.

COMPLETED-DAYS.md
contains today's date
+
File mtime
< 2 hours ago
=
COMPLETE
📋
Why Two Checks?
The date line ensures the correct day. The freshness check prevents stale completions from previous runs being accepted.
🔒
Ack File
On success, orchestrator writes .orchestrator-ack-{date} to prevent re-runs on the same day.