# Daily AI Digest

An automated daily digest generator that curates content from Product Hunt and Reddit, filtered to your interests. Built to run with Claude Code as the AI agent.

## Overview

This system generates a personalized daily digest by:
1. Fetching top products from Product Hunt
2. Browsing relevant subreddits for AI, GenAI, and Apps/Software news
3. Filtering content based on your interests and current tech stack
4. Generating a styled HTML/PHP page with the curated items
5. Validating all links
6. Sending a Telegram notification when ready

## File Structure

```
daily-ai-digest/
├── CLAUDE.md                    # Execution instructions for Claude Code
├── README.md                    # This file
├── validate-links.sh            # Link validation script
├── SUGGESTIONS-FROM-USER.MD     # User feedback collected from digests
│
├── DIGESTS/                     # Source configuration
│   ├── 0-PRODUCTHUNT/
│   │   ├── INTERESTS.md         # What to look for on Product Hunt
│   │   ├── MEMORY.md            # Your current tech stack (for comparison)
│   │   └── fetch-producthunt.sh # GraphQL fetch script
│   │
│   ├── 1-AI-LLMS/
│   │   ├── INTERESTS.md         # AI/LLM topics of interest
│   │   ├── MEMORY.md            # AI subscriptions, context
│   │   └── SUBREDDITS.md        # r/openai, r/singularity, r/LocalLLaMA, r/artificial
│   │
│   ├── 2-GENAI-MEDIA/
│   │   ├── INTERESTS.md         # Image/video generation interests
│   │   ├── MEMORY.md            # Tools you use (ComfyUI, Midjourney, etc.)
│   │   └── SUBREDDITS.md        # r/StableDiffusion, r/midjourney, r/comfyui
│   │
│   └── 3-APPS-SOFTWARE/
│       ├── INTERESTS.md         # App categories of interest
│       ├── MEMORY.md            # Extensive list of installed apps
│       └── SUBREDDITS.md        # r/macapps, r/selfhosted, r/opensource, r/SideProject
│
├── digest-archive/              # Generated digests
│   └── YYYY-MM-DD/              # One folder per digest
│       ├── checkpoint.md        # State file (raw data, analysis, decisions)
│       └── index.php            # Final output with feedback form
│
└── WEB/             # Frontend assets
    ├── template.php             # Reference template for HTML generation
    ├── css/
    │   └── styles.css           # Digest styling
    ├── js/
    │   └── app.js               # Animations, favicon loading, theme
    └── assets/
        ├── producthunt-logo.svg
        └── reddit-logo.svg
```

## How It Works

### 1. Initialization
- Creates `digest-archive/YYYY-MM-DD/` folder
- Reads all `INTERESTS.md` and `MEMORY.md` files for filtering context
- Creates `checkpoint.md` to track progress

### 2. Product Hunt (Source 0)
- Runs `fetch-producthunt.sh` to get top 20 products via GraphQL API
- Filters based on interests (automation, AI, macOS, hardware, self-hosted)
- Deep analyzes candidates with WebFetch and WebSearch
- Saves decisions to checkpoint

### 3. Reddit Sources (1-3)
- Uses Reddit MCP to browse subreddits (`sort: top`, `time: week`)
- Filters posts against interests, skipping memes/complaints/duplicates
- Analyzes images (content is often in screenshots, not titles)
- Reads comments for context and corrections
- Extracts specific URLs (never generic profiles)

### 4. Final Selection
- Reviews all included items across sources
- Merges duplicates (same story from multiple subreddits)
- Selects 8-16 items total, quality over quantity
- Apps/Software section aims for 3-5 items minimum

### 5. HTML Generation
- Generates `index.php` using card templates
- Includes PHP feedback form handler
- Uses relative paths to `WEB/` for CSS/JS

### 6. Link Validation
- Runs `validate-links.sh` on generated file
- Fixes any broken (404) links via WebSearch
- **Verifies all 403 warnings** (especially X.com) via WebSearch
- X.com returns 403 for both bot blocking AND invalid URLs

### 7. Telegram Notification
- Sends notification to configured Telegram group
- Format: "📝 Daily digest ready" + description + link

## Automation Setup (Linux VPS)

The digest runs daily at 9:00 PM via systemd timer, using an orchestrator script with retry logic and completion detection.

### How It Works

1. Orchestrator starts Claude Code with the digest prompt
2. Polls `COMPLETED-DAYS.md` for today's date (completion signal)
3. If not complete within 20 minutes, kills Claude and retries (up to 4 attempts)
4. On success, writes ack file and exits
5. All attempts logged to `logs/raw/` and summarized in `logs/past-logs/{date}.md`

### Systemd Units

**Timer:** `/etc/systemd/system/digest-orchestrator.timer`
```ini
[Unit]
Description=Run Claude Digest Orchestrator daily at 21:00

[Timer]
OnCalendar=*-*-* 21:00:00
Persistent=true

[Install]
WantedBy=timers.target
```

**Service:** `/etc/systemd/system/digest-orchestrator.service`
```ini
[Unit]
Description=Claude Digest Orchestrator
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
User=agent
WorkingDirectory=/workspace/digest
ExecStart=/workspace/digest/run-digest-orchestrator.sh
```

### Management Commands

```bash
# Check timer status
sudo systemctl status digest-orchestrator.timer

# Check when next run is scheduled
sudo systemctl list-timers | grep digest

# Run manually (ignores today's ack if exists)
./run-digest-orchestrator.sh --ignore-ack

# Zero-token test (simulates Claude)
./run-digest-orchestrator.sh --simulate --ignore-ack --attempt-timeout-seconds 15
```

## Web Server Setup

### Nginx Configuration

```nginx
server {
    listen 80;
    server_name digest.yourdomain.com;
    root /path/to/daily-ai-digest;
    index index.php;

    location / {
        try_files $uri $uri/ =404;
    }

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock;
    }

    # Serve latest digest at root
    location = / {
        # Redirect to most recent digest
        rewrite ^ /digest-archive/latest/index.php redirect;
    }
}
```

### Create "Latest" Symlink (add to cron or service)

```bash
# After each digest generation, update the symlink
cd /path/to/daily-ai-digest/digest-archive
ln -sfn $(ls -d */ | sort -r | head -1 | tr -d '/') latest
```

## Configuration Files

### INTERESTS.md
Defines what topics to look for in each source. Example:
- Automation tools, CLI utilities, macOS apps
- New AI models, research breakthroughs
- Image/video generation workflows
- Self-hosted alternatives

### MEMORY.md
Your current tech stack for comparison. The system uses this to:
- Avoid recommending apps you already have
- Compare new tools against your existing solutions
- Provide context-aware summaries

### SUBREDDITS.md
List of subreddits to browse for each source category.

## Telegram Integration

Notifications are sent via Telegram Bot API when a digest completes.

### Bot Configuration
- **Bot Username:** @claudecode_tonkidev_bot
- **Chat ID:** -5063753582 (Daily Digest group)

### Message Format
```
📝 Daily digest ready

Your digest for December 27, 2025 is ready. Fresh picks from Product Hunt and Reddit.

🔗 https://tonki.dev/digest/digest-archive/2025-12-27/
```

## Checkpoint System

The `checkpoint.md` file tracks all state, enabling:
- **Crash recovery:** Resume from last completed step
- **Audit trail:** See what was fetched, filtered, and why
- **URL persistence:** All URLs are saved before HTML generation

### Phases
1. `starting` → Begin at Product Hunt
2. `source-0-complete` → Begin at Reddit AI-LLMs
3. `source-1-complete` → Begin at Reddit GenAI-Media
4. `source-2-complete` → Begin at Reddit Apps-Software
5. `source-3-complete` → Begin at Final Selection
6. `selection-complete` → Begin at HTML Generation
7. `complete` → Done

## Link Validation

The `validate-links.sh` script checks all `<a href>` URLs:

```bash
./validate-links.sh digest-archive/2025-12-27/index.php
```

### Output
- **OK (200, 301, 302):** Link works
- **BROKEN (404, 401, etc.):** Must fix with WebSearch
- **WARNING (403):** Must verify with WebSearch (especially X.com)

### Why Verify 403s?
X.com/Twitter returns 403 for both:
- Bot protection (link is valid)
- Invalid URLs (link is wrong)

Always search to confirm the URL matches what you expect.

## Feedback System

Each digest includes a feedback form that saves to `SUGGESTIONS-FROM-USER.MD`:
- Users can suggest changes for future digests
- Feedback includes digest date and timestamp
- Review suggestions to improve filtering

## API Keys Required

| Service | Purpose | Location |
|---------|---------|----------|
| Anthropic | Claude Code API | Environment variable |
| Product Hunt | GraphQL API | `fetch-producthunt.sh` |
| Reddit | MCP Server | Claude Code MCP config |
| Telegram | Notifications | `CLAUDE.md` Step 8 |

## Local Development

```bash
# Start PHP server
php -S localhost:8000

# View latest digest
open https://tonki.dev/digest/digest-archive/2025-12-27/

# Validate links
./validate-links.sh digest-archive/2025-12-27/index.php
```

## Manual Execution

```bash
# Run via orchestrator (recommended - has retry logic)
./run-digest-orchestrator.sh --ignore-ack

# Run for a specific date
./run-digest-orchestrator.sh --ignore-ack --date-iso 2025-12-27

# Direct Claude call (no retries, no completion detection)
claude "Read CLAUDE.md and generate the digest for December 27, 2025"
```

## Version History

- **v1.1** - Orchestrator with retry logic, completion detection, systemd automation
- **v0.9** - Complete system before VPS deployment
- **v0.7** - Frontend with Product Hunt section
- **v0.5** - Initial frontend + project structure

## License

Private project. Not for redistribution.
