# Reddit Digest System - Technical Specification

## Executive Summary

Automated Reddit curation system that runs daily via Claude Code on a VPS. Analyzes specific subreddits, filters content based on user interests, and generates a daily newsletter-style web digest.

---

## General Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                           VPS                                    │
│                                                                  │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────────┐  │
│  │   Cron Job   │───▶│  Claude Code │───▶│  SQLite + PHP    │  │
│  │  (8-9 PM)    │    │   + Reddit   │    │   (Web Digest)   │  │
│  │              │    │     MCP      │    │                  │  │
│  └──────────────┘    └──────────────┘    └──────────────────┘  │
│                             │                                    │
│                             ▼                                    │
│                    ┌──────────────┐                             │
│                    │  /DIGESTS/   │                             │
│                    │  structure   │                             │
│                    └──────────────┘                             │
└─────────────────────────────────────────────────────────────────┘
```

---

## Directory Structure

```
/reddit-mcp/
├── SPEC.md                              # This file
├── CLAUDE.md                            # Instructions for Claude Code agent
│
├── DIGESTS/
│   ├── 1-AI-LLMS/
│   │   ├── INTERESTS.md                 # What to look for in this category
│   │   ├── MEMORY.md                    # Persistent context + developing topics
│   │   ├── SUBREDDITS.md                # Subreddits to monitor (full URLs)
│   │   └── ENTRIES/
│   │       ├── 2026-01-15.md            # Daily digest (with checkpoint)
│   │       └── ...
│   │
│   ├── 2-GENAI-MEDIA/
│   │   ├── INTERESTS.md
│   │   ├── MEMORY.md
│   │   ├── SUBREDDITS.md
│   │   └── ENTRIES/
│   │
│   └── 3-APPS-SOFTWARE/
│       ├── INTERESTS.md
│       ├── MEMORY.md
│       ├── SUBREDDITS.md
│       └── ENTRIES/
│
├── ARCHIVE/
│   ├── CHECKPOINT-EXAMPLE.md            # Example of checkpoint format
│   └── EXAMPLES/
│       └── FORMAT-OPTIONS.md            # Different format styles to choose from
│
└── web/
    ├── index.php                        # List of digests
    ├── entry.php                        # Single digest view
    ├── schema.sql                       # Database schema
    └── digest.sqlite                    # Database
```

---

## The 3 Categories

### 1. AI-LLMS
**Focus**: News, launches, techniques, and AI/LLM tools
**Typical subreddits**: r/openai, r/anthropic, r/LocalLLaMA, r/MachineLearning, r/ClaudeAI

### 2. GENAI-MEDIA
**Focus**: Image, video, and audio generation with AI
**Typical subreddits**: r/StableDiffusion, r/midjourney, r/comfyui, r/runwayml, r/singularity

### 3. APPS-SOFTWARE
**Focus**: Mac/iPhone apps, new software, self-hosting, productivity
**Typical subreddits**: r/macapps, r/iosapps, r/selfhosted, r/opensource, r/SideProject

---

## Execution Flow

See `CLAUDE.md` for detailed step-by-step instructions. Summary:

1. **Initialize**: Check for existing checkpoint, read configuration files
2. **Fetch**: Get top posts from each subreddit via Reddit MCP
3. **Analyze**: Review all content (title + image + text), identify candidates
4. **Deep Dive**: For candidates, analyze images, fetch links, search web for context
5. **Merge**: Consolidate duplicate topics across subreddits
6. **Select**: Choose 2-5 best items total
7. **Generate**: Create digest with checkpoint
8. **Save**: Insert into SQLite
9. **Update**: Update MEMORY.md if there are developing topics

---

## Checkpoint System

The daily entry file (`DIGESTS/1-AI-LLMS/ENTRIES/2026-01-15.md`) serves as checkpoint.

See `ARCHIVE/CHECKPOINT-EXAMPLE.md` for detailed format.

Key points:
- Checkpoint is at the top of the file
- Updated after each significant operation
- If process interrupts, next execution continues from last checkbox

---

## Resilience

- **Retries**: Until completion (no limit), with exponential backoff
- **Checkpoint**: State saved in the daily .md file
- **Context exhausted**: Process terminates, cron re-executes and continues from checkpoint
- **Network errors**: Retry with backoff
- **Reddit rate limits**: Respect, wait, continue

---

## SQLite Database

### Table: `digests`
```sql
CREATE TABLE digests (
    id INTEGER PRIMARY KEY,
    date TEXT UNIQUE NOT NULL,
    content_md TEXT NOT NULL,
    items_count INTEGER,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### Table: `processed_posts`
```sql
CREATE TABLE processed_posts (
    id INTEGER PRIMARY KEY,
    reddit_id TEXT UNIQUE NOT NULL,
    processed_date TEXT NOT NULL,
    category INTEGER NOT NULL
);
```

---

## Web (PHP)

### `index.php`
- Lists all digests ordered by date
- Link to each entry

### `entry.php?date=2026-01-15`
- Renders the digest markdown as HTML
- Clean newsletter style

---

## Content Format

**Principle**: High signal, no noise. Each item should be concise but complete.

See `ARCHIVE/EXAMPLES/FORMAT-OPTIONS.md` for format styles.

---

## Configuration Files

### INTERESTS.md (per category)
Narrative markdown describing:
- What topics interest you
- What type of posts to look for
- What to ignore
- Desired depth level

### MEMORY.md (per category)
- **Personal context**: Apps installed, tools tried, current stack
- **Developing topics**: Stories/launches to follow
- **Notes**: Any relevant context

### SUBREDDITS.md (per category)
```markdown
# Subreddits: [Category name]

- https://www.reddit.com/r/openai
- https://www.reddit.com/r/anthropic
```
(Full URLs, one per line)

---

## Execution (Cron)

```bash
# Run every day at 8:30 PM
30 20 * * * cd /path/to/reddit-mcp && claude-code CLAUDE.md
```

(Exact command depends on Claude Code VPS configuration)

---

## Implementation Notes

1. **Images are critical**: Reddit has many posts where the title is vague and the image contains all the information. ALWAYS analyze images.

2. **Aggressive web search**: For news/launches, always search for additional context. The value of the digest is having complete information, not just mentions.

3. **Smart merge**: Subreddits in the same category will have significant overlap. Detect and consolidate.

4. **2-5 items is the limit**: Better fewer items with more depth than many superficial items.

5. **The .md file is the checkpoint**: Don't over-engineer the state system. Markdown with checklist is sufficient and human-readable.
