# Testing Log

Status: `executed`.

This file records real execution tests for Phase 1 scripts:
- `PHASES/PHASE-01-INGEST/scrape_slot.sh`
- `PHASES/PHASE-01-INGEST/process_day.sh`

## 1) Test Goal

Validate that both Day 1 scripts run end-to-end and produce expected artifacts:
1. A slot run CSV + merged `raw_feed.csv`
2. Processed outputs (`with_images.csv`, `no_images.csv`, `manifest.json`, `images/*`)

## 2) Test Date and Context

- Execution date: `2026-02-10`
- Server time zone at execution: `UTC-06:00`
- Repo root: `/workspace/twitter-digest`
- Cookies file present: `PHASES/PHASE-01-INGEST/cookies.json`

## 3) Commands Executed

### 3.1 Scrape test (fast controlled run)

```bash
SCRAPE_BATCHES=1 SCRAPE_DELAY=1-1 bash PHASES/PHASE-01-INGEST/scrape_slot.sh morning 2026-02-10
```

Reason for overrides:
- `SCRAPE_BATCHES=1`: fast validation run.
- `SCRAPE_DELAY=1-1`: avoids long waits.

### 3.2 Process test

```bash
bash PHASES/PHASE-01-INGEST/process_day.sh 2026-02-10
```

## 4) Runtime Results

### 4.1 `scrape_slot.sh` result

- Exit code: `0`
- Captured tweets in run CSV: `31`
- Merged `raw_feed.csv` unique rows: `31`
- Duplicate rows removed during merge: `0`

Primary output files:
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/raw_feed_runs/1519_morning.csv`
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/raw_feed.csv`

Log file:
- `PHASES/PHASE-01-INGEST/logs/scrape/test_2026-02-10_morning.log`

### 4.2 `process_day.sh` result

- Exit code: `0`
- Tweets processed from `raw_feed.csv`: `31`
- `with_images.csv` rows: `17` data rows (`18` lines including header)
- `no_images.csv` rows: `14` data rows (`15` lines including header)
- Downloaded images: `18`
- Manifest download failures: `0`

Primary output files:
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/with_images.csv`
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/no_images.csv`
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/manifest.json`
- `PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/images/*`

Log file:
- `PHASES/PHASE-01-INGEST/logs/process/test_2026-02-10.log`

## 5) Artifact Verification Commands Used

```bash
DATE=2026-02-10
wc -l PHASES/PHASE-01-INGEST/tweets/archive/$DATE/raw_feed.csv
wc -l PHASES/PHASE-01-INGEST/tweets/archive/$DATE/with_images.csv
wc -l PHASES/PHASE-01-INGEST/tweets/archive/$DATE/no_images.csv
find PHASES/PHASE-01-INGEST/tweets/archive/$DATE/images -type f | wc -l
python3 - <<'PY'
import json
from pathlib import Path
p=Path("PHASES/PHASE-01-INGEST/tweets/archive/2026-02-10/manifest.json")
m=json.loads(p.read_text())
print(m["total_tweets"], m["tweets_with_images"], m["tweets_without_images"], m["total_images"], m["download_failures"])
PY
```

Observed values:
- `raw_feed.csv` lines: `32` (`31` data + header)
- `with_images.csv` lines: `18` (`17` data + header)
- `no_images.csv` lines: `15` (`14` data + header)
- image files on disk: `18`
- manifest counters: `31, 17, 14, 18, 0`

## 6) Errors and Warnings

- No blocking errors.
- No script-level failures observed.
- No data integrity mismatch observed between CSV counts and `manifest.json`.

## 7) Limits of This Test

- This was a single-slot validation (`morning`) with reduced batch/delay settings.
- It does not yet validate:
  - full 3-slot daily accumulation (`morning` + `midday` + `evening`)
  - cron-triggered execution at scheduled times
  - downstream Phase 2+ pipeline

## 8) Next Test to Run

1. Run full-day simulation manually:
   - `scrape_slot.sh morning DATE`
   - `scrape_slot.sh midday DATE`
   - `scrape_slot.sh evening DATE`
   - `process_day.sh DATE` after each desired checkpoint
2. Install cron bundle from `CRON.md`.
3. Validate logs and artifacts after first real scheduled day.
