# CLI Pipeline Architecture: Article/YouTube Summarizer

## Context
Building a Python CLI tool that processes URLs (articles or YouTube videos) through a pipeline of nodes. Each node extracts/transforms data, saves to a temp file, then passes context to the next node.

---

## Architecture Overview

### Pipeline Flow

```
┌─────────┐
│   URL   │ ──→ validate & detect type (regex)
└─────────┘
    ↓
┌───────────────────────────┐
│   Type Router             │
│   ├─ YouTube URL → yt-dlp │
│   └─ Article URL → fetch  │
└───────────────────────────┘
    ↓
┌─────────────────────────────┐
│  Node 1: Metadata Extractor │
│  (yt-dlp metadata + content)│
│  ↓ saves to temp file       │
└─────────────────────────────┘
    ↓
┌────────────────────────┐
│  Node 2: Summarizer    │
│  (Ollama: summary)     │
│  ↓ reads temp file     │
│  ↓ saves to temp file  │
└────────────────────────┘
    ↓
┌───────────────────────┐
│  Node 3: Mindmap      │
│  (Ollama: mindmap)    │
│  ↓ reads temp file    │
│  ↓ saves to temp file │
└───────────────────────┘
    ↓
┌────────────────────────┐
│  Node 4: TTS Generator │
│  (Ollama: tts)         │
│  ↓ reads temp file     │
│  ↓ saves to temp file  │
└────────────────────────┘
    ↓
┌────────────────────────────┐
│  Node 5: Markdown Renderer │
│  (assembles all outputs)   │
└────────────────────────────┘
```

---

## Node Details

### Node 1: Metadata Extractor
**Input:** YouTube URL (extracted via regex)
**Tools:** `yt-dlp` metadata API
**Output files:**
- `temp_meta.json` - metadata (title, author, description, duration)
- `temp_content.txt` - full transcript/text content
**Error handling:** Stop if yt-dlp fails (invalid URL, private video, etc.)

### Node 2: Summarizer
**Input:** `temp_meta.json` + `temp_content.txt`
**Tools:** API call
**Output files:**
- `temp_summary.txt` - brief summary paragraph

```python
import requests

def ask_gemma4(prompt: str, model: str = "gemma4") -> str:
    response = requests.post(
        "http://localhost:11434/api/generate",
        json={
            "model": model,
            "prompt": prompt,
            "stream": False
        }
    )
    response.raise_for_status()
    return response.json()["response"]

result = ask_gemma4("Summarize")
print(result)
```



### Node 3: Mindmap Creator
**Input:** `temp_summary.txt` + `temp_content.txt`
**Tools:** Ollama API call
**Output files:**
- `temp_mindmap.txt` - structured mindmap (markdown hierarchy)

```python
import requests

def ask_gemma4(prompt: str, model: str = "gemma4") -> str:
    response = requests.post(
        "http://localhost:11434/api/generate",
        json={
            "model": model,
            "prompt": prompt,
            "stream": False
        }
    )
    response.raise_for_status()
    return response.json()["response"]

result = ask_gemma4("Generate a mindmap")
print(result)
```


### Node 4: TTS Generator
**Input:** `temp_summary.txt` (or full content if preferred)
**Tools:** OpenAI API call
**Output files:**
- `temp_audio.wav` or `.mp3` - audio file

```bash
openai_audio_speech() {
  # $1: request body
  # $2: output file
  curl -s --fail-with-body \
    -X POST "https://api.openai.com/v1/audio/speech" \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -H "Content-Type: application/json" \
    -d "$1" \
    --output "$2"
  local exit=$?
  if [ $exit -ne 0 ]; then
    error_message "An error occured while processing the request"
    return 1
  fi
  return 0
}
# needs to be converted to Python requests call
```


### Node 5: Markdown Renderer
**Input:** All temp files
**Output:** Single markdown file with:
- Title, author, duration (if YouTube)
- Summary section
- Mindmap section
- Audio player link (local path)

---

## File Structure

```
summarize/
├── summarize.py          # Main CLI entry point
├── nodes/                 # Node implementations
│   ├── __init__.py
│   ├── base.py           # Abstract base class for nodes
│   ├── router.py         # URL type detection & routing
│   ├── metadata.py       # yt-dlp extraction
│   ├── summarizer.py     # Ollama summary
│   ├── mindmap.py        # Ollama mindmap
│   ├── tts.py            # Ollama TTS
│   └── renderer.py       # Markdown assembly
├── temp/                  # Temp directory for intermediate files
└── requirements.txt       # Dependencies
```

---

## Key Design Patterns

### 1. Temp File Pipeline
Each node:
1. Reads from input temp files
2. Processes data
3. Writes to output temp files
4. Passes file paths to next node

### 2. Abstract Base Class
```python
class Node(ABC):
    @abstractmethod
    def execute(self, temp_dir: str) -> dict[str, str]:
        """Execute node and return list of output file paths"""
```

### 3. URL Detection
```python
# Auto-detect YouTube URLs using regex
YOUTUBE_PATTERNS = [
    r'https?://www\.youtube\.com/watch\?v=\S+',
    r'https?://youtu\.be/\S+',
    r'https?://m\.youtube\.com/watch\?v=\S+',
]
```

### 4. Error Propagation
- Each node catches its own exceptions
- Raises `PipelineError` on failure
- Main loop stops on first exception
- Prints error message and exits with code 1

---

## Dependencies

```text
yt-dlp>=2024.4.0
requests>=2.28.0
```

---

## Verification Plan

1. Test with YouTube URL:
   ```bash
   python summarize.py https://www.youtube.com/watch?v=dQw4w9WgXcQ
   ```

2. Test with article URL:
   ```bash
   python summarize.py https://blog.annimon.com/tui-tiki/
   ```

3. Verify temp files created in `./temp/`

4. Verify markdown output contains all sections

---

## Critical Files

- `summarize.py` - CLI entry point
- `nodes/base.py` - Node abstraction
- `nodes/router.py` - URL detection
- `nodes/metadata.py` - yt-dlp extraction
- `nodes/summarizer.py` - Ollama API call summary
- `nodes/mindmap.py` - Ollama mindmap
- `nodes/tts.py` - OpenAI API call TTS
- `nodes/renderer.py` - Markdown assembly