diff --git a/docs/superpowers/specs/2026-03-19-vod-merge-split-design.md b/docs/superpowers/specs/2026-03-19-vod-merge-split-design.md new file mode 100644 index 0000000..14b40fa --- /dev/null +++ b/docs/superpowers/specs/2026-03-19-vod-merge-split-design.md @@ -0,0 +1,432 @@ +# VOD Merge+Split Feature — Design Spec + +**Date:** 2026-03-19 +**Status:** Approved +**Revision:** 3 (second spec-review fixes) + +## Summary + +Allow users to select 2+ pending VODs in the download queue, combine them into a single "Merge Group" queue item, and have the system automatically: download all VODs, merge them chronologically via FFmpeg, split the merged result into time-based parts (using the existing `part_minutes` setting), and clean up all temporary files. + +## Requirements + +1. User selects 2+ pending queue items via checkboxes +2. Clicking "Merge & Split" creates a single merge-group queue item +3. VODs are auto-sorted chronologically by full ISO timestamp (date + time) +4. The entire download → merge → split → cleanup runs as one automated job +5. Part naming uses the existing `filename_template_parts` setting +6. Date for naming is taken from the first (earliest) VOD +7. Temporary files (individual downloads, merged file) are deleted after successful split +8. On failure, the entire group fails; retry resumes from where it left off + +--- + +## 1. Data Model + +### New Interfaces + +```typescript +interface MergeGroupItem { + url: string; + title: string; + date: string; // Full ISO timestamp from Twitch API (includes time) + streamer: string; + duration_str: string; +} + +interface MergeGroup { + items: MergeGroupItem[]; // VODs, sorted chronologically by date + mergePhase: 'downloading' | 'merging' | 'splitting' | 'cleanup' | 'done'; + currentItemIndex: number; // Index of VOD currently being downloaded + downloadedFiles: Record; // Sparse map: VOD index → file path (not array) + mergedFile?: string; // Path of concatenated file + splitFiles?: string[]; // Paths of final split parts + totalDurationSec?: number; // Cached total duration for progress calculation +} +``` + +**Note on `downloadedFiles`:** Uses `Record` (sparse map keyed by VOD index) instead of `string[]` to avoid ambiguity during retry. On retry, `downloadedFiles[i]` is checked — if the key exists AND the file exists on disk, that VOD is skipped. A dense array would cause index misalignment when earlier downloads succeed but later ones fail. + +### QueueItem Extension + +```typescript +interface QueueItem { + // ... all existing fields unchanged ... + mergeGroup?: MergeGroup; // Present → this is a merge group +} +``` + +**Important:** The `mergeGroup` field must be added to ALL THREE copies of the `QueueItem` interface: +- `src/main.ts` (lines ~156-173) +- `src/preload.ts` (lines ~12-26) +- `src/renderer-globals.d.ts` (lines ~40-58) + +**Type sharing strategy:** The `MergeGroup` and `MergeGroupItem` interfaces must be defined in ALL locations where `QueueItem` is defined: +- **`src/main.ts`**: Full interface definitions (canonical source) +- **`src/renderer-globals.d.ts`**: Full interface definitions (for renderer TypeScript) +- **`src/preload.ts`**: Full interface definitions (for preload TypeScript — required because `preload.ts` has its own copy of `QueueItem` and TypeScript strict mode requires `MergeGroup` to be in scope) + +This follows the existing pattern: `QueueItem` and `CustomClip` are already duplicated across all three files. + +### Conventions + +- When `mergeGroup` is set, the top-level `url`, `title`, `date`, `streamer`, `duration_str` are populated from the **first** (chronologically earliest) item in the group, for backwards compatibility with existing queue code. +- `title` is generated: `"Merge: {title1} + {title2}"` (3+ items: `"Merge: {title1} + {n-1} weitere"`) +- `duration_str` is the sum of all individual durations. +- `url` is set to the first VOD's URL (used as identifier only, not for downloading — each item in `mergeGroup.items` has its own URL). + +--- + +## 2. Download Flow — `processDownloadMergeGroup()` + +A new function handling the 4-phase pipeline. Called from `processQueue()` when `item.mergeGroup` is present. + +**Critical: Download mode override.** Merge group VODs are ALWAYS downloaded in full-file mode, regardless of the user's `config.download_mode` setting. This ensures each VOD produces exactly one file. The user's parts setting is only applied during Phase 3 (splitting the merged result). Without this override, `download_mode === 'parts'` would produce multiple files per VOD, breaking the merge pipeline. + +**Implementation approach:** `processDownloadMergeGroup()` does NOT call `downloadVOD()` (which reads `config.download_mode` from the global config and would split into parts). Instead, it calls `downloadVODPart()` directly with `startTime=null, endTime=null` (meaning "download the entire VOD as one file"). The merge-group function replicates the necessary setup from `downloadVOD()` itself: +- Tool verification (`ensureStreamlinkInstalled()`, `ensureFfmpegInstalled()`) +- Output folder creation (`ensureDownloadFolder()`) +- Disk space pre-check (`ensureDiskSpace()` with 3x estimate) +- Filename generation (using first VOD's date, streamer, and a temp naming scheme like `merge_tmp_{vodIndex}_{timestamp}.mp4`) + +This avoids mutating the global `config.download_mode` (which would be a race condition in async code) and avoids refactoring `downloadVOD()` to accept an override parameter. + +### Phase 1: Downloading (`mergePhase: 'downloading'`) + +``` +For each VOD in mergeGroup.items (chronological order): + 0. Reset currentDownloadCancelled = false (CRITICAL: prevents stale cancel state + from a prior VOD or previous retry from aborting the current download) + 1. Set currentItemIndex = i + 2. Skip if downloadedFiles[i] exists in map AND file exists on disk (retry recovery) + 3. Call downloadVODPart() directly with startTime=null, endTime=null + (bypasses downloadVOD() which checks config.download_mode) + 5. Store resulting file path: downloadedFiles[i] = path + 6. Report weighted progress: each VOD's share proportional to its duration + 7. On error → entire group status = 'error', phase stays 'downloading' + 8. saveQueue() after each completed VOD (crash recovery) +``` + +### Phase 2: Merging (`mergePhase: 'merging'`) + +``` +1. Use existing mergeVideos() function (FFmpeg concat demuxer) + BUT pass totalDurationSec so progress can be calculated correctly +2. Input: files sorted explicitly by index: + Object.keys(downloadedFiles).sort((a, b) => Number(a) - Number(b)).map(k => downloadedFiles[Number(k)]) + (Do NOT rely on implicit Object.values() ordering — VOD order is correctness-critical) +3. Output: temporary file "merged_{timestamp}.mp4" in download directory +4. Two-stage strategy: stream copy first, re-encode fallback +5. Progress: mergeVideos() must use actual total duration for correct percentage: + percent = (out_time_us / (totalDurationSec * 1_000_000)) * 100 + (The existing formula `currentUs / 10_000_000` is a bug — it caps at 99% after + 16.5 minutes regardless of actual file length. Must be fixed.) +6. On error → status = 'error', phase stays 'merging' +7. Store path in mergeGroup.mergedFile, saveQueue() +``` + +**Note on existing mergeVideos() progress bug and signature change:** The current implementation at main.ts:2440 uses `Math.min(99, currentUs / 10000000)` which hits 99% after ~16 minutes. The fix: +- Add an **optional** `totalDurationSec?: number` parameter to `mergeVideos()` (4th parameter after `onProgress`) +- When provided: `percent = Math.min(99, (currentUs / (totalDurationSec * 1_000_000)) * 100)` +- When NOT provided (standalone Merge Videos tab): use `ffprobe` to determine total duration of all input files before starting the merge. This is a lightweight call (~100ms) that runs once before the merge begins. +- The existing IPC handler `merge-videos` does NOT need signature changes — it calls `mergeVideos()` internally and simply omits the `totalDurationSec` parameter, triggering the ffprobe fallback. +- `processDownloadMergeGroup()` passes `totalDurationSec` directly (already computed from VOD durations), skipping the ffprobe call. + +**Note on Windows path escaping:** The existing `mergeVideos()` function uses Unix-style single-quote escaping (`'\\''`) in the FFmpeg concat file. This is a pre-existing issue that should be verified on Windows during testing. FFmpeg's concat demuxer on Windows typically handles forward-slash paths correctly, so the implementation should normalize backslashes to forward slashes in the concat file. + +### Phase 3: Splitting (`mergePhase: 'splitting'`) + +``` +1. Calculate total duration from mergeGroup.items (or use cached totalDurationSec) +2. Calculate numParts = ceil(totalDuration / (config.part_minutes * 60)) +3. For each part: + ffmpeg -ss -i mergedFile -t -c copy -y part_X.mp4 + (Note: -ss BEFORE -i for fast seeking, matching existing cutVideo() pattern) +4. Naming via existing filename_template_parts, date = first VOD's date +5. Progress: simple part counter (copy mode with fast-seek is near-instant) +6. Store paths in mergeGroup.splitFiles +7. On error → status = 'error', phase stays 'splitting' +``` + +**Split approach:** `-c copy` (stream copy, no re-encoding) with `-ss` before `-i` (fast seek) +- Near-instant execution (fast seek skips directly to the target position) +- Quality-lossless +- Minor keyframe-boundary imprecision at split points (irrelevant for 60+ min parts) +- Matches the existing `cutVideo()` pattern at main.ts:2303-2306 + +### Phase 4: Cleanup (`mergePhase: 'cleanup'`) + +``` +1. Delete all individual downloads: all values from downloadedFiles +2. Delete merged file: mergedFile +3. Only split parts remain on disk +4. Set mergePhase = 'done', status = 'completed', progress = 100 +5. saveQueue() +``` + +### Progress Weighting + +| Phase | Weight | Display | +|-------------|--------|--------------------------------------------| +| Downloading | 70% | "VOD 1/2 wird heruntergeladen — 45%" | +| Merging | 20% | "Zusammenfugen... 60%" | +| Splitting | 10% | "Part 2/5 wird erstellt..." | +| Cleanup | 0% | "Aufraumen..." (instant) | + +Overall progress = phase_base + (phase_progress × phase_weight) + +Example for 2 VODs: During download of VOD 2 at 50%: +- VOD 1 complete = 35% (50% of 70%) +- VOD 2 at 50% = 17.5% (50% of remaining 35%) +- Total: 52.5% + +--- + +## 3. UI Changes + +### 3a) Queue Item Checkboxes + +- Each queue item gets a checkbox to the left of the status indicator +- Checkbox only visible/enabled for items with `status === 'pending'` +- Hidden for: downloading, paused, completed, error items +- Hidden for items that are already merge groups + +### 3b) "Merge & Split" Button + +- Appears in the queue action bar when 2+ checkboxes are checked +- Label: "Merge & Split (N)" where N = number of selected items +- Clicking it: + 1. Collects selected items + 2. Sorts them chronologically by `date` (ISO timestamp comparison) + 3. Creates a new merge-group QueueItem + 4. Removes the individual items from the queue + 5. Adds the merge-group item in their place + 6. Clears all checkboxes + 7. Calls `window.api.createMergeGroup(selectedIds)` → main process + +### 3c) Merge Group Rendering + +A merge group in the queue looks like a regular item but visually distinct: + +``` +┌─────────────────────────────────────────────────┐ +│ [merge-icon] Merge: VOD Title 1 + VOD Title 2 │ +│ 4h15m (2 VODs) | 01.03.2026 Running │ +│ VOD 1/2 wird heruntergeladen — 45.0% │ +│ ████████████░░░░░░░░░░░░░░░░░ 45% │ +│ [x]│ +└─────────────────────────────────────────────────┘ +``` + +- Merge icon via CSS/SVG (chain-link or similar), no emoji +- Meta line shows total duration and VOD count +- Progress text shows current phase with detail +- Remove button cancels and cleans up all temp files + +### 3d) Localization Strings + +**English (`renderer-locale-en.ts`):** +```typescript +mergeGroup: { + btn: 'Merge & Split', + phaseDownloading: 'Downloading VOD {current}/{total}', + phaseMerging: 'Merging...', + phaseSplitting: 'Splitting Part {current}/{total}...', + phaseCleanup: 'Cleaning up...', + titleTwo: 'Merge: {title1} + {title2}', + titleMany: 'Merge: {title1} + {count} more', + needMinTwo: 'Select at least 2 VODs', + metaLabel: '{duration} ({count} VODs)', +} +``` + +**German (`renderer-locale-de.ts`):** +```typescript +mergeGroup: { + btn: 'Zusammenfugen & Splitten', + phaseDownloading: 'VOD {current}/{total} wird heruntergeladen', + phaseMerging: 'Zusammenfugen...', + phaseSplitting: 'Part {current}/{total} wird erstellt...', + phaseCleanup: 'Aufraumen...', + titleTwo: 'Merge: {title1} + {title2}', + titleMany: 'Merge: {title1} + {count} weitere', + needMinTwo: 'Mindestens 2 VODs auswahlen', + metaLabel: '{duration} ({count} VODs)', +} +``` + +--- + +## 4. IPC Interface + +### New IPC Channel: `create-merge-group` + +```typescript +// Preload +createMergeGroup: (itemIds: string[]): Promise => + ipcRenderer.invoke('create-merge-group', itemIds), + +// Main process handler +ipcMain.handle('create-merge-group', async (_, itemIds: string[]) => { + // 1. Find items in downloadQueue by IDs + // 2. Validate all are 'pending' + // 3. Sort chronologically by date (ISO timestamp — handles same-day different times) + // 4. Create MergeGroup object with downloadedFiles = {} + // 5. Create new QueueItem with mergeGroup field + // - url = first item's url + // - title = generated merge title + // - date = first item's date + // - duration_str = sum of durations + // 6. Remove individual items from queue + // 7. Insert merge-group item at position of first removed item + // 8. saveQueue() and emit 'queue-updated' + // 9. Return updated queue +}); +``` + +No other new IPC channels needed — progress reuses existing `download-progress` event. + +--- + +## 5. Error Handling + +| Scenario | Behavior | +|----------|----------| +| VOD download fails | Entire group → `error`. Retry skips already-downloaded files. | +| FFmpeg merge fails | Group → `error`, phase stays `merging`. Retry re-attempts merge. | +| FFmpeg split fails | Group → `error`, phase stays `splitting`. Retry checks if merged file exists. | +| Disk space insufficient | Pre-check before download. Estimate: **3x** sum of VOD sizes needed (downloads + merged + split parts all on disk simultaneously before cleanup). | +| User deletes group during download | Cancel running download, delete all temp files. | +| App crash / restart | Queue persisted with `mergeGroup` data. Retry resumes from last phase. | +| VODs from different streamers | Allowed. Naming uses first VOD's streamer. | +| Different qualities/resolutions | FFmpeg re-encode fallback handles codec mismatches. | + +### Disk Space Calculation + +Peak disk usage occurs just before cleanup, when all three file sets coexist: +- Individual downloads: ~1x total VOD size +- Merged file: ~1x total VOD size +- Split parts: ~1x total VOD size +- **Total: ~3x sum of VOD sizes** + +The pre-check uses `3x * estimated_total_size` with a minimum of 256 MB. + +### Retry Logic + +``` +On retry of a merge-group item: + status is reset to 'pending', progress to 0 + mergeGroup.mergePhase and mergeGroup.downloadedFiles are PRESERVED + + processDownloadMergeGroup() checks mergePhase on entry: + + if mergePhase === 'downloading': + For each i in items: + if downloadedFiles[i] exists AND file on disk → skip + else → download and store + Then proceed to merging phase + + if mergePhase === 'merging': + Check if all downloadedFiles values exist on disk + If any missing → reset mergePhase to 'downloading', restart + If all present → re-attempt merge + Then proceed to splitting phase + + if mergePhase === 'splitting': + Check if mergedFile exists on disk + If missing → check downloadedFiles → reset to appropriate phase + If present → re-attempt split + Then proceed to cleanup + + Progress is recalculated from phase on first tick of processDownloadMergeGroup(): + 'downloading' → base = 0% (count already-downloaded files for offset) + 'merging' → base = 70% + 'splitting' → base = 90% + + Note: retryFailedDownloads resets item.progress to 0. There will be a brief + 0% flash in the UI between retry-click and processDownloadMergeGroup() starting. + This is a minor UX imperfection (< 1 second) — acceptable since the progress + corrects itself immediately when processing begins. Fixing it would require + special-casing merge groups in retryFailedDownloads, adding complexity for + negligible benefit. +``` + +### Cancel/Remove During Active Merge Group + +``` +When user removes a merge-group item during processing: + 1. Set currentDownloadCancelled = true (kills active streamlink/ffmpeg process) + 2. Wait for process to exit + 3. Delete all temp files: + - All files in downloadedFiles (if they exist) + - mergedFile (if it exists) + - Any partial output files + 4. Remove item from queue +``` + +--- + +## 6. Testing Strategy + +### Unit Tests +- Duration summation across multiple VODs +- Chronological sorting by ISO timestamp (same day, different times: 16:00 vs 18:00) +- Merge group title generation (2 items, 3+ items) +- Progress weighting calculation (70/20/10) +- Retry logic: phase detection, file existence checks +- FFmpeg split argument construction (verify `-ss` before `-i`) +- `currentDownloadCancelled` reset between VOD downloads + +### Integration Tests +- Create merge group via IPC → verify queue structure +- Persist merge group → reload → verify mergeGroup data intact (including `downloadedFiles` as Record) +- FFmpeg concat file generation (correct paths, escaping, forward slashes on Windows) +- `download_mode` override: verify merge group downloads use 'full' mode regardless of config + +### E2E Tests (Playwright) +- Checkbox visibility: only on pending items +- Button appears/disappears based on selection count +- Merge group renders with correct title, meta, and phase text +- Localization: all new strings present in EN and DE + +### Manual Test +- 2 short VODs → Merge & Split with 5 min parts → verify parts created, temp files deleted, progress correct + +--- + +## 7. Files to Modify + +| File | Changes | +|------|---------| +| `src/main.ts` | Add `processDownloadMergeGroup()`, `splitMergedFile()`, IPC handler `create-merge-group`, extend `processQueue()` branching, fix `mergeVideos()` progress formula (pass totalDurationSec), force `download_mode='full'` for merge groups, reset `currentDownloadCancelled` per sub-download | +| `src/preload.ts` | Add `createMergeGroup` to API surface, add `mergeGroup?: MergeGroup` to QueueItem interface | +| `src/renderer-queue.ts` | Add checkbox rendering, selection state, "Merge & Split" button, merge-group display logic, phase-aware progress text | +| `src/renderer-shared.ts` | Add `selectedQueueIds: Set` global state | +| `src/renderer-locale-en.ts` | Add `mergeGroup` string block | +| `src/renderer-locale-de.ts` | Add `mergeGroup` string block | +| `src/renderer-globals.d.ts` | Add `MergeGroup`, `MergeGroupItem` interfaces, extend `QueueItem` with `mergeGroup?` | +| `src/index.html` | Add merge-group button to queue action bar, CSS for checkboxes and merge-group items | + +--- + +## 8. Review Changelog + +### Revision 2 (first spec review — 9 issues fixed): + +1. **`currentDownloadCancelled` reset** — Added explicit reset to `false` before each VOD download in Phase 1 +2. **Windows path escaping** — Added note to normalize backslashes to forward slashes in concat file +3. **`mergeVideos()` progress bug** — Fixed: pass `totalDurationSec` for correct percentage calculation +4. **Retry progress reset** — Defined phase-based progress recalculation on retry resume +5. **`-ss` after `-i` slow seeking** — Fixed: `-ss` before `-i` matching existing `cutVideo()` pattern +6. **`downloadedFiles` indexing** — Changed from `string[]` to `Record` (sparse map by VOD index) +7. **Missing `mergeGroup` in preload.ts** — Added to files-to-modify table +8. **Disk space estimate** — Changed from 2.2x to 3x to account for all three file sets coexisting +9. **`download_mode` conflict** — Force `download_mode = 'full'` for merge group downloads + +### Revision 3 (second spec review — 5 issues fixed): + +10. **`download_mode` override race condition** — Changed approach: call `downloadVODPart()` directly instead of `downloadVOD()`, avoiding global config mutation. Merge-group function replicates necessary setup (tool checks, folder, disk space, filename). +11. **`mergeVideos()` signature change breaks standalone IPC** — Made `totalDurationSec` an optional parameter with `ffprobe` fallback for the standalone Merge Videos tab. Existing IPC surface unchanged. +12. **Retry progress 0% flash** — Acknowledged as minor UX gap (< 1 second). Documenting rather than adding complexity to fix. +13. **`MergeGroup` types not in preload.ts scope** — Added explicit type-sharing strategy: duplicate interfaces in all three files (follows existing `QueueItem`/`CustomClip` pattern). +14. **`Object.values()` implicit key ordering** — Changed to explicit sort: `Object.keys().sort().map()`.