Twitch-VOD-Manager/docs/superpowers/specs/2026-03-19-vod-merge-split-design.md
xRangerDE 1abc87d17d docs: add VOD merge+split feature design spec
Comprehensive design for combining multiple queue VODs into a merge group,
downloading them, merging via FFmpeg, and splitting into time-based parts.
Reviewed through two spec-review iterations fixing 14 issues.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:56:38 +01:00

21 KiB
Raw Blame History

VOD Merge+Split Feature — Design Spec

Date: 2026-03-19 Status: Approved Revision: 3 (second spec-review fixes)

Summary

Allow users to select 2+ pending VODs in the download queue, combine them into a single "Merge Group" queue item, and have the system automatically: download all VODs, merge them chronologically via FFmpeg, split the merged result into time-based parts (using the existing part_minutes setting), and clean up all temporary files.

Requirements

  1. User selects 2+ pending queue items via checkboxes
  2. Clicking "Merge & Split" creates a single merge-group queue item
  3. VODs are auto-sorted chronologically by full ISO timestamp (date + time)
  4. The entire download → merge → split → cleanup runs as one automated job
  5. Part naming uses the existing filename_template_parts setting
  6. Date for naming is taken from the first (earliest) VOD
  7. Temporary files (individual downloads, merged file) are deleted after successful split
  8. On failure, the entire group fails; retry resumes from where it left off

1. Data Model

New Interfaces

interface MergeGroupItem {
    url: string;
    title: string;
    date: string;           // Full ISO timestamp from Twitch API (includes time)
    streamer: string;
    duration_str: string;
}

interface MergeGroup {
    items: MergeGroupItem[];           // VODs, sorted chronologically by date
    mergePhase: 'downloading' | 'merging' | 'splitting' | 'cleanup' | 'done';
    currentItemIndex: number;          // Index of VOD currently being downloaded
    downloadedFiles: Record<number, string>;  // Sparse map: VOD index → file path (not array)
    mergedFile?: string;               // Path of concatenated file
    splitFiles?: string[];             // Paths of final split parts
    totalDurationSec?: number;         // Cached total duration for progress calculation
}

Note on downloadedFiles: Uses Record<number, string> (sparse map keyed by VOD index) instead of string[] to avoid ambiguity during retry. On retry, downloadedFiles[i] is checked — if the key exists AND the file exists on disk, that VOD is skipped. A dense array would cause index misalignment when earlier downloads succeed but later ones fail.

QueueItem Extension

interface QueueItem {
    // ... all existing fields unchanged ...
    mergeGroup?: MergeGroup;           // Present → this is a merge group
}

Important: The mergeGroup field must be added to ALL THREE copies of the QueueItem interface:

  • src/main.ts (lines ~156-173)
  • src/preload.ts (lines ~12-26)
  • src/renderer-globals.d.ts (lines ~40-58)

Type sharing strategy: The MergeGroup and MergeGroupItem interfaces must be defined in ALL locations where QueueItem is defined:

  • src/main.ts: Full interface definitions (canonical source)
  • src/renderer-globals.d.ts: Full interface definitions (for renderer TypeScript)
  • src/preload.ts: Full interface definitions (for preload TypeScript — required because preload.ts has its own copy of QueueItem and TypeScript strict mode requires MergeGroup to be in scope)

This follows the existing pattern: QueueItem and CustomClip are already duplicated across all three files.

Conventions

  • When mergeGroup is set, the top-level url, title, date, streamer, duration_str are populated from the first (chronologically earliest) item in the group, for backwards compatibility with existing queue code.
  • title is generated: "Merge: {title1} + {title2}" (3+ items: "Merge: {title1} + {n-1} weitere")
  • duration_str is the sum of all individual durations.
  • url is set to the first VOD's URL (used as identifier only, not for downloading — each item in mergeGroup.items has its own URL).

2. Download Flow — processDownloadMergeGroup()

A new function handling the 4-phase pipeline. Called from processQueue() when item.mergeGroup is present.

Critical: Download mode override. Merge group VODs are ALWAYS downloaded in full-file mode, regardless of the user's config.download_mode setting. This ensures each VOD produces exactly one file. The user's parts setting is only applied during Phase 3 (splitting the merged result). Without this override, download_mode === 'parts' would produce multiple files per VOD, breaking the merge pipeline.

Implementation approach: processDownloadMergeGroup() does NOT call downloadVOD() (which reads config.download_mode from the global config and would split into parts). Instead, it calls downloadVODPart() directly with startTime=null, endTime=null (meaning "download the entire VOD as one file"). The merge-group function replicates the necessary setup from downloadVOD() itself:

  • Tool verification (ensureStreamlinkInstalled(), ensureFfmpegInstalled())
  • Output folder creation (ensureDownloadFolder())
  • Disk space pre-check (ensureDiskSpace() with 3x estimate)
  • Filename generation (using first VOD's date, streamer, and a temp naming scheme like merge_tmp_{vodIndex}_{timestamp}.mp4)

This avoids mutating the global config.download_mode (which would be a race condition in async code) and avoids refactoring downloadVOD() to accept an override parameter.

Phase 1: Downloading (mergePhase: 'downloading')

For each VOD in mergeGroup.items (chronological order):
    0. Reset currentDownloadCancelled = false (CRITICAL: prevents stale cancel state
       from a prior VOD or previous retry from aborting the current download)
    1. Set currentItemIndex = i
    2. Skip if downloadedFiles[i] exists in map AND file exists on disk (retry recovery)
    3. Call downloadVODPart() directly with startTime=null, endTime=null
       (bypasses downloadVOD() which checks config.download_mode)
    5. Store resulting file path: downloadedFiles[i] = path
    6. Report weighted progress: each VOD's share proportional to its duration
    7. On error → entire group status = 'error', phase stays 'downloading'
    8. saveQueue() after each completed VOD (crash recovery)

Phase 2: Merging (mergePhase: 'merging')

1. Use existing mergeVideos() function (FFmpeg concat demuxer)
   BUT pass totalDurationSec so progress can be calculated correctly
2. Input: files sorted explicitly by index:
   Object.keys(downloadedFiles).sort((a, b) => Number(a) - Number(b)).map(k => downloadedFiles[Number(k)])
   (Do NOT rely on implicit Object.values() ordering — VOD order is correctness-critical)
3. Output: temporary file "merged_{timestamp}.mp4" in download directory
4. Two-stage strategy: stream copy first, re-encode fallback
5. Progress: mergeVideos() must use actual total duration for correct percentage:
   percent = (out_time_us / (totalDurationSec * 1_000_000)) * 100
   (The existing formula `currentUs / 10_000_000` is a bug — it caps at 99% after
    16.5 minutes regardless of actual file length. Must be fixed.)
6. On error → status = 'error', phase stays 'merging'
7. Store path in mergeGroup.mergedFile, saveQueue()

Note on existing mergeVideos() progress bug and signature change: The current implementation at main.ts:2440 uses Math.min(99, currentUs / 10000000) which hits 99% after ~16 minutes. The fix:

  • Add an optional totalDurationSec?: number parameter to mergeVideos() (4th parameter after onProgress)
  • When provided: percent = Math.min(99, (currentUs / (totalDurationSec * 1_000_000)) * 100)
  • When NOT provided (standalone Merge Videos tab): use ffprobe to determine total duration of all input files before starting the merge. This is a lightweight call (~100ms) that runs once before the merge begins.
  • The existing IPC handler merge-videos does NOT need signature changes — it calls mergeVideos() internally and simply omits the totalDurationSec parameter, triggering the ffprobe fallback.
  • processDownloadMergeGroup() passes totalDurationSec directly (already computed from VOD durations), skipping the ffprobe call.

Note on Windows path escaping: The existing mergeVideos() function uses Unix-style single-quote escaping ('\\'') in the FFmpeg concat file. This is a pre-existing issue that should be verified on Windows during testing. FFmpeg's concat demuxer on Windows typically handles forward-slash paths correctly, so the implementation should normalize backslashes to forward slashes in the concat file.

Phase 3: Splitting (mergePhase: 'splitting')

1. Calculate total duration from mergeGroup.items (or use cached totalDurationSec)
2. Calculate numParts = ceil(totalDuration / (config.part_minutes * 60))
3. For each part:
    ffmpeg -ss <startSec> -i mergedFile -t <partDuration> -c copy -y part_X.mp4
    (Note: -ss BEFORE -i for fast seeking, matching existing cutVideo() pattern)
4. Naming via existing filename_template_parts, date = first VOD's date
5. Progress: simple part counter (copy mode with fast-seek is near-instant)
6. Store paths in mergeGroup.splitFiles
7. On error → status = 'error', phase stays 'splitting'

Split approach: -c copy (stream copy, no re-encoding) with -ss before -i (fast seek)

  • Near-instant execution (fast seek skips directly to the target position)
  • Quality-lossless
  • Minor keyframe-boundary imprecision at split points (irrelevant for 60+ min parts)
  • Matches the existing cutVideo() pattern at main.ts:2303-2306

Phase 4: Cleanup (mergePhase: 'cleanup')

1. Delete all individual downloads: all values from downloadedFiles
2. Delete merged file: mergedFile
3. Only split parts remain on disk
4. Set mergePhase = 'done', status = 'completed', progress = 100
5. saveQueue()

Progress Weighting

Phase Weight Display
Downloading 70% "VOD 1/2 wird heruntergeladen — 45%"
Merging 20% "Zusammenfugen... 60%"
Splitting 10% "Part 2/5 wird erstellt..."
Cleanup 0% "Aufraumen..." (instant)

Overall progress = phase_base + (phase_progress × phase_weight)

Example for 2 VODs: During download of VOD 2 at 50%:

  • VOD 1 complete = 35% (50% of 70%)
  • VOD 2 at 50% = 17.5% (50% of remaining 35%)
  • Total: 52.5%

3. UI Changes

3a) Queue Item Checkboxes

  • Each queue item gets a checkbox to the left of the status indicator
  • Checkbox only visible/enabled for items with status === 'pending'
  • Hidden for: downloading, paused, completed, error items
  • Hidden for items that are already merge groups

3b) "Merge & Split" Button

  • Appears in the queue action bar when 2+ checkboxes are checked
  • Label: "Merge & Split (N)" where N = number of selected items
  • Clicking it:
    1. Collects selected items
    2. Sorts them chronologically by date (ISO timestamp comparison)
    3. Creates a new merge-group QueueItem
    4. Removes the individual items from the queue
    5. Adds the merge-group item in their place
    6. Clears all checkboxes
    7. Calls window.api.createMergeGroup(selectedIds) → main process

3c) Merge Group Rendering

A merge group in the queue looks like a regular item but visually distinct:

┌─────────────────────────────────────────────────┐
│ [merge-icon] Merge: VOD Title 1 + VOD Title 2  │
│    4h15m (2 VODs) | 01.03.2026          Running │
│    VOD 1/2 wird heruntergeladen — 45.0%         │
│    ████████████░░░░░░░░░░░░░░░░░ 45%            │
│                                              [x]│
└─────────────────────────────────────────────────┘
  • Merge icon via CSS/SVG (chain-link or similar), no emoji
  • Meta line shows total duration and VOD count
  • Progress text shows current phase with detail
  • Remove button cancels and cleans up all temp files

3d) Localization Strings

English (renderer-locale-en.ts):

mergeGroup: {
    btn: 'Merge & Split',
    phaseDownloading: 'Downloading VOD {current}/{total}',
    phaseMerging: 'Merging...',
    phaseSplitting: 'Splitting Part {current}/{total}...',
    phaseCleanup: 'Cleaning up...',
    titleTwo: 'Merge: {title1} + {title2}',
    titleMany: 'Merge: {title1} + {count} more',
    needMinTwo: 'Select at least 2 VODs',
    metaLabel: '{duration} ({count} VODs)',
}

German (renderer-locale-de.ts):

mergeGroup: {
    btn: 'Zusammenfugen & Splitten',
    phaseDownloading: 'VOD {current}/{total} wird heruntergeladen',
    phaseMerging: 'Zusammenfugen...',
    phaseSplitting: 'Part {current}/{total} wird erstellt...',
    phaseCleanup: 'Aufraumen...',
    titleTwo: 'Merge: {title1} + {title2}',
    titleMany: 'Merge: {title1} + {count} weitere',
    needMinTwo: 'Mindestens 2 VODs auswahlen',
    metaLabel: '{duration} ({count} VODs)',
}

4. IPC Interface

New IPC Channel: create-merge-group

// Preload
createMergeGroup: (itemIds: string[]): Promise<QueueItem[]> =>
    ipcRenderer.invoke('create-merge-group', itemIds),

// Main process handler
ipcMain.handle('create-merge-group', async (_, itemIds: string[]) => {
    // 1. Find items in downloadQueue by IDs
    // 2. Validate all are 'pending'
    // 3. Sort chronologically by date (ISO timestamp — handles same-day different times)
    // 4. Create MergeGroup object with downloadedFiles = {}
    // 5. Create new QueueItem with mergeGroup field
    //    - url = first item's url
    //    - title = generated merge title
    //    - date = first item's date
    //    - duration_str = sum of durations
    // 6. Remove individual items from queue
    // 7. Insert merge-group item at position of first removed item
    // 8. saveQueue() and emit 'queue-updated'
    // 9. Return updated queue
});

No other new IPC channels needed — progress reuses existing download-progress event.


5. Error Handling

Scenario Behavior
VOD download fails Entire group → error. Retry skips already-downloaded files.
FFmpeg merge fails Group → error, phase stays merging. Retry re-attempts merge.
FFmpeg split fails Group → error, phase stays splitting. Retry checks if merged file exists.
Disk space insufficient Pre-check before download. Estimate: 3x sum of VOD sizes needed (downloads + merged + split parts all on disk simultaneously before cleanup).
User deletes group during download Cancel running download, delete all temp files.
App crash / restart Queue persisted with mergeGroup data. Retry resumes from last phase.
VODs from different streamers Allowed. Naming uses first VOD's streamer.
Different qualities/resolutions FFmpeg re-encode fallback handles codec mismatches.

Disk Space Calculation

Peak disk usage occurs just before cleanup, when all three file sets coexist:

  • Individual downloads: ~1x total VOD size
  • Merged file: ~1x total VOD size
  • Split parts: ~1x total VOD size
  • Total: ~3x sum of VOD sizes

The pre-check uses 3x * estimated_total_size with a minimum of 256 MB.

Retry Logic

On retry of a merge-group item:
    status is reset to 'pending', progress to 0
    mergeGroup.mergePhase and mergeGroup.downloadedFiles are PRESERVED

    processDownloadMergeGroup() checks mergePhase on entry:

    if mergePhase === 'downloading':
        For each i in items:
            if downloadedFiles[i] exists AND file on disk → skip
            else → download and store
        Then proceed to merging phase

    if mergePhase === 'merging':
        Check if all downloadedFiles values exist on disk
        If any missing → reset mergePhase to 'downloading', restart
        If all present → re-attempt merge
        Then proceed to splitting phase

    if mergePhase === 'splitting':
        Check if mergedFile exists on disk
        If missing → check downloadedFiles → reset to appropriate phase
        If present → re-attempt split
        Then proceed to cleanup

    Progress is recalculated from phase on first tick of processDownloadMergeGroup():
        'downloading' → base = 0% (count already-downloaded files for offset)
        'merging' → base = 70%
        'splitting' → base = 90%

    Note: retryFailedDownloads resets item.progress to 0. There will be a brief
    0% flash in the UI between retry-click and processDownloadMergeGroup() starting.
    This is a minor UX imperfection (< 1 second) — acceptable since the progress
    corrects itself immediately when processing begins. Fixing it would require
    special-casing merge groups in retryFailedDownloads, adding complexity for
    negligible benefit.

Cancel/Remove During Active Merge Group

When user removes a merge-group item during processing:
    1. Set currentDownloadCancelled = true (kills active streamlink/ffmpeg process)
    2. Wait for process to exit
    3. Delete all temp files:
       - All files in downloadedFiles (if they exist)
       - mergedFile (if it exists)
       - Any partial output files
    4. Remove item from queue

6. Testing Strategy

Unit Tests

  • Duration summation across multiple VODs
  • Chronological sorting by ISO timestamp (same day, different times: 16:00 vs 18:00)
  • Merge group title generation (2 items, 3+ items)
  • Progress weighting calculation (70/20/10)
  • Retry logic: phase detection, file existence checks
  • FFmpeg split argument construction (verify -ss before -i)
  • currentDownloadCancelled reset between VOD downloads

Integration Tests

  • Create merge group via IPC → verify queue structure
  • Persist merge group → reload → verify mergeGroup data intact (including downloadedFiles as Record)
  • FFmpeg concat file generation (correct paths, escaping, forward slashes on Windows)
  • download_mode override: verify merge group downloads use 'full' mode regardless of config

E2E Tests (Playwright)

  • Checkbox visibility: only on pending items
  • Button appears/disappears based on selection count
  • Merge group renders with correct title, meta, and phase text
  • Localization: all new strings present in EN and DE

Manual Test

  • 2 short VODs → Merge & Split with 5 min parts → verify parts created, temp files deleted, progress correct

7. Files to Modify

File Changes
src/main.ts Add processDownloadMergeGroup(), splitMergedFile(), IPC handler create-merge-group, extend processQueue() branching, fix mergeVideos() progress formula (pass totalDurationSec), force download_mode='full' for merge groups, reset currentDownloadCancelled per sub-download
src/preload.ts Add createMergeGroup to API surface, add mergeGroup?: MergeGroup to QueueItem interface
src/renderer-queue.ts Add checkbox rendering, selection state, "Merge & Split" button, merge-group display logic, phase-aware progress text
src/renderer-shared.ts Add selectedQueueIds: Set<string> global state
src/renderer-locale-en.ts Add mergeGroup string block
src/renderer-locale-de.ts Add mergeGroup string block
src/renderer-globals.d.ts Add MergeGroup, MergeGroupItem interfaces, extend QueueItem with mergeGroup?
src/index.html Add merge-group button to queue action bar, CSS for checkboxes and merge-group items

8. Review Changelog

Revision 2 (first spec review — 9 issues fixed):

  1. currentDownloadCancelled reset — Added explicit reset to false before each VOD download in Phase 1
  2. Windows path escaping — Added note to normalize backslashes to forward slashes in concat file
  3. mergeVideos() progress bug — Fixed: pass totalDurationSec for correct percentage calculation
  4. Retry progress reset — Defined phase-based progress recalculation on retry resume
  5. -ss after -i slow seeking — Fixed: -ss before -i matching existing cutVideo() pattern
  6. downloadedFiles indexing — Changed from string[] to Record<number, string> (sparse map by VOD index)
  7. Missing mergeGroup in preload.ts — Added to files-to-modify table
  8. Disk space estimate — Changed from 2.2x to 3x to account for all three file sets coexisting
  9. download_mode conflict — Force download_mode = 'full' for merge group downloads

Revision 3 (second spec review — 5 issues fixed):

  1. download_mode override race condition — Changed approach: call downloadVODPart() directly instead of downloadVOD(), avoiding global config mutation. Merge-group function replicates necessary setup (tool checks, folder, disk space, filename).
  2. mergeVideos() signature change breaks standalone IPC — Made totalDurationSec an optional parameter with ffprobe fallback for the standalone Merge Videos tab. Existing IPC surface unchanged.
  3. Retry progress 0% flash — Acknowledged as minor UX gap (< 1 second). Documenting rather than adding complexity to fix.
  4. MergeGroup types not in preload.ts scope — Added explicit type-sharing strategy: duplicate interfaces in all three files (follows existing QueueItem/CustomClip pattern).
  5. Object.values() implicit key ordering — Changed to explicit sort: Object.keys().sort().map().