Compare commits

...

3 Commits

Author SHA1 Message Date
xRangerDE
9d57c03e74 release: 4.5.9 release pipeline + defensive parsing
- scripts/release_gitea.mjs: --skip-build flag and auto-skip when
  artifacts for the version already exist on disk
- playwright in devDependencies; test:e2e* scripts call node directly
- loadConfig isPlainObject shape check + loadQueue sanitizes every
  entry incl. customClip/mergeGroup; stale status="downloading" is
  demoted to "pending" so the queue can actually be resumed

See docs/IMPROVEMENT_LOG.md (Cycle 2, 2026-05-03) for the dated
rationale and regression run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:29:37 +02:00
xRangerDE
379048f191 harden: defensive parsing for config + queue, normalize stale downloading
- loadConfig now checks isPlainObject(parsed) before spreading over
  defaults. Non-object JSON (array, primitive, null) is logged and the
  app falls back to defaults instead of silently polluting the config
  with array indices or dropping values.

- loadQueue runs every entry through sanitizeQueueItem which validates
  the status enum, clamps progress to [0, 100], validates customClip
  and mergeGroup shapes (with sanitizeCustomClip / sanitizeMergeGroup
  helpers), and demotes stale status="downloading" entries to "pending"
  with progress=0 on cold start. The previous filter only checked
  typeof id/url/status === "string" and let through whatever shape
  customClip / mergeGroup happened to have.

- The stale-downloading normalisation fixes a real user trap: after a
  hard kill mid-download, the queue persisted status="downloading", but
  no download was running on next launch and start-download only resumed
  paused items, leaving "downloading" entries stuck.

- Bonus: CustomClip and MergeGroupItem imports now have call sites
  (previously unused-import warnings).

docs/IMPROVEMENT_LOG.md gains a Cycle 2 dated section.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:29:28 +02:00
xRangerDE
b4faf67db7 infra: release_gitea.mjs --skip-build + playwright in devDeps
Two release-pipeline fixes that previously forced manual workarounds.

- scripts/release_gitea.mjs no longer unconditionally runs npm run dist:win.
  New --skip-build flag, plus auto-skip when all 3 required artifacts
  (Setup-<v>.exe, Setup-<v>.exe.blockmap, latest.yml) already exist for
  the requested version. The previous behaviour re-ran the entire test
  suite + electron-builder on every release attempt — unusable when the
  test path was broken.
- playwright ^1.59.1 added to devDependencies. test:e2e / test:e2e:guide
  / test:e2e:full now invoke node scripts/smoke-test*.js directly instead
  of "npm exec --yes --package=playwright -- node ...", which failed with
  MODULE_NOT_FOUND when npm exec could not resolve playwright on the fly.
  No browser binaries needed — the smoke tests drive Electron via
  _electron, not a browser.

All test paths verified after the change: test:e2e, test:e2e:guide,
test:e2e:full, test:merge-split, test:e2e:update-logic — all pass with
the simplified scripts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 15:29:17 +02:00
5 changed files with 250 additions and 19 deletions

View File

@ -2,6 +2,37 @@
Dated entries from improvement cycles. Newest at top. Dated entries from improvement cycles. Newest at top.
## 2026-05-03 — Cycle 2: release pipeline + defensive parsing
Three independent improvements landed this cycle.
### 1. `scripts/release_gitea.mjs` skips rebuild when artifacts exist (release pipeline)
- **File**: `scripts/release_gitea.mjs`.
- **Problem**: The script unconditionally ran `npm run dist:win` (full test suite + electron-builder) even when the version's artifacts were already on disk under `release/`. When `npm run test:e2e` was broken (cycle 1 follow-up), the release path was unusable — the previous cycle had to bypass the script with direct API uploads via PowerShell. Every future agent would hit the same wall.
- **Fix**: New `--skip-build` flag. The script now also auto-detects whether all 3 required artifacts (`Setup-<v>.exe`, `Setup-<v>.exe.blockmap`, `latest.yml`) exist for the requested version and skips `dist:win` accordingly. The auto-skip is the safe default — explicit `--skip-build` documents intent. Help text updated to describe the new flag and the auto-skip behaviour.
### 2. `playwright` in `devDependencies` + simplified test scripts (release pipeline)
- **Files**: `package.json` (+ `package-lock.json`).
- **Problem**: `npm exec --yes --package=playwright -- node scripts/smoke-test*.js` failed with `MODULE_NOT_FOUND` in environments where `npm exec` couldn't resolve playwright on the fly (clean caches, locked CI runners). Cycle 1 worked around it with `npm install --no-save playwright`. Result: the documented test path was unreliable.
- **Fix**: `playwright ^1.59.1` added to `devDependencies`. `test:e2e`, `test:e2e:guide`, `test:e2e:full` now invoke `node scripts/smoke-test*.js` directly — `require('playwright')` resolves locally. No browser binary install needed because the smoke tests drive Electron via `_electron`, not a browser.
### 3. Defensive parsing in `loadConfig` and `loadQueue` (server-side correctness)
- **File**: `src/main.ts` — new `isPlainObject` / `isValidQueueStatus` / `sanitizeCustomClip` / `sanitizeMergeGroup` / `sanitizeQueueItem` helpers; rewritten `loadConfig` and `loadQueue`.
- **Problem**: `loadConfig` blindly spread `JSON.parse(data)` over the defaults. If the config file ever held a non-object (corrupt, manually edited to an array, partial write before Cycle 1's fsync landed), the spread either dropped values silently (primitives) or polluted the config object (arrays became numeric keys). `loadQueue` only validated `id`, `url`, `status` are strings — it accepted `customClip` / `mergeGroup` of any shape, never validated `progress` was a finite number, and notably never normalized stale `status: 'downloading'` items. After a hard kill mid-download, those items came back marked as still downloading with no actual download running, and `start-download` only resurrected `paused` items, leaving them stuck.
- **Fix**: `loadConfig` checks `isPlainObject(parsed)` before spread; non-objects are logged and ignored, defaults used. `loadQueue` runs every entry through `sanitizeQueueItem` which validates the `status` enum, normalizes `progress` to `[0, 100]`, validates and normalizes `customClip` / `mergeGroup` shapes, and demotes stale `status: 'downloading'` to `pending` with `progress = 0` so the user can actually resume the queue. Invalid items are dropped with a count logged. As a bonus, the previously-unused `CustomClip` and `MergeGroupItem` type imports now have call sites.
### Regression
- `npm run build` — clean (TypeScript strict, 0 errors).
- `npm run test:e2e:update-logic` — passed.
- `npm run test:e2e` — passed via the new direct script path (no `npm exec` workaround), `issues: []`.
- `npm run test:e2e:guide` — passed.
- `npm run test:merge-split` — passed.
- `npm run test:e2e:full` — passed (`failures: []`, `runtimeIssues: []`; flows: language switch, queue, duplicate prevention, runtime metrics, clip queue, pause/resume, retry, reorder, media cut/merge, update check).
## 2026-05-03 — Cycle 1: stability & UX polish ## 2026-05-03 — Cycle 1: stability & UX polish
Three independent improvements landed this cycle. Three independent improvements landed this cycle.

52
package-lock.json generated
View File

@ -1,12 +1,12 @@
{ {
"name": "twitch-vod-manager", "name": "twitch-vod-manager",
"version": "4.5.8", "version": "4.5.9",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "twitch-vod-manager", "name": "twitch-vod-manager",
"version": "4.5.8", "version": "4.5.9",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"axios": "^1.6.0", "axios": "^1.6.0",
@ -19,6 +19,7 @@
"electron-builder": "^24.9.0", "electron-builder": "^24.9.0",
"eslint": "^10.1.0", "eslint": "^10.1.0",
"eslint-plugin-security": "^4.0.0", "eslint-plugin-security": "^4.0.0",
"playwright": "^1.59.1",
"typescript": "^5.3.0", "typescript": "^5.3.0",
"typescript-eslint": "^8.57.1" "typescript-eslint": "^8.57.1"
} }
@ -3218,6 +3219,21 @@
"dev": true, "dev": true,
"license": "ISC" "license": "ISC"
}, },
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/function-bind": { "node_modules/function-bind": {
"version": "1.1.2", "version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
@ -4366,6 +4382,38 @@
"url": "https://github.com/sponsors/jonschlinkert" "url": "https://github.com/sponsors/jonschlinkert"
} }
}, },
"node_modules/playwright": {
"version": "1.59.1",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.59.1.tgz",
"integrity": "sha512-C8oWjPR3F81yljW9o5OxcWzfh6avkVwDD2VYdwIGqTkl+OGFISgypqzfu7dOe4QNLL2aqcWBmI3PMtLIK233lw==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.59.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.59.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.59.1.tgz",
"integrity": "sha512-HBV/RJg81z5BiiZ9yPzIiClYV/QMsDCKUyogwH9p3MCP6IYjUFu/MActgYAvK0oWyV9NlwM3GLBjADyWgydVyg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/plist": { "node_modules/plist": {
"version": "3.1.0", "version": "3.1.0",
"resolved": "https://registry.npmjs.org/plist/-/plist-3.1.0.tgz", "resolved": "https://registry.npmjs.org/plist/-/plist-3.1.0.tgz",

View File

@ -1,6 +1,6 @@
{ {
"name": "twitch-vod-manager", "name": "twitch-vod-manager",
"version": "4.5.8", "version": "4.5.9",
"description": "Twitch VOD Manager - Download Twitch VODs easily", "description": "Twitch VOD Manager - Download Twitch VODs easily",
"main": "dist/main.js", "main": "dist/main.js",
"author": "xRangerDE", "author": "xRangerDE",
@ -9,9 +9,9 @@
"build": "tsc", "build": "tsc",
"start": "npm run build && electron .", "start": "npm run build && electron .",
"test:e2e:update-logic": "node scripts/smoke-test-update-version-logic.js", "test:e2e:update-logic": "node scripts/smoke-test-update-version-logic.js",
"test:e2e": "npm exec --yes --package=playwright -- node scripts/smoke-test.js", "test:e2e": "node scripts/smoke-test.js",
"test:e2e:guide": "npm exec --yes --package=playwright -- node scripts/smoke-test-template-guide.js", "test:e2e:guide": "node scripts/smoke-test-template-guide.js",
"test:e2e:full": "npm exec --yes --package=playwright -- node scripts/smoke-test-full.js", "test:e2e:full": "node scripts/smoke-test-full.js",
"test:e2e:release": "npm run build && npm run test:e2e:update-logic && npm run test:e2e && npm run test:e2e:guide && npm run test:e2e:full", "test:e2e:release": "npm run build && npm run test:e2e:update-logic && npm run test:e2e && npm run test:e2e:guide && npm run test:e2e:full",
"test:e2e:stress": "npm run test:e2e:release && npm run test:e2e:release && npm run test:e2e:release", "test:e2e:stress": "npm run test:e2e:release && npm run test:e2e:release && npm run test:e2e:release",
"pack": "npm run build && electron-builder --dir", "pack": "npm run build && electron-builder --dir",
@ -31,6 +31,7 @@
"electron-builder": "^24.9.0", "electron-builder": "^24.9.0",
"eslint": "^10.1.0", "eslint": "^10.1.0",
"eslint-plugin-security": "^4.0.0", "eslint-plugin-security": "^4.0.0",
"playwright": "^1.59.1",
"typescript": "^5.3.0", "typescript": "^5.3.0",
"typescript-eslint": "^8.57.1" "typescript-eslint": "^8.57.1"
}, },

View File

@ -28,10 +28,13 @@ function parseArgs(argv) {
if (args.includes("--help") || args.includes("-h")) { if (args.includes("--help") || args.includes("-h")) {
return { help: true }; return { help: true };
} }
const FLAGS = new Set(["--dry-run", "--skip-build"]);
const dryRun = args.includes("--dry-run"); const dryRun = args.includes("--dry-run");
const version = args.find((arg) => arg !== "--dry-run") || ""; const skipBuild = args.includes("--skip-build");
const notes = args.filter((arg) => arg !== "--dry-run").slice(1).join(" ").trim(); const positional = args.filter((arg) => !FLAGS.has(arg));
return { help: false, dryRun, version, notes }; const version = positional[0] || "";
const notes = positional.slice(1).join(" ").trim();
return { help: false, dryRun, skipBuild, version, notes };
} }
function ensureVersion(version) { function ensureVersion(version) {
@ -122,10 +125,22 @@ async function uploadAssets(baseApi, releaseId, authHeader, releaseDir, files) {
} }
} }
function hasAllArtifactsForVersion(version) {
const releaseDir = path.join(process.cwd(), "release");
const files = [
`Twitch-VOD-Manager-Setup-${version}.exe`,
`Twitch-VOD-Manager-Setup-${version}.exe.blockmap`,
"latest.yml"
];
return files.every((f) => fs.existsSync(path.join(releaseDir, f)));
}
async function main() { async function main() {
const args = parseArgs(process.argv); const args = parseArgs(process.argv);
if (args.help) { if (args.help) {
process.stdout.write("Usage: npm run release:gitea -- <version> [release notes] [--dry-run]\n"); process.stdout.write("Usage: npm run release:gitea -- <version> [release notes] [--skip-build] [--dry-run]\n");
process.stdout.write(" --skip-build skip dist:win when release/ already has the 3 required artifacts\n");
process.stdout.write(" (auto-skipped when artifacts already exist for this version)\n");
process.stdout.write("Env: GITEA_BASE_URL, GITEA_REPO_OWNER, GITEA_REPO_NAME, GITEA_TOKEN\n"); process.stdout.write("Env: GITEA_BASE_URL, GITEA_REPO_OWNER, GITEA_REPO_NAME, GITEA_TOKEN\n");
return; return;
} }
@ -141,7 +156,20 @@ async function main() {
run("git", ["push", "origin", tag]); run("git", ["push", "origin", tag]);
} }
// Skip the rebuild when the user passed --skip-build OR when all artifacts
// for this version are already on disk. The original unconditional dist:win
// re-ran the full test suite + electron-builder even when the .exe already
// existed, which made the script unusable when test:e2e was broken.
const artifactsExist = hasAllArtifactsForVersion(version);
const shouldBuild = !args.skipBuild && !artifactsExist;
if (shouldBuild) {
run(NPM_EXECUTABLE, ["run", "dist:win"]); run(NPM_EXECUTABLE, ["run", "dist:win"]);
} else if (artifactsExist) {
process.stdout.write(`Skipping dist:win — artifacts for ${tag} already exist in release/\n`);
} else {
process.stdout.write(`Skipping dist:win (--skip-build)\n`);
}
const assets = ensureAssets(version); const assets = ensureAssets(version);
if (args.dryRun) { if (args.dryRun) {
process.stdout.write(`Dry run complete for ${tag}\n`); process.stdout.write(`Dry run complete for ${tag}\n`);

View File

@ -256,11 +256,20 @@ function normalizeConfigTemplates(input: Config): Config {
}; };
} }
function isPlainObject(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function loadConfig(): Config { function loadConfig(): Config {
try { try {
if (fs.existsSync(CONFIG_FILE)) { if (fs.existsSync(CONFIG_FILE)) {
const data = fs.readFileSync(CONFIG_FILE, 'utf-8'); const data = fs.readFileSync(CONFIG_FILE, 'utf-8');
return normalizeConfigTemplates({ ...defaultConfig, ...JSON.parse(data) }); const parsed = JSON.parse(data);
if (!isPlainObject(parsed)) {
console.error('Config file is not a JSON object — using defaults');
return normalizeConfigTemplates(defaultConfig);
}
return normalizeConfigTemplates({ ...defaultConfig, ...parsed });
} }
} catch (e) { } catch (e) {
console.error('Error loading config:', e); console.error('Error loading config:', e);
@ -303,6 +312,111 @@ function saveConfig(config: Config): void {
// ========================================== // ==========================================
// QUEUE MANAGEMENT // QUEUE MANAGEMENT
// ========================================== // ==========================================
const VALID_QUEUE_STATUSES: ReadonlyArray<QueueItem['status']> = ['pending', 'downloading', 'paused', 'completed', 'error'];
const VALID_MERGE_PHASES: ReadonlyArray<MergeGroup['mergePhase']> = ['downloading', 'merging', 'splitting', 'cleanup', 'done'];
function isValidQueueStatus(status: unknown): status is QueueItem['status'] {
return typeof status === 'string' && (VALID_QUEUE_STATUSES as readonly string[]).includes(status);
}
function sanitizeMergeGroup(raw: unknown): MergeGroup | undefined {
if (!isPlainObject(raw)) return undefined;
if (!Array.isArray(raw.items) || raw.items.length < 2) return undefined;
const items: MergeGroupItem[] = [];
for (const mi of raw.items) {
if (!isPlainObject(mi)) continue;
if (typeof mi.url !== 'string' || typeof mi.title !== 'string'
|| typeof mi.date !== 'string' || typeof mi.streamer !== 'string'
|| typeof mi.duration_str !== 'string') continue;
items.push({ url: mi.url, title: mi.title, date: mi.date, streamer: mi.streamer, duration_str: mi.duration_str });
}
if (items.length < 2) return undefined;
const phase: MergeGroup['mergePhase'] = (VALID_MERGE_PHASES as readonly string[]).includes(String(raw.mergePhase))
? raw.mergePhase as MergeGroup['mergePhase']
: 'downloading';
const downloadedFiles: Record<number, string> = {};
if (isPlainObject(raw.downloadedFiles)) {
for (const [k, v] of Object.entries(raw.downloadedFiles)) {
const idx = Number(k);
if (Number.isFinite(idx) && typeof v === 'string') downloadedFiles[idx] = v;
}
}
return {
items,
mergePhase: phase,
currentItemIndex: typeof raw.currentItemIndex === 'number' && Number.isFinite(raw.currentItemIndex) ? raw.currentItemIndex : 0,
downloadedFiles,
mergedFile: typeof raw.mergedFile === 'string' ? raw.mergedFile : undefined,
splitFiles: Array.isArray(raw.splitFiles) ? raw.splitFiles.filter((f): f is string => typeof f === 'string') : undefined,
totalDurationSec: typeof raw.totalDurationSec === 'number' && Number.isFinite(raw.totalDurationSec) ? raw.totalDurationSec : undefined
};
}
function sanitizeCustomClip(raw: unknown): CustomClip | undefined {
if (!isPlainObject(raw)) return undefined;
const startSec = Number(raw.startSec);
const durationSec = Number(raw.durationSec);
const startPart = Number(raw.startPart);
if (!Number.isFinite(startSec) || !Number.isFinite(durationSec) || durationSec <= 0 || !Number.isFinite(startPart)) return undefined;
const filenameFormat = raw.filenameFormat;
if (filenameFormat !== 'simple' && filenameFormat !== 'timestamp' && filenameFormat !== 'template') return undefined;
return {
startSec: Math.max(0, startSec),
durationSec: Math.max(1, durationSec),
startPart: Math.max(1, Math.floor(startPart)),
filenameFormat,
filenameTemplate: typeof raw.filenameTemplate === 'string' ? raw.filenameTemplate : undefined
};
}
function sanitizeQueueItem(raw: unknown): QueueItem | null {
if (!isPlainObject(raw)) return null;
if (typeof raw.id !== 'string' || !raw.id) return null;
if (typeof raw.url !== 'string' || !raw.url) return null;
if (!isValidQueueStatus(raw.status)) return null;
// 'downloading' on cold start is stale — no download is actually running
// and the user expects to resume from start, so map it back to 'pending'
const isStaleDownloading = raw.status === 'downloading';
const finalStatus: QueueItem['status'] = isStaleDownloading ? 'pending' : raw.status;
const progressNum = Number(raw.progress);
const safeProgress = Number.isFinite(progressNum) ? Math.max(0, Math.min(100, progressNum)) : 0;
const item: QueueItem = {
id: raw.id,
url: raw.url,
title: typeof raw.title === 'string' ? raw.title : '',
date: typeof raw.date === 'string' ? raw.date : '',
streamer: typeof raw.streamer === 'string' ? raw.streamer : '',
duration_str: typeof raw.duration_str === 'string' ? raw.duration_str : '0s',
status: finalStatus,
progress: isStaleDownloading ? 0 : safeProgress
};
if (typeof raw.currentPart === 'number' && Number.isFinite(raw.currentPart)) item.currentPart = raw.currentPart;
if (typeof raw.totalParts === 'number' && Number.isFinite(raw.totalParts)) item.totalParts = raw.totalParts;
if (typeof raw.speed === 'string') item.speed = raw.speed;
if (typeof raw.eta === 'string') item.eta = raw.eta;
if (typeof raw.last_error === 'string') item.last_error = raw.last_error;
if (typeof raw.downloadedBytes === 'number' && Number.isFinite(raw.downloadedBytes)) item.downloadedBytes = raw.downloadedBytes;
if (typeof raw.totalBytes === 'number' && Number.isFinite(raw.totalBytes)) item.totalBytes = raw.totalBytes;
const customClip = sanitizeCustomClip(raw.customClip);
if (customClip) item.customClip = customClip;
const mergeGroup = sanitizeMergeGroup(raw.mergeGroup);
if (mergeGroup) item.mergeGroup = mergeGroup;
return item;
}
function loadQueue(): QueueItem[] { function loadQueue(): QueueItem[] {
if (config.persist_queue_on_restart === false) { if (config.persist_queue_on_restart === false) {
return []; return [];
@ -312,16 +426,25 @@ function loadQueue(): QueueItem[] {
if (fs.existsSync(QUEUE_FILE)) { if (fs.existsSync(QUEUE_FILE)) {
const data = fs.readFileSync(QUEUE_FILE, 'utf-8'); const data = fs.readFileSync(QUEUE_FILE, 'utf-8');
const parsed = JSON.parse(data); const parsed = JSON.parse(data);
if (!Array.isArray(parsed)) return []; if (!Array.isArray(parsed)) {
return parsed.filter((item: any) => console.error('Queue file is not a JSON array — ignoring');
item && typeof item.id === 'string' && return [];
typeof item.url === 'string' && }
typeof item.status === 'string'
); const items: QueueItem[] = [];
let droppedCount = 0;
for (const raw of parsed) {
const sanitized = sanitizeQueueItem(raw);
if (sanitized) items.push(sanitized);
else droppedCount++;
}
if (droppedCount > 0) {
console.error(`loadQueue: dropped ${droppedCount} invalid queue item(s)`);
}
return items;
} }
} catch (e) { } catch (e) {
console.error('Error loading queue:', e); console.error('Error loading queue:', e);
console.error('queue-load-error', { error: String(e) });
} }
return []; return [];
} }