c3381d360f
The v3.3.98 instrument exposed the actual cause of the 1-2s UI button lag, and it was none of the read-path suspects. electron-config.json had grown to 38.5MB and was being load()/structuredClone()/JSON.stringify()'d ~137 times in a 73s window (140-592ms each) on the Electron main thread — roughly 47% main-thread occupancy. That is the lag: a button click lands while the thread is mid-clone of a 38.5MB object. The 1MB read-ahead could not touch it. The bulk is history, not the queue. Each batch-done appended the upload-manager summary verbatim, including the full per-file result list (per-hoster URLs), into config.history; default historyRetention='all' never prunes, so it grew unbounded (75 batches ≈ 38.5MB). The "queue=undefined" in the perf log was a logging bug (reading .length on the pendingQueue object), not an empty queue. Writes drove the storm: queue-persistence (save-global-settings) did two loads + one 38.5MB serialize per call, and _atomicWrite nulls the cache so the next load is a full 38.5MB reparse; even cache hits structuredCloned the whole 38.5MB. The per-job upload path makes zero config calls, so pending=1280 was never the driver. Fix — move history into its own file so it is never parsed/cloned/serialized on a config op (a 5-agent investigation + adversarial review chose this over three read-side half-fixes; it is the only change that removes the clone AND the serialize AND the post-write reparse at once): - History lives in electron-history.json. _migrateHistory() runs once at init (packaged only) and is fail-safe: it writes history.json (tmp → fsync → rename), re-reads and verifies the entry count, and keeps a permanent electron-config.json.pre-history-split.bak BEFORE the config is ever allowed to drop its history. If verification fails it leaves history in the config (retry next launch). _loadImpl returns history:[] once migrated, so the cached object is tiny (cheap clones); the config file shrinks to ~KB on the first save (cheap reparse) and _serializeForDisk writes ~KB (cheap serialize). loadHistory/appendHistory/pruneHistory/clearHistory go through history.json on their own write-queue with a no-clobber guard; the legacy config path stays as a fallback when migration did not run. - Validated against the real 185MB / 30000-entry bench fixture: migrate 1.5s once, every entry preserved + .bak kept; load() 631ms cold once → 0.1ms after the first save strips the file (185MB → 2.1KB); loadHistory() still returns all 30000. Dropped per review (one-variable + risk): loadShallow (moot after the split), cache-repopulate (its gate can never fire), and a per-batch resolution cache (stale account pools → the rotation/byse failover-regression class — the one thing that could silently corrupt uploads). Instrumentation (the user asked to measure everything; all additive, threshold- gated, MHU_PERF=0 disables): - ipcMain.handle/.on are centrally wrapped to log `ipc <channel> wall=Xms sync=Yms` over 50ms — the button-press→response latency — hardened (Promise.resolve(p) .finally + try/catch'd logging) so a logging failure can never break IPC. - A 100ms main-process drift monitor logs `main-longtask blocked=Xms lastIpc=… gc=…` for any single main-thread turn over 100ms (catches GC, fs scans, serialize that the IPC timing structurally cannot see). - config-store perf lines gain via=<caller> and wqDepth=, and the queue= logging bug is fixed (now reads pendingQueue.queueJobs.length). 405 tests pass (9 new migration tests: preserve-count, round-trip, save() never loses history, crash-window fallback, idempotency). Clean Electron boot, no repo pollution (migration is packaged-only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
57 lines
1.4 KiB
JSON
57 lines
1.4 KiB
JSON
{
|
|
"name": "multi-hoster-uploader",
|
|
"version": "3.3.99",
|
|
"description": "Upload files to doodstream, voe, vidmoly, byse simultaneously",
|
|
"main": "main.js",
|
|
"scripts": {
|
|
"start": "electron .",
|
|
"test": "node --test tests/*.test.js tests/ui-smoke.js",
|
|
"dist": "electron-builder --win",
|
|
"release:win": "electron-builder --publish never --win nsis portable",
|
|
"release:gitea": "node scripts/release_gitea.mjs"
|
|
},
|
|
"dependencies": {
|
|
"chokidar": "^3.6.0",
|
|
"undici": "^7.16.0",
|
|
"ws": "^8.19.0"
|
|
},
|
|
"devDependencies": {
|
|
"electron": "^41.3.0",
|
|
"electron-builder": "^26.8.1",
|
|
"eslint": "^10.1.0",
|
|
"eslint-plugin-security": "^4.0.0",
|
|
"rcedit": "^4.0.1"
|
|
},
|
|
"build": {
|
|
"appId": "com.multihoster.uploader",
|
|
"productName": "Multi-Hoster-Upload",
|
|
"directories": {
|
|
"buildResources": "assets",
|
|
"output": "release"
|
|
},
|
|
"files": [
|
|
"main.js",
|
|
"preload.js",
|
|
"lib/**/*",
|
|
"renderer/**/*",
|
|
"assets/app_icon.ico",
|
|
"assets/app_icon.png"
|
|
],
|
|
"win": {
|
|
"target": [
|
|
"nsis",
|
|
"portable"
|
|
],
|
|
"icon": "assets/app_icon.ico",
|
|
"signAndEditExecutable": false
|
|
},
|
|
"nsis": {
|
|
"oneClick": false,
|
|
"perMachine": false,
|
|
"allowToChangeInstallationDirectory": true,
|
|
"createDesktopShortcut": true
|
|
},
|
|
"afterPack": "scripts/afterPack.cjs"
|
|
}
|
|
}
|