a5b835f76a
The v3.3.96 eventloop-delay logs from a real 70-connection run pinpointed the
high-concurrency lag to the file-read path. At a CONSTANT active-job count the
process flips between two clean regimes:
HEALTHY ELD ~11ms, rss 268-308MB: SimpleWriteWrap ≈ active, FSReqCallback ≈ 0-1
BLOCKED ELD 49-217ms, rss 540-610MB: FSReqCallback ≈ active (62-71 reads in
flight), SimpleWriteWrap ≈ 0-4
Both the ELD spike and the rss balloon track FSReqCallback (libuv-threadpool file
reads) exactly — not crypto, not the renderer, not GC alone. All five uploaders
read identically via fs.createReadStream({highWaterMark: 256KB}); byse/doodstream/
voe run through the generic uploadFile in hosters.js (no dedicated module).
This build is both a candidate fix and a discriminator, per advisor review:
- UV_THREADPOOL_SIZE 64→8 (main.js:1). One reversible line, NOT an upload cap —
70 uploads still run. 8 concurrent 256KB reads sustain ~100MB/s, far above the
41MB/s aggregate, so it cannot bottleneck throughput even on the slow VM disk.
Strong suspicion that tp=64 made it worse: it removed the natural read-
serialization (default 4 threads) and let all 70 streams' reads fire at once,
flooding the loop with completion callbacks in lock-step bursts.
- ELD line now also logs heap=heapUsed ext=external ab=arrayBuffers and
gc=/gcTotal=/gcMax=ms (PerformanceObserver entryTypes:['gc'], reset per window).
The 70×256KB ≈ 18MB of read buffers cannot account for the ~300MB rss swing —
that is heap/object churn, so GC must be measured directly.
Decision rule for the next real-run log:
- ELD drops with tp=8 → read over-parallelism confirmed (keep 8 or add
a dedicated read-semaphore).
- ELD high + GC pauses align → heap churn, hunt the allocator.
- ELD high + GC flat → causation was reversed, pivot.
No upload-behavior change; the concurrency cap the user explicitly rejected is
untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
57 lines
1.4 KiB
JSON
57 lines
1.4 KiB
JSON
{
|
|
"name": "multi-hoster-uploader",
|
|
"version": "3.3.97",
|
|
"description": "Upload files to doodstream, voe, vidmoly, byse simultaneously",
|
|
"main": "main.js",
|
|
"scripts": {
|
|
"start": "electron .",
|
|
"test": "node --test tests/*.test.js tests/ui-smoke.js",
|
|
"dist": "electron-builder --win",
|
|
"release:win": "electron-builder --publish never --win nsis portable",
|
|
"release:gitea": "node scripts/release_gitea.mjs"
|
|
},
|
|
"dependencies": {
|
|
"chokidar": "^3.6.0",
|
|
"undici": "^7.16.0",
|
|
"ws": "^8.19.0"
|
|
},
|
|
"devDependencies": {
|
|
"electron": "^41.3.0",
|
|
"electron-builder": "^26.8.1",
|
|
"eslint": "^10.1.0",
|
|
"eslint-plugin-security": "^4.0.0",
|
|
"rcedit": "^4.0.1"
|
|
},
|
|
"build": {
|
|
"appId": "com.multihoster.uploader",
|
|
"productName": "Multi-Hoster-Upload",
|
|
"directories": {
|
|
"buildResources": "assets",
|
|
"output": "release"
|
|
},
|
|
"files": [
|
|
"main.js",
|
|
"preload.js",
|
|
"lib/**/*",
|
|
"renderer/**/*",
|
|
"assets/app_icon.ico",
|
|
"assets/app_icon.png"
|
|
],
|
|
"win": {
|
|
"target": [
|
|
"nsis",
|
|
"portable"
|
|
],
|
|
"icon": "assets/app_icon.ico",
|
|
"signAndEditExecutable": false
|
|
},
|
|
"nsis": {
|
|
"oneClick": false,
|
|
"perMachine": false,
|
|
"allowToChangeInstallationDirectory": true,
|
|
"createDesktopShortcut": true
|
|
},
|
|
"afterPack": "scripts/afterPack.cjs"
|
|
}
|
|
}
|