Administrator/Multi-Hoster-Upload
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 260 Wiki Activity

perf(clouddrop): stream chunk reads off the main event loop (async fh.read)

Browse Source 
_uploadChunked read each 16 MB chunk with fs.readSync on the main JS thread — the only one of the five uploaders that blocks synchronously (the other four stream async). Each readSync stalls the whole event loop ~5-9 ms on SSD, 30-100 ms on a slow disk, freezing all progress emits, IPC, renders and every other concurrent upload for that window. The stall scales with the number of simultaneous clouddrop uploads, matching the 'feels laggy while uploading, worse with more at once' symptom at modest CPU (one core pinned, ~40% of 8).

Swap fs.openSync/readSync/closeSync for fs.promises.open + await fh.read + await fh.close. Buffer reuse and the partial-last-chunk subarray view are unchanged. Verified byte-identical to the old loop via SHA-256 over every chunk-boundary case (full chunk, partial last chunk, 2/3/4-chunk files, single byte) before shipping — a chunk-read bug would corrupt the upload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Administrator 2026-06-21 03:44:31 +02:00
parent ea14d11ee2
commit c6a67f6f2f
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
lib/clouddrop-upload.js
View File

@ -159,7 +159,7 @@ class ClouddropUploader {
// Reuse a single buffer for all chunks (only the last chunk may be smaller, // Reuse a single buffer for all chunks (only the last chunk may be smaller,
// in which case we slice a view). Avoids 64× 16 MB allocations on a 1 GB // in which case we slice a view). Avoids 64× 16 MB allocations on a 1 GB
// file — real GC pressure during busy uploads. // file — real GC pressure during busy uploads.
const fd = fs.openSync(filePath, 'r'); const fh = await fs.promises.open(filePath, 'r');
let bytesSent = 0; let bytesSent = 0;
const reusableBuf = Buffer.allocUnsafe(chunkSize); const reusableBuf = Buffer.allocUnsafe(chunkSize);
try { try {
@ -169,7 +169,7 @@ class ClouddropUploader {
const offset = i * chunkSize; const offset = i * chunkSize;
const remaining = fileSize - offset; const remaining = fileSize - offset;
const thisChunkSize = Math.min(chunkSize, remaining); const thisChunkSize = Math.min(chunkSize, remaining);
fs.readSync(fd, reusableBuf, 0, thisChunkSize, offset); await fh.read(reusableBuf, 0, thisChunkSize, offset);
const body = thisChunkSize === chunkSize const body = thisChunkSize === chunkSize
? reusableBuf ? reusableBuf
: reusableBuf.subarray(0, thisChunkSize); : reusableBuf.subarray(0, thisChunkSize);
@ -194,7 +194,7 @@ class ClouddropUploader {
if (progressCb) progressCb(bytesSent, fileSize); if (progressCb) progressCb(bytesSent, fileSize);
} }
} finally { } finally {
try { fs.closeSync(fd); } catch {} try { await fh.close(); } catch {}
} }
// 3. Complete session — all bytes are already on the server at this point. // 3. Complete session — all bytes are already on the server at this point.