Multi-Hoster-Upload/lib/throttle-timer.js
Administrator eeec1d150c fix(queue): completed files no longer reappear in the queue after restart
Closing the app (especially during an active upload or on a hard kill) and
reopening sometimes left already-uploaded files sitting in the queue as if
still pending. Root cause is three layers stacked:

1. Persist-starvation: persistQueueStateSoon() reset a 10s debounce on every
   progress event, so during an upload the on-disk queue snapshot was never
   rewritten and stayed frozen at the pre-upload state (all jobs "preview").
2. The beforeunload sync flush only covers a clean close; a hard kill / crash
   leaves that stale snapshot on disk.
3. Startup auto-dedup only dropped jobs with status "done". The completed
   files were stored as "preview" in the stale snapshot, so they survived and
   reappeared.

Fix (mechanism-independent — holds whether the stale snapshot came from
starvation, a mid-upload close race, or a hard kill):

FIX A (core, durable): the upload log is the source of truth. Each persisted
snapshot is now stamped with savedAt; on restart any restored job whose newest
matching log entry is timestamped at/after floor(savedAt) is dropped regardless
of status — it provably completed after the snapshot, so a "preview" row for it
is a ghost. lib/queue-dedup.js gains an additive 3rd savedAt param; without
savedAt or without log timestamps it behaves exactly as before (the 5 canary
tests stay green, so intentional re-uploads of older files still survive).

FIX B: new lib/throttle-timer.js with a max-wait. During uploads the snapshot
is now written at most ~20s into a continuous progress burst instead of never;
idle stays a pure debounce. The fallback shim honors max-wait too, so a missing
library can never silently reintroduce the starvation.

FIX C: the synchronous close-write retries renameSync on EBUSY/EPERM/EACCES and
uses a pid-unique tmp (de-conflicts it from config-store._atomicWrite's fixed
.tmp). A startup sweep reclaims orphaned <config>.<pid>.tmp files left by a hard
kill between write and rename.

lib/upload-log.js extracts formatUploadLogLine + parseUploadLogLine from main.js
so the real writer -> reader -> gate seam is unit-tested (a future log-format or
epoch-basis change can no longer pass green while breaking the fix).

Verified: 334/334 tests green (incl. throttle fake-clock starvation/maxWait,
ts-gate multi-hoster partial-completion, and the real-format seam tests), ESLint
clean, smoke-boot identical to baseline, and an adversarial multi-agent review
(15 findings, 14 refuted, 1 low — the tmp orphan, now swept) on the diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 04:35:27 +02:00

60 lines
1.8 KiB
JavaScript

(function (root) {
'use strict';
function makeThrottleTimer(opts) {
const o = opts || {};
const now = typeof o.now === 'function' ? o.now : (() => Date.now());
const schedule = typeof o.schedule === 'function'
? o.schedule
: ((cb, ms) => setTimeout(cb, ms));
const clear = typeof o.clear === 'function' ? o.clear : ((h) => clearTimeout(h));
let handle = null;
let burstStart = null;
let pendingFn = null;
function fire() {
handle = null;
burstStart = null;
const fn = pendingFn;
pendingFn = null;
if (typeof fn === 'function') fn();
}
function request(fn, delay, maxWait) {
if (typeof fn === 'function') pendingFn = fn;
const t = now();
if (burstStart === null) burstStart = t;
let wait = typeof delay === 'number' && delay >= 0 ? delay : 0;
if (typeof maxWait === 'number' && maxWait >= 0) {
const remaining = maxWait - (t - burstStart);
wait = Math.min(wait, remaining < 0 ? 0 : remaining);
}
if (handle !== null) clear(handle);
handle = schedule(fire, wait);
}
function flushSync() {
if (handle !== null) { clear(handle); handle = null; }
burstStart = null;
const fn = pendingFn;
pendingFn = null;
if (typeof fn === 'function') fn();
}
function cancel() {
if (handle !== null) { clear(handle); handle = null; }
burstStart = null;
pendingFn = null;
}
function isPending() { return handle !== null; }
return { request, flushSync, cancel, isPending };
}
const api = { makeThrottleTimer };
if (typeof module !== 'undefined' && module.exports) module.exports = api;
else if (root) root.ThrottleTimer = api;
})(typeof window !== 'undefined' ? window : this);