66ae240794
74 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Administrator
|
66ae240794 |
feat(logs): session log filename → DD-MM-YYYY-mdu-session-HH-MM (v3.3.107)
Per user request, the per-session log file is renamed from
fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log to
DD-MM-YYYY-mdu-session-HH-MM.log (hour and minute only, no seconds or pid).
lib/log-mode.js:
- formatSessionStamp(date) returns `${DD}-${MM}-${YYYY}-mdu-session-${HH}-${MM}`
(the pid argument is dropped; main.js still passes process.pid, harmlessly
ignored). Same-minute restarts now share a session file, which is the intended
human-readable trade.
- the session branch of resolveLogFileName returns `${sessionId}${ext}` — the stamp
is the full app-defined stem and baseName is intentionally ignored for session
mode (single/daily still use the 'fileuploader' base).
- stripModeStampFromFileName recognizes the new format and resets to the default
'fileuploader' base (the new stem embeds no base, so the configured base is not
recoverable from it); the existing daily and old-session strip regexes are kept
for backward-compat with any persisted old paths, and the persist/re-resolve
round-trip stays idempotent (no compounding stamps).
Tests updated for the new format (formatSessionStamp, session resolveLogFileName,
the new-format strip, and the idempotency regression). 410 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
7a40afbe7e |
fix(config): fsync config writes + guard against account-wipe on a corrupt read (v3.3.105)
A user's server crashed hard during an upload and lost all configured accounts. It
was NOT the v3.3.104 update (a second server updated fine and kept its accounts) —
it was a data-durability hole exposed by the crash:
- Config writes were atomic (tmp + rename) but never fsync'd, so a hard crash could
leave electron-config.json truncated/unflushed on disk.
- On restart, load() reads the truncated file, falls back to .bak, and if that is
also bad returns empty DEFAULTS. The next settings/queue save then persists EMPTY
hosters — permanently wiping the accounts. Worse, the async _atomicWrite blindly
copied the (now truncated) live file over .bak, so an empty live could clobber a
good backup.
Hardening (lib/config-store.js + main.js; no behavior change in the happy path):
- fsync before rename in both write paths — _atomicWrite (openSync/writeSync/
fsyncSync/closeSync) and the synchronous save-global-settings-sync on window close.
A hard crash can no longer leave a truncated config.
- _atomicWrite only refreshes .bak when the current live file is non-trivial
(trim length > 2), so an empty/truncated live can never overwrite a good backup
(the sync-save path already did this).
- Wipe-guard (_guardHosters): save(), saveRotationCursors() and the sync close-save
never intend to change hosters; if after a load() the hosters are all empty and
the write did not explicitly provide hosters, recover them from disk
(_recoverHostersFromDisk: live -> .bak -> .pre-history-split.bak) instead of
persisting the wipe. An explicit save({hosters: {}}) (user deleted all accounts)
is still allowed. Restored hosters are already-encrypted on disk and
encryptCredentials skips already-encrypted fields, so re-serializing is safe.
- load() gained a third fallback tier — the permanent pre-history-split.bak snapshot
(which still holds the accounts) — so load() itself recovers after corruption.
Recovery for the already-affected server: copy
%APPDATA%/multi-hoster-uploader/electron-config.json.pre-history-split.bak (or .bak)
over electron-config.json with the app closed.
2 new regression tests (post-wipe valid-empty live + .bak → guard restores accounts;
an explicit empty-hosters save is not blocked). 409 tests pass; clean boot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
7f636258d4 |
perf(uploads): make the batch-start file-stat non-blocking — kill the 336ms main stall (v3.3.103)
The v3.3.102 log (real 224-job batch) confirmed History virtualization works and
steady-state uploads are pristine (fps=32, event-loop mean 11.8ms). The remaining
residual was a ~6s spin-up burst at batch start: the main event loop blocked for
336ms with cpu=0%core (i.e. blocked on I/O, not computing) and the renderer janked
196-391ms, then everything settled clean.
A multi-agent investigation plus an adversarial review corrected the obvious-looking
hypothesis. The renderer's uncapped progress-batch drain is NOT the cause:
handleProgress only mutates plain JS state and schedules already-coalesced renders
(one per frame), and main coalesces progress to ~50 latest-per-job entries per
100ms. Chunking that drain would fix nothing — and the reviewer showed it would
REGRESS correctness: requestAnimationFrame throttles to ~0 when the window is
minimized (the common state for a background uploader), so a rAF-chunked drain would
grow an unbounded backlog and defer persistQueueStateSoon for every buffered item,
losing terminal 'done' events on close (the queue-persistence ghost-fix class). So
that path is deliberately not taken.
The real cause (cpu=0%core = blocked on I/O) is a synchronous fs.statSync storm in
UploadManager.startBatch: the dedup loop ran up to DEDUP_CHUNK=200 synchronous
fs.statSync calls in a single tick before yielding (200 x ~1.68ms on the user's VM
= the exact 336ms), on a disk already saturated by the 1MB read-ahead.
Fix — make the batch-start stats non-blocking:
- The dedup loop now dedupes synchronously (cheap Map work) and then stats the
unique files in parallel via await Promise.all(fs.promises.stat ...) per chunk, so
the stat I/O runs on the libuv threadpool and the main thread never blocks. The
results-Map shape ({name,size,results:[]}) and dedup semantics (size 0 on failure)
are unchanged.
- The per-job statSync fallback is converted to await fs.promises.stat for
consistency (it sits in an async function before the first real await; the cached
size from dedup already lets nearly every job skip it).
Tests: the upload-manager mocks override fs.statSync; they now also override
fs.promises.stat with the same fake sizes (upload-manager.test.js x2,
suspect-reject-alternates.test.js). 407 tests pass; clean Electron boot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
bad1c665f5 |
perf(history): gate the unconditional History-tab reload + fix the diagnostics history regression (v3.3.101)
v3.3.100's interaction instrument pinpointed the residual UI lag exactly: every slow click was a tab switch into the History view (button.tab / nav.tab-bar, 200-840ms), each coupled to `ipc get-history wall=150-200ms sync` + `main-longtask blocked=242-289ms lastIpc=get-history` + a 200-248ms renderer long task. Uploads themselves are pristine (event-loop mean 11.7ms; the only spikes line up with the get-history tab switches). Root cause: the History tab handler called loadHistory() UNCONDITIONALLY on every activation (renderer/app.js) even though it tracks a `_historyDirty` flag it never checked. Each call synchronously readFileSync + JSON.parse's the ~185MB / 30000-entry electron-history.json (no cache), ships all 30000 batches across IPC, and the renderer flattens ~120000 row objects before .slice(-2000) for the DOM (the DOM is already capped at 2000). Fix (the adversary-verified safe subset): - Gate the History-tab load: only reload when `_historyDirty || !_historyEverLoaded`. Added `_historyEverLoaded`, and both flags are now set inside loadHistory() after the fetch succeeds (so a failed load retries). Dirty-coverage is complete — every history append routes through batch-done -> appendHistory and upload-batch-done -> handleBatchDone which sets _historyDirty=true. Result: repeat History tab switches with no new uploads do zero IPC/parse/flatten and are instant. (The first open after a new batch still parses once ~450ms; removing that needs a parse-cache or JSONL storage, deferred.) - Diagnostics history regression: getHistory() read loadConfig().history, but since the v3.3.99 history split load() returns history:[] in packaged mode, so remote diagnostics reported totalBatches:0 despite 30000 real batches. It now reads loadHistory() (injected via the collector deps), with a backward-compatible fallback to loadConfig().history when not provided. 407 tests pass (2 new diagnostics regression tests); clean Electron boot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
c3381d360f |
perf(config): split history out of the hot config file — the real 38.5MB main-thread thrash (v3.3.99)
The v3.3.98 instrument exposed the actual cause of the 1-2s UI button lag, and it was none of the read-path suspects. electron-config.json had grown to 38.5MB and was being load()/structuredClone()/JSON.stringify()'d ~137 times in a 73s window (140-592ms each) on the Electron main thread — roughly 47% main-thread occupancy. That is the lag: a button click lands while the thread is mid-clone of a 38.5MB object. The 1MB read-ahead could not touch it. The bulk is history, not the queue. Each batch-done appended the upload-manager summary verbatim, including the full per-file result list (per-hoster URLs), into config.history; default historyRetention='all' never prunes, so it grew unbounded (75 batches ≈ 38.5MB). The "queue=undefined" in the perf log was a logging bug (reading .length on the pendingQueue object), not an empty queue. Writes drove the storm: queue-persistence (save-global-settings) did two loads + one 38.5MB serialize per call, and _atomicWrite nulls the cache so the next load is a full 38.5MB reparse; even cache hits structuredCloned the whole 38.5MB. The per-job upload path makes zero config calls, so pending=1280 was never the driver. Fix — move history into its own file so it is never parsed/cloned/serialized on a config op (a 5-agent investigation + adversarial review chose this over three read-side half-fixes; it is the only change that removes the clone AND the serialize AND the post-write reparse at once): - History lives in electron-history.json. _migrateHistory() runs once at init (packaged only) and is fail-safe: it writes history.json (tmp → fsync → rename), re-reads and verifies the entry count, and keeps a permanent electron-config.json.pre-history-split.bak BEFORE the config is ever allowed to drop its history. If verification fails it leaves history in the config (retry next launch). _loadImpl returns history:[] once migrated, so the cached object is tiny (cheap clones); the config file shrinks to ~KB on the first save (cheap reparse) and _serializeForDisk writes ~KB (cheap serialize). loadHistory/appendHistory/pruneHistory/clearHistory go through history.json on their own write-queue with a no-clobber guard; the legacy config path stays as a fallback when migration did not run. - Validated against the real 185MB / 30000-entry bench fixture: migrate 1.5s once, every entry preserved + .bak kept; load() 631ms cold once → 0.1ms after the first save strips the file (185MB → 2.1KB); loadHistory() still returns all 30000. Dropped per review (one-variable + risk): loadShallow (moot after the split), cache-repopulate (its gate can never fire), and a per-batch resolution cache (stale account pools → the rotation/byse failover-regression class — the one thing that could silently corrupt uploads). Instrumentation (the user asked to measure everything; all additive, threshold- gated, MHU_PERF=0 disables): - ipcMain.handle/.on are centrally wrapped to log `ipc <channel> wall=Xms sync=Yms` over 50ms — the button-press→response latency — hardened (Promise.resolve(p) .finally + try/catch'd logging) so a logging failure can never break IPC. - A 100ms main-process drift monitor logs `main-longtask blocked=Xms lastIpc=… gc=…` for any single main-thread turn over 100ms (catches GC, fs scans, serialize that the IPC timing structurally cannot see). - config-store perf lines gain via=<caller> and wqDepth=, and the queue= logging bug is fixed (now reads pendingQueue.queueJobs.length). 405 tests pass (9 new migration tests: preserve-count, round-trip, save() never loses history, crash-window fallback, idempotency). Clean Electron boot, no repo pollution (migration is packaged-only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
29d1944328 |
perf(config): cache parsed config + lean serialize + drop load() from log flush (long-run lag)
A 13-agent hunt pinned the "wird mit der Zeit laggy" symptom (CPU/RAM normal, UI sluggish after many uploads) to the main process re-doing config I/O that scales with the ever-growing history, stalling the synchronous main event loop so the renderer's IPC round-trips feel laggy. The renderer render path was already optimized (virtualized queue, capped panels) — confirmed clean. This commit lands the two contained fixes (T1 + T3); the queue-persist rewrite (T2) follows separately. config-store (T1): - load() now has an in-memory cache keyed on the file mtime+size. The processed config (merged + credential-decrypted) is re-read/re-parsed/re-DPAPI-decrypted ONLY when the file actually changes; our writes invalidate it, external edits change mtime/size so the cache misses. Eliminates a full disk read + JSON.parse of the whole growing history + per-credential decrypt on the vast majority of the ~38 load() call sites. load() always returns a structuredClone so callers can mutate freely without corrupting the cache. - _serializeForDisk clones ONLY the hosters subtree (the only thing encryptCredentials touches) instead of JSON.parse(JSON.stringify(whole config)) — no more deep-cloning an 8 MB history on every write. - _atomicWrite refreshes the .bak with a raw fs.copyFileSync instead of read + JSON.parse + write (it was re-parsing the full config a 2nd time per write). main.js (T3): - The log-flush paths resolved the log file via configStore.load() ~8x/second during uploads (re-reading + cloning the whole config just to read logMode/logFilePath). Cache those two strings in module scope, invalidate on the settings-save handlers. Verified: 26 config-store tests (incl. new cache-correctness: independent clones, external-change invalidation, save invalidation) + full 394-test suite green, lint 0 errors. Benchmark (8000-batch / 4.6 MB history): log flush no longer calls load() at all; the remaining per-write history serialize is what T2 removes from the hot path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
b25b51840d |
test(diagnostics): live network-bind path — allowlisted non-loopback peer connects over a real 0.0.0.0 socket
Closes the one link the unit/wiring tests covered only by composition: binds 0.0.0.0, allowlists a real LAN IPv4, and asserts auth-ok over a real socket (the Tailscale path). Skips when no non-internal IPv4 interface exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
0c6c502aab |
feat(diagnostics): network bind + fail-closed IP allowlist + host-in-code (Tailscale, like rd-diagnostics-mcp)
Matches the Real-Debrid-Downloader's rd-diagnostics-mcp model so the read-only
diagnostics agent is reachable over Tailscale (or any private tunnel) the same way
the downloader is, instead of requiring an SSH local-forward.
- lib/ip-allowlist.js (NEW): fail-closed IP allowlist — normalizeIp strips
::ffff:, loopback is always allowed, an empty allowlist accepts loopback ONLY
(fail-closed), exact IP + CIDR (incl. the Tailscale CGNAT range 100.64.0.0/10) +
wildcard rules. The real socket peer IP is the authority (never a forwarded header).
- remote-server.js: rejects non-allowlisted peers at connection (close 4005). Opt-in
via config.allowlist (the existing remote-control server, which passes none, is
unaffected). Loopback always passes, so local + SSH-forward use keeps working.
- Two bind modes (config diagnostics.bindMode): "local" -> 127.0.0.1 (default),
"network" -> 0.0.0.0 but ONLY when a non-empty allowlist is set (else it stays
loopback, fail-closed). The allowlist + token gate access; the tunnel
(Tailscale/WireGuard) is the confidentiality layer (transport is still plaintext ws://).
- The connection code now carries the host: mhu1_<base64url{v,h,p,t,n,fp?,s?}>. The
gateway decode is tolerant of the legacy {port,token,label} keys; connect_server
takes the host from the code (host arg is an optional override). Proven end-to-end:
the integration harness now connects with NO host arg and resolves it from the code.
- Renderer: Sichtbarkeit selector (local/network), public-host input with
suggested-host chips (os.networkInterfaces — the Tailscale IP shows up there),
allowlist textarea (network mode), and network-requires-allowlist validation.
- main.js: bindMode->host, getSuggestedRemoteHosts, host-in-code, allowlist plumbed
into startDiagnosticAgent + the diagnostics IPC (get/save/status).
- docs: rewritten for Tailscale (set the allowlist to your tailnet, put the Tailscale
IP/MagicDNS in the code address — no SSH forward needed).
This supersedes the v3.3.85 hard loopback-lock with the downloader's allowlist model.
Tests: lib/ip-allowlist (8) + remote-server allowlist wiring/loopback (2) + gateway
decode (host short-key + legacy tolerance). 393 app tests + 9 gateway tests + e2e +
host-in-code integration + adversarial all green; lint 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
8d757a99dd |
fix(diagnostics): harden read-only agent — grep ReDoS, prototype-chain whitelist bypass, redaction gaps
Intensive end-to-end testing (a live gateway-MCP <-> agent integration harness +
an adversarial redaction/abuse probe + an independent security audit) surfaced
three real issues in the shipped read-only diagnostic agent. All run in lib/**,
which is packaged in the app.
1. grep ReDoS froze the Electron main process. read_log compiled the
client-supplied grep into `new RegExp(grep, 'i')` and ran it synchronously over
the log tail IN the main process. A catastrophic pattern (e.g. "(a+)+$" against
a long line) hangs the whole app — empirically confirmed (8s timeout, killed).
JS regex is synchronous and uncancellable, so grep is now a case-insensitive
literal substring filter with "|" alternation ("error|timeout|502"). Provably
linear-time; covers the real diagnostic need.
2. Prototype-chain whitelist bypass. The op table was a plain object literal, so
handle("constructor" | "toString" | "valueOf", ...) resolved an inherited
Object.prototype function, passed the `typeof fn === 'function'` guard and
returned {ok:true}. Harmless functions today, but a whitelist-integrity hole.
Now guarded with a string check + Object.prototype.hasOwnProperty.
3. Redaction defense-in-depth gaps. redactLogText now also scrubs: basic-auth URL
passwords (scheme://user:pass@host), Authorization: Basic, JWTs (eyJ...x.y.z),
and bare/JSON session= values. Mostly theoretical in today's readable logs
(secret-bearing bodies go to the excluded doodstream-debug.log; other hosters
throw static strings) but matters as the verbose-logging surface grows.
Verified: 383 app tests (incl. new regression tests for all three), the live
gateway-MCP integration harness (all 14 tools, zero leaks, error paths), the
adversarial probe (14/14+ secret shapes scrubbed, ReDoS 1ms, lockout, malformed
args), e2e gate, lint 0 errors. Only residual: a standalone high-entropy blob with
zero key/Bearer/URL context — inherent to any denylist, acknowledged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
7b5420eeaa |
fix(diagnostics): deep-redact queue jobs + rotation state — one collector still leaked
get_queue_state with includeJobs:true (the DEFAULT path) scrubbed the job list
with value-scrub only, so an opaque token a hoster returns inside a job error
(token=... that is NOT one of the user's stored credentials) survived in the
response. Same leak class already fixed for get_config_redacted and
server_health, still open on the queue collector's default path.
The first end-to-end gate missed it on two coincidences: server_health calls
getQueueState with includeJobs:false (no job error ever serialized), and the
fixture's queue error used a value that WAS a config secret (so value-scrub
caught it anyway). The direct get_queue_state{includeJobs:true} path with a
non-config token was never exercised.
- getQueueState job list and getRotationState now go through _deepRedact
(per-leaf pattern + value scrub), matching the other collectors.
- e2e-verify.mjs now plants a non-config token in a queue job error and asserts
both get_queue_state{includeJobs:true} and the default-args call leak nothing.
- Added a main-suite regression test for the default includeJobs path.
386 app tests + 9 gateway tests + e2e gate pass; lint 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
d69e5c39bf |
feat(diagnostics): MCP gateway + harden redaction so no secret ever leaves the box
Adds the connect-by-code side of remote diagnostics and closes two real secret-leak vectors that an end-to-end gateway<->agent test surfaced. Gateway (gateway/, local stdio MCP, Claude connects once): - 14 read-only tools (server_health hub, read_log, list_logs, list_errors, get_queue_state, get_history, get_config_redacted, get_system_info, get_rotation_state, get_app_events + connect/disconnect/list/current). - The HOST is always supplied by the operator, never taken from the code. - TLS fingerprint pinning is enforced in the socket 'open' handler BEFORE the token is sent (wss opt-in); plain ws is loopback-only. - registry.json (holds bearer tokens) is gitignored; only an empty example ships. Security hardening (gates every off-box payload): - redactLogText now scrubs opaque bearer/token-family secrets that are NOT stored config credentials (e.g. a session token a hoster returns inside an error string): bare token/auth_token/refresh_token/session_token + standalone "Bearer <opaque>". Benign "token bucket" prose is left intact. - get_config_redacted deep-redacts every string leaf (JSON-safe, per-leaf, so the cookie/sess line patterns can't gobble across a compact-JSON field) and drops the history subtree (served by get_history with its own per-error redaction). This plugs leaks via globalSettings.pendingQueue[].error etc. Bind-address safety: - _safeDiagBindAddress() forces the diagnostic agent to 127.0.0.1/::1; the 0.0.0.0 UI option is removed. Direct LAN/Internet bind stays disabled until encrypted transport (wss) exists — remote access goes through an SSH/VPN tunnel to loopback. (Never plaintext ws:// on all interfaces.) Tests: end-to-end gateway<->agent gate (connect -> server_health/read_log/ get_config_redacted, asserts zero secret leakage, rejects doodstream log, path traversal and write ops); + redaction regression tests in the main suite. 385 app tests + 9 gateway tests pass; lint 0 errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
ab7313f32c |
feat(diagnostics): read-only remote diagnostics agent over the existing WS transport (app side)
Adds the server/app half of a remote-diagnostics system so Claude (via a local MCP gateway, added separately) can read a server's full state to diagnose problems: logs, errors, queue/app state, redacted config, system health. All read-only; security review folded in as hard requirements. Architecture (design-verified): - A SECOND, independent RemoteServer instance (diagnosticMode) on its own port (default 9110, bind 127.0.0.1) with NO capture/input callbacks, so screen capture and sendInputEvent are structurally unreachable from the diagnostic path. The existing screen-share server (port 9100) is byte-for-byte unchanged. - lib/remote-server.js: a `host` bind option, a post-auth `diag-request` branch that delegates to onDiagnosticRequest and replies with a reqId-correlated `diag-response`, a `diagnosticMode` guard (capture never spawns), a timing-safe token compare (length-guarded), and getLastAccess(). - lib/diagnostics-agent.js: handle(op,args) enforces a HARDCODED read-only op whitelist as the sole authority — no write/exec op, no run_health_check. - lib/diagnostics-collectors.js: pure, dependency-injected collectors (server_health one-shot hub, read_log, list_errors, get_queue_state, get_history, get_config_redacted, get_system_info, list_logs, get_app_events, get_rotation_state, get_health) reusing support-bundle + stats. Security (all mandatory, implemented): - Redaction gates every off-box payload. support-bundle now adds webhookUrl + diagToken to CRED_KEYS, and exports redactLogText (value-scrub of live secret strings + pattern-scrub of Discord webhooks / Bearer / api_key= / cookies / sess ids) + valueScrub + collectSecretValues. read_log takes a logical NAME (no path traversal); doodstream-debug.log is excluded from the readable set (it logs live api-key-bearing HTML). grep is length-capped (ReDoS guard). - diagnostics config subtree (enabled/port/token/label/codeIssuedAt/bindAddress) defaults OFF, bind 127.0.0.1; deep-merge makes it migration-free. The server-owned token is preserved against renderer clobber in both save-global-settings handlers; the diagnostics:* IPC is the only mutator. - app.requestSingleInstanceLock() (also the approved fix #6) so a relaunch can't EADDRINUSE-kill the agent and two instances can't clobber the config. main.js: buildDiagnosticCode (mhu1_ base64url, no host embedded), start/stop + auto-start-on-launch + stop-on-quit, the four diagnostics:* IPC handlers. renderer: a "Diagnose-Zugriff" settings subtab (toggle, port, bind-address, copyable connection code + regenerate, status, read-only security note). 382 tests pass (incl. 11 new: collectors+redaction, agent whitelist, and a live RemoteServer protocol test proving auth->diag-response correlation, that a diagnostic client never triggers the capture window, and brute-force lockout). Lint clean (0 errors). Smoke boots identically to baseline. The standalone MCP gateway package + operator setup docs land in a follow-up commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
0a607adb29 |
fix(queue): stop finished uploads from re-appearing as pending ghosts across restart
The completed-upload dedup guard (_completedUploadKeys, "file|hoster") is the single source of truth that keeps a finished job from being re-materialised as a "Bereit" preview by buildQueuePreview(). Three independent paths leaked ghosts back into the queue: A) removeJobFromIndex() unconditionally DELETED the dedup key. The removeFromQueueOnDone auto-remove path (handleProgress), the batch-done sweep, and the terminal-job prune all call removeJobFromIndex on a *finished* job, so the guard handleProgress had just added was immediately wiped and buildQueuePreview re-created the job as a preview both mid-session and after restart. removeJobFromIndex now takes keepCompletedKey; the three auto-removal call sites pass true (keep the guard), while the manual-delete sites keep the old behaviour (drop the guard so the user can re-add the file). D/E) The dedup guard lived only in memory, so every restart began with an empty set and buildQueuePreview rebuilt ghosts from the persisted selectedFiles. buildPersistedQueueState() now serialises the keys whose file is still in the snapshot (completedKeys) and restoreQueueStateFromConfig() re-seeds them. To keep deliberate re-uploads working, applyHosterSelection() clears the persisted key for every freshly (re-)added file: re-adding through the hoster modal is the explicit "upload this again" signal. Both retry paths (retrySelectedJobs, _retryFailedFromBuckets) mutate the existing job in place instead of relying on buildQueuePreview, so they are unaffected. H) buildQueuePreview() excluded error jobs from its existingKeys set, so a file+hoster that already had an 'error' row got a second 'preview' row stacked beside it. Error jobs now count as existing. B/C) parseUploadLogLine() took the filename from a fixed field index and trimmed it. A pipe inside the link shifted the field (entry lost from the log-based dedup) and trimming broke matching against the untrimmed OS basename used as the queue-job key. The filename is now the last non-empty field and is no longer trimmed, so log lines with pipes in the URL and leading-space filenames both match end-to-end. 362 tests pass, lint clean on all touched files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
6b0c515a9b |
fix(queue): close two lost-work / sticky-ghost edge cases found by intensive testing
Follow-up hardening on the v3.3.80 queue-persistence fix after a deeper adversarial sweep + reviewer pass over the WHOLE subsystem (not just the diff). Two real defects: 1. Basename-collision lost work (regression introduced by v3.3.80 FIX A). restoreQueueStateFromConfig collapses jobs on the FULL path while the ts-gate keys on basename|hoster. Two genuinely different files with the same basename queued to the same hoster from different folders therefore share a gate key: if one was logged after savedAt, the ts-rule dropped BOTH — silently losing the still-pending one. Pre-FIX-A only 'done' jobs were dropped, so a pending file was never at risk. Fix: an ambiguity guard in partitionRestoredJobsByLog — the ts-rule is suppressed when a basename|hoster key maps to more than one distinct file path (the log records only basenames, so it can't say which physical file completed). The done-in-log rule is unchanged. Fails safe: worst case a visible ghost survives, never silent data loss. 2. selectedFiles re-materialization with removeFromQueueOnDone=ON (second mechanism, independent of the stale snapshot). When that setting is on, a completed job is stripped from queueJobs but its path stays in selectedFiles (syncSelectedFilesFromQueue only runs at batch-done, never on a mid-upload close). On restart the ts-gate operates on queueJobs and never sees it, then the startup updateUploadView -> buildQueuePreview re-creates it as a preview ghost AFTER the gate ran, and it re-persists with a fresh savedAt — sticky. Fix: completedSelectionKeys() seeds _completedUploadKeys (the set buildQueuePreview already consults) from the log at startup, keyed on full path, with the same ambiguity guard. Log-based so it survives a hard kill, consistent with FIX A. Also extracts the orphan-tmp sweep decision into lib/orphan-tmp.js (was untested inline code in main.js; behavior-preserving) and adds executable coverage for the paths that were previously only argued from logic: orphan-tmp sweep, config-store pendingQueue+savedAt round-trip, an end-to-end scenario in the exact user-reported shape (300 queued / ~200 finished mid-session), and a 3000+500-iteration property fuzz of the gate invariant including the lost-work guarantee. 359/359 green, ESLint clean, smoke-boot unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
eeec1d150c |
fix(queue): completed files no longer reappear in the queue after restart
Closing the app (especially during an active upload or on a hard kill) and reopening sometimes left already-uploaded files sitting in the queue as if still pending. Root cause is three layers stacked: 1. Persist-starvation: persistQueueStateSoon() reset a 10s debounce on every progress event, so during an upload the on-disk queue snapshot was never rewritten and stayed frozen at the pre-upload state (all jobs "preview"). 2. The beforeunload sync flush only covers a clean close; a hard kill / crash leaves that stale snapshot on disk. 3. Startup auto-dedup only dropped jobs with status "done". The completed files were stored as "preview" in the stale snapshot, so they survived and reappeared. Fix (mechanism-independent — holds whether the stale snapshot came from starvation, a mid-upload close race, or a hard kill): FIX A (core, durable): the upload log is the source of truth. Each persisted snapshot is now stamped with savedAt; on restart any restored job whose newest matching log entry is timestamped at/after floor(savedAt) is dropped regardless of status — it provably completed after the snapshot, so a "preview" row for it is a ghost. lib/queue-dedup.js gains an additive 3rd savedAt param; without savedAt or without log timestamps it behaves exactly as before (the 5 canary tests stay green, so intentional re-uploads of older files still survive). FIX B: new lib/throttle-timer.js with a max-wait. During uploads the snapshot is now written at most ~20s into a continuous progress burst instead of never; idle stays a pure debounce. The fallback shim honors max-wait too, so a missing library can never silently reintroduce the starvation. FIX C: the synchronous close-write retries renameSync on EBUSY/EPERM/EACCES and uses a pid-unique tmp (de-conflicts it from config-store._atomicWrite's fixed .tmp). A startup sweep reclaims orphaned <config>.<pid>.tmp files left by a hard kill between write and rename. lib/upload-log.js extracts formatUploadLogLine + parseUploadLogLine from main.js so the real writer -> reader -> gate seam is unit-tested (a future log-format or epoch-basis change can no longer pass green while breaking the fix). Verified: 334/334 tests green (incl. throttle fake-clock starvation/maxWait, ts-gate multi-hoster partial-completion, and the real-format seam tests), ESLint clean, smoke-boot identical to baseline, and an adversarial multi-agent review (15 findings, 14 refuted, 1 low — the tmp orphan, now swept) on the diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
4f2bd25426 |
revert(byse): drop the v3.3.78 truncated-host rejection (added cost, no benefit)
v3.3.78 rejected upload-server hosts ending in the bare ".filemoon" label so the lookup would retry for a valid server. In practice byse is currently returning a truncated host for its WHOLE pool, so the rejection found nothing to fall back to and turned every byse upload into a slow "Kein Upload-Server erhalten: OK" (6x2.5s server retry per file) instead of the fast ENOTFOUND fail it had before. That is a worse UX with zero upside while byse is down, so revert to the v3.3.77 behavior. The premise (byse returns a MIX of good and truncated hosts, skip the bad ones) is unverified; the evidence is byse returning truncated hosts for everything. And the correct upload domain is NOT safely determinable from outside: a swarm of filemoon-brand TLDs resolve (filemoon.eu/.me/.xyz/.art/.nl, kerapoxy.cc, moonmov.pro, ...) but the ones probed are parked/squatted (parklogic landers, shared catch-all certs), and byse's real response is only visible with a valid key. Guessing a TLD would point uploads at a parking page or a third party, so we do not. lib/hosters.js is now byte-identical to v3.3.77. The v3.3.77 fixes (5xx/ECONNRESET treated as transient -> no failover cascade/blacklist; the /api/file/list path fix) are kept. ENOTFOUND remains transient, so byse's outage fails clean (retry same account, no cascade) with the actual unresolvable host shown in the message. Suite 311/311. When byse restores its server pool (returns complete hostnames), uploads resume automatically with no client change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
46888c0220 |
fix(byse): reject truncated upload-server hostnames so the lookup retries for a valid one
In v3.3.77 the byse uploads started failing with "getaddrinfo ENOTFOUND s1065.filemoon" / "s1070.filemoon" / "s1075.filemoon". The hostname has no TLD, so DNS cannot resolve it. Root cause: byse's GET /upload/server is INTERMITTENTLY handing back a truncated upload-server host — "sNNNN.filemoon" with the TLD dropped (consistent with byse's ongoing filemoon migration; their own docs already expose an old_domain/new_domain embed switch). Some uploads still work because byse returns the complete host on those; the failing ones got truncated. byse did not change the API contract — this is malformed data from their server pool. The correct upload domain is NOT externally verifiable: every TLD that resolves for these server IDs is a parked/squatter domain (filemoon.art -> ParkLogic "lander" PTR; filemoon.nl -> a shared catch-all cert for kaobei.cc/babesex.xyz/...). Appending a TLD would point uploads at a parking page (or a third party) — so we do NOT guess one. Instead, treat an obviously-truncated host as "no valid server": normalizeAbsoluteUrl now returns null for any host ending in the bare ".filemoon" label (never a real TLD). extractUploadServerUrl then yields nothing for that response, so getUploadServer falls through to its existing machinery — it retries the lookup (SERVER_RETRY_ATTEMPTS) to get a different server and, crucially, returns the last-known-good server from LAST_UPLOAD_SERVERS once one has been cached. So after any single complete response, the truncated ones transparently reuse the good server instead of uploading into a DNS void. If byse's whole pool is truncating (cold cache), the lookup fails CLEAN as hosterTransient (no cascade, no blacklist — the v3.3.77 behavior), and the next batch tries again. This composes with v3.3.77: ENOTFOUND was already transient (retry same account, no failover); this stops the unresolvable host from ever reaching the POST in the first place when a working server is available. Tests: extractUploadServerUrl rejects https://sNNNN.filemoon / bare sNNNN.filemoon / https://filemoon and keeps a complete host (https://s1065.filemoon.sx/...) untouched. Suite 313/313. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
8b2a1d7c1f |
fix(byse): treat 5xx/gateway upload failures as transient infra, not account faults
A single byse.sx gateway hiccup (HTTP 502 "<!doctype html>..." or a mid-upload
ECONNRESET) was cascading through the whole failover chain and blacklisting every
account for the batch. Root cause: the upload-POST throw sites threw PLAIN errors
with no classification, so a 502 was treated as a GENERIC error -> mark-failed ->
emit('account-failed') -> failover to the next account (which hits the same
gateway) -> repeat until the chain is exhausted, then poison sibling files in the
batch via the blacklist. The screenshots showed exactly this: "Primär ... Fallback
#3" all failing with the same 502. byse did NOT change their API (the upload
contract still matches their docs and was live-probed: GET /upload/server -> POST
{key}+file -> files[0].filecode); these are transient infrastructure failures.
A 5xx gateway error is not an account fault: every account hits the same gateway,
so failing over is pointless and blacklisting is harmful. Fail open instead — retry
the SAME account and, if byse stays down, fail the file cleanly without touching the
account or the rotation cursor.
- upload-manager: _isTransientNetworkError() now honors an explicit err.transientNetwork
flag (checked before the empty-message guard) and, as a defensive fallback, matches
/HTTP 5\d\d/, Bad Gateway, Service Unavailable, Gateway Time-out. The flag is made
authoritative in _isFileRejectedError and _shouldSkipRetryOnAccountError (both return
early when it is set) so a 5xx whose HTML body happens to contain a rejection/auth
keyword can never be mis-binned as file/account. The post-rotation retry loop also
breaks on a transient error (parity with the primary loop) to avoid burning the
retry budget re-uploading on a fallback.
- hosters: the upload POST throw sites tag err.transientNetwork when statusCode >= 500
(non-JSON body and non-2xx-with-JSON) and when the 2xx status-envelope carries
status:500; 401/403/429 stay PLAIN so they remain account errors. The server-lookup
path (apiGet/getUploadServer) is hardened symmetrically: a 5xx there tags
transientNetwork and getUploadServer preserves the flag onto its wrapped error, so
the heuristic shouldRetryServerLookup() can no longer be defeated by a 5xx body that
contains an auth keyword.
- hosters: fixed a separate real bug surfaced during investigation — _fetchByseFileList
built the wrong URL https://api.byse.sx/api/file/list (the /api/ prefix is correct
only for doodstream's doodapi.co host; byse already carries the api. subdomain).
Live-probed: /api/file/list 302-redirects to the docs page, /file/list returns 200
JSON. The wrong path made the byse async-recovery poll silently dead (always []),
removing the safety net that reclaims a large file that registered despite a 502.
Now https://api.byse.sx/file/list.
ECONNRESET was already transient and is unchanged. doodstream's 2xx empty-form stays
hosterTransient (one attempt, no re-upload). vidmoly/clouddrop use their own uploaders
and are unaffected; doodstream/voe apiKey uploads share uploadFile and correctly
benefit from the same 5xx-is-infra logic.
Deferred follow-up: poll-first-on-5xx dedup (route a 5xx through the now-working
recovery poll before retrying, to reclaim a registered-but-502 file instead of
re-uploading). It does not add new dupe risk vs the prior cascade and a pure 502
means the backend was never reached, so retry-same-account is dupe-safe for it.
Investigated and reviewed by two multi-agent workflows (4-lens investigation with a
synthesized fix spec; 4-lens adversarial review with per-finding verification): no
API drift, 0 confirmed defects. Tests: classifier units (flag-above-message-guard,
5xx-transient, 4xx-stay-account), an end-to-end 502/503/500-envelope tag through
uploadFile, and a 502-retries-same-account-no-cascade integration case. Suite 311/311.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
7a04060e9b |
feat(rotation): per-hoster toggle to disable the "Bekanntes Größen-Limit" pre-skip
The byse suspect size-memo (softened in v3.3.73 to arm only after two confirmed suspect rejections on the same account and only for strictly larger files) still occasionally pre-skips a primary account with "Bekanntes Größen-Limit auf diesem Account", which the user finds annoying when they would rather just let every file attempt the upload for real. Add a per-hoster opt-out: a new "Größen-Limit merken" checkbox (Accounts tab, default ON so existing behavior is preserved for everyone) that, when unchecked, disables the memo-based pre-skip entirely for that hoster. - config-store.js / upload-manager.js DEFAULT_SETTINGS: sizeMemoEnabled: true. - upload-manager.js _suspectMemoBlocks(): returns false up front when the hoster has sizeMemoEnabled === false. That single guard covers BOTH consult sites — the pre-job block (skips the guaranteed-fail upload) and the alternate-account skip — since both route through _suspectMemoBlocks. The memo is still recorded; it is simply never allowed to block. This moves the flow toward "fail open": with the toggle off, every file always gets a real attempt on its account. - The genuine suspect-reject alternates walk is gated on err.suspectReject, not the memo, so disabling the memo does not weaken real failover — a file the account actually rejects still rolls over to the next account. - renderer/app.js: checkbox data-hs="sizeMemoEnabled", checked when hs.sizeMemoEnabled !== false; persisted by the existing generic checkbox path in saveHosterSettingsFromDom(). The running manager picks it up on the next batch (fresh UploadManager) and immediately on save (updateSettings). Tests: suspect-reject-alternates gains a disabled-memo case proving the larger third file still gets a real primary attempt (key1 tried 3x, no suspect-memo-skip event); config-store gains a default-true + persist-false case. Full suite 305/305. Reviewed by a 4-lens adversarial workflow (completeness / composition / persistence-ui / conventions): 0 findings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
644ed712e0 |
fix(rotation): persist the round-robin cursor so drip-fed uploads keep rotating
v3.3.74 created a fresh account picker inside every buildUploadTasks /
buildUploadTasksFromJobs call, so its in-memory rotation index reset to 0
on each call. That is correct for a one-shot batch (drag-drop N files at
once distributes fine), but it silently no-ops in exactly the pattern the
byse 80-uploads/30-days quota cares about: the folder monitor feeds new
files into a *running* batch one detection at a time via add-jobs-to-batch
(renderer -> add-jobs-to-batch IPC), and per-detection autoStart likewise
fires start-upload per file. Each of those calls rebuilt the picker from
index 0, so account 1 won every single time and the secondary accounts
never received anything.
The rotation cursor now lives outside the picker and survives across calls
and across app restarts (the quota spans a rolling 30-day window, so a
session-only counter would still starve secondaries for users who restart
between small uploads):
- account-rotation.js: createAccountPicker() accepts a seed { indices } map,
resumes the per-hoster cursor from it, and exposes indices() (the advanced
cursors) + dirty() (whether any rotation actually happened this call). The
cursor is a monotonic counter taken mod the current enabled-account count,
so it keeps wrapping correctly even if an account is later enabled/disabled.
- config-store.js: new top-level rotationCursors map (added to DEFAULTS, read
back in load() which otherwise reconstructs the result and would drop
unknown keys) plus a saveRotationCursors() method. The existing
read-modify-write save() preserves it across unrelated settings/credential
saves; secret-store never touches it.
- main.js: a module-level _rotationCursors is the authoritative source of
truth (seeded once from disk on first use, updated synchronously per batch),
with config as restart-survival backing. makeAccountPicker() seeds the
picker from it; persistRotation() folds the advance back and flushes to
config only when dirty. Authoritative in-memory state also closes the
disk-read race two rapid batches would otherwise hit. Both task builders
now receive the picker instead of constructing their own.
Composition with failover is unchanged and verified: the picker only chooses
each file's *initial* account and is never called on the failover path. The
pre-job-swap reroutes a task only when that task's own account is in
_failedAccounts, so a dead account's jobs reroute while healthy accounts keep
their rotation share. No double-advance of the cursor.
Tests: account-rotation gains seeded-resume, drip-feed-across-pickers (the
regression), dirty()-semantics, carry-forward, and count-shrink-wrap cases;
config-store gains rotationCursors default, save round-trip, no-clobber, and
credentials-undisturbed cases. Full suite 303/303.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
3204629fef |
feat(rotation): opt-in per-hoster account rotation to keep all accounts active
byse.sx requires each account to upload >=80 videos in the last 30 days (measured across a rolling 3-month window) or it goes dormant. With a single primary account doing all the work, the other configured accounts decay and eventually get suspended. This adds an opt-in "Accounts rotieren" toggle per hoster that round-robins files across every enabled account that has credentials: file 1 -> account 1, file 2 -> account 2, file 3 -> account 3, then wraps. Off by default, so existing single-account behavior is unchanged. Mechanics: - New lib/account-rotation.js: createAccountPicker() returns a stateful pick(hoster) closure holding a per-hoster round-robin index. It only rotates when rotateAccounts === true AND more than one usable account exists; otherwise it returns the first enabled account (the old primary). enabledAccountsFor() filters disabled + credential-less accounts while preserving configured order, so a disabled account is simply skipped in the cycle rather than leaving a gap. - main.js: both buildUploadTasks() and buildUploadTasksFromJobs() now build one picker per call and use pick(hoster) instead of getPrimaryAccount(), which is now removed (dead code). A fresh picker per batch means each upload session starts the cycle at account 1, matching "die erste Datei auf Account eins". - config-store.js: rotateAccounts: false added to HOSTER_SETTINGS_DEFAULTS. - renderer/app.js: "Accounts rotieren" checkbox in the per-hoster upload settings (Accounts tab). Persisted by the existing generic checkbox path in saveHosterSettingsFromDom(). Composes with failover: rotation only chooses the *initial* account per file. On a hard account failure the existing failover (_failedAccounts + pre-job account swap) reroutes just that account's jobs; the healthy accounts keep their rotation share. Single enabled account (or rotation off) = no behavior change. Tests: tests/account-rotation.test.js (10 cases) covers off=primary, on=round-robin+wrap, single-account no-op, skip disabled, skip no-creds, null when none usable, per-hoster index independence, byse-only rotation, 100-file 50/50 split, and the enabledAccountsFor filter. Full suite 294/294. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Administrator
|
87f976faf0 |
fix(rotation): soften the byse suspect size-memo so one blip cannot poison a batch
The per-batch size memo previously armed on a SINGLE confirmed suspect rejection
and blocked every same-or-larger file on that account (fileSize >= memoSize).
A single spurious byse "Not video file format" at e.g. 1.3GB therefore pre-failed
the whole series of ~1.3GB files in that batch with "Bekanntes Größen-Limit auf
diesem Account", even when byse could actually take that size.
Now the memo:
- arms only after the 2nd confirmed rejection on the same account (count >= 2),
so a lone byse aussetzer no longer short-circuits anything; and
- blocks only STRICTLY LARGER files (fileSize > memoSize), so a same-size file
always still gets one real attempt.
_suspectSizeMemo now stores { size: smallest-rejected, count } instead of a bare
number. Still per-batch (cleared at batch start, never re-primed across batches).
Updated the two memo tests to the new semantics (added a size-aware statSync mock
so a strictly-larger file can be exercised). Full suite 284 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
497317dc32 |
fix(rotation): suspect byse rejections now reach every account instead of dying on the first
Root cause of "4 accounts configured, rotation only ever uses 1-3": byse returns HTTP 200 + per-file status "Not video file format" for valid MKVs above an account's size tier (proven from the live rotation log: account 4ha0 accepted 1158 files up to 2.62 GB and deterministically rejected all 6 files >= 2.81 GB with that status, each costing a full ~20 min upload). The parser tagged this fileRejected, which (a) skipped the 30s recovery poll that was originally BUILT for this misleading status (guard regression from "byse skips 30s recovery poll on explicit reject") and (b) made the upload manager skip rotation entirely - so the remaining accounts were never tried and the files failed forever. Account selection is primary-first failover, so a later account only ever runs when every earlier one is hard-failed or disabled, which matches the reported "account 4 only gets used when I disable the others". The fix, layered: - parseByseResult flags "Not video file format" as err.suspectReject (alongside fileRejected). Genuine rejections (Duplicate, too small, account-level disk-full) behave exactly as before. - uploadFile runs the byse recovery poll again for suspect rejections (the file often registers asynchronously despite the status), but skips it when the caller's file probe says the upload is genuinely not a video (new opts.probeIsVideoLike, threaded from the upload manager's probe). Poll helpers now swallow an abort during their sleep and return null so a cancelled poll surfaces the original error instead of a bare AbortError. - upload-manager: on a suspect rejection of a probe-verified video, the file gets ONE attempt on each remaining pool account (new accountPools from main.js, refreshed on save-config) - WITHOUT blacklisting the rejecting account, which keeps working for smaller files. A per-batch size memo (hoster:account -> smallest rejected size) plus a known-good account preference prevent re-uploading every following oversized file through the whole chain: the second file skips known-limited accounts outright and goes straight to the account that took the first one. Alternates that fail with a genuine account error (quota/ban/full) are marked failed in-batch so no later file burns a multi-GB upload on them; deliberately no account-failed emit there, as repointing the hoster-wide override would reroute normal-sized files away from a healthy primary. - Rotation hardening from the adversarial review: the rotated-to retry loop now fast-breaks on file-rejected/hoster-transient errors instead of burning maxAttempts full uploads, and a file-class error on a rotated account no longer blacklists that account (it fails only the file). Cancelling mid-alternates or mid-rotation now records the job as aborted instead of error (this also stops the batch webhook from firing on fully user-cancelled batches). New upload-failure rot-logs are guarded against logging user aborts as failures; rotation retries are now visible in the rotation log at all (previously failures on a rotated-to account were never logged, which is why account 2's death was invisible in the support log). - file-probe: the mpeg-ts signature required only a single leading 0x47 byte, so GIFs and "G"-prefixed text classified as video and would have qualified junk for pool-wide alternate uploads; it now requires TS sync-byte periodicity (0x47 at offsets 0/188/376) and the probe head read grew from 64 to 512 bytes to make that check possible. GIF gets its own non-video signature. Tests: 9 new/reworked across suspect-reject-alternates.test.js (alternate walk, no blacklist, memo short-circuit, account-error marking, abort labeling, probe gating) and byse-reject-recovery.test.js (suspect polls + recovers, empty poll throws suspectReject, Duplicate still fast-fails, probe-confirmed non-video skips the poll) plus a TS/GIF probe regression test. Full suite 276 green. |
||
|
Administrator
|
05ad08433c |
feat(history): configurable retention + render cap so the Verlauf tab loads fast
The Upload-Verlauf tab took 30s+ to open on large, never-cleared histories.
Measured on the live config: 18.9 MB config / 41 batches / 53,403 rendered rows.
Two independent problems, fixed together:
1. Render side (the 30s killer): renderHistoryTable built ALL rows into one
innerHTML. Now the build is capped to the newest 2,000 rows (HISTORY_RENDER_CAP)
after slicing the chronological tail, then sorted for display. A notice shows
"Zeige neueste 2000 von N" so nothing looks deleted; full history stays on disk
and "Verlauf exportieren" still emits everything (export reads loadHistory()).
2. Storage side: history grew unbounded because appendHistory only ever pushed.
New globalSettings.historyRetention ('all' default, non-destructive) with
policies: 7d / 30d / 90d (time-based) and 1000 / 100 (newest-uploads-based).
appendHistory now prunes after each batch; a new prune-history IPC re-applies
the policy immediately when the user changes it, gated behind a confirm() that
shows the exact removal count (dry-run first).
Prune model keeps whole batches (atomic): count policies accumulate rendered rows
from the newest batch backward and always keep >= the newest batch even if it
alone exceeds the target; time policies keep batches whose timestamp is missing
or unparseable (old entries have none) so ambiguous data is never silently dropped.
New retention dropdown lives in the Verlauf header. 9 unit tests cover the prune
edge cases incl. a realistic 41-batch fixture. Full suite: 272 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Administrator
|
6df00a0948 | revert(network): remove network-outage auto-pause entirely — false positives froze batches | ||
|
Administrator
|
0e5eaa89e6 | fix(webhook): retry+429+status handling, await before shutdown, error-path notify, abort/auto-retry suppress, Discord limits | ||
|
Administrator
|
e1d04d0838 | feat(webhook): optional Discord ping (user-id / role / @here / @everyone) so batch-done actually notifies | ||
|
Administrator
|
1d116ac4bf | fix(ui+byse): live newest-on-top in files panel; byse skips 30s recovery poll on explicit reject | ||
|
Administrator
|
34aaa36571 | feat(unattended): network outage auto-pause/resume, post-batch auto-retry rounds, webhook notifications | ||
|
Administrator
|
d59c5c1df8 | perf: per-batch baseline cache, async folder walk, history-table fast path, progress IPC batching | ||
|
Administrator
|
125e5f55ea | fix(perf): kill per-progress renderer-to-main IPC + drop redundant queued emit + cache fileSize | ||
|
Administrator
|
cf35f4401d | feat(ui): per-hoster success rate, session-paused badge, post-batch retry, link export formats | ||
|
Administrator
|
5fb313273d | feat(diagnostics): file-format probe + structured upload-start/failure rot-log | ||
|
Administrator
|
f42c55c521 | feat(diagnostics): log levels, support bundle export, verbose toggle, log paths panel | ||
|
Administrator
|
e26b7ea8ed |
fix(accounts): never persist unverified creds + dedupe-proof modal + label + perf
User reported three coupled bugs in account add/edit:
(1) Invalid logins still create the account
(2) Doodstream gets created multiple times when "Prüfen & Anlegen" is
double-clicked or repeatedly OTP-retried
(3) Add/Delete in the accounts panel feel laggy
Plus a UX/feature request: account label + two-step "Prüfen → Anlegen" flow.
Map (workflow wf44zpud4, 3 parallel subagents + adversarial verify) confirmed:
- saveAccount() persisted to disk BEFORE the health check (lines 3407-3409)
- saveBtn.disabled was set AFTER two awaited IPC roundtrips → 5-100ms race window
- OTP-retry path generated a new accountId on every click (editingAccountId
stayed null in ADD mode) → DETERMINISTIC duplication on every OTP attempt
- runHealthCheck IPC required the account to be already persisted → that's
why the old code wrote-first-check-second
Fix architecture (advisor: Option A — make the invariant real, not cleanup-based):
- main.js + preload.js: NEW `validate-credentials` IPC. Accepts ephemeral
{hoster, authType, username, password, apiKey, otp} payload, builds an
ephemeral hosterConfig, runs the same per-hoster checker via a shared
_dispatchHealthCheck helper. Nothing touches config.hosters.
- renderer: two-step modal state machine.
- "Prüfen" click → validateCredentials (ephemeral) → green flips button to
"Anlegen"/"Speichern" AND caches a snapshot of the validated creds.
- "Anlegen"/"Speichern" click → only fires if cached snapshot matches the
currently-typed credential-identity (username+password or apiKey;
label and OTP are not part of the snapshot key).
- Input listeners on the identity fields drop the snapshot the moment any
cred is edited post-green → user can't sneak unverified creds through.
- _accountModalBusy is set SYNCHRONOUSLY at the top of the click handler,
before any await, so a double-click is a no-op.
- _accountModalSession token bumps on every modal reset → a stale late
response from a closed-and-reopened modal can't stomp the new session's
busy flag or UI (lens-2 review fix).
- Edit mode flows through the same path → bad edits never reach disk
before being validated (fixes the silent good-creds clobber).
- closeAccountModal cancels the auto-close timer + clears modal state so
a stale 600 ms timer can't close a freshly-reopened modal.
- Label field (new): persisted on the account, shown in the card subtitle as
"Label: XYZ • API: ABC… — API Key gültig" so identical-looking API accounts
are disambiguable. Excluded from snapshot key on purpose — label is metadata.
- Perf: drop the redundant `await getConfig()` round-trip in commit+delete
(in-memory state was already the source of truth and the old reload was the
main lag source). deleteAccount fires-and-forgets the saveConfig and closes
the modal synchronously. Commit path uses updateAccountCard for the
single-card edit case instead of a 4-panel cascade.
Multi-lens review (workflow wyoc3iq4k, 3 reviewers): OTP-correctness SHIP,
race-guard SHIP-WITH-FIXES (session-id token + busy-inside-try applied),
edit-mode+label SHIP. No blockers.
Tests: 6 new regression tests (tests/validate-credentials.test.js) covering
the three reported bugs as executable spec:
(a) failed validation persists nothing to config.hosters
(b) second click with guard set persists exactly one entry
(c) OTP-required persists nothing; OTP retry re-validates ephemerally
plus snapshot-key identity, post-validation edit invalidation, and the
ephemeral hosterConfig shape contract. 210/210 green, lint clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Administrator
|
ca35c2a6a4 |
fix(log): persist BARE log path (no compounded daily/session stamps)
User report: in session mode the upload-log lines split across two files — the first few before the auto-persist fallback fired, the rest after into a path with the session-stamp DOUBLED. Root cause in main.js _persistFallbackLogPath: 1. The strip was gated on the legacy `sessionLog` boolean, which 3.3.35 retired in favour of `logMode`. So in session/daily mode the gate was false and the resolved path got persisted with its stamp intact. 2. Even when the gate triggered, its regex matched only the daily YYYY-MM-DD suffix, not the session "session-YYYY-MM-DD_HH-MM-SS-pid" suffix. The next getLogFilePath() call read that saved path as the "base", treated the already-stamped filename as the base name, and re-applied another stamp on top. First flush hit the original session file; everything after hit a doubly- stamped one — exactly the symptom (top file: 2 lines, bottom file: the rest). - lib/log-mode.js: new pure stripModeStampFromFileName helper that removes both the daily and the session suffix patterns. Anchored to $, no nested quantifiers (linear). - main.js: gate on logMode (not sessionLog) and call the helper for daily AND session, so logFilePath always persists as a bare base. - Tests: 4 new — strip behaviour + an idempotence regression that locks in "resolve → strip → resolve = same path" so this can't silently come back. 204/200. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
d720ba295a |
feat(log): add per-session log mode (one file per app launch)
Adds a third choice next to the existing single-file and per-day modes: a new log file is created at every app start (process boot) and used until the app is closed. A close → reopen of the app starts a new session, hence a new file. File pattern: fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log. The boolean sessionLog field — misnamed: it actually toggled daily mode — is replaced by a logMode enum: "single" | "daily" | "session". The misnomer made the migration the trap to watch: existing users with sessionLog:true must land on "daily", NOT "session". normalizeLogMode handles this and is unit-tested. - lib/log-mode.js (new, pure, dual CJS/window export): normalizeLogMode + resolveLogFileName + format helpers. No fs, no Date.now() at call time. - config-store.js: normalize at the single load() boundary so downstream readers consume logMode only. logMode is deliberately NOT seeded in DEFAULTS (would beat the legacy migration after merge). - main.js: stamp SESSION_ID once at process start (with pid hedge against same-second restart collisions); getLogFilePath and buildFallbackLogName switch on mode via the lib. _resolveUploadLogTarget cache key is now just the primary path, which already encodes mode/date/session — self-invalidates. - renderer: <select> with three German labels replaces the old checkbox; saveSettings writes logMode; index.html loads the lib so window.LogMode is available in renderSettings. - Tests: 14 log-mode tests (incl. legacy-migration regression), 3 config-store tests (defaults, legacy migration, round-trip all three values). 200/200. End-to-end simulated locally: two launches → two distinct session files; PID hedge produces distinct names even within the same second. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
9ae5d312e1 |
fix(doodstream): web upload submits the live form's fields (not stale hardcoded)
Direct improvement to the login/web path (no API key needed): we were POSTing a stale field set — sess_id + utype=reg + file — but doodstream's CURRENT upload form dropped `utype` and added file_title / fakefilepc / submit_btn. Submitting an incomplete/stale field set can make the CDN node accept the bytes but skip the registration step (→ the empty result form with no fn). Now we parse the live upload form (already fetched in _getUploadServer) and replicate ALL its non-file fields faithfully — exactly what the browser submits — while keeping sess_id (the fresh node token) and utype as a harmless compatibility extra. - _parseUploadFormFields(html): pull every named input/button from the upload form, excluding the file input (streamed separately). Adapts to whatever fields doodstream uses now rather than hardcoding. - upload() builds the multipart from those fields; minimal known-good fallback if the form wasn't parsed. - Tests: real-form extraction (incl. file-input exclusion) + no-form safety. 183/183. Low regression risk (superset of the previously-working fields). Whether it resolves the large-file empty-form is for the server run; the API path (3.3.31/32) remains the reliable route when a key is available/derivable. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
d24fd54e83 |
test(doodstream): end-to-end integration test for the API upload + recovery path
Closes the gap between the unit-tested parseDoodstreamResult and the real
uploadFile orchestration. Mocks the undici transport (reassign undici.request +
refresh the hosters cache; mock.module needs an experimental flag npm test
doesn't pass) and global fetch, then drives the full doodstream API path against
the doc-verified response shapes:
- filecode returned directly in result[0].filecode → used.
- codeless 2xx → recovered by polling file/list and name-matching the title.
- codeless + file never appears → throws with err.hosterTransient=true (so the
account is not blacklisted).
Verified live this session: doodapi.co returns {"status":400,"msg":"Invalid
key"} for a bad key, so validation/list logic keys off status correctly.
Also makes the recovery poll count/delay tunable via __test.DOODSTREAM_POLL
(same 12 × 2.5 s defaults — non-behavioral) so the exhaustion test runs in ms.
Full suite 181/181.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Administrator
|
84c48ad7d6 |
fix(doodstream): login path auto-derives the API key → uploads via reliable API
The user uploads with username/password (login), so 3.3.30's "use API when a key is configured" did nothing for them — and the web-form upload keeps failing with empty forms on large files. Fix the LOGIN path itself: after logging in, pull the account's API key out of the logged-in session and upload via the official doodapi API (which returns result[0].filecode directly, no empty form). The user keeps using login and configures nothing. How the key is derived without knowing doodstream's (cookie-gated, unseen) settings DOM: brute-force candidate extraction + API validation. - DoodstreamUploader.deriveApiKey(): fetch the logged-in settings page (?op=my_account / /settings), pull every plausible long token from form-field values + element contents (ranked: tokens near an "api" mention first), and validate each against doodapi.co/api/account/info — only the account's real key returns status 200. A wrong guess is therefore harmless (fails validation → web fallback). Logs the raw settings HTML when nothing validates, so the scrape can be refined from a real capture if doodstream's markup differs. - upload-manager: doodstream login-path now resolves the key ONCE per batch (cached by accountId; '' = tried-none) and routes to the API when found, else the existing web-form upload. Keyless accounts: one extra probe-login per batch, then unchanged. - Tests: candidate extraction (value/textarea/api_key shapes, api-context ranking), validate-then-pick, null→web-fallback, preset short-circuit. 178/178. If derivation works the login path now uploads via the API. It does NOT change doodstream's backend; the server run confirms. Falls back safely if no key. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
a8d81cbf0d |
fix(doodstream): upload via the doodapi API when an API key exists
Root cause of the recurring "kein Filecode — Server gab leeren Link zurueck": the web-session upload flow gets the filecode back inside an XFileSharing HTML form, and on long/large uploads that form comes back empty (no fn). Verified research: doodstream's server-side file-registration callback times out under large-file load, so the upload "succeeds" (bytes sent, HTTP 200) but no filecode is minted — and because registration failed, the file is NOT in the file list either, so polling can't recover it. The web path also rides a per-page-load sess_id token that ages over the multi-minute upload. The official doodapi.co JSON API has no such failure mode for result retrieval: the upload response returns result[0].filecode directly, and it authenticates with a persistent api_key (no aging sess_id). Git history confirms the API was doodstream's ORIGINAL upload path (initial commit); web login was added later only "as an alternative to API key" — so preferring the key restores the intended primary path rather than fighting a deliberate choice. - lib/account-auth.js (new, pure, unit-tested): selectUploadAuth() prefers the doodstream API key over username/password; all other hosters unchanged. - main.js buildTaskFromAccount delegates to it → a doodstream account with an apiKey now routes through hosters.uploadFile (doodapi API) instead of the web uploader; keyless accounts keep using web login. - hosters.js: drop the stale hardcoded fallback node from the doodstream API config (same dead tr1128ve host removed from the web path) so a failed server lookup throws cleanly instead of uploading into a dead end. - Tests: 8 routing cases (doodstream key-preference, keyless fallback, voe unaffected, authType=api, null-safety). Full suite 173/173. This eliminates the empty-form failure mode for result retrieval when a key is configured. It does NOT change doodstream's backend — whether the large-file timeout recurs (now as a structured JSON error, not a silent empty form) is for the server run to confirm. Requires a doodstream API key on the account. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
166b04c526 |
fix(upload): classify doodstream empty-form as hoster-transient (don't kill account)
The "kein Filecode — Server gab leeren Link zurueck" error was treated as a generic upload failure → after retries exhausted, the manager called mark-failed and added the account to _failedAccounts → next batch re-primed with primedFailed=1 → pre-job-swap-blocked because no fallback override exists for a single-account hoster. One server-side flake permanently poisoned the session. It's not an account problem — same account + same file works on a later try. This is a doodstream-backend processing flake (empty CDN form, no fn / no st), the same class as a transient network error: don't blacklist, just fail this file cleanly. - doodstream-upload.js: tag the empty-form throw with err.hosterTransient=true (explicit flag, primary signal — matches the err.accountError / err.fileRejected pattern already used elsewhere). - upload-manager.js: new _isHosterTransientError classifier (flag first, message regex as defensive fallback). In the retry loop: break on first hit (server flake won't clear in 3 s, re-uploading the file 4× is pure bandwidth waste). Post-loop: dedicated branch that emits the final error WITHOUT blacklisting the account — same shape as the existing transient-network branch. - Tests: classifier unit tests (flag path, regex path, negatives) + regression test that proves the account is NOT added to _failedAccounts and mark-failed does NOT fire. Drops the hoster-transient test from ~19 s to ~1.5 ms, confirming the in-loop fast-break works. We now fail fast on this error class instead of retrying — the next-batch manual retry is the recovery path, and the account stays usable for it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
af51bebaf7 |
fix(queue): stop auto-dedup from deleting pending jobs on restart/update
Reproduced from a real saved config: pendingQueue held 4 'preview' jobs (one file across 4 hosters); the queue saved + restored correctly. But _autoDeduplicateFromLog (runs at init after restore) removed jobs whose fileName|hoster appeared ANYWHERE in the lifetime fileuploader.log, regardless of status — so all 4 pending previews were deleted and the queue showed the empty "Dateien hierhin ziehen" state. Looked update-specific only because the server restarts on update; a plain restart did the same. - New lib/queue-dedup.js (pure, dual CJS/window export like queue-prune.js): partitionRestoredJobsByLog drops ONLY 'done' jobs that match the log. Pending (preview/queued) and failed (error/aborted) jobs always survive — they're intentional queued work (often a deliberate re-upload of a previously uploaded file). Manual importUploadLog stays separate/explicit. - renderer wires it in; index.html loads the module before app.js. - Tests: 5 cases incl. the exact reproduced scenario (4 previews all in log -> 0 removed). Full suite 162/162. Verified against the user's real electron-config.json + fileuploader.log: old logic removed 4/4 (empty queue), new logic removes 0/4 (queue preserved). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
f237d0f97a |
fix(doodstream): survive transient network blips around the upload
After 3.3.26 fixed the filecode parsing, the remaining intermittent failure is a generic "fetch failed" — a transient network error on one of the requests around the multi-minute upload. Can't tell from one log line whether it's the server-discovery GET or the post-upload result-submit, so harden both: - _fetch (the native-fetch chokepoint for discovery, redirects, result-submit): retry up to 3x with short backoff on a thrown network error, each attempt bounded by a 20s timeout (Node fetch has none by default). Caller aborts are not retried. The big file upload (undici) is retried at the upload-manager level, not here. - result-submit is now best-effort: if it still fails after retries but we already hold the filecode from the CDN response, return that instead of discarding a completed upload. - label the undici upload-POST error with phase + MB sent + node, preserving the original message so transient classification still matches. - eslint: add AbortSignal to globals. - Tests: _fetch transient-retry path (10 doodstream tests total). "fetch failed" is already classified transient by upload-manager, so this is additive resilience; next logs will show if anything still slips through. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
18a875a764 |
fix(doodstream): use current page format (form action + matching sess_id)
The 3.3.25 diagnostics captured the live upload page: doodstream moved the upload server from a `srv_url` JS variable into the multipart form's action, e.g. action="https://xxx.cloudatacdn.com/upload/01?SESSID", with a per-page session token in the query that matches the page's hidden sess_id input. The old parser found neither and fell through to the stale hardcoded node, which returns an empty filecode. - Parse the upload server from the form action (matched via the /upload/ path), un-escaping & in the query string. - Refresh this.sessId from the SAME page (only on action match) so the multipart sess_id field matches the node URL's token; login-time and node tokens otherwise diverge. Keep the existing sessId if the input is absent. - Keep the legacy ?op=upload_server JSON and srv_url paths as fallbacks; the fail-fast throw from 3.3.25 stays as the last resort. - Tests: form-action parse, sess_id refresh, & un-escape (9 total). Whether this fully resolves the uploads is for the next server logs to confirm; both the node and sess_id fixes are individually correct. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
52751df735 |
fix(doodstream): fail fast instead of uploading into a dead hardcoded node
Real root cause from the 3.3.24 diagnostics: the failing upload used CDN "tr1128ve.cloudatacdn.com/upload/01" — character-for-character the hardcoded last-resort fallback in _getUploadServer(). The CDN form came back with only op=upload_result and NO fn/NO st, i.e. the bytes went into a stale node that returns an empty form. So _getUploadServer can no longer extract the current upload server (Doodstream likely changed the upload_server response/format) and silently fell back to a dead node — wasting ~90s/95MB per attempt. - Remove the silent hardcoded-node fallback; throw a clear error when discovery fails so the upload fails instantly instead of 90s later with a cryptic msg. - Embed the raw upload_server response (status, content-type, body) and upload-page URL hints in the error AND debug log, to pin the format change. - Tests: getUploadServer JSON path, srv_url HTML fallback, and the no-silent- fallback throw (asserts the hardcoded node never leaks into the error). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
ce5f20b1e1 |
fix(doodstream): surface real upload-failure reason + fix dead prod debug log
The "upload_result Seite hat keinen filecode" error fired with no actionable detail when Doodstream's CDN returned an empty filecode (fn). Root cause is server-side: the page structure is unchanged, the link is just missing — Doodstream's backend refused the file (copyright/hash match, duplicate, size, quota). XFileSharing reports the reason in the `st` field, which we ignored. - Surface `st`: non-OK status now throws "Doodstream lehnt Datei ab (Status: …)". - Enrich the generic error with st, fn-state, and the CDN node for diagnosis. - Fix debug-log path: wrote to __dirname/.. which is read-only (app.asar) in packaged builds, so production captured zero traces. Now uses Electron's writable userData dir, with repo-root fallback for tests/plain node. - Add tests/doodstream-upload.test.js (4 tests) pinning the parse/error paths. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Administrator
|
57f8f0876e |
feat(log): per-hoster toggle for writing links to fileuploader.log
New per-hoster setting "Links in Log schreiben" (logToFile, default on). When unchecked for a hoster, that hoster's successful upload links are no longer written to fileuploader.log — other hosters keep logging independently. - lib/config-store.js: logToFile: true added to HOSTER_SETTINGS_DEFAULTS; merge-on-load gives every hoster the key (old configs included). - renderer/app.js: checkbox per hoster panel + collection loop now handles type=checkbox (boolean) alongside the numeric fields. The autosave bind already special-cased checkboxes (change event). - lib/log-policy.js (new): hosterLogToFileEnabled() — pure, opt-out semantics. Only an explicit logToFile===false disables; missing/ malformed/non-true values all default ON so links are never silently dropped. - main.js: shouldLogHosterToFile() reads the LIVE uploadManager .hosterSettings (so a mid-batch toggle takes effect at once), falls back to persisted config, then to enabled. Guards appendUploadLog in the done handler; skipped writes get a debugLog line. Tests: 8 log-policy (defaults, opt-out, per-hoster independence, malformed input) + 2 config-store (default true, persisted false survives reload). 147/147 green, eslint clean. |
||
|
Administrator
|
166a49dd0c |
test(coalesce): extract done-removal coalescer + 11 unit tests
The microtask-coalesce path from 3.3.1 (queueMicrotask + Set so 500
finishing jobs become one queueJobs.filter pass instead of 500) lived
inline in renderer/app.js. Pulled out into lib/coalesced-set.js with
an injectable scheduler so a Node test can drive timing without
async waits.
API: makeCoalescedSet({ apply, scheduler? }) returns
add(id) — queue an id for the next batch
drainSync() — flush synchronously (used by beforeunload)
pendingSize() — diagnostics
isScheduled() — diagnostics
Renderer rewires the previous _pendingDoneRemovalIds + manual
queueMicrotask plumbing to the new helper. Optional-chained: if the
script fails to load, a slower per-event filter runs as fallback.
Coverage:
- multiple adds same tick → 1 apply, all ids deduped
- duplicate ids deduped
- batches between flushes stay independent
- add after flush re-schedules
- drainSync flushes synchronously, queued microtask becomes a no-op
- empty drainSync is a no-op
- throwing apply doesn't lock out subsequent batches
- default scheduler (queueMicrotask) runs eventually
- 5000-id burst still coalesces to 1 apply
137/137 green.
|
||
|
Administrator
|
0ba8bd3a2c |
fix(hosters): defensive null-payload guards in result parsers + 7 tests
When a hoster server replies with a body that JSON-parses to a
non-object (literal "null", a bare string, a number, a top-level
array), uploadFile's downstream code crashed:
payload.msg → TypeError on null
payload.status → TypeError on null
config.parseResult() → TypeError inside parseDoodstreamResult
(payload.result) and parseByseResult
(payload.files / payload.result)
The user saw a confusing "Cannot read properties of null" instead of
a useful "server returned no JSON object". Found by deep-audit pass.
Fix in three places:
1. uploadFile (lib/hosters.js): after JSON.parse, normalise non-object
payloads to {}. Subsequent `payload.X` accesses then return
undefined and the existing fallback paths handle the empty case.
2. parseDoodstreamResult: defensive `payload && payload.result` so
direct callers (tests, hypothetical future callers) get the same
guarantee instead of relying on uploadFile to have normalised.
3. parseByseResult: same `payload || typeof payload !== 'object'`
short-circuit at entry, plus null-checks on `f` (the first files
entry) so a server returning [null] in files doesn't crash either.
Tests: 7 new unit tests covering null/undefined/string/number/array
payloads, malformed files entries, the fileRejected/accountError
classification (regression-pinning the 3.1.4 phrasing tweaks), and
the valid-filecode happy path. 126/126 green.
|