Multi-Hoster-Upload/tests
Administrator 29d1944328 perf(config): cache parsed config + lean serialize + drop load() from log flush (long-run lag)
A 13-agent hunt pinned the "wird mit der Zeit laggy" symptom (CPU/RAM normal, UI
sluggish after many uploads) to the main process re-doing config I/O that scales
with the ever-growing history, stalling the synchronous main event loop so the
renderer's IPC round-trips feel laggy. The renderer render path was already
optimized (virtualized queue, capped panels) — confirmed clean.

This commit lands the two contained fixes (T1 + T3); the queue-persist rewrite (T2)
follows separately.

config-store (T1):
- load() now has an in-memory cache keyed on the file mtime+size. The processed
  config (merged + credential-decrypted) is re-read/re-parsed/re-DPAPI-decrypted
  ONLY when the file actually changes; our writes invalidate it, external edits
  change mtime/size so the cache misses. Eliminates a full disk read + JSON.parse of
  the whole growing history + per-credential decrypt on the vast majority of the ~38
  load() call sites. load() always returns a structuredClone so callers can mutate
  freely without corrupting the cache.
- _serializeForDisk clones ONLY the hosters subtree (the only thing encryptCredentials
  touches) instead of JSON.parse(JSON.stringify(whole config)) — no more deep-cloning
  an 8 MB history on every write.
- _atomicWrite refreshes the .bak with a raw fs.copyFileSync instead of
  read + JSON.parse + write (it was re-parsing the full config a 2nd time per write).

main.js (T3):
- The log-flush paths resolved the log file via configStore.load() ~8x/second during
  uploads (re-reading + cloning the whole config just to read logMode/logFilePath).
  Cache those two strings in module scope, invalidate on the settings-save handlers.

Verified: 26 config-store tests (incl. new cache-correctness: independent clones,
external-change invalidation, save invalidation) + full 394-test suite green, lint 0
errors. Benchmark (8000-batch / 4.6 MB history): log flush no longer calls load() at
all; the remaining per-write history serialize is what T2 removes from the hot path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 00:49:55 +02:00
..
account-auth.test.js fix(doodstream): upload via the doodapi API when an API key exists 2026-05-28 21:42:19 +02:00
account-rotation.test.js fix(rotation): persist the round-robin cursor so drip-fed uploads keep rotating 2026-06-17 03:33:20 +02:00
backup-crypto.test.js feat(backup): import legacy password-encrypted backups 2026-04-17 11:22:33 +02:00
byse-reject-recovery.test.js fix(byse): treat 5xx/gateway upload failures as transient infra, not account faults 2026-06-19 01:56:11 +02:00
coalesced-set.test.js test(coalesce): extract done-removal coalescer + 11 unit tests 2026-04-28 11:59:32 +02:00
config-store.test.js perf(config): cache parsed config + lean serialize + drop load() from log flush (long-run lag) 2026-06-21 00:49:55 +02:00
diagnostics-agent.test.js fix(diagnostics): harden read-only agent — grep ReDoS, prototype-chain whitelist bypass, redaction gaps 2026-06-19 18:40:57 +02:00
diagnostics-collectors.test.js fix(diagnostics): harden read-only agent — grep ReDoS, prototype-chain whitelist bypass, redaction gaps 2026-06-19 18:40:57 +02:00
diagnostics-protocol.test.js test(diagnostics): live network-bind path — allowlisted non-loopback peer connects over a real 0.0.0.0 socket 2026-06-19 19:38:25 +02:00
doodstream-api-upload.test.js test(doodstream): end-to-end integration test for the API upload + recovery path 2026-05-28 22:20:21 +02:00
doodstream-upload.test.js fix(doodstream): web upload submits the live form's fields (not stale hardcoded) 2026-05-28 22:28:24 +02:00
file-probe.test.js fix(rotation): suspect byse rejections now reach every account instead of dying on the first 2026-06-15 01:11:37 +02:00
history-retention.test.js feat(history): configurable retention + render cap so the Verlauf tab loads fast 2026-06-15 00:26:20 +02:00
hosters.test.js revert(byse): drop the v3.3.78 truncated-host rejection (added cost, no benefit) 2026-06-19 02:31:57 +02:00
ip-allowlist.test.js feat(diagnostics): network bind + fail-closed IP allowlist + host-in-code (Tailscale, like rd-diagnostics-mcp) 2026-06-19 19:32:29 +02:00
log-mode.test.js fix(log): persist BARE log path (no compounded daily/session stamps) 2026-06-04 22:08:15 +02:00
log-policy.test.js feat(log): per-hoster toggle for writing links to fileuploader.log 2026-05-23 15:29:25 +02:00
log-rotation.test.js test(log): extract log-rotation into testable module + 10 unit tests 2026-04-28 05:10:53 +02:00
orphan-tmp.test.js fix(queue): close two lost-work / sticky-ghost edge cases found by intensive testing 2026-06-19 06:51:48 +02:00
queue-dedup-property.test.js fix(queue): close two lost-work / sticky-ghost edge cases found by intensive testing 2026-06-19 06:51:48 +02:00
queue-dedup.test.js fix(queue): close two lost-work / sticky-ghost edge cases found by intensive testing 2026-06-19 06:51:48 +02:00
queue-persistence-scenario.test.js fix(queue): close two lost-work / sticky-ghost edge cases found by intensive testing 2026-06-19 06:51:48 +02:00
queue-prune.test.js test(queue): extract terminal-job prune into testable module + 10 tests 2026-04-28 06:41:47 +02:00
remote-config.test.js feat(remote): add remote control defaults to config store 2026-03-12 06:54:46 +01:00
remote-server.test.js feat(remote): add WebSocket server with auth, signaling relay, and rate limiting 2026-03-12 06:54:51 +01:00
semaphore.test.js test: add edge case tests for throttle and semaphore 2026-03-21 15:16:49 +01:00
stats.test.js feat(ui): per-hoster success rate, session-paused badge, post-batch retry, link export formats 2026-06-07 20:32:35 +02:00
support-bundle.test.js fix(diagnostics): harden read-only agent — grep ReDoS, prototype-chain whitelist bypass, redaction gaps 2026-06-19 18:40:57 +02:00
suspect-reject-alternates.test.js fix(byse): treat 5xx/gateway upload failures as transient infra, not account faults 2026-06-19 01:56:11 +02:00
throttle-timer.test.js fix(queue): completed files no longer reappear in the queue after restart 2026-06-19 04:35:27 +02:00
throttle.test.js test: add edge case tests for throttle and semaphore 2026-03-21 15:16:49 +01:00
throttled-cache.test.js test(sort): extract throttled-cache utility + 12 unit tests 2026-04-28 07:12:52 +02:00
ui-smoke.js feat: improve account-driven uploads 2026-03-11 02:41:32 +01:00
upload-log.test.js fix(queue): stop finished uploads from re-appearing as pending ghosts across restart 2026-06-19 14:53:04 +02:00
upload-manager.test.js fix(byse): treat 5xx/gateway upload failures as transient infra, not account faults 2026-06-19 01:56:11 +02:00
validate-credentials.test.js fix(accounts): never persist unverified creds + dedupe-proof modal + label + perf 2026-06-07 03:11:13 +02:00
webhook-notify.test.js fix(webhook): retry+429+status handling, await before shutdown, error-path notify, abort/auto-retry suppress, Discord limits 2026-06-10 00:08:02 +02:00