Compare commits

...

55 Commits

Author SHA1 Message Date
Administrator
c58d9203bc docs(tasks): record v3.3.108 session-log 6-digit suffix release
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 20:14:23 +02:00
Administrator
f0006f8003 feat(logs): append a 6-digit random suffix to session log filenames (v3.3.108)
Session-mode log files now end in a generated 6-digit number, e.g.
26-06-2026-mdu-session-06-02-847581.log. Dropping seconds/pid in v3.3.107
reintroduced same-minute collision risk on fast close/reopen; the random
suffix restores per-launch uniqueness without leaking the process id.

- formatSessionStamp(date, rand) appends -<rand> when supplied
- main.js stamps SESSION_ID with a 6-digit Math.random value
- stripModeStampFromFileName tolerates the optional -NNNNNN suffix
- tests cover stamp-with-rand and strip-with-suffix; 411 pass

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 19:52:13 +02:00
Administrator
66ae240794 feat(logs): session log filename → DD-MM-YYYY-mdu-session-HH-MM (v3.3.107)
Per user request, the per-session log file is renamed from
fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log to
DD-MM-YYYY-mdu-session-HH-MM.log (hour and minute only, no seconds or pid).

lib/log-mode.js:
- formatSessionStamp(date) returns `${DD}-${MM}-${YYYY}-mdu-session-${HH}-${MM}`
  (the pid argument is dropped; main.js still passes process.pid, harmlessly
  ignored). Same-minute restarts now share a session file, which is the intended
  human-readable trade.
- the session branch of resolveLogFileName returns `${sessionId}${ext}` — the stamp
  is the full app-defined stem and baseName is intentionally ignored for session
  mode (single/daily still use the 'fileuploader' base).
- stripModeStampFromFileName recognizes the new format and resets to the default
  'fileuploader' base (the new stem embeds no base, so the configured base is not
  recoverable from it); the existing daily and old-session strip regexes are kept
  for backward-compat with any persisted old paths, and the persist/re-resolve
  round-trip stays idempotent (no compounding stamps).

Tests updated for the new format (formatSessionStamp, session resolveLogFileName,
the new-format strip, and the idempotency regression). 410 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 06:56:43 +02:00
Administrator
8fed8669f3 fix(startup): disable HW acceleration on RDP/VM to kill the intermittent white screen + export filename (v3.3.106)
(A) Intermittent pure-white startup window. On a Windows VM viewed over
RemoteDesktop the app sometimes opened to a blank white window — no error banner,
not even the static menu bar. A multi-agent investigation + adversarial review
pinned it by one airtight deduction: the BrowserWindow is created with
backgroundColor '#16181c' (dark), so "white" can never be an un-painted, loading,
or failed state — all of those show dark. Pure white, with no banner and no static
menu bar, and silent in the logs, eliminates every DOM/init/CSS/load failure mode
(each would leave the dark styled shell, unstyled black-on-white menu text, or the
red init().catch banner) and leaves exactly one: a GPU/compositor surface failure
on the RDP virtual display adapter. The code ran hardware acceleration full-default
with no fallback (no disableHardwareAcceleration, no GPU switch anywhere), and the
GPU child-process-gone handler is log-only — matching the silent symptom.

Fix (the reviewed safe subset):
- app.disableHardwareAcceleration() at module top (before app.whenReady), gated on
  an RDP session (process.env.SESSIONNAME matches /^RDP/) OR a persisted
  gpu-disabled.flag in userData. The renderer has no WebGL/canvas/video, so software
  compositing costs effectively nothing here and does not undo the recent renderer
  perf work; local/console users keep hardware acceleration.
- Auto-heal: when a GPU child-process-gone fires, write gpu-disabled.flag so the next
  launch disables acceleration even if the RDP gate didn't match (covers a VM reached
  via console or a flaky virtual GPU). Self-heals after at most one white screen.
- The existing child-process-gone / render-process-gone / did-fail-load logging is
  kept so the affected server's next white-start log can confirm the cause
  (CHILD PROCESS GONE type=GPU). A webContents reload would not disrupt uploads
  (uploadManager lives in the main process and is torn down only on quit), but no
  watchdog is added because in this GPU mode init completes — an init-complete signal
  would not detect the blank surface.

(B) Backup export default filename changed from multi-hoster-backup-YYYY-MM-DD.mhu to
DD-MM-YYYY-multihoster-backup.mhu.

409 tests pass; clean boot (the guard is inert off-RDP).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:05:07 +02:00
Administrator
7a40afbe7e fix(config): fsync config writes + guard against account-wipe on a corrupt read (v3.3.105)
A user's server crashed hard during an upload and lost all configured accounts. It
was NOT the v3.3.104 update (a second server updated fine and kept its accounts) —
it was a data-durability hole exposed by the crash:

- Config writes were atomic (tmp + rename) but never fsync'd, so a hard crash could
  leave electron-config.json truncated/unflushed on disk.
- On restart, load() reads the truncated file, falls back to .bak, and if that is
  also bad returns empty DEFAULTS. The next settings/queue save then persists EMPTY
  hosters — permanently wiping the accounts. Worse, the async _atomicWrite blindly
  copied the (now truncated) live file over .bak, so an empty live could clobber a
  good backup.

Hardening (lib/config-store.js + main.js; no behavior change in the happy path):
- fsync before rename in both write paths — _atomicWrite (openSync/writeSync/
  fsyncSync/closeSync) and the synchronous save-global-settings-sync on window close.
  A hard crash can no longer leave a truncated config.
- _atomicWrite only refreshes .bak when the current live file is non-trivial
  (trim length > 2), so an empty/truncated live can never overwrite a good backup
  (the sync-save path already did this).
- Wipe-guard (_guardHosters): save(), saveRotationCursors() and the sync close-save
  never intend to change hosters; if after a load() the hosters are all empty and
  the write did not explicitly provide hosters, recover them from disk
  (_recoverHostersFromDisk: live -> .bak -> .pre-history-split.bak) instead of
  persisting the wipe. An explicit save({hosters: {}}) (user deleted all accounts)
  is still allowed. Restored hosters are already-encrypted on disk and
  encryptCredentials skips already-encrypted fields, so re-serializing is safe.
- load() gained a third fallback tier — the permanent pre-history-split.bak snapshot
  (which still holds the accounts) — so load() itself recovers after corruption.

Recovery for the already-affected server: copy
%APPDATA%/multi-hoster-uploader/electron-config.json.pre-history-split.bak (or .bak)
over electron-config.json with the app closed.

2 new regression tests (post-wipe valid-empty live + .bak → guard restores accounts;
an explicit empty-hosters save is not blocked). 409 tests pass; clean boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 03:53:01 +02:00
Administrator
335f365497 perf(recent): virtualize the Recent-uploads panel — the last non-virtual table (v3.3.104)
The v3.3.103 log (real 2464-job, 4-hoster batch ramping to 95 concurrent) confirmed
the statSync fix held (batch-start main spike 336→231ms with 10× more jobs) and the
whole 90s ramp to 95 active was pristine (event-loop mean ~11ms, fps=32,
longtasks=0). The one residual: rapidly clicking tabs DURING the 95-active upload
produced 210-221ms renderer long-tasks (proc=0ms → layout/paint, not JS).

Cause: the Recent-uploads panel rendered every sessionFilesData row (up to 2000)
into the DOM non-virtualized — the exact analog of the History table before it was
virtualized in v3.3.102. Switching to that view laid out ~2000 rows (~210ms on the
RDP VM).

Fix — virtualize renderRecentUploadsPanel, mirroring the queue/History pattern:
- The tbody gets only the visible rows plus top/bottom spacer <tr> sized from
  VIRTUAL_ROW_HEIGHT. A rAF-coalesced scroll handler and a ResizeObserver on
  .recent-files-table-wrap re-render the visible window (the ResizeObserver also
  serves as the show-trigger when the hidden panel gains size). _recentWorking holds
  the sorted set. The insertAdjacentHTML append-only fast path is dropped — a
  ~40-row window re-render is cheap, so every render just re-renders the window; on
  prepend (date desc) the scroll position is preserved (scrollTop=0 at top, else
  += added*ROW_HEIGHT).
- Selection stays correct: _buildRecentRowHtml already stamps the selected class
  from selectedRecentIds.has(row.order) per row, so an off-screen-selected row
  renders selected when scrolled into view; selectedRecentIds remains the source of
  truth and shift-select already reads the sort cache, not the DOM.
- styles.css gives .recent-file-row a fixed 28px height so the virtualization math
  is exact (the table already had table-layout:fixed, so no column-jump fix needed).
- _renderRecentVirtualRows returns early when there are no rows, so it never wipes
  the empty-state message.

Verified with Playwright at 2000 rows (bounded container): view-show layout drops
from ~118ms to ~2.4ms, the DOM stays at 29-39 rows, scrolling maps to the correct
rows, the scrollbar height is exact, an off-screen-selected row renders with the
selected class, and the row height is exactly 28. Every large table (Queue,
History, Recent) is now virtualized.

407 tests pass; clean Electron boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 03:15:44 +02:00
Administrator
7f636258d4 perf(uploads): make the batch-start file-stat non-blocking — kill the 336ms main stall (v3.3.103)
The v3.3.102 log (real 224-job batch) confirmed History virtualization works and
steady-state uploads are pristine (fps=32, event-loop mean 11.8ms). The remaining
residual was a ~6s spin-up burst at batch start: the main event loop blocked for
336ms with cpu=0%core (i.e. blocked on I/O, not computing) and the renderer janked
196-391ms, then everything settled clean.

A multi-agent investigation plus an adversarial review corrected the obvious-looking
hypothesis. The renderer's uncapped progress-batch drain is NOT the cause:
handleProgress only mutates plain JS state and schedules already-coalesced renders
(one per frame), and main coalesces progress to ~50 latest-per-job entries per
100ms. Chunking that drain would fix nothing — and the reviewer showed it would
REGRESS correctness: requestAnimationFrame throttles to ~0 when the window is
minimized (the common state for a background uploader), so a rAF-chunked drain would
grow an unbounded backlog and defer persistQueueStateSoon for every buffered item,
losing terminal 'done' events on close (the queue-persistence ghost-fix class). So
that path is deliberately not taken.

The real cause (cpu=0%core = blocked on I/O) is a synchronous fs.statSync storm in
UploadManager.startBatch: the dedup loop ran up to DEDUP_CHUNK=200 synchronous
fs.statSync calls in a single tick before yielding (200 x ~1.68ms on the user's VM
= the exact 336ms), on a disk already saturated by the 1MB read-ahead.

Fix — make the batch-start stats non-blocking:
- The dedup loop now dedupes synchronously (cheap Map work) and then stats the
  unique files in parallel via await Promise.all(fs.promises.stat ...) per chunk, so
  the stat I/O runs on the libuv threadpool and the main thread never blocks. The
  results-Map shape ({name,size,results:[]}) and dedup semantics (size 0 on failure)
  are unchanged.
- The per-job statSync fallback is converted to await fs.promises.stat for
  consistency (it sits in an async function before the first real await; the cached
  size from dedup already lets nearly every job skip it).

Tests: the upload-manager mocks override fs.statSync; they now also override
fs.promises.stat with the same fake sizes (upload-manager.test.js x2,
suspect-reject-alternates.test.js). 407 tests pass; clean Electron boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 20:53:48 +02:00
Administrator
d5bb97aefe perf(history): virtualize the History table — kill the last tab-switch layout cost (v3.3.102)
v3.3.101's gate removed the get-history parse on tab switch, but the log still
showed tab clicks at ~216ms with a ~197ms renderer long-task and NO get-history
(proc=0ms → pure browser layout, not JS). Cause: `.view{display:none}` ->
`.active{display:flex}` and the History table built up to 2000 non-virtualized
<tr>, so making the view visible laid out 2000 rows (~197ms on the RDP VM). The
queue was already virtualized; the History table was the only large non-virtual
one.

Measured the options with Playwright on the real DOM (this machine; the user's VM
is ~1.7x slower):
- current 2000 rows (auto layout): 118ms
- content-visibility + table-layout:fixed: 117ms (no help — rows still laid out)
- cap to 300 rows: 16ms, but rejected: History rows are per-file-per-hoster, so a
  single 1280-file batch is ~3840 rows and a small cap would hide a recent batch's
  links
- virtualize (40 visible of 2000): 2.3ms

Fix — virtualize renderHistoryTable, mirroring the queue's _renderVirtualRows:
- The header is always rendered; tbody#historyBody gets only the visible rows plus
  top/bottom spacer <tr> sized from VIRTUAL_ROW_HEIGHT. A rAF-coalesced scroll
  handler and a ResizeObserver on #historyContainer re-render the visible window;
  the ResizeObserver also serves as the show-trigger (a hidden 0x0 container that
  gains size on tab activation re-renders at the correct height). Sorting resets
  scrollTop and re-renders; the copy-link / sort-header click delegation is
  unchanged. _historyWorking holds the sorted working set.
- styles.css: .history-table gets table-layout:fixed plus scoped column widths so
  columns do not jump as rows scroll in and out. This is scoped to .history-table
  and does not touch the .col-* classes the queue shares.

Verified end-to-end with Playwright at 6000 rows: show cost 1.1ms (was 118ms), DOM
stays 32-42 rows, scroll maps to the right rows (top/middle/bottom), scrollbar
height exact, column widths stable, rows update on scroll. All rows remain
scrollable (no UX loss); the show cost drops ~100x.

407 tests pass; clean Electron boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 20:04:10 +02:00
Administrator
bad1c665f5 perf(history): gate the unconditional History-tab reload + fix the diagnostics history regression (v3.3.101)
v3.3.100's interaction instrument pinpointed the residual UI lag exactly: every
slow click was a tab switch into the History view (button.tab / nav.tab-bar,
200-840ms), each coupled to `ipc get-history wall=150-200ms sync` +
`main-longtask blocked=242-289ms lastIpc=get-history` + a 200-248ms renderer long
task. Uploads themselves are pristine (event-loop mean 11.7ms; the only spikes line
up with the get-history tab switches).

Root cause: the History tab handler called loadHistory() UNCONDITIONALLY on every
activation (renderer/app.js) even though it tracks a `_historyDirty` flag it never
checked. Each call synchronously readFileSync + JSON.parse's the ~185MB /
30000-entry electron-history.json (no cache), ships all 30000 batches across IPC,
and the renderer flattens ~120000 row objects before .slice(-2000) for the DOM (the
DOM is already capped at 2000).

Fix (the adversary-verified safe subset):
- Gate the History-tab load: only reload when `_historyDirty || !_historyEverLoaded`.
  Added `_historyEverLoaded`, and both flags are now set inside loadHistory() after
  the fetch succeeds (so a failed load retries). Dirty-coverage is complete — every
  history append routes through batch-done -> appendHistory and upload-batch-done ->
  handleBatchDone which sets _historyDirty=true. Result: repeat History tab switches
  with no new uploads do zero IPC/parse/flatten and are instant. (The first open
  after a new batch still parses once ~450ms; removing that needs a parse-cache or
  JSONL storage, deferred.)
- Diagnostics history regression: getHistory() read loadConfig().history, but since
  the v3.3.99 history split load() returns history:[] in packaged mode, so remote
  diagnostics reported totalBatches:0 despite 30000 real batches. It now reads
  loadHistory() (injected via the collector deps), with a backward-compatible
  fallback to loadConfig().history when not provided.

407 tests pass (2 new diagnostics regression tests); clean Electron boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 19:45:50 +02:00
Administrator
003e14dfe9 feat(diagnostics): renderer interaction timing — measure every UI switch/click, not just main-thread (v3.3.100)
Closes the last measurement gap. The main process was already fully instrumented
(every ipcMain handler timed at >=50ms, a 100ms main-thread long-task monitor with
channel attribution, config load/serialize timing). Account switches were already
covered too: switchAccount is a trivial synchronous Map-set and the rotation work
is async, so a switch cannot block the main thread, and any block that did occur
would surface in the long-task monitor.

The real gap was the renderer side: the renderer-perf line was gated on active
uploads (idle clicks were never logged) and only reported an aggregate longtask
count — no per-interaction latency and no element attribution. So a switch/sort/tab
that janked in the renderer (not main) was invisible.

renderer/app.js (additive, self-silencing, wrapped in try/catch):
- An Event Timing observer (PerformanceObserver type:'event', durationThreshold:50,
  buffered) logs `renderer-interaction <type> dur=Xms proc=Yms target=<el>` for every
  user interaction whose latency exceeds 50ms — always on, idle or under load — and
  names the element (id / first class / data-action / aria-label / title). This is
  the direct click->reaction latency the user feels.
- The longtask observer now also logs `renderer-longtask dur=Xms` immediately for any
  single renderer long task >=100ms, regardless of upload state.

405 tests pass; clean Electron boot. Every action — main or renderer, idle or under
load — now names itself in the log if it is slow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 19:21:41 +02:00
Administrator
c3381d360f perf(config): split history out of the hot config file — the real 38.5MB main-thread thrash (v3.3.99)
The v3.3.98 instrument exposed the actual cause of the 1-2s UI button lag, and it
was none of the read-path suspects. electron-config.json had grown to 38.5MB and
was being load()/structuredClone()/JSON.stringify()'d ~137 times in a 73s window
(140-592ms each) on the Electron main thread — roughly 47% main-thread occupancy.
That is the lag: a button click lands while the thread is mid-clone of a 38.5MB
object. The 1MB read-ahead could not touch it.

The bulk is history, not the queue. Each batch-done appended the upload-manager
summary verbatim, including the full per-file result list (per-hoster URLs), into
config.history; default historyRetention='all' never prunes, so it grew unbounded
(75 batches ≈ 38.5MB). The "queue=undefined" in the perf log was a logging bug
(reading .length on the pendingQueue object), not an empty queue. Writes drove the
storm: queue-persistence (save-global-settings) did two loads + one 38.5MB
serialize per call, and _atomicWrite nulls the cache so the next load is a full
38.5MB reparse; even cache hits structuredCloned the whole 38.5MB. The per-job
upload path makes zero config calls, so pending=1280 was never the driver.

Fix — move history into its own file so it is never parsed/cloned/serialized on a
config op (a 5-agent investigation + adversarial review chose this over three
read-side half-fixes; it is the only change that removes the clone AND the
serialize AND the post-write reparse at once):

- History lives in electron-history.json. _migrateHistory() runs once at init
  (packaged only) and is fail-safe: it writes history.json (tmp → fsync → rename),
  re-reads and verifies the entry count, and keeps a permanent
  electron-config.json.pre-history-split.bak BEFORE the config is ever allowed to
  drop its history. If verification fails it leaves history in the config (retry
  next launch). _loadImpl returns history:[] once migrated, so the cached object is
  tiny (cheap clones); the config file shrinks to ~KB on the first save (cheap
  reparse) and _serializeForDisk writes ~KB (cheap serialize).
  loadHistory/appendHistory/pruneHistory/clearHistory go through history.json on
  their own write-queue with a no-clobber guard; the legacy config path stays as a
  fallback when migration did not run.
- Validated against the real 185MB / 30000-entry bench fixture: migrate 1.5s once,
  every entry preserved + .bak kept; load() 631ms cold once → 0.1ms after the first
  save strips the file (185MB → 2.1KB); loadHistory() still returns all 30000.

Dropped per review (one-variable + risk): loadShallow (moot after the split),
cache-repopulate (its gate can never fire), and a per-batch resolution cache (stale
account pools → the rotation/byse failover-regression class — the one thing that
could silently corrupt uploads).

Instrumentation (the user asked to measure everything; all additive, threshold-
gated, MHU_PERF=0 disables):
- ipcMain.handle/.on are centrally wrapped to log `ipc <channel> wall=Xms sync=Yms`
  over 50ms — the button-press→response latency — hardened (Promise.resolve(p)
  .finally + try/catch'd logging) so a logging failure can never break IPC.
- A 100ms main-process drift monitor logs `main-longtask blocked=Xms lastIpc=… gc=…`
  for any single main-thread turn over 100ms (catches GC, fs scans, serialize that
  the IPC timing structurally cannot see).
- config-store perf lines gain via=<caller> and wqDepth=, and the queue= logging
  bug is fixed (now reads pendingQueue.queueJobs.length).

405 tests pass (9 new migration tests: preserve-count, round-trip, save() never
loses history, crash-window fallback, idempotency). Clean Electron boot, no repo
pollution (migration is packaged-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 18:37:12 +02:00
Administrator
121eac5f14 perf(uploads): 1MB read-ahead to absorb read-bursts + instrument the config-persist/load path (v3.3.98)
v3.3.97 (UV_THREADPOOL_SIZE 64→8) was a decisive win — mean event-loop-delay at
70 active uploads dropped 200ms→~11ms (18×), rss 577→287MB, renderer healthy in
14/15 windows. But the user reports it is still not perfectly smooth. A focused
multi-agent investigation plus an adversarial review localized the residual to
TWO distinct, separately-measured spike sources:

1. Read-bursts. In the tail windows the file-read histogram inverts: FSReqCallback
   climbs to 66-70 against threadpool=8 (~8.75× queue depth) while SimpleWriteWrap
   (socket writes) collapses to 4-24 and mean delay rises to 30-42ms. GC is ruled
   out (gcMax ≤27ms in every window). The clean inversion at a stable active=70 /
   pending=1287 shows the reads are causal, not a symptom of a block elsewhere.

2. A suspected synchronous config-persist stall. save() → load() reparses the whole
   electron-config.json — which now carries the 1287-job pending queue nested in
   globalSettings plus full history — on every persist (because _atomicWrite nulls
   the read cache), then _serializeForDisk JSON.stringify(…, null, 2) of all of it.
   One tail sample (max 1021ms, heap spiking to 142MB) fits a large synchronous
   structuredClone+stringify, but it is a single confounded point, so this build
   only INSTRUMENTS the path rather than asserting the cause.

This release ships one behavioral change (kept to a single variable so the next
log attributes cleanly) plus measurement:

- highWaterMark 256KB→1MB in all five streaming read loops (lib/hosters.js,
  doodstream/voe/vidmoly CHUNK_SIZE consts, and the inline value in
  clouddrop-upload.js:108 — NOT the 16MB server chunk at clouddrop-upload.js:12).
  UV_THREADPOOL_SIZE stays 8. This deepens each stream's read-ahead cushion from
  ~0.43s to ~1.7s at the per-stream rate, so a stream tolerates the threadpool
  queue without starving its socket write, and cuts read-completion callbacks and
  per-chunk Buffer allocations ~4×. Byte-correctness is unaffected: Content-Length
  is preamble+fileSize+epilogue, independent of chunk size, and the chunk size
  never touches the multipart boundaries. Fully reversible; a dedicated read-
  concurrency semaphore is held in reserve if 1MB does not clear the bursts.

- config-store.js now times load() (the full reparse, which the account-failed
  handler also hits per failure) and the _commit serialize, logging
  `config-load …` / `config-serialize wall=…ms bytes=… hist=… queue=…` when the
  synchronous work exceeds 20ms. load() is split into a timing wrapper + _loadImpl;
  the timer is a no-op until main.js wires configStore.setPerfLog → logInfo.

The renderer batch-drain fix for the one observed 243ms longtask is intentionally
deferred: that jank is downstream of the main-thread read-burst flooding IPC, so
fix #1 should make it self-heal; bundling it would confound the measurement and
touch the progress hot path. All 397 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 17:55:34 +02:00
Administrator
a5b835f76a perf(uploads): threadpool 64→8 + GC/heap instrumentation (decisive high-concurrency lag build, v3.3.97)
The v3.3.96 eventloop-delay logs from a real 70-connection run pinpointed the
high-concurrency lag to the file-read path. At a CONSTANT active-job count the
process flips between two clean regimes:

  HEALTHY  ELD ~11ms,  rss 268-308MB:  SimpleWriteWrap ≈ active, FSReqCallback ≈ 0-1
  BLOCKED  ELD 49-217ms, rss 540-610MB: FSReqCallback ≈ active (62-71 reads in
                                        flight), SimpleWriteWrap ≈ 0-4

Both the ELD spike and the rss balloon track FSReqCallback (libuv-threadpool file
reads) exactly — not crypto, not the renderer, not GC alone. All five uploaders
read identically via fs.createReadStream({highWaterMark: 256KB}); byse/doodstream/
voe run through the generic uploadFile in hosters.js (no dedicated module).

This build is both a candidate fix and a discriminator, per advisor review:

- UV_THREADPOOL_SIZE 64→8 (main.js:1). One reversible line, NOT an upload cap —
  70 uploads still run. 8 concurrent 256KB reads sustain ~100MB/s, far above the
  41MB/s aggregate, so it cannot bottleneck throughput even on the slow VM disk.
  Strong suspicion that tp=64 made it worse: it removed the natural read-
  serialization (default 4 threads) and let all 70 streams' reads fire at once,
  flooding the loop with completion callbacks in lock-step bursts.

- ELD line now also logs heap=heapUsed ext=external ab=arrayBuffers and
  gc=/gcTotal=/gcMax=ms (PerformanceObserver entryTypes:['gc'], reset per window).
  The 70×256KB ≈ 18MB of read buffers cannot account for the ~300MB rss swing —
  that is heap/object churn, so GC must be measured directly.

Decision rule for the next real-run log:
  - ELD drops with tp=8           → read over-parallelism confirmed (keep 8 or add
                                     a dedicated read-semaphore).
  - ELD high + GC pauses align    → heap churn, hunt the allocator.
  - ELD high + GC flat            → causation was reversed, pivot.

No upload-behavior change; the concurrency cap the user explicitly rejected is
untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 17:09:46 +02:00
Administrator
59dbd4b41c release: v3.3.96 2026-06-21 16:48:21 +02:00
Administrator
54dbba223d fix(diag): log the full eventloop-delay payload + harden tray icon load
The v3.3.94 measurement computed the main-process event-loop delay, CPU%, per-hoster connection distribution and transient error counts, but logged only '[INFO ] perf': logInfo(ctx, msg) treats a string first arg as the whole message and discards the second, so the payload was thrown away (confirmed in the user's upload-debug.log). Pass the line as a single argument so the real numbers reach the log.

Also: assets/ was missing from the electron-builder files list, so app_icon.ico was never packaged into the asar — new Tray() threw an unhandled rejection on every startup and left no tray icon. Package the icons and wrap createTray in try/catch with a nativeImage fallback so a missing/invalid icon can never crash tray creation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 16:47:48 +02:00
Administrator
7f7da8a619 release: v3.3.95 2026-06-21 16:33:51 +02:00
Administrator
8d1d641f97 fix(update): cancel active uploads before downloading + add download stall-timeout
Clicking 'install update' while many uploads were running left the update stuck at 'Update wird heruntergeladen...' forever, requiring a manual restart. Two causes, both fixed:

1) The 95 MB installer download competed with the active uploads for the (saturated) main process, CPU, network and connection pool, so it barely progressed. The install handler now calls uploadManager.cancel() first — the app quits and relaunches for the update anyway, and the queue is already persisted (non-terminal jobs restore as 'Bereit'), so freeing the resources is correct and makes the download fast.

2) The download read loop had no timeout, so a stalled stream hung indefinitely with no error. Added a 45 s stall-timeout: if no data arrives, the download aborts with a clear message ('Download haengt — seit 45 s keine Daten ... bitte erneut versuchen') instead of freezing, so the retry button works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 16:33:20 +02:00
Administrator
58db913275 release: v3.3.94 2026-06-21 16:24:24 +02:00
Administrator
958bc35c14 docs(tasks): v3.3.94 measurement build + confirmed localization (renderer innocent, lag from active uploads/oversubscription)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 16:23:43 +02:00
Administrator
53c3448836 feat(diag): comprehensive high-concurrency measurement build
User asked to measure everything so we can pinpoint the lag/ECONNRESET at 70 concurrent uploads. All additive instrumentation, no upload-behavior change.

main process: the eventloop-delay log line now also reports cpu=X%core (process.cpuUsage delta over wall time; >100% means multiple cores), rss, per-hoster live connection distribution (active-by-hoster), cumulative transient (ECONNRESET-class) error count, and pending count — pulled from a new UploadManager.getDiagnostics(). This shows whether the main thread is CPU-bound and which hoster is oversubscribed.

renderer: a PerformanceObserver('longtask') plus a requestAnimationFrame frame-time monitor log 'renderer-perf' every 5s while uploading — fps, jankFrames (>33ms), worstFrame, longtask count and max. This is the direct renderer ground truth (real frame rate under load), which the component-timing harness could not capture. Low fps / high jank => renderer is blocked; ~60fps while it still feels laggy => the cpu/eld numbers in the same log decide CPU-bound vs IO-bound in the main process.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 16:23:43 +02:00
Administrator
ad87e36a8f release: v3.3.93 2026-06-21 05:05:38 +02:00
Administrator
87886a5e8b docs(tasks,lessons): v3.3.93 renderer lag knot — formatDateTime per progress event; ask the regime before measuring
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 05:05:01 +02:00
Administrator
6b5349c5f7 perf(renderer): stop formatting a timestamp on every progress event
maybeAddSessionFile computed formatDateTime(new Date()) at the top, before the status==='done' guard that early-returns for every other status. formatDateTime runs two Intl locale formats (~83us measured), so it executed on every progress event — onUploadProgressBatch loops the M-item batch into handleProgress -> maybeAddSessionFile, i.e. 10xM times per second — and threw the result away for all non-done events (the vast majority while uploading).

That made each progress batch a synchronous main-thread block scaling with the active-upload count: ~2.4ms at 25 concurrent, ~5ms at 61, every 100ms, on top of render and sort — enough to blow the 16ms frame budget and stutter scrolling. It is per-event, not per-render, so it janks regardless of scroll position, matching the user's report that the lag appears above ~50 connections and when the uploading rows are scrolled out of view.

Move the formatDateTime call inside the dedup block so it runs once per genuinely-new completed upload. A faithful Blink benchmark at the user's regime (500 rows virtualized, dynamic progress sort, scrolling) shows the per-batch cost drop from 1.7/3.2/4.1ms at 25/50/61 concurrent to a flat 0ms, and frame P95 at 61 concurrent from 7.3ms to 4.2ms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 05:05:01 +02:00
Administrator
10854a8f43 release: v3.3.92 2026-06-21 04:21:01 +02:00
Administrator
5e8a34de40 docs(tasks): v3.3.92 decisive instrument + breadth audit (main.js log/progress/IPC + 5 uploaders all clean)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:20:32 +02:00
Administrator
901cd823dd perf(diag): enrich event-loop-delay log with active-resource histogram
Make the single high-concurrency measurement decisive. The ELD line now appends process.getActiveResourcesInfo() as a compact type-histogram, so one run splits all three readings in a single line: high mean/p99 means the main thread is CPU-blocked (workers/cap justified); low delay with many TCPSocketWrap/FSReqCallback/GetAddrInfoReqWrap resources means IO-bound queueing (threadpool/socket tuning, not workers); low delay with few resources means it is not saturated at all. Pure metrics — no credential-redaction path touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:20:32 +02:00
Administrator
854740d57c docs(lessons): instrument-before-refactor when two hypotheses imply opposite hard-to-reverse fixes (v3.3.91)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:14:32 +02:00
Administrator
f63ca53f2f release: v3.3.91 2026-06-21 04:11:21 +02:00
Administrator
e135655c95 docs(tasks): high-concurrency discriminator answered — instrument-first (v3.3.91), gate worker refactor on real-app ELD number
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:10:47 +02:00
Administrator
2fd26add1a perf(main): bump UV_THREADPOOL_SIZE to 64 + instrument event-loop delay
The user runs 50+ concurrent uploads (parallel counts raised deliberately). Every async uploader feeds undici from fs.createReadStream and resolves DNS via getaddrinfo — both go through the libuv threadpool, whose default size is 4. At 50 concurrent uploads, file reads and lookups serialize 4-at-a-time: a hard cliff at a small connection count that matches the 'lags from X connections onward' symptom. Raise the cap to 64 as the first statement (before require('electron'), so libuv reads it when it lazily inits the pool; an explicit env override still wins). Threads are created on demand, so a higher max costs nothing when unused — reversible, zero upload-core change.

Also enable perf_hooks.monitorEventLoopDelay and log mean/p99/max/stddev every ~5s while uploading. This is the ground-truth instrument that splits the two competing explanations for the lag: a high event-loop delay means the main thread is CPU-blocked (TLS/crypto) and only workers or a concurrency cap will help; a low delay while uploads stall means the work is IO-bound and the threadpool/socket config is the lever, not workers. The numbers are pure metrics — they never touch the credential-redaction path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:10:43 +02:00
Administrator
b85efcfe3d release: v3.3.90 2026-06-21 03:45:30 +02:00
Administrator
e0f789f56f docs(tasks): high-concurrency lag audit — renderer measured-refuted, sync-fs was the blocker (v3.3.90)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:44:41 +02:00
Administrator
c7e884fdec perf(upload-manager): throttle progress emits on rotation/suspect paths
The rotation-retry and suspect-alternate progressCb callbacks called _emitProgress (a synchronous EventEmitter emit plus a fresh object spread) on every stream chunk — hundreds per second per job — because they lacked the 250 ms lastEmitTime gate the primary upload path already has. With many concurrent uploads in rotation or suspect mode that is real main-thread emit amplification.

Mirror the primary path's gate exactly: the activeEntry speed/bytes mutation stays ungated so the stats timer and speed monitor keep seeing fresh values; only the _emitProgress call is throttled to ~4/sec. Behavior-preserving — identical progress data, fewer emits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:44:37 +02:00
Administrator
c6a67f6f2f perf(clouddrop): stream chunk reads off the main event loop (async fh.read)
_uploadChunked read each 16 MB chunk with fs.readSync on the main JS thread — the only one of the five uploaders that blocks synchronously (the other four stream async). Each readSync stalls the whole event loop ~5-9 ms on SSD, 30-100 ms on a slow disk, freezing all progress emits, IPC, renders and every other concurrent upload for that window. The stall scales with the number of simultaneous clouddrop uploads, matching the 'feels laggy while uploading, worse with more at once' symptom at modest CPU (one core pinned, ~40% of 8).

Swap fs.openSync/readSync/closeSync for fs.promises.open + await fh.read + await fh.close. Buffer reuse and the partial-last-chunk subarray view are unchanged. Verified byte-identical to the old loop via SHA-256 over every chunk-boundary case (full chunk, partial last chunk, 2/3/4-chunk files, single byte) before shipping — a chunk-read bug would corrupt the upload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:44:31 +02:00
Administrator
ea14d11ee2 release: v3.3.89 2026-06-21 03:06:44 +02:00
Administrator
8df6de06f1 docs(todo,lessons): diagnostics audit — healthy for real usage; ship 2 transport one-liners, defer the redaction-surface freeze
52-agent audit of the v3.3.84/85 remote-diagnostics code. App is healthy for this
user's actual usage. Shipped the two zero-redaction-surface fixes (ws maxPayload,
sendToClient guard). Deferred the cold-path server_health O(historySize) freeze
because the fix refactors the credential-redaction collectors (leaked twice before).

Lesson: "not persistence, so safe" is a fallacy — the redaction layer is an equally
catastrophic guarantee surface (silent secret leak). At an audit goal, find+document
is the deliverable; don't cut into a twice-leaked redaction pipeline under a Stop hook.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:06:08 +02:00
Administrator
0809c75d50 fix(remote-server): cap WS maxPayload (256 KiB) + guard sendToClient — close a pre-auth parse freeze-sink and a send-throw crash
Two isolated hardenings of the opt-in remote/diagnostics WS server, surfaced by
the session-wide diagnostics audit:

1. WebSocketServer was created with no maxPayload, so ws defaults to 100 MiB per
   message. The connection handler runs JSON.parse(raw) on the FIRST message
   (the auth frame) before authentication, so any peer past the IP allowlist
   could send a huge payload and force a synchronous multi-MB JSON.parse on the
   main-process event loop — an unbounded freeze/DoS sink. Diag, auth and WebRTC
   signaling messages are all small; cap maxPayload at 256 KiB to close it.

2. sendToClient did ws.send(JSON.stringify(data)) with no readyState/try guard
   (unlike broadcast, which checks ws.readyState === 1). A send on a closing
   socket, or a stringify throw, escaped the diag-response callback as an
   uncaughtException — a potential crash. Mirror broadcast: send only when
   readyState === 1, wrapped in try/catch.

Both are isolated to the transport layer with zero redaction surface. The audit's
larger finding — server_health doing O(historySize) synchronous work per request
(6-7 full-config clones + unbounded history walks) — is a real freeze, but ONLY on
the cold opt-in diagnostics path with a large history (this user: 23 rows), and the
safe fix cuts into the credential-redaction collectors (which have leaked twice);
deferred and documented in tasks/todo.md rather than operated under risk.

397/397 tests pass, eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:06:08 +02:00
Administrator
6dcc98f52d release: v3.3.88 2026-06-21 02:30:40 +02:00
Administrator
15a4509ad5 docs(todo,lessons): session-wide lag audit — clean bill of health, doodstream gate shipped, config-store findings deferred
Records the 18-agent line-by-line audit of every line written this session: the
reported lag was v3.3.87 (recent-panel cliff); the audit found no second cause
affecting this user. doodstream _debugLog sync-fs gated (shipped). config-store
load()/serialize history-scaling costs are real but sub-ms at this user's scale
and the fix is risky persistence surgery — deferred, documented with measurements.

Lesson: "audit every line" = look + measure + risk-appropriate decision, NOT
fix-everything; the load() perf win and its corruption risk are the same coin
(shared batch refs), so there is no safe version — defer, don't ship.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 02:29:59 +02:00
Administrator
233952af8f perf(doodstream): gate per-upload _debugLog behind logVerbose (default off) — drop ~8-15 sync fs syscalls/upload off the main loop
doodstream-upload.js's _debugLog ran with NO verbose gate: every call did a
synchronous statSync (via maybeRotateLogFile) + appendFileSync directly on the
main-process event loop, ~8-15x per upload (server probe, response, redirect,
result page, filecode parse, hidden fields, submit/follow, plus retry branches).
DoodstreamUploader runs in the main process, so each pair of sync fs syscalls
blocked the event loop while uploading — delaying IPC, upload-progress-batch
forwarding, tray-tooltip and webhook handling for every other concurrent upload.
A constant per-upload main-thread tax (not history-scaling), surfaced by the
session-wide lag audit and confirmed as the one finding that bites in the real
upload scenario.

Fix: gate _debugLog behind the existing globalSettings.logVerbose setting
(default false), exactly mirroring main.js logDebug/_logVerbose. A module-level
_debugVerbose flag + setDebugVerbose() setter, an early-return at the top of
_debugLog, and one wire at main.js's single setLogVerbose chokepoint (covers
boot + save-config + the verbose toggle). When verbose is off the doodstream
trace simply isn't written — same contract as the main debug.log — and the
per-upload sync fs disappears. When a doodstream issue needs tracing, enabling
verbose restores the full trace.

The config-store audit findings (load() history-clone cost, per-write history
serialize) are real but scale only with history size — tens of microseconds at
this user's 8-batch config, and the safe fix is risky persistence surgery on
credential-bearing code for a latent micro-cost; deferred and documented in
tasks/todo.md rather than shipped.

397/397 tests pass, eslint clean, gate wiring verified (shared module instance,
default-off, toggles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 02:29:59 +02:00
Administrator
c37c8e3906 release: v3.3.87 2026-06-21 01:49:23 +02:00
Administrator
9cc8fee02c docs(todo,lessons): long-run lag root cause was the recent-panel append-gate cliff (measured 80ms->7ms)
Records the verified root cause + the dismissed-with-evidence findings:
- _sessionFileKeys "separator mismatch" = FALSE POSITIVE (real U+0001 chars, verifier Read rendered them invisibly)
- queueJobs O(N) per-render scan = real but Blink-measured <0.1ms at 3000 jobs -> skipped
- standing 2000-row relayout = median 0.4ms -> no virtualization needed
- doodstream sync _debugLog = constant freeze, deferred to a separate change

Lesson: measure magnitudes at the real artifact before fixing a "scales-with-X" cause;
profile in real Blink (Playwright) not jsdom; verify multi-agent findings against primary
evidence (char-code dump caught the control-char false positive).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 01:47:59 +02:00
Administrator
0f9096be3c perf(renderer): keep recent-uploads panel append-only past the cap (kill ~80ms per-completion freeze)
The recent-uploads panel had a cheap append-only fast path, but it was gated on
`rows.length > _recentLastRenderedLen`. maybeAddSessionFile caps sessionFilesData
by push-then-slice (2000 -> 2001 -> sliced back to 2000), so once a session
produces more than SESSION_FILES_CAP rows the length is pinned at the cap and the
gate is false forever. Every subsequent completion then fell through to the full
`tbody.innerHTML = rows.map(...).join('')` rebuild of all ~2000 rows.

The cap is per (link x file x hoster), so with 4-5 selected hosters the 2000 cap
is hit at only ~400-500 distinct files — very reachable in a long folder-monitor
session. Profiled in Chromium (same Blink engine as Electron, table-layout:fixed):
the full 2000-row rebuild costs ~80ms and ran on EVERY completion past the cap — a
repeating ~80ms main-thread freeze. That is the "fine on a fresh start, gets laggy
after many uploads while CPU (~40%) and RAM (~6GB, stable) stay normal" symptom: a
render-thread stall, not CPU saturation or a memory leak.

Fix: track newly-pushed rows in _recentPendingAppends (incremented in
maybeAddSessionFile, consumed every render) and gate the fast path on
`pendingAppends > 0` instead of length growth, so it survives the cap. Prepend the
new rows, then evict the same overflow count from the DOM bottom (oldest rows,
which is where the date-desc view places the front-of-array entries the cap slices
off). DOM work is O(added) again. The fast path is gated behind an explicit
`appendOnly` flag passed only by scheduleRecentRender's rAF, so selection / delete
/ clear / sort / batch-done renders stay full rebuilds and cannot wrong-evict or
double-prepend.

Verified in Blink over a simulated 5000-completion session (3/frame, far past the
cap): per-frame render 80ms -> median 7.4ms (>10x), and the DOM stays exactly equal
to the data (cap held at 2000, newest-on-top, oldest evicted, zero duplicates).
397/397 tests pass, eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 01:47:59 +02:00
Administrator
29d1944328 perf(config): cache parsed config + lean serialize + drop load() from log flush (long-run lag)
A 13-agent hunt pinned the "wird mit der Zeit laggy" symptom (CPU/RAM normal, UI
sluggish after many uploads) to the main process re-doing config I/O that scales
with the ever-growing history, stalling the synchronous main event loop so the
renderer's IPC round-trips feel laggy. The renderer render path was already
optimized (virtualized queue, capped panels) — confirmed clean.

This commit lands the two contained fixes (T1 + T3); the queue-persist rewrite (T2)
follows separately.

config-store (T1):
- load() now has an in-memory cache keyed on the file mtime+size. The processed
  config (merged + credential-decrypted) is re-read/re-parsed/re-DPAPI-decrypted
  ONLY when the file actually changes; our writes invalidate it, external edits
  change mtime/size so the cache misses. Eliminates a full disk read + JSON.parse of
  the whole growing history + per-credential decrypt on the vast majority of the ~38
  load() call sites. load() always returns a structuredClone so callers can mutate
  freely without corrupting the cache.
- _serializeForDisk clones ONLY the hosters subtree (the only thing encryptCredentials
  touches) instead of JSON.parse(JSON.stringify(whole config)) — no more deep-cloning
  an 8 MB history on every write.
- _atomicWrite refreshes the .bak with a raw fs.copyFileSync instead of
  read + JSON.parse + write (it was re-parsing the full config a 2nd time per write).

main.js (T3):
- The log-flush paths resolved the log file via configStore.load() ~8x/second during
  uploads (re-reading + cloning the whole config just to read logMode/logFilePath).
  Cache those two strings in module scope, invalidate on the settings-save handlers.

Verified: 26 config-store tests (incl. new cache-correctness: independent clones,
external-change invalidation, save invalidation) + full 394-test suite green, lint 0
errors. Benchmark (8000-batch / 4.6 MB history): log flush no longer calls load() at
all; the remaining per-write history serialize is what T2 removes from the hot path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 00:49:55 +02:00
Administrator
939d30abfe docs(lessons): 'do it like <other project>' = find + map it exactly, fail-closed allowlist model (v3.3.86)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:43:07 +02:00
Administrator
3f8854693a release: v3.3.86 2026-06-19 19:39:15 +02:00
Administrator
b25b51840d test(diagnostics): live network-bind path — allowlisted non-loopback peer connects over a real 0.0.0.0 socket
Closes the one link the unit/wiring tests covered only by composition: binds 0.0.0.0,
allowlists a real LAN IPv4, and asserts auth-ok over a real socket (the Tailscale path).
Skips when no non-internal IPv4 interface exists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:38:25 +02:00
Administrator
8dff455062 docs(todo): Tailscale network-bind + fail-closed allowlist plan + review
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:35:38 +02:00
Administrator
0c6c502aab feat(diagnostics): network bind + fail-closed IP allowlist + host-in-code (Tailscale, like rd-diagnostics-mcp)
Matches the Real-Debrid-Downloader's rd-diagnostics-mcp model so the read-only
diagnostics agent is reachable over Tailscale (or any private tunnel) the same way
the downloader is, instead of requiring an SSH local-forward.

- lib/ip-allowlist.js (NEW): fail-closed IP allowlist — normalizeIp strips
  ::ffff:, loopback is always allowed, an empty allowlist accepts loopback ONLY
  (fail-closed), exact IP + CIDR (incl. the Tailscale CGNAT range 100.64.0.0/10) +
  wildcard rules. The real socket peer IP is the authority (never a forwarded header).
- remote-server.js: rejects non-allowlisted peers at connection (close 4005). Opt-in
  via config.allowlist (the existing remote-control server, which passes none, is
  unaffected). Loopback always passes, so local + SSH-forward use keeps working.
- Two bind modes (config diagnostics.bindMode): "local" -> 127.0.0.1 (default),
  "network" -> 0.0.0.0 but ONLY when a non-empty allowlist is set (else it stays
  loopback, fail-closed). The allowlist + token gate access; the tunnel
  (Tailscale/WireGuard) is the confidentiality layer (transport is still plaintext ws://).
- The connection code now carries the host: mhu1_<base64url{v,h,p,t,n,fp?,s?}>. The
  gateway decode is tolerant of the legacy {port,token,label} keys; connect_server
  takes the host from the code (host arg is an optional override). Proven end-to-end:
  the integration harness now connects with NO host arg and resolves it from the code.
- Renderer: Sichtbarkeit selector (local/network), public-host input with
  suggested-host chips (os.networkInterfaces — the Tailscale IP shows up there),
  allowlist textarea (network mode), and network-requires-allowlist validation.
- main.js: bindMode->host, getSuggestedRemoteHosts, host-in-code, allowlist plumbed
  into startDiagnosticAgent + the diagnostics IPC (get/save/status).
- docs: rewritten for Tailscale (set the allowlist to your tailnet, put the Tailscale
  IP/MagicDNS in the code address — no SSH forward needed).

This supersedes the v3.3.85 hard loopback-lock with the downloader's allowlist model.
Tests: lib/ip-allowlist (8) + remote-server allowlist wiring/loopback (2) + gateway
decode (host short-key + legacy tolerance). 393 app tests + 9 gateway tests + e2e +
host-in-code integration + adversarial all green; lint 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:32:29 +02:00
Administrator
0ee874ba99 docs(lessons): a diagnostic tool must never freeze the process it diagnoses (v3.3.85 hardening)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:46:07 +02:00
Administrator
70e7f2a9fd release: v3.3.85 2026-06-19 18:45:22 +02:00
Administrator
cfd5ca07ec chore(gateway/tooling): verify harnesses, Windows token ACL, ESM lint, honest transport docs
Repo-side hardening and tooling from the intensive test round (none of this ships
in the app installer).

- gateway: registry.json (holds bearer tokens) now gets a best-effort owner-only
  NTFS ACL on Windows via `icacls /inheritance:r /grant:r <user>:F` (the chmod
  0600 is a no-op on NTFS); verified the file ends up <user>:(F) only.
- gateway: connect_server now reads the app version from the real get_system_info
  shape (data.app.version / data.agent.version), so "connected to vX.Y.Z" works.
- gateway: read_log tool description documents grep as a case-insensitive substring
  filter with "|" alternation (not a regex), matching the agent-side change.
- gateway: standalone verification harnesses moved to gateway/verify/ (so
  `node --test` only sweeps real unit tests) and exposed via `npm run verify`:
  e2e-verify, integration-mcp (live gateway-MCP <-> agent, all 14 tools), and
  adversarial-probe (redaction fuzz + ReDoS + lockout). `npm test` runs the units.
- eslint: gateway/** now lints as ESM (sourceType module) via a dedicated block;
  global ignores fixed so `eslint .` is clean across the whole project (0 errors).
- docs/remote-diagnostics-setup.md: made the transport story honest — the agent
  speaks plaintext ws:// over enforced loopback; the SSH/WireGuard tunnel is the
  ONLY confidentiality layer (wss/TLS + cert-pin is a documented future mode, not
  active). Removed the stale "bind to a LAN/VPN IP" guidance (loopback is enforced).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:41:21 +02:00
Administrator
8d757a99dd fix(diagnostics): harden read-only agent — grep ReDoS, prototype-chain whitelist bypass, redaction gaps
Intensive end-to-end testing (a live gateway-MCP <-> agent integration harness +
an adversarial redaction/abuse probe + an independent security audit) surfaced
three real issues in the shipped read-only diagnostic agent. All run in lib/**,
which is packaged in the app.

1. grep ReDoS froze the Electron main process. read_log compiled the
   client-supplied grep into `new RegExp(grep, 'i')` and ran it synchronously over
   the log tail IN the main process. A catastrophic pattern (e.g. "(a+)+$" against
   a long line) hangs the whole app — empirically confirmed (8s timeout, killed).
   JS regex is synchronous and uncancellable, so grep is now a case-insensitive
   literal substring filter with "|" alternation ("error|timeout|502"). Provably
   linear-time; covers the real diagnostic need.

2. Prototype-chain whitelist bypass. The op table was a plain object literal, so
   handle("constructor" | "toString" | "valueOf", ...) resolved an inherited
   Object.prototype function, passed the `typeof fn === 'function'` guard and
   returned {ok:true}. Harmless functions today, but a whitelist-integrity hole.
   Now guarded with a string check + Object.prototype.hasOwnProperty.

3. Redaction defense-in-depth gaps. redactLogText now also scrubs: basic-auth URL
   passwords (scheme://user:pass@host), Authorization: Basic, JWTs (eyJ...x.y.z),
   and bare/JSON session= values. Mostly theoretical in today's readable logs
   (secret-bearing bodies go to the excluded doodstream-debug.log; other hosters
   throw static strings) but matters as the verbose-logging surface grows.

Verified: 383 app tests (incl. new regression tests for all three), the live
gateway-MCP integration harness (all 14 tools, zero leaks, error paths), the
adversarial probe (14/14+ secret shapes scrubbed, ReDoS 1ms, lockout, malformed
args), e2e gate, lint 0 errors. Only residual: a standalone high-entropy blob with
zero key/Bearer/URL context — inherent to any denylist, acknowledged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:40:57 +02:00
Administrator
602e48cb01 fix(gateway): write the token registry with owner-only permissions (0600)
registry.json holds each server's bearer token (the connection secret). It was
written with the process default umask, leaving it group/world-readable on POSIX
multi-user hosts. Write with mode 0o600 and chmod the existing file (writeFile
only applies mode on creation). No-op on Windows (NTFS uses ACLs, and the file
already sits under the user profile and is gitignored), effective on Linux/macOS
where the gateway may run.

Gateway-only change — not part of the app installer or auto-updater, so no
version bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 17:57:39 +02:00
Administrator
20b53f9587 docs(lessons): security-E2E must drive every collector default path, not just the aggregate hub
Captures the two redaction leaks from the remote-diagnostics build: a hoster-
returned opaque token surviving because value-scrub only covers stored config
secrets, and the get_config_redacted/get_queue_state default paths skipping
pattern-scrub entirely. The discriminator the first E2E missed, plus the rule:
exercise each collector with its DEFAULT args and seed fixtures with a NON-config
secret so you test pattern-scrub, not value-scrub by accident.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 17:49:14 +02:00
40 changed files with 2467 additions and 393 deletions

3
.gitignore vendored
View File

@ -6,7 +6,10 @@ __pycache__/
electron-config.json
electron-config.json.bak
electron-config.json.tmp
electron-config.json.pre-history-split.bak
electron-config.pre-import-*.json
electron-history.json
electron-history.json.tmp
*.log
debug.log
fileuploader.log

View File

@ -17,38 +17,60 @@ On the **server** (the machine running the app):
2. Enable **"Diagnose-Zugriff"**.
3. Copy the connection **code**. It looks like `mhu1_<base64url...>`.
The code carries a one-time auth **token** (and, for TLS, the server cert fingerprint). It does
**not** carry a host — you supply the host yourself when you connect. Treat the code as a
The code carries the **host**, **port** and a one-time auth **token** (`mhu1_<base64url{v,h,p,t,n}>`).
The bridge dials the host from the code, so you usually just hand over the code. Treat the code as a
**secret**: anyone with the code and network reach to the agent can read diagnostics.
The agent's **safe default is to bind to `127.0.0.1`** (loopback only). It is not exposed to the
network. You reach it through a tunnel (next step).
**Two visibility modes** (Settings → Diagnose-Zugriff → Sichtbarkeit):
- **Nur lokal** (default): the agent binds to `127.0.0.1`. Reach it through a tunnel (next step).
- **Im Netzwerk**: the agent binds to `0.0.0.0` but is gated by a **fail-closed IP allowlist** — only
source IPs/CIDRs you list may connect (loopback is always allowed), *in addition to* the token. An
empty allowlist means loopback only. This is the mode to use with **Tailscale**: set the allowlist
to your tailnet (e.g. `100.64.0.0/10`) and put the server's Tailscale IP / MagicDNS name into the
code address — then the bridge connects straight over the tailnet, no SSH forward needed.
**Transport note:** the agent speaks **plaintext `ws://`**. The token and the diagnostic data are
*not* encrypted on the wire by the agent itself — confidentiality comes from the **tunnel**
(Tailscale/WireGuard/SSH) you reach it through. In network mode the IP allowlist + token are the
access gate; **only bind to the network behind a private tunnel you trust** (a tailnet, a VPN, or a
trusted LAN). (A future build may add `wss`/TLS with cert pinning via an `fp` field in the code; it
is not active today.)
---
## 2. Reach the agent over a tunnel (SSH local port-forward or WireGuard)
## 2. Reach the agent over a tunnel
Because the agent binds to `127.0.0.1` on the server, open a tunnel from your machine to the
server's loopback. The agent's default port is `9110`.
The agent's default port is `9110`. Pick the path that matches your setup.
### SSH local port-forward (recommended)
### Tailscale (recommended for many servers)
Put every server and your gateway machine on the same tailnet. On each server, set **Sichtbarkeit =
Im Netzwerk**, allowlist your tailnet (`100.64.0.0/10`, or the specific Tailscale IPs you'll connect
from), and set the **code address** to that server's Tailscale IP or MagicDNS name. The bridge then
connects straight to `<tailscale-name>:9110` — WireGuard (Tailscale) encrypts the transport, and the
allowlist + token gate access. No SSH forward, no per-session tunnel command.
### SSH local port-forward (keep the agent loopback-only)
With **Sichtbarkeit = Nur lokal**:
```
ssh -L 9110:127.0.0.1:9110 user@server
```
Leave that session open. Now `127.0.0.1:9110` on **your** machine is forwarded to
`127.0.0.1:9110` on the **server**. You connect Claude to **`127.0.0.1`** (your local end of the
tunnel), not the server's public IP.
Leave that session open. Now `127.0.0.1:9110` on **your** machine is forwarded to the server's
loopback. The code address is `127.0.0.1` (your local end of the tunnel).
### WireGuard (alternative)
### WireGuard (manual, alternative)
Bring up a WireGuard tunnel to the server, then connect to the server's WireGuard address (or, if
you forward loopback over the tunnel, `127.0.0.1`). Use whichever address resolves to the agent's
`127.0.0.1:9110` on the server.
Bring up a WireGuard tunnel and either bind the agent to the network with the peer's WG IP in the
allowlist, or forward loopback over the tunnel and connect to `127.0.0.1`.
> Only expose the agent directly on a LAN/VPN IP if you fully trust that network segment. The
> default loopback + tunnel is the secure choice.
> In **Nur lokal** mode the agent is unreachable except through a tunnel to loopback. In **Im
> Netzwerk** mode the fail-closed IP allowlist (plus the token) is the access gate — only bind to
> the network behind a tunnel/VPN you trust (a tailnet, WireGuard, or a trusted LAN). The transport
> is plaintext; the tunnel is what encrypts it.
---
@ -110,7 +132,7 @@ When a connect or a request fails, the gateway returns a human-readable cause. Q
| close code **4002** | stale or rotated code — re-copy the current code from the server |
| close code **4003** | brute-force lockout, wait 60s |
| connected but no `auth-ok` | old app version without the diagnostic agent — update that server |
| wss fingerprint mismatch | server cert changed (reinstalled?) — re-copy the code |
| wss fingerprint mismatch | reserved for the future TLS mode (not active today) — re-copy the code |
If you tunnel and still get `ECONNREFUSED`, check that the SSH session is up and that the agent is
actually listening on `127.0.0.1:9110` on the server (Diagnose-Zugriff enabled).

View File

@ -1,47 +1,6 @@
import security from 'eslint-plugin-security';
export default [
{
files: ['**/*.js'],
ignores: ['node_modules/**', 'release/**', 'tests/**'],
plugins: { security },
languageOptions: {
ecmaVersion: 2022,
sourceType: 'commonjs',
globals: {
require: 'readonly',
module: 'readonly',
exports: 'readonly',
__dirname: 'readonly',
__filename: 'readonly',
process: 'readonly',
console: 'readonly',
setTimeout: 'readonly',
clearTimeout: 'readonly',
setInterval: 'readonly',
clearInterval: 'readonly',
setImmediate: 'readonly',
Buffer: 'readonly',
URL: 'readonly',
fetch: 'readonly',
AbortController: 'readonly',
AbortSignal: 'readonly',
navigator: 'readonly',
document: 'readonly',
window: 'readonly',
localStorage: 'readonly',
HTMLElement: 'readonly',
alert: 'readonly',
confirm: 'readonly',
requestAnimationFrame: 'readonly',
queueMicrotask: 'readonly',
Intl: 'readonly',
crypto: 'readonly',
URLSearchParams: 'readonly',
EventSource: 'readonly',
}
},
rules: {
const sharedRules = {
// Security rules
// detect-object-injection disabled: 78 false positives from config lookups like obj[hosterName]
'security/detect-object-injection': 'off',
@ -80,6 +39,66 @@ export default [
'no-unsafe-finally': 'error',
'no-unmodified-loop-condition': 'warn',
'no-template-curly-in-string': 'warn',
};
const nodeGlobals = {
process: 'readonly',
console: 'readonly',
setTimeout: 'readonly',
clearTimeout: 'readonly',
setInterval: 'readonly',
clearInterval: 'readonly',
setImmediate: 'readonly',
Buffer: 'readonly',
URL: 'readonly',
URLSearchParams: 'readonly',
fetch: 'readonly',
crypto: 'readonly',
structuredClone: 'readonly',
};
export default [
{ ignores: ['**/node_modules/**', 'release/**', 'tests/**'] },
{
files: ['**/*.js'],
ignores: ['gateway/**'],
plugins: { security },
languageOptions: {
ecmaVersion: 2022,
sourceType: 'commonjs',
globals: {
require: 'readonly',
module: 'readonly',
exports: 'readonly',
__dirname: 'readonly',
__filename: 'readonly',
...nodeGlobals,
AbortController: 'readonly',
AbortSignal: 'readonly',
navigator: 'readonly',
document: 'readonly',
window: 'readonly',
localStorage: 'readonly',
HTMLElement: 'readonly',
alert: 'readonly',
confirm: 'readonly',
requestAnimationFrame: 'readonly',
queueMicrotask: 'readonly',
Intl: 'readonly',
EventSource: 'readonly',
}
},
rules: sharedRules
},
{
files: ['gateway/**/*.js', 'gateway/**/*.mjs'],
ignores: ['gateway/node_modules/**'],
plugins: { security },
languageOptions: {
ecmaVersion: 2022,
sourceType: 'module',
globals: nodeGlobals
},
rules: sharedRules
}
];

View File

@ -42,18 +42,26 @@ export function decode(code) {
if (payload.v !== 1) {
throw new Error(`Invalid code: unsupported version (expected v=1, got ${payload.v})`);
}
if (typeof payload.port !== 'number' || !Number.isFinite(payload.port)) {
const host = payload.h !== undefined ? payload.h : payload.host;
const port = payload.p !== undefined ? payload.p : payload.port;
const token = payload.t !== undefined ? payload.t : payload.token;
const label = payload.n !== undefined ? payload.n : payload.label;
const scheme = payload.s === 'wss' ? 'wss' : 'ws';
if (host !== undefined && typeof host !== 'string') {
throw new Error('Invalid code: "host" must be a string when present');
}
if (typeof port !== 'number' || !Number.isFinite(port)) {
throw new Error('Invalid code: "port" must be a number');
}
if (typeof payload.token !== 'string' || payload.token.length === 0) {
if (typeof token !== 'string' || token.length === 0) {
throw new Error('Invalid code: "token" must be a non-empty string');
}
if (typeof payload.label !== 'string') {
if (label !== undefined && typeof label !== 'string') {
throw new Error('Invalid code: "label" must be a string');
}
if (payload.fp !== undefined && typeof payload.fp !== 'string') {
throw new Error('Invalid code: "fp" must be a string when present');
}
return payload;
return { v: 1, host: host ? String(host) : undefined, port, token, label: label !== undefined ? String(label) : undefined, fp: payload.fp, scheme };
}

View File

@ -48,7 +48,7 @@ const DIAGNOSTIC_TOOLS = [
{
name: 'read_log',
title: 'Read a log file',
description: 'Read a tail of one of the app log files, optionally grep-filtered, optionally a rotated backup.',
description: 'Read a tail of one of the app log files, optionally a rotated backup. grep is a case-insensitive substring filter; separate alternatives with "|" (e.g. "error|timeout|502") to keep any line matching at least one term. Not a regular expression.',
op: 'read_log',
inputSchema: {
name: z.enum(['debug', 'fileuploader', 'accountRotation', 'crash']),
@ -140,7 +140,7 @@ async function doConnect({ code, host, port, label }) {
target = { host: e.host, port: e.port, token: e.token, fp: e.fp, label: e.label };
} else {
if (!code) {
return { ok: false, error: 'provide a known label, or a code plus host' };
return { ok: false, error: 'provide a known label, or a code (the host is taken from the code; pass host only to override)' };
}
let payload;
try {
@ -148,15 +148,16 @@ async function doConnect({ code, host, port, label }) {
} catch (e) {
return { ok: false, error: String(e.message ?? e) };
}
if (!host) {
return { ok: false, error: 'host is required when connecting with a code' };
const effHost = host || payload.host;
if (!effHost) {
return { ok: false, error: 'no host in the code and none provided — pass host (e.g. the Tailscale IP/MagicDNS name)' };
}
target = {
host,
host: effHost,
port: typeof port === 'number' ? port : payload.port,
token: payload.token,
fp: payload.fp,
label: label || payload.label,
label: label || payload.label || effHost,
};
}
@ -172,7 +173,8 @@ async function doConnect({ code, host, port, label }) {
let version;
const info = await client.request('get_system_info', {});
if (info && info.ok && info.data) {
version = info.data.version ?? info.data.appVersion ?? undefined;
const d = info.data;
version = d.version ?? d.appVersion ?? (d.app && d.app.version) ?? (d.agent && d.agent.version) ?? undefined;
}
const id = `${target.label}@${target.host}:${target.port}`;
@ -223,7 +225,7 @@ export function buildServer() {
{
title: 'Connect to a diagnostic server',
description:
'Connect to a remote diagnostic agent. Use label for a known server, or code+host for a new one. The host is always supplied by you, never taken from the code.',
'Connect to a remote diagnostic agent. Use label for a known server, or a code for a new one — the host (e.g. a Tailscale IP/MagicDNS name) is taken from the code. Pass host only to override what the code carries.',
inputSchema: {
code: z.string().optional(),
host: z.string().optional(),

View File

@ -10,6 +10,10 @@
"engines": {
"node": ">=18"
},
"scripts": {
"test": "node --test \"test/**/*.test.js\"",
"verify": "node verify/e2e-verify.mjs && node verify/integration-mcp.mjs && node verify/adversarial-probe.mjs"
},
"dependencies": {
"@modelcontextprotocol/sdk": "~1.29.0",
"ws": "^8",

View File

@ -1,6 +1,8 @@
import { readFile, writeFile } from 'node:fs/promises';
import { readFile, writeFile, chmod } from 'node:fs/promises';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
import { execFile } from 'node:child_process';
import { userInfo } from 'node:os';
const HERE = dirname(fileURLToPath(import.meta.url));
const REGISTRY_PATH = join(HERE, 'registry.json');
@ -18,9 +20,20 @@ export async function loadRegistry() {
}
}
function tightenWindowsAcl(path) {
return new Promise((resolve) => {
let user;
try { user = userInfo().username; } catch { resolve(); return; }
if (!user) { resolve(); return; }
execFile('icacls', [path, '/inheritance:r', '/grant:r', `${user}:F`], { windowsHide: true }, () => resolve());
});
}
export async function saveRegistry(registry) {
const data = registry && typeof registry === 'object' ? registry : {};
await writeFile(REGISTRY_PATH, JSON.stringify(data, null, 2) + '\n', 'utf8');
await writeFile(REGISTRY_PATH, JSON.stringify(data, null, 2) + '\n', { encoding: 'utf8', mode: 0o600 });
try { await chmod(REGISTRY_PATH, 0o600); } catch {}
if (process.platform === 'win32') { try { await tightenWindowsAcl(REGISTRY_PATH); } catch {} }
}
export async function upsertEntry(entry) {

View File

@ -2,23 +2,25 @@ import { test } from 'node:test';
import assert from 'node:assert/strict';
import { encode, decode } from '../code.js';
test('decode(encode(x)) round-trips a full payload', () => {
const payload = {
v: 1,
port: 9110,
token: 'deadbeefcafe1234',
label: 'prod-3',
fp: 'AB:CD:EF:01:23:45:67:89',
};
const code = encode(payload);
test('decode reads the host-bearing short-key format (h/p/t/n/s/fp)', () => {
const code = encode({ v: 1, h: '100.64.0.5', p: 9110, t: 'deadbeefcafe1234', n: 'prod-3', s: 'wss', fp: 'AB:CD:EF:01' });
assert.ok(code.startsWith('mhu1_'));
assert.deepEqual(decode(code), payload);
const d = decode(code);
assert.equal(d.host, '100.64.0.5');
assert.equal(d.port, 9110);
assert.equal(d.token, 'deadbeefcafe1234');
assert.equal(d.label, 'prod-3');
assert.equal(d.scheme, 'wss');
assert.equal(d.fp, 'AB:CD:EF:01');
});
test('decode(encode(x)) round-trips a payload without fp (ws://)', () => {
const payload = { v: 1, port: 9110, token: 'token-abc', label: 'localhost' };
const code = encode(payload);
assert.deepEqual(decode(code), payload);
test('decode is tolerant of the legacy long-key format (port/token/label, no host -> ws)', () => {
const d = decode(encode({ v: 1, port: 9110, token: 'token-abc', label: 'localhost' }));
assert.equal(d.host, undefined);
assert.equal(d.port, 9110);
assert.equal(d.token, 'token-abc');
assert.equal(d.label, 'localhost');
assert.equal(d.scheme, 'ws');
});
test('decode rejects a string without the mhu1_ prefix', () => {

View File

@ -0,0 +1,137 @@
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
import { writeFileSync, mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import WebSocket from 'ws';
const require = createRequire(import.meta.url);
const appRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const support = require(join(appRoot, 'lib', 'support-bundle.js'));
const stats = require(join(appRoot, 'lib', 'stats.js'));
const RemoteServer = require(join(appRoot, 'lib', 'remote-server.js'));
const { createCollectors } = require(join(appRoot, 'lib', 'diagnostics-collectors.js'));
const { createAgent } = require(join(appRoot, 'lib', 'diagnostics-agent.js'));
const findings = [];
function leakCheck(label, text, needle, { realistic }) {
if (text.includes(needle)) findings.push({ label, needle: needle.slice(0, 24), realistic });
}
console.log('=== A. redactLogText pattern-scrub battery (no config secrets; pure shape detection) ===');
const battery = [
{ line: 'Authorization: Bearer abcDEF123456ghiJKL789', needle: 'abcDEF123456ghiJKL789', realistic: true },
{ line: 'using Bearer eyJhbGciOiJIUzI1Nib-longtokenvalue-0099', needle: 'eyJhbGciOiJIUzI1Nib-longtokenvalue-0099', realistic: true },
{ line: 'resp token=sess_9f8e7d6c5b4a3210ffee', needle: 'sess_9f8e7d6c5b4a3210ffee', realistic: true },
{ line: 'access_token: ya29.A0ARrdaM-longgoogletoken-123', needle: 'ya29.A0ARrdaM-longgoogletoken-123', realistic: true },
{ line: 'refresh_token = 1//0ggLongRefreshToken_abcdef', needle: '1//0ggLongRefreshToken_abcdef', realistic: true },
{ line: 'x-api-key: SuperSecretApiKeyValue99', needle: 'SuperSecretApiKeyValue99', realistic: true },
{ line: 'api_key=AKIAIOSFODNN7EXAMPLEKEY', needle: 'AKIAIOSFODNN7EXAMPLEKEY', realistic: true },
{ line: 'GET /up?key=querykeysecret12345 HTTP/1.1', needle: 'querykeysecret12345', realistic: true },
{ line: 'POST https://discord.com/api/webhooks/123456789012345678/WEBHOOKTOKENsecretvalue', needle: 'WEBHOOKTOKENsecretvalue', realistic: true },
{ line: 'set-cookie: session=COOKIEsecretvalue9988; HttpOnly', needle: 'COOKIEsecretvalue9988', realistic: true },
{ line: 'Cookie: sess_id=ABCcookievalue12345', needle: 'ABCcookievalue12345', realistic: true },
{ line: 'sessionId: SESSIONsecret009988aa', needle: 'SESSIONsecret009988aa', realistic: true },
{ line: 'two leaks: token=firsttok12345678 and api_key=secondkey87654321', needle: 'secondkey87654321', realistic: true },
{ line: 'proxy https://admin:Sup3rProxyPass@proxy.internal:8080/path', needle: 'Sup3rProxyPass', realistic: true },
{ line: 'Authorization: Basic dXNlcjpwYXNzd29yZF9zZWNyZXQ=', needle: 'dXNlcjpwYXNzd29yZF9zZWNyZXQ', realistic: true },
{ line: 'jwt eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhYmMxMjMifQ.SflKxwRJSMeKKF2QT4fwpMabc', needle: 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhYmMxMjMifQ', realistic: true },
{ line: 'session=BareSessionSecret998877', needle: 'BareSessionSecret998877', realistic: true },
{ line: 'password: PlainTextPassword123 was used', needle: 'PlainTextPassword123', realistic: true },
{ line: 'random high-entropy blob 9f8e7d6c5b4a3210ffeeddccbbaa with no key context', needle: '9f8e7d6c5b4a3210ffeeddccbbaa', realistic: false },
];
for (const t of battery) {
const out = support.redactLogText(t.line, []);
leakCheck('redactLogText: ' + t.line.slice(0, 40), out, t.needle, t);
console.log(` ${out.includes(t.needle) ? 'LEAK ' : 'scrub'} ${t.line.slice(0, 52)}`);
}
console.log('\n=== B. value-scrub: config secret in odd encodings (deepRedact via collectors) ===');
const SECRET = 'CFGsecret_aabbccddeeff';
const tmp = mkdtempSync(join(tmpdir(), 'mhu-adv-'));
writeFileSync(join(tmp, 'debug.log'), `plain ${SECRET}\nurlenc CFGsecret_aabbccddeeff also\n`);
const cfg = {
hosters: { doodstream: [{ accountId: 'a', apiKey: SECRET }] }, hosterSettings: {},
globalSettings: { diagnostics: { enabled: true, token: 'd'.repeat(64) },
pendingQueue: { savedAt: 1, selectedUploadHosters: [], selectedFiles: [], queueJobs: [
{ file: 'x', fileName: 'x', hoster: 'doodstream', status: 'error', error: `failed ${SECRET}` }] } },
history: [{ timestamp: 't', files: [{ name: 'x', results: [{ hoster: 'doodstream', status: 'error', error: `e ${SECRET}` }] }] }],
rotationCursors: {},
};
const cols = createCollectors({
loadConfig: () => JSON.parse(JSON.stringify(cfg)),
getAllLogPaths: () => ({ fileuploader: join(tmp, 'f.log'), debug: join(tmp, 'debug.log'), accountRotation: join(tmp, 'r.log'), doodstreamDebug: join(tmp, 'doodstream-debug.log'), crashLog: join(tmp, 'c.log'), logDir: tmp }),
support, stats, appInfo: () => ({ version: '3.3.84' }), systemInfo: () => ({}), agentInfo: () => ({}),
});
for (const [name, fn] of [
['getConfigRedacted(all)', () => cols.getConfigRedacted({ section: 'all' })],
['getQueueState(includeJobs)', () => cols.getQueueState({ includeJobs: true })],
['getQueueState(default)', () => cols.getQueueState({})],
['listErrors', () => cols.listErrors({})],
['getHistory(files)', () => cols.getHistory({ includeFiles: true })],
['serverHealth', () => cols.serverHealth({})],
['readLog(debug)', () => cols.readLog({ name: 'debug' })],
]) {
const text = JSON.stringify(fn());
leakCheck('collector:' + name, text, SECRET, { realistic: true });
console.log(` ${text.includes(SECRET) ? 'LEAK ' : 'scrub'} ${name}`);
}
console.log('\n=== C. abuse / DoS: ReDoS grep, oversized tailKb, malformed args (must not hang/crash) ===');
const t0 = Date.now();
writeFileSync(join(tmp, 'debug.log'), 'a'.repeat(80) + '! catastrophic-bait line\n' + `plain ${SECRET}\nurlenc CFGsecret_aabbccddeeff also\n`);
const redos = cols.readLog({ name: 'debug', grep: '(a+)+$', tailKb: 1 });
const redosMs = Date.now() - t0;
if (redosMs > 1500) findings.push({ label: 'grep ReDoS hang (' + redosMs + 'ms) on 80-a line', needle: '(a+)+$', realistic: true });
console.log(` grep "(a+)+$" vs 80-a line returned in ${redosMs}ms (must be <1500: ${redosMs < 1500})`);
const longGrep = cols.readLog({ name: 'debug', grep: 'a'.repeat(5000) });
console.log(` grep 5000-char pattern: ${longGrep && (longGrep.matchedLines !== undefined || longGrep.content !== undefined) ? 'handled' : 'handled'}`);
const bigTail = cols.readLog({ name: 'debug', tailKb: 9999999 });
console.log(` tailKb 9999999 clamped to: ${bigTail.tailKb} (<=1024:${bigTail.tailKb <= 1024})`);
let crashed = false;
for (const bad of [null, undefined, 42, [], { name: 123 }, { name: ['debug'] }, { name: 'debug', backup: 'evil' }, { name: 'debug', tailKb: -5 }, { limit: 'NaN' }]) {
try { cols.readLog(bad); cols.listErrors(bad); cols.getQueueState(bad); cols.getHistory(bad); cols.getAppEvents(bad); }
catch (e) { crashed = true; findings.push({ label: 'collector THREW on malformed args: ' + JSON.stringify(bad), needle: String(e.message), realistic: true }); }
}
console.log(` malformed-args battery: ${crashed ? 'THREW (bad)' : 'no throw (good)'}`);
console.log('\n=== D. agent whitelist: write/exec/unknown ops rejected, never throws ===');
const agent = createAgent(cols);
let agentThrew = false;
for (const op of ['save_config', 'run_health_check', 'exec', 'eval', 'delete_log', '__proto__', 'constructor', 'getConfigRedacted', '', null, 'get_config_redacted; drop']) {
try { const r = agent.handle(op, {}); if (r && r.ok === true && !['get_config_redacted'].includes(op)) findings.push({ label: 'agent ACCEPTED non-whitelisted op: ' + op, needle: op, realistic: true }); }
catch (e) { agentThrew = true; findings.push({ label: 'agent THREW on op ' + op, needle: String(e.message), realistic: true }); }
}
console.log(` non-whitelisted ops: ${agentThrew ? 'THREW (bad)' : 'all returned {ok:false} (good)'}`);
console.log('\n=== E. transport abuse: brute-force lockout + concurrent clients (live RemoteServer) ===');
const TOKEN = 'z'.repeat(64);
const srv = new RemoteServer();
await srv.start({ port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true, onDiagnosticRequest: (m, _c, reply) => reply(agent.handle(m.op, m.args)) });
const port = srv.getPort();
function wsOnce(sendToken) {
return new Promise((resolve) => {
const ws = new WebSocket(`ws://127.0.0.1:${port}`);
let authed = false;
ws.on('open', () => ws.send(JSON.stringify({ type: 'auth', token: sendToken, role: 'diagnostic' })));
ws.on('message', (raw) => { try { const m = JSON.parse(raw); if (m.type === 'auth-ok') { authed = true; ws.close(); resolve({ authed: true }); } } catch {} });
ws.on('close', (code) => resolve({ authed, code }));
ws.on('error', () => {});
});
}
const okClients = await Promise.all([wsOnce(TOKEN), wsOnce(TOKEN), wsOnce(TOKEN)]);
console.log(` 3 concurrent valid clients all authed: ${okClients.every(c => c.authed)}`);
let lastCode = null;
for (let i = 0; i < 6; i++) lastCode = (await wsOnce('wrongtoken')).code;
console.log(` after 6 bad-token attempts, close code = ${lastCode} (4003 lockout expected: ${lastCode === 4003})`);
const afterLock = await wsOnce(TOKEN);
console.log(` valid token DURING lockout window: authed=${afterLock.authed} closeCode=${afterLock.code} (locked out even with right token: ${!afterLock.authed})`);
srv.stop();
console.log('\n=== SUMMARY ===');
const real = findings.filter(f => f.realistic);
const theo = findings.filter(f => !f.realistic);
if (theo.length) console.log(` ${theo.length} THEORETICAL (acknowledged denylist limit): ${theo.map(f => f.label).join(' | ')}`);
if (real.length) { console.log(` ${real.length} REAL finding(s):`); for (const f of real) console.log(` - ${f.label} :: ${f.needle}`); process.exit(2); }
console.log(' No REAL leaks/crashes found. (Theoretical = standalone secret with zero key/Bearer/URL context — inherent to denylist.)');
process.exit(0);

View File

@ -0,0 +1,218 @@
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
import { writeFileSync, mkdtempSync, readFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import assert from 'node:assert';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
import { encode } from '../code.js';
const require = createRequire(import.meta.url);
const here = dirname(fileURLToPath(import.meta.url));
const gwRoot = join(here, '..');
const appRoot = join(gwRoot, '..');
const indexPath = join(gwRoot, 'index.js');
const registryPath = join(gwRoot, 'registry.json');
const RemoteServer = require(join(appRoot, 'lib', 'remote-server.js'));
const support = require(join(appRoot, 'lib', 'support-bundle.js'));
const stats = require(join(appRoot, 'lib', 'stats.js'));
const { createCollectors } = require(join(appRoot, 'lib', 'diagnostics-collectors.js'));
const { createAgent } = require(join(appRoot, 'lib', 'diagnostics-agent.js'));
const SECRET_API = 'APIKEY_live_77ffee0011aabb';
const SECRET_PW = 'pw_S3cr3t_zzqqww';
const SECRET_DIAGTOK = 'd'.repeat(64);
const OPAQUE_TOK = 'OPAQUE_session_tok_5a4b3c2d1e';
const WEBHOOK = 'https://discord.com/api/webhooks/987654321098765432/IntegrationWebhookSecretXyz';
const SECRETS = [SECRET_API, SECRET_PW, SECRET_DIAGTOK, OPAQUE_TOK, 'IntegrationWebhookSecretXyz'];
const tmp = mkdtempSync(join(tmpdir(), 'mhu-int-'));
const debugLog = join(tmp, 'debug.log');
const rotLog = join(tmp, 'account-rotation.log');
const dood = join(tmp, 'doodstream-debug.log');
writeFileSync(debugLog, [
`[2026-06-19T10:00:00Z] boot ok`,
`[2026-06-19T10:00:01Z] upload with apiKey=${SECRET_API}`,
`[2026-06-19T10:00:02Z] Authorization: Bearer ${OPAQUE_TOK}`,
`[2026-06-19T10:00:03Z] grepneedle marker line`,
].join('\n'));
writeFileSync(rotLog, `[2026-06-19T10:00:00Z] rotating to acc2\n`);
writeFileSync(dood, `[dood] api_key=${SECRET_API}\n`);
const config = {
hosters: { doodstream: [{ accountId: 'acc1', apiKey: SECRET_API, password: SECRET_PW }] },
hosterSettings: {},
globalSettings: {
webhookUrl: WEBHOOK,
diagnostics: { enabled: true, port: 9110, token: SECRET_DIAGTOK },
pendingQueue: { savedAt: '2026-06-19T09:00:00Z', selectedUploadHosters: ['doodstream'], selectedFiles: ['a.mp4', 'b.mp4'], queueJobs: [
{ file: 'a.mp4', fileName: 'a.mp4', hoster: 'doodstream', status: 'error', error: `rejected token=${OPAQUE_TOK}` },
{ file: 'b.mp4', fileName: 'b.mp4', hoster: 'doodstream', status: 'uploading', error: null },
] },
},
rotationCursors: { doodstream: 1 },
history: [{ timestamp: '2026-06-19T08:00:00Z', files: [{ name: 'a.mp4', results: [
{ hoster: 'doodstream', status: 'error', error: `boom apiKey=${SECRET_API}` },
{ hoster: 'voe.sx', status: 'done', url: 'https://voe.sx/x' },
] }] }],
};
function getAllLogPaths() {
return { fileuploader: join(tmp, 'fileuploader.log'), debug: debugLog, accountRotation: rotLog, doodstreamDebug: dood, crashLog: join(tmp, 'crash.log'), logDir: tmp };
}
const collectors = createCollectors({
loadConfig: () => JSON.parse(JSON.stringify(config)),
getAllLogPaths, support, stats,
appInfo: () => ({ name: 'mhu', version: '3.3.84' }),
systemInfo: () => ({ platform: 'win32', hostname: 'INT-HOST', cpuCount: 8 }),
agentInfo: () => ({ version: '3.3.84', port: 9110, clientCount: 1, lastAccess: null }),
});
const agent = createAgent(collectors);
const TOKEN = 't'.repeat(64);
const failures = [];
function check(name, cond, detail) {
if (cond) { console.log(` PASS ${name}`); }
else { console.log(` FAIL ${name}${detail ? ' :: ' + detail : ''}`); failures.push(name); }
}
function noLeak(name, payloadText) {
for (const s of SECRETS) {
if (payloadText.includes(s)) { check(`${name} (no-leak:${s.slice(0, 10)})`, false, 'secret present'); return; }
}
check(`${name} (no-leak)`, true);
}
const srv = new RemoteServer();
let client, transport;
const savedRegistry = existsSync(registryPath) ? readFileSync(registryPath, 'utf8') : null;
async function callJSON(name, args) {
try {
const r = await client.callTool({ name, arguments: args || {} });
const text = r.content ? r.content.map(c => c.text || '').join('') : '';
let data; try { data = JSON.parse(text); } catch { data = null; }
return { text, data, isError: !!r.isError, threw: false };
} catch (e) {
return { text: String(e && e.message || e), data: null, isError: true, threw: true };
}
}
function rejected(r) { return r.threw || r.isError || (r.data && r.data.ok === false); }
(async () => {
await srv.start({
port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true,
onDiagnosticRequest: (msg, _c, reply) => {
let res; try { res = agent.handle(msg.op, msg.args); } catch (e) { res = { ok: false, error: String(e && e.message || e) }; }
reply(res);
},
});
const port = srv.getPort();
const code = encode({ v: 1, h: '127.0.0.1', p: port, t: TOKEN, n: 'integration' });
transport = new StdioClientTransport({ command: process.execPath, args: [indexPath] });
client = new Client({ name: 'mhu-int-test', version: '1.0.0' });
await client.connect(transport);
const toolList = await client.listTools();
const names = toolList.tools.map(t => t.name).sort();
const expected = ['connect_server', 'current_server', 'disconnect_server', 'get_app_events', 'get_config_redacted', 'get_history', 'get_queue_state', 'get_rotation_state', 'get_system_info', 'list_errors', 'list_logs', 'list_servers', 'read_log', 'server_health'].sort();
check('listTools returns all 14 tools', JSON.stringify(names) === JSON.stringify(expected), names.join(','));
let r = await callJSON('connect_server', { code });
check('connect_server ok (host taken from the code, no host arg)', r.data && r.data.ok === true, r.text.slice(0, 120));
check('connect_server resolved host from code', r.data && r.data.server && r.data.server.host === '127.0.0.1');
check('connect_server reports version 3.3.84', r.data && r.data.server && r.data.server.version === '3.3.84');
r = await callJSON('current_server');
check('current_server shows host 127.0.0.1', r.data && r.data.current && r.data.current.host === '127.0.0.1');
r = await callJSON('server_health', { errorLimit: 10 });
check('server_health ok + sections', r.data && r.data.ok && r.data.data.errors && r.data.data.queue && r.data.data.logs);
noLeak('server_health', r.text);
r = await callJSON('read_log', { name: 'debug', tailKb: 64 });
check('read_log debug ok + benign content kept', r.data && r.data.ok && r.data.data.content.includes('grepneedle marker line'));
noLeak('read_log:debug', r.text);
r = await callJSON('read_log', { name: 'debug', grep: 'grepneedle' });
check('read_log grep filters to matching line', r.data && r.data.ok && r.data.data.content.includes('grepneedle') && !r.data.data.content.includes('boot ok'));
r = await callJSON('read_log', { name: 'doodstream' });
check('read_log doodstream REJECTED (live keys)', rejected(r), r.text.slice(0, 100));
noLeak('read_log:doodstream-reject', r.text);
r = await callJSON('read_log', { name: '../../../etc/passwd' });
check('read_log path traversal REJECTED', rejected(r), r.text.slice(0, 100));
noLeak('read_log:traversal-reject', r.text);
r = await callJSON('list_logs');
check('list_logs lists debug+fileuploader+accountRotation+crash', r.data && r.data.ok && r.data.data.files.length === 4);
check('list_logs marks doodstream NOT readable (not in files)', r.data && !r.data.data.files.some(f => f.name === 'doodstream'));
r = await callJSON('list_errors', { limit: 50 });
check('list_errors finds the 1 history error', r.data && r.data.ok && r.data.data.total === 1);
noLeak('list_errors', r.text);
r = await callJSON('get_queue_state', { includeJobs: true });
check('get_queue_state jobs present (2)', r.data && r.data.ok && Array.isArray(r.data.data.jobs) && r.data.data.jobs.length === 2);
check('get_queue_state stale flag set', r.data && r.data.data.stale === true);
noLeak('get_queue_state:includeJobs', r.text);
r = await callJSON('get_queue_state', {});
noLeak('get_queue_state:default-args', r.text);
r = await callJSON('get_history', { limit: 10, includeFiles: true, includeUrls: true });
check('get_history returns batches + perHoster', r.data && r.data.ok && Array.isArray(r.data.data.batches) && r.data.data.perHoster);
noLeak('get_history', r.text);
r = await callJSON('get_config_redacted', { section: 'all' });
check('get_config_redacted ok + history omitted', r.data && r.data.ok && r.data.data.config && r.data.data.config.history === undefined);
noLeak('get_config_redacted:all', r.text);
r = await callJSON('get_config_redacted', { section: 'hosters' });
noLeak('get_config_redacted:hosters', r.text);
r = await callJSON('get_rotation_state');
check('get_rotation_state returns cursors', r.data && r.data.ok && r.data.data.rotationCursors);
noLeak('get_rotation_state', r.text);
r = await callJSON('get_system_info');
check('get_system_info returns app+system+agent', r.data && r.data.ok && r.data.data.app && r.data.data.system);
noLeak('get_system_info', r.text);
r = await callJSON('get_app_events', { limit: 20 });
check('get_app_events ok', r.data && r.data.ok);
noLeak('get_app_events', r.text);
r = await callJSON('list_servers');
check('list_servers includes the connected one', r.data && r.data.ok && r.data.current && r.data.current.includes('integration'));
r = await callJSON('disconnect_server');
check('disconnect_server ok', r.data && r.data.ok && r.data.disconnected === true);
r = await callJSON('server_health');
check('tool after disconnect returns guidance (no server connected)', r.data && r.data.ok === false && /connect_server/.test(r.data.error || ''));
r = await callJSON('connect_server', { code: 'mhu1_not_valid_base64!!', host: '127.0.0.1' });
check('connect with garbage code REJECTED', r.data && r.data.ok === false);
r = await callJSON('connect_server', { code, host: '127.0.0.1', port: 1 });
check('connect to dead port REJECTED with guidance', r.data && r.data.ok === false && /not running|refused|firewall|closed|timeout/i.test(r.data.error || ''));
await client.close();
srv.stop();
if (savedRegistry !== null) writeFileSync(registryPath, savedRegistry, 'utf8');
console.log('');
if (failures.length) { console.log(`INTEGRATION FAIL: ${failures.length} check(s) failed: ${failures.join(' | ')}`); process.exit(1); }
console.log('INTEGRATION PASS: full gateway-MCP <-> live agent stack verified, every tool, no leaks, error paths correct.');
process.exit(0);
})().catch((e) => {
console.error('INTEGRATION ERROR:', e && e.stack || e);
try { srv.stop(); } catch {}
try { if (savedRegistry !== null) writeFileSync(registryPath, savedRegistry, 'utf8'); } catch {}
process.exit(1);
});

View File

@ -105,7 +105,7 @@ class ClouddropUploader {
let bytesRead = 0;
async function* generate() {
yield preambleBuf;
const fileStream = fs.createReadStream(filePath, { highWaterMark: 256 * 1024 });
const fileStream = fs.createReadStream(filePath, { highWaterMark: 1024 * 1024 });
for await (const chunk of fileStream) {
if (signal && signal.aborted) throw new Error('Aborted');
if (throttle) await throttle.consume(chunk.length, signal);
@ -159,7 +159,7 @@ class ClouddropUploader {
// Reuse a single buffer for all chunks (only the last chunk may be smaller,
// in which case we slice a view). Avoids 64× 16 MB allocations on a 1 GB
// file — real GC pressure during busy uploads.
const fd = fs.openSync(filePath, 'r');
const fh = await fs.promises.open(filePath, 'r');
let bytesSent = 0;
const reusableBuf = Buffer.allocUnsafe(chunkSize);
try {
@ -169,7 +169,7 @@ class ClouddropUploader {
const offset = i * chunkSize;
const remaining = fileSize - offset;
const thisChunkSize = Math.min(chunkSize, remaining);
fs.readSync(fd, reusableBuf, 0, thisChunkSize, offset);
await fh.read(reusableBuf, 0, thisChunkSize, offset);
const body = thisChunkSize === chunkSize
? reusableBuf
: reusableBuf.subarray(0, thisChunkSize);
@ -194,7 +194,7 @@ class ClouddropUploader {
if (progressCb) progressCb(bytesSent, fileSize);
}
} finally {
try { fs.closeSync(fd); } catch {}
try { await fh.close(); } catch {}
}
// 3. Complete session — all bytes are already on the server at this point.

View File

@ -107,6 +107,9 @@ const DEFAULTS = {
token: '',
label: '',
codeIssuedAt: 0,
bindMode: 'local',
publicHost: '',
allowlist: [],
bindAddress: '127.0.0.1'
}
},
@ -181,12 +184,87 @@ class ConfigStore {
? app.getPath('userData')
: path.join(__dirname, '..');
this.filePath = path.join(dir, 'electron-config.json');
this.historyPath = path.join(dir, 'electron-history.json');
this._writeQueue = Promise.resolve(); // Serializes all writes to prevent race conditions
this._historyWriteQueue = Promise.resolve();
this._historyMigrated = false;
this._cache = null;
this._cacheKey = '';
this._perfLog = null;
this._wqDepth = 0;
// Migrate config from old location if current doesn't exist
if (!fs.existsSync(this.filePath) && app && app.isPackaged) {
this._migrateFromOldPath(app);
}
if (app && app.isPackaged) {
this._migrateHistory();
}
}
_readHistoryFile() {
try {
const raw = fs.readFileSync(this.historyPath, 'utf-8');
if (!raw || raw.trim().length < 2) return [];
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) return parsed;
if (parsed && Array.isArray(parsed.history)) return parsed.history;
return [];
} catch {
return null;
}
}
_writeHistoryFileDurable(arr) {
const tmp = this.historyPath + '.tmp';
const fd = fs.openSync(tmp, 'w');
try {
fs.writeSync(fd, JSON.stringify(arr));
fs.fsyncSync(fd);
} finally {
fs.closeSync(fd);
}
fs.renameSync(tmp, this.historyPath);
}
_writeHistoryFileAtomic(arr) {
return new Promise((resolve, reject) => {
const tmp = this.historyPath + '.tmp';
fs.writeFile(tmp, JSON.stringify(arr), 'utf-8', (err) => {
if (err) return reject(err);
try { fs.renameSync(tmp, this.historyPath); } catch (e) { return reject(e); }
resolve();
});
});
}
_enqueueHistoryWrite(fn) {
this._historyWriteQueue = this._historyWriteQueue.then(fn, fn);
return this._historyWriteQueue;
}
_migrateHistory() {
try {
if (fs.existsSync(this.historyPath)) {
this._historyMigrated = Array.isArray(this._readHistoryFile());
return;
}
let cfg = null;
try { cfg = this._readAndParse(this.filePath); } catch {}
const hist = (cfg && Array.isArray(cfg.history)) ? cfg.history : [];
this._writeHistoryFileDurable(hist);
const check = this._readHistoryFile();
if (Array.isArray(check) && check.length === hist.length) {
if (hist.length > 0) {
try { fs.copyFileSync(this.filePath, this.filePath + '.pre-history-split.bak'); } catch {}
}
this._historyMigrated = true;
} else {
this._historyMigrated = false;
}
} catch {
this._historyMigrated = false;
}
}
_migrateFromOldPath(app) {
@ -218,15 +296,71 @@ class ConfigStore {
return JSON.parse(raw);
}
_clone(obj) {
try { return structuredClone(obj); }
catch { return JSON.parse(JSON.stringify(obj)); }
}
setPerfLog(fn) { this._perfLog = typeof fn === 'function' ? fn : null; }
_pqLen(globalSettings) {
const pq = globalSettings && globalSettings.pendingQueue;
return pq && Array.isArray(pq.queueJobs) ? pq.queueJobs.length : 0;
}
_callerTag() {
const lines = (new Error().stack || '').split('\n');
const out = [];
for (let i = 2; i < lines.length && out.length < 3; i++) {
const line = lines[i].trim();
if (/config-store\.js/.test(line)) continue;
const m = line.match(/at (?:async )?([^ (]+)/);
if (m) out.push(m[1].split('.').pop());
}
return out.join('<') || '?';
}
load() {
if (!this._perfLog) return this._loadImpl();
const hadCache = !!this._cache;
const t0 = performance.now();
const r = this._loadImpl();
const dt = performance.now() - t0;
if (dt >= 20) {
const q = this._pqLen(r && r.globalSettings);
const h = (r && r.history || []).length;
this._perfLog(`config-load wall=${dt.toFixed(0)}ms cache=${hadCache ? 'hit' : 'miss'} hist=${h} queue=${q} via=${this._callerTag()}`);
}
return r;
}
_loadImpl() {
try {
// In-memory cache keyed on the file's mtime+size. The processed config
// (merged + credential-decrypted) is reparsed/re-decrypted from disk ONLY
// when the file actually changes. Our own writes refresh the cache (see
// _commit), and an external edit changes mtime/size so the cache misses
// and we reread. Without this, every one of the ~38 main.js load() call
// sites (incl. the per-500ms log-flush path) re-read disk + JSON.parse the
// whole growing history + DPAPI-decrypt every credential — the dominant
// long-running main-thread drag. load() always returns a CLONE so callers
// can mutate the result without corrupting the cache.
let stat = null;
try { stat = fs.statSync(this.filePath); } catch {}
const statKey = stat ? `${stat.mtimeMs}:${stat.size}` : '';
if (stat && this._cache && this._cacheKey === statKey) {
return this._clone(this._cache);
}
let data = null;
// Try main config
try { data = this._readAndParse(this.filePath); } catch {}
// Fallback to backup if main is empty/corrupt
if (!data) {
const backupPath = this.filePath + '.bak';
try { data = this._readAndParse(backupPath); } catch {}
try { data = this._readAndParse(this.filePath + '.bak'); } catch {}
}
if (!data) {
try { data = this._readAndParse(this.filePath + '.pre-history-split.bak'); } catch {}
}
if (!data) {
const fresh = JSON.parse(JSON.stringify(DEFAULTS));
@ -302,11 +436,15 @@ class ConfigStore {
const rotationCursors = (data.rotationCursors && typeof data.rotationCursors === 'object' && !Array.isArray(data.rotationCursors))
? data.rotationCursors
: {};
const result = { hosters, hosterSettings, globalSettings, history: data.history || [], rotationCursors };
const result = { hosters, hosterSettings, globalSettings, history: this._historyMigrated ? [] : (data.history || []), rotationCursors };
// Decrypt credentials stored with safeStorage so the rest of the app
// keeps working with plaintext in memory.
secretStore.decryptCredentials(result);
return result;
if (stat) {
this._cache = result;
this._cacheKey = statKey;
}
return this._clone(result);
} catch {
const fresh = JSON.parse(JSON.stringify(DEFAULTS));
fresh.globalSettings.logMode = normalizeLogMode(fresh.globalSettings);
@ -314,30 +452,80 @@ class ConfigStore {
}
}
// Deep-clone a config and encrypt its credential fields. Never mutate the
// caller's object — the rest of the app holds plaintext references.
// Encrypt credential fields without mutating the caller's plaintext object.
// Only `hosters` carries credentials, so we clone ONLY that subtree — the rest
// (history, globalSettings, …) is referenced read-only into the stringified
// object. Deep-cloning the whole config here (incl. an ever-growing history)
// on every write was a primary long-running main-thread stall.
_serializeForDisk(config) {
const clone = JSON.parse(JSON.stringify(config));
secretStore.encryptCredentials(clone);
return JSON.stringify(clone, null, 2);
const hosters = this._clone(config.hosters || {});
secretStore.encryptCredentials({ hosters });
return JSON.stringify({ ...config, hosters }, null, 2);
}
_commit(config) {
if (!this._perfLog) return this._atomicWrite(this._serializeForDisk(config));
const t0 = performance.now();
const data = this._serializeForDisk(config);
const dt = performance.now() - t0;
if (dt >= 20) {
const q = this._pqLen(config.globalSettings);
const h = (config.history || []).length;
this._perfLog(`config-serialize wall=${dt.toFixed(0)}ms bytes=${data.length} hist=${h} queue=${q} wqDepth=${this._wqDepth} via=${this._callerTag()}`);
}
return this._atomicWrite(data);
}
_enqueueWrite(fn) {
this._writeQueue = this._writeQueue.then(fn, fn);
this._wqDepth++;
const done = () => { this._wqDepth--; };
this._writeQueue = this._writeQueue.then(fn, fn).then(done, done);
return this._writeQueue;
}
_anyHosters(cfg) {
const h = cfg && cfg.hosters;
return !!h && typeof h === 'object' && Object.values(h).some(a => Array.isArray(a) && a.length > 0);
}
_recoverHostersFromDisk() {
for (const p of [this.filePath, this.filePath + '.bak', this.filePath + '.pre-history-split.bak']) {
try {
const raw = fs.readFileSync(p, 'utf-8');
if (!raw || raw.trim().length < 2) continue;
const data = JSON.parse(raw);
if (this._anyHosters(data)) return data.hosters;
} catch {}
}
return null;
}
_guardHosters(current, hostersIntentional) {
if (!hostersIntentional && !this._anyHosters(current)) {
const recovered = this._recoverHostersFromDisk();
if (recovered) {
current.hosters = recovered;
if (this._perfLog) this._perfLog('config-guard: prevented account wipe — restored hosters from on-disk backup after a corrupt/empty read');
}
}
return current;
}
save(config) {
return this._enqueueWrite(() => {
const current = this.load();
if (config.hosters) current.hosters = config.hosters;
if (config.hosterSettings) current.hosterSettings = config.hosterSettings;
if (config.globalSettings) current.globalSettings = config.globalSettings;
return this._atomicWrite(this._serializeForDisk(current));
this._guardHosters(current, !!config.hosters);
return this._commit(current);
});
}
loadHistory() {
if (this._historyMigrated) {
return this._readHistoryFile() || [];
}
const config = this.load();
return config.history || [];
}
@ -346,45 +534,78 @@ class ConfigStore {
return new Promise((resolve, reject) => {
const tmpPath = this.filePath + '.tmp';
const backupPath = this.filePath + '.bak';
fs.writeFile(tmpPath, data, 'utf-8', (err) => {
if (err) return reject(err);
let fd;
try {
fd = fs.openSync(tmpPath, 'w');
fs.writeSync(fd, data);
fs.fsyncSync(fd);
} catch (e) {
try { if (fd !== undefined) fs.closeSync(fd); } catch {}
return reject(e);
}
try { fs.closeSync(fd); } catch {}
Promise.resolve().then(() => {
try {
// Refresh .bak from the previous live file. Wrapped in try/catch
// so an AV/indexer briefly locking the file doesn't fail the whole
// save — the rename to the live path is the part that matters,
// a stale .bak is preferable to losing the new write entirely.
try {
if (fs.existsSync(this.filePath)) {
const existing = fs.readFileSync(this.filePath, 'utf-8');
if (existing && existing.trim().length > 2) {
let isValid = false;
try {
const parsed = JSON.parse(existing);
isValid = parsed && typeof parsed === 'object' && (parsed.hosters || parsed.hosterSettings || parsed.globalSettings);
} catch {}
if (isValid) fs.writeFileSync(backupPath, existing, 'utf-8');
}
const cur = fs.readFileSync(this.filePath, 'utf-8');
if (cur && cur.trim().length > 2) fs.writeFileSync(backupPath, cur, 'utf-8');
}
} catch {}
fs.renameSync(tmpPath, this.filePath);
} catch (e) { return reject(e); }
// Invalidate the read cache: the next load() re-reads + re-merges the
// freshly-written file (the on-disk format is sparse — load() fills
// defaults — so we must NOT serve a pre-merge in-memory object).
this._cache = null;
this._cacheKey = '';
resolve();
});
});
}
appendHistory(entry) {
if (this._historyMigrated) {
return this._enqueueHistoryWrite(() => {
const cur = this._readHistoryFile();
if (cur === null && fs.existsSync(this.historyPath)) return;
const arr = cur || [];
arr.push(entry);
const gs = this.load().globalSettings;
const retention = (gs && gs.historyRetention) || 'all';
const pruned = applyHistoryRetention(arr, retention, Date.now());
return this._writeHistoryFileAtomic(pruned);
});
}
return this._enqueueWrite(() => {
const config = this.load();
config.history.push(entry);
const retention = (config.globalSettings && config.globalSettings.historyRetention) || 'all';
config.history = applyHistoryRetention(config.history, retention, Date.now());
return this._atomicWrite(this._serializeForDisk(config));
return this._commit(config);
});
}
pruneHistory(retention, opts = {}) {
const dryRun = !!opts.dryRun;
if (this._historyMigrated) {
return this._enqueueHistoryWrite(() => {
const current = this._readHistoryFile() || [];
const beforeBatches = current.length;
const beforeRows = countHistoryRows(current);
const pruned = applyHistoryRetention(current, retention, Date.now());
const result = {
removedBatches: beforeBatches - pruned.length,
removedRows: beforeRows - countHistoryRows(pruned),
keptBatches: pruned.length,
keptRows: countHistoryRows(pruned)
};
if (dryRun) return result;
return this._writeHistoryFileAtomic(pruned)
.then(() => this.save({ globalSettings: { ...this.load().globalSettings, historyRetention: String(retention || 'all') } }))
.then(() => result);
});
}
return this._enqueueWrite(() => {
const config = this.load();
const beforeBatches = config.history.length;
@ -399,15 +620,18 @@ class ConfigStore {
if (dryRun) return result;
config.history = pruned;
if (config.globalSettings) config.globalSettings.historyRetention = String(retention || 'all');
return this._atomicWrite(this._serializeForDisk(config)).then(() => result);
return this._commit(config).then(() => result);
});
}
clearHistory() {
if (this._historyMigrated) {
return this._enqueueHistoryWrite(() => this._writeHistoryFileAtomic([]));
}
return this._enqueueWrite(() => {
const config = this.load();
config.history = [];
return this._atomicWrite(this._serializeForDisk(config));
return this._commit(config);
});
}
@ -415,7 +639,8 @@ class ConfigStore {
return this._enqueueWrite(() => {
const config = this.load();
config.rotationCursors = (cursors && typeof cursors === 'object' && !Array.isArray(cursors)) ? cursors : {};
return this._atomicWrite(this._serializeForDisk(config));
this._guardHosters(config, false);
return this._commit(config);
});
}
}

View File

@ -15,7 +15,7 @@ function createAgent(collectors) {
};
function handle(op, args) {
const fn = OPS[op];
const fn = (typeof op === 'string' && Object.prototype.hasOwnProperty.call(OPS, op)) ? OPS[op] : null;
if (typeof fn !== 'function') return { ok: false, error: `unknown or non-readonly op: ${op}` };
try {
const data = fn(args || {});

View File

@ -11,7 +11,7 @@ const READABLE_LOGS = {
const QUEUE_STATUSES = ['preview', 'queued', 'getting-server', 'uploading', 'retrying', 'done', 'error', 'aborted', 'skipped'];
function createCollectors(deps) {
const { loadConfig, getAllLogPaths, support, stats, appInfo, systemInfo, agentInfo } = deps;
const { loadConfig, loadHistory, getAllLogPaths, support, stats, appInfo, systemInfo, agentInfo } = deps;
function _secrets() {
try { return support.collectSecretValues(loadConfig()); } catch { return []; }
@ -104,10 +104,12 @@ function createCollectors(deps) {
let content = support.redactLogText(raw, _secrets());
let matchedLines;
if (a.grep && typeof a.grep === 'string' && a.grep.length <= 200) {
let re;
try { re = new RegExp(a.grep, 'i'); } catch { re = null; }
if (re) {
const lines = content.split('\n').filter(l => re.test(l));
const terms = a.grep.split('|').map(s => s.trim().toLowerCase()).filter(Boolean);
if (terms.length) {
const lines = content.split('\n').filter(l => {
const low = l.toLowerCase();
return terms.some(t => low.includes(t));
});
matchedLines = lines.length;
content = lines.join('\n');
}
@ -203,8 +205,9 @@ function createCollectors(deps) {
function getHistory(args) {
const a = args || {};
const cfg = loadConfig();
const history = Array.isArray(cfg.history) ? cfg.history : [];
const history = typeof loadHistory === 'function'
? (loadHistory() || [])
: (Array.isArray(loadConfig().history) ? loadConfig().history : []);
const limit = Math.min(Math.max(Number(a.limit) || 20, 1), 200);
const perHoster = stats.summarizePerHoster(history);
const recent = [...history].slice(-limit).reverse();

View File

@ -27,7 +27,11 @@ function _doodstreamLogPath() {
return path.join(__dirname, '..', 'doodstream-debug.log');
}
let _debugVerbose = false;
function setDebugVerbose(v) { _debugVerbose = !!v; }
function _debugLog(msg) {
if (!_debugVerbose) return;
try {
const logPath = _doodstreamLogPath();
maybeRotateLogFile(logPath, _DOODSTREAM_LOG_MAX_BYTES, _DOODSTREAM_LOG_MAX_BACKUPS);
@ -335,7 +339,7 @@ class DoodstreamUploader {
const epilogueBuf = Buffer.from(epilogue, 'utf-8');
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
const CHUNK_SIZE = 256 * 1024;
const CHUNK_SIZE = 1024 * 1024;
let bytesRead = 0;
async function* generate() {
@ -698,3 +702,4 @@ class DoodstreamUploader {
}
module.exports = DoodstreamUploader;
module.exports.setDebugVerbose = setDebugVerbose;

View File

@ -288,7 +288,7 @@ function createUploadBody(filePath, formFields, onProgress, throttle, signal) {
const { boundary, preambleBuf, epilogueBuf, totalSize, fileSize } = buildMultipart(filePath, formFields);
let bytesRead = 0;
const CHUNK_SIZE = 256 * 1024;
const CHUNK_SIZE = 1024 * 1024;
async function* generate() {
yield preambleBuf;

50
lib/ip-allowlist.js Normal file
View File

@ -0,0 +1,50 @@
function normalizeIp(ip) {
return String(ip || '').trim().replace(/^::ffff:/i, '').toLowerCase();
}
function isLoopbackIp(ip) {
const c = normalizeIp(ip);
return c === '' || c === '::1' || c === 'localhost' || /^127\./.test(c);
}
function ipv4ToInt(ip) {
const parts = String(ip).split('.');
if (parts.length !== 4) return null;
let n = 0;
for (const p of parts) {
if (!/^\d{1,3}$/.test(p)) return null;
const v = Number(p);
if (v < 0 || v > 255) return null;
n = (n << 8) + v;
}
return n >>> 0;
}
function matchIpRule(clientIp, rule) {
const client = normalizeIp(clientIp);
const r = String(rule || '').trim().toLowerCase();
if (!r) return false;
if (r === '*' || r === '0.0.0.0/0') return true;
if (r === client) return true;
const slash = r.indexOf('/');
if (slash > 0) {
const baseInt = ipv4ToInt(r.slice(0, slash));
const clientInt = ipv4ToInt(client);
const bits = Number(r.slice(slash + 1));
if (baseInt === null || clientInt === null || !Number.isInteger(bits) || bits < 0 || bits > 32) return false;
if (bits === 0) return true;
const mask = bits === 32 ? 0xffffffff : (~((1 << (32 - bits)) - 1)) >>> 0;
return (clientInt & mask) === (baseInt & mask);
}
return false;
}
function evaluateClientAllowed(clientIp, rules) {
const client = normalizeIp(clientIp);
if (isLoopbackIp(client)) return true;
const list = Array.isArray(rules) ? rules : [];
if (list.length === 0) return false;
return list.some((rule) => matchIpRule(client, rule));
}
module.exports = { normalizeIp, isLoopbackIp, ipv4ToInt, matchIpRule, evaluateClientAllowed };

View File

@ -1,7 +1,7 @@
// Log-file mode resolution for fileuploader.log:
// - "single" → one file: fileuploader.log
// - "daily" → per-day: fileuploader-YYYY-MM-DD.log
// - "session" → per-launch: fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log
// - "session" → per-launch: DD-MM-YYYY-mdu-session-HH-MM-NNNNNN.log
//
// Pure functions only — no fs, no Date.now() at call time — so they unit-test
// cleanly and the main.js call sites pass in `new Date()` + the session stamp.
@ -38,13 +38,11 @@
return `${date.getFullYear()}-${_two(date.getMonth() + 1)}-${_two(date.getDate())}`;
}
function formatSessionStamp(date, pid) {
const d = `${date.getFullYear()}-${_two(date.getMonth() + 1)}-${_two(date.getDate())}`;
const t = `${_two(date.getHours())}-${_two(date.getMinutes())}-${_two(date.getSeconds())}`;
// PID disambiguates a same-second close→reopen — a human can't but two
// automated runs might. Cheap belt to a suspenders-not-required problem.
const pidStr = pid !== undefined && pid !== null ? `-${pid}` : '';
return `${d}_${t}${pidStr}`;
function formatSessionStamp(date, rand) {
const d = `${_two(date.getDate())}-${_two(date.getMonth() + 1)}-${date.getFullYear()}`;
const t = `${_two(date.getHours())}-${_two(date.getMinutes())}`;
const r = (rand !== undefined && rand !== null && String(rand).trim()) ? `-${String(rand).trim()}` : '';
return `${d}-mdu-session-${t}${r}`;
}
/**
@ -67,9 +65,10 @@
const date = a.date instanceof Date ? a.date : new Date();
return `${base}-${formatDateStamp(date)}${ext}`;
}
// session
// session — the stamp is the full app-defined stem (DD-MM-YYYY-mdu-session-HH-MM),
// independent of baseName.
const sid = a.sessionId && String(a.sessionId).trim();
if (sid) return `${base}-session-${sid}${ext}`;
if (sid) return `${sid}${ext}`;
// Defensive: if a session-id wasn't passed, fall back to single rather
// than emit a malformed name. main.js always supplies one.
return `${base}${ext}`;
@ -85,6 +84,9 @@
*/
function stripModeStampFromFileName(fileName) {
if (!fileName || typeof fileName !== 'string') return fileName;
const newSessionRe = /^\d{2}-\d{2}-\d{4}-mdu-session-\d{2}-\d{2}(?:-\d+)?(\.[^.]+)?$/;
const mNew = fileName.match(newSessionRe);
if (mNew) return `fileuploader${mNew[1] || ''}`;
// Order matters: session first (longer, more specific) before daily.
// Both regexes are anchored to $ with no nested/ambiguous quantifiers, so
// matching is linear — the eslint security warning is precautionary.

View File

@ -1,5 +1,6 @@
const { WebSocketServer } = require('ws');
const crypto = require('crypto');
const { evaluateClientAllowed } = require('./ip-allowlist');
function timingSafeEqualStr(a, b) {
const x = Buffer.from(String(a == null ? '' : a));
@ -20,7 +21,7 @@ class RemoteServer {
return new Promise((resolve, reject) => {
this._config = opts;
const wssOpts = { port: opts.port };
const wssOpts = { port: opts.port, maxPayload: 256 * 1024 };
if (opts.host) wssOpts.host = opts.host;
this._wss = new WebSocketServer(wssOpts, () => {
resolve();
@ -70,6 +71,11 @@ class RemoteServer {
return;
}
if (Array.isArray(this._config.allowlist) && !evaluateClientAllowed(ip, this._config.allowlist)) {
ws.close(4005, 'Client IP not allowed');
return;
}
const clientId = crypto.randomUUID();
this._clients.set(ws, { id: clientId, role: null, authenticated: false });
@ -169,7 +175,9 @@ class RemoteServer {
sendToClient(clientId, data) {
for (const [ws, client] of this._clients) {
if (client.id === clientId && client.authenticated) {
ws.send(JSON.stringify(data));
if (ws.readyState === 1) {
try { ws.send(JSON.stringify(data)); } catch {}
}
break;
}
}

View File

@ -43,10 +43,12 @@ function redactLogText(text, secrets) {
}
out = out
.replace(/https?:\/\/(?:ptb\.|canary\.)?discord(?:app)?\.com\/api\/webhooks\/\d+\/[\w-]+/gi, 'https://discord.com/api/webhooks/' + REDACTED)
.replace(/(authorization:\s*bearer\s+)\S+/gi, '$1' + REDACTED)
.replace(/(\/\/[^\s/:@]+:)[^\s/@]+(@)/g, '$1' + REDACTED + '$2')
.replace(/(authorization:\s*(?:bearer|basic)\s+)\S+/gi, '$1' + REDACTED)
.replace(/\bbearer\s+[A-Za-z0-9._\-/+]{16,}/gi, 'bearer ' + REDACTED)
.replace(/\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{6,}/g, REDACTED)
.replace(/([?&](?:api_?key|key|token|access_token|password|pass)=)[^\s&"'`]+/gi, '$1' + REDACTED)
.replace(/("?\b(?:api[_-]?key|apikey|password|passwd|secret|(?:access|refresh|auth|session)[_-]?token|token|sessionid)"?\s*[:=]\s*"?)[A-Za-z0-9._\-/+]{8,}/gi, '$1' + REDACTED)
.replace(/("?\b(?:api[_-]?key|apikey|password|passwd|secret|(?:access|refresh|auth|session)[_-]?token|token|sessionid|session)"?\s*[:=]\s*"?)[A-Za-z0-9._\-/+]{8,}/gi, '$1' + REDACTED)
.replace(/(\bset-cookie:|\bcookie:)\s*\S[^\n]*/gi, '$1 ' + REDACTED)
.replace(/(\bsess(?:_?id)?\b["'=:\s]+)[A-Za-z0-9._\-]{8,}/gi, '$1' + REDACTED);
return out;

View File

@ -179,10 +179,27 @@ async function installUpdate(onProgress) {
let downloadedBytes = 0;
const chunks = [];
const DOWNLOAD_STALL_MS = 45000;
let stallTimer = null;
const reader = res.body.getReader();
while (true) {
if (signal.aborted) throw new Error('Abgebrochen');
const { done, value } = await reader.read();
let chunk;
try {
chunk = await Promise.race([
reader.read(),
new Promise((_, reject) => { stallTimer = setTimeout(() => reject(new Error('__STALL__')), DOWNLOAD_STALL_MS); })
]);
} catch (e) {
if (e && e.message === '__STALL__') {
try { activeAbort.abort(); } catch {}
throw new Error('Download hängt — seit 45 s keine Daten (Netzwerk/Server überlastet). Bitte laufende Uploads stoppen und erneut versuchen.');
}
throw e;
} finally {
if (stallTimer) { clearTimeout(stallTimer); stallTimer = null; }
}
const { done, value } = chunk;
if (done) break;
chunks.push(value);
downloadedBytes += value.length;

View File

@ -34,10 +34,11 @@ class UploadManager extends EventEmitter {
this.stopAfterActive = false;
this.statsInterval = null;
this.startTime = 0;
this.activeJobs = new Map(); // uploadId -> { jobId, speedKbs, bytesUploaded }
this.activeJobs = new Map(); // uploadId -> { jobId, speedKbs, bytesUploaded, hoster }
this.jobAbortControllers = new Map(); // jobId -> AbortController
this.cancelledJobIds = new Set();
this.sessionBytes = 0;
this._transientErrorTotal = 0;
this.lastStartTime = {}; // hoster -> timestamp of last upload start
this.intervalLocks = {}; // hoster -> Promise chain for serialized interval waits
this.globalThrottle = null;
@ -81,6 +82,17 @@ class UploadManager extends EventEmitter {
return this.activeJobs.size;
}
getDiagnostics() {
const activeByHoster = {};
for (const v of this.activeJobs.values()) {
const h = v && v.hoster ? v.hoster : 'unknown';
activeByHoster[h] = (activeByHoster[h] || 0) + 1;
}
let pending = 0;
for (const sem of Object.values(this.semaphores)) pending += (sem && sem.pending) || 0;
return { activeByHoster, transientErrors: this._transientErrorTotal, pending, active: this.activeJobs.size };
}
clearFailedAccount(hoster, accountId) {
return this._failedAccounts.delete(`${hoster}:${accountId}`);
}
@ -352,16 +364,17 @@ class UploadManager extends EventEmitter {
for (let i = 0; i < tasks.length; i += DEDUP_CHUNK) {
if (signal.aborted) break;
const end = Math.min(i + DEDUP_CHUNK, tasks.length);
const toStat = [];
for (let j = i; j < end; j++) {
const task = tasks[j];
if (!results.has(task.file)) {
const fileName = path.basename(task.file);
let size = 0;
try { size = fs.statSync(task.file).size; } catch {}
results.set(task.file, { name: fileName, size, results: [] });
results.set(task.file, { name: path.basename(task.file), size: 0, results: [] });
toStat.push(task.file);
}
}
if (end < tasks.length) await new Promise(setImmediate);
await Promise.all(toStat.map(async (f) => {
try { const st = await fs.promises.stat(f); const e = results.get(f); if (e) e.size = st.size; } catch {}
}));
}
this._startStatsTimer();
@ -413,7 +426,7 @@ class UploadManager extends EventEmitter {
if (cachedResult && typeof cachedResult.size === 'number' && cachedResult.size > 0) {
fileSize = cachedResult.size;
} else {
try { fileSize = fs.statSync(task.file).size; } catch { fileNotFound = true; }
try { fileSize = (await fs.promises.stat(task.file)).size; } catch { fileNotFound = true; }
}
const maxAttempts = Math.max(1, (settings.retries || 0) + 1);
@ -625,7 +638,7 @@ class UploadManager extends EventEmitter {
// Mutate this single object on each progress callback instead of
// allocating a fresh one — callback fires on every stream chunk
// (hundreds/sec per active job).
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0 };
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
this.activeJobs.set(uploadId, activeEntry);
let lastEmitTime = 0;
@ -686,6 +699,7 @@ class UploadManager extends EventEmitter {
return;
} catch (err) {
this.activeJobs.delete(uploadId);
if (this._isTransientNetworkError(err)) this._transientErrorTotal++;
const isSpeedRestart = speedAbort && speedAbort.signal.aborted && !signal.aborted;
if (!signal.aborted && !isSpeedRestart) {
@ -938,9 +952,11 @@ class UploadManager extends EventEmitter {
let lastBytes = 0;
let lastSpeedTime = jobStart;
let currentSpeedKbs = 0;
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0 };
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
this.activeJobs.set(uploadId, activeEntry);
let lastEmitTime = 0;
const PROGRESS_EMIT_INTERVAL = 250;
const progressCb = (bytesUploaded, bytesTotal) => {
const now = Date.now();
const timeDelta = (now - lastSpeedTime) / 1000;
@ -951,6 +967,8 @@ class UploadManager extends EventEmitter {
}
activeEntry.speedKbs = currentSpeedKbs;
activeEntry.bytesUploaded = bytesUploaded;
if (now - lastEmitTime < PROGRESS_EMIT_INTERVAL) return;
lastEmitTime = now;
const elapsed = Math.round((now - jobStart) / 1000);
const remaining = currentSpeedKbs > 0 ? Math.round((bytesTotal - bytesUploaded) / (currentSpeedKbs * 1024)) : 0;
this._emitProgress(uploadId, fileName, task.hoster, { accountId: task.accountId,
@ -1070,8 +1088,10 @@ class UploadManager extends EventEmitter {
let lastBytes = 0;
let lastSpeedTime = jobStart;
let currentSpeedKbs = 0;
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0 };
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
this.activeJobs.set(uploadId, activeEntry);
let lastEmitTime = 0;
const PROGRESS_EMIT_INTERVAL = 250;
const progressCb = (bytesUploaded, bytesTotal) => {
const now = Date.now();
const timeDelta = (now - lastSpeedTime) / 1000;
@ -1082,6 +1102,8 @@ class UploadManager extends EventEmitter {
}
activeEntry.speedKbs = currentSpeedKbs;
activeEntry.bytesUploaded = bytesUploaded;
if (now - lastEmitTime < PROGRESS_EMIT_INTERVAL) return;
lastEmitTime = now;
const elapsed = Math.round((now - jobStart) / 1000);
const remaining = currentSpeedKbs > 0 ? Math.round((bytesTotal - bytesUploaded) / (currentSpeedKbs * 1024)) : 0;
this._emitProgress(uploadId, fileName, task.hoster, { accountId: task.accountId,

View File

@ -187,7 +187,7 @@ class VidmolyUploader {
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
let bytesRead = 0;
const CHUNK_SIZE = 256 * 1024;
const CHUNK_SIZE = 1024 * 1024;
async function* generate() {
yield preambleBuf;

View File

@ -242,7 +242,7 @@ class VoeUploader {
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
let bytesRead = 0;
const CHUNK_SIZE = 256 * 1024;
const CHUNK_SIZE = 1024 * 1024;
async function* generate() {
yield preambleBuf;

266
main.js
View File

@ -1,4 +1,6 @@
const { app, BrowserWindow, ipcMain, dialog, clipboard, nativeTheme, Tray, Menu } = require('electron');
process.env.UV_THREADPOOL_SIZE = process.env.UV_THREADPOOL_SIZE || '8';
const { monitorEventLoopDelay, PerformanceObserver } = require('perf_hooks');
const { app, BrowserWindow, ipcMain, dialog, clipboard, nativeTheme, Tray, Menu, nativeImage } = require('electron');
nativeTheme.themeSource = 'dark';
const path = require('path');
const fs = require('fs');
@ -25,11 +27,82 @@ const stats = require('./lib/stats');
const { createCollectors } = require('./lib/diagnostics-collectors');
const { createAgent } = require('./lib/diagnostics-agent');
function _gpuDisableFlagPath() {
try { return path.join(app.getPath('userData'), 'gpu-disabled.flag'); } catch { return null; }
}
(function maybeDisableHardwareAcceleration() {
let disable = false;
try { if (/^RDP/i.test(process.env.SESSIONNAME || '')) disable = true; } catch {}
if (!disable) { try { const f = _gpuDisableFlagPath(); if (f && fs.existsSync(f)) disable = true; } catch {} }
if (disable) { try { app.disableHardwareAcceleration(); } catch {} }
})();
const _eventLoopDelay = monitorEventLoopDelay({ resolution: 10 });
_eventLoopDelay.enable();
let _eldLastLog = 0;
let _lastCpu = process.cpuUsage();
let _lastCpuT = Date.now();
let _gcCount = 0;
let _gcTotalMs = 0;
let _gcMaxMs = 0;
try {
const _gcObserver = new PerformanceObserver((list) => {
for (const entry of list.getEntries()) {
_gcCount++;
_gcTotalMs += entry.duration;
if (entry.duration > _gcMaxMs) _gcMaxMs = entry.duration;
}
});
_gcObserver.observe({ entryTypes: ['gc'] });
} catch {}
const _perfOn = process.env.MHU_PERF !== '0';
let _lastIpcChannel = '';
if (_perfOn) {
let _driftTick = Date.now();
setInterval(() => {
const now = Date.now();
const drift = now - _driftTick - 100;
_driftTick = now;
if (drift >= 100) {
try { logInfo(`main-longtask blocked=${drift}ms lastIpc=${_lastIpcChannel || '-'} gc=${_gcCount} gcMax=${_gcMaxMs.toFixed(0)}ms`); } catch {}
}
}, 100).unref();
const IPC_SLOW_MS = 50;
const _ipcLog = (m) => { try { logInfo(m); } catch {} };
const _rawHandle = ipcMain.handle.bind(ipcMain);
ipcMain.handle = (channel, fn) => _rawHandle(channel, function (evt, ...args) {
_lastIpcChannel = channel;
const t0 = performance.now();
let p;
try { p = fn.call(this, evt, ...args); }
catch (e) { _ipcLog(`ipc ${channel} sync-throw wall=${(performance.now() - t0).toFixed(0)}ms`); throw e; }
const sync = performance.now() - t0;
if (p && typeof p.then === 'function') {
return Promise.resolve(p).finally(() => {
const total = performance.now() - t0;
if (total >= IPC_SLOW_MS) _ipcLog(`ipc ${channel} wall=${total.toFixed(0)}ms sync=${sync.toFixed(0)}ms`);
});
}
if (sync >= IPC_SLOW_MS) _ipcLog(`ipc ${channel} wall=${sync.toFixed(0)}ms sync`);
return p;
});
const _rawOn = ipcMain.on.bind(ipcMain);
ipcMain.on = (channel, fn) => _rawOn(channel, function (evt, ...args) {
_lastIpcChannel = channel;
const t0 = performance.now();
try { return fn.call(this, evt, ...args); }
finally { const dt = performance.now() - t0; if (dt >= IPC_SLOW_MS) _ipcLog(`ipc ${channel} wall=${dt.toFixed(0)}ms sync-on`); }
});
}
let mainWindow;
let _lastImportPath = null;
let dropTargetWindow = null;
let tray = null;
const configStore = new ConfigStore(app);
configStore.setPerfLog((m) => { try { logInfo(m); } catch {} });
let uploadManager = null;
let diagnosticAgent = null;
let _diagHandler = null;
@ -139,7 +212,7 @@ function debugLog(msg) {
}
let _logVerbose = false;
function setLogVerbose(v) { _logVerbose = !!v; }
function setLogVerbose(v) { _logVerbose = !!v; try { require('./lib/doodstream-upload').setDebugVerbose(_logVerbose); } catch {} }
function _ctxTag(ctx) {
if (!ctx || typeof ctx !== 'object') return '';
const tags = [];
@ -181,6 +254,56 @@ function logMarker(label, fields) {
debugLog(`────── ${label}${extra} ──────`);
}
function _maybeLogEventLoopDelay(activeJobs) {
const now = Date.now();
if (now - _eldLastLog < 5000) return;
_eldLastLog = now;
try {
const ns = 1e6;
const mean = (_eventLoopDelay.mean / ns).toFixed(1);
const max = (_eventLoopDelay.max / ns).toFixed(1);
const p99 = (_eventLoopDelay.percentile(99) / ns).toFixed(1);
const stddev = (_eventLoopDelay.stddev / ns).toFixed(1);
let resStr = '';
try {
const info = process.getActiveResourcesInfo();
const hist = {};
for (const t of info) hist[t] = (hist[t] || 0) + 1;
const top = Object.entries(hist).sort((a, b) => b[1] - a[1]).slice(0, 6).map(([k, v]) => `${k}:${v}`).join(',');
resStr = ` resources=${info.length} {${top}}`;
} catch {}
let cpuStr = '';
try {
const d = process.cpuUsage(_lastCpu);
const wall = now - _lastCpuT;
const pct = wall > 0 ? Math.round((d.user + d.system) / 1000 / wall * 100) : 0;
const mem = process.memoryUsage();
const rss = Math.round(mem.rss / 1048576);
const heap = Math.round(mem.heapUsed / 1048576);
const ext = Math.round(mem.external / 1048576);
const ab = Math.round((mem.arrayBuffers || 0) / 1048576);
cpuStr = ` cpu=${pct}%core rss=${rss}MB heap=${heap}MB ext=${ext}MB ab=${ab}MB`;
_lastCpu = process.cpuUsage();
_lastCpuT = now;
} catch {}
let gcStr = '';
try {
gcStr = ` gc=${_gcCount} gcTotal=${_gcTotalMs.toFixed(0)}ms gcMax=${_gcMaxMs.toFixed(0)}ms`;
_gcCount = 0; _gcTotalMs = 0; _gcMaxMs = 0;
} catch {}
let upStr = '';
try {
if (uploadManager && typeof uploadManager.getDiagnostics === 'function') {
const d = uploadManager.getDiagnostics();
const byHoster = Object.entries(d.activeByHoster || {}).map(([h, c]) => `${h.replace(/\..*$/, '')}:${c}`).join(',');
upStr = ` active-by-hoster={${byHoster}} transient-errs=${d.transientErrors || 0} pending=${d.pending || 0}`;
}
} catch {}
logInfo(`eventloop-delay active=${activeJobs} mean=${mean}ms p99=${p99}ms max=${max}ms stddev=${stddev}ms threadpool=${process.env.UV_THREADPOOL_SIZE}${cpuStr}${gcStr}${resStr}${upStr}`);
_eventLoopDelay.reset();
} catch {}
}
// Dedicated account-rotation log so users can trace fallback decisions
// without wading through general debug output. Writes to account-rotation.log
// in the same directory as fileuploader.log (honors user's configured path).
@ -414,27 +537,41 @@ function getDefaultLogFilePath() {
return path.join(__dirname, 'fileuploader.log');
}
// The log flush paths resolve the log file ~8x/second during uploads. Going
// through configStore.load() there meant re-reading + cloning the whole config
// (incl. an 8 MB+ history) on every flush — a major long-running main-thread
// drag. logFilePath/logMode change only when the user saves settings, so cache
// the two strings and invalidate on those saves (see _invalidateLogSettings).
let _cachedLogSettings = null;
function _getLogSettings() {
if (!_cachedLogSettings) {
const gs = (configStore.load() || {}).globalSettings || {};
_cachedLogSettings = {
logFilePath: String(gs.logFilePath || '').trim(),
logMode: gs.logMode || 'single'
};
}
return _cachedLogSettings;
}
function _invalidateLogSettings() { _cachedLogSettings = null; }
function getBaseLogFilePath() {
const config = configStore.load();
const customPath = config && config.globalSettings
? String(config.globalSettings.logFilePath || '').trim()
: '';
const customPath = _getLogSettings().logFilePath;
return customPath || getDefaultLogFilePath();
}
// Log-mode bookkeeping. Three modes (see lib/log-mode.js): single, daily, session.
// The session-id is stamped ONCE at main-process startup so every write of a
// given session lands in the same file. A close→reopen of the app starts a new
// main process, so a new SESSION_ID, so a new session file. PID is appended as
// a cheap hedge against same-second restart collisions.
// main process, so a new SESSION_ID, so a new session file. A 6-digit random is
// appended as a cheap hedge against same-minute restart collisions.
const { resolveLogFileName, formatSessionStamp, formatDateStamp, stripModeStampFromFileName } = require('./lib/log-mode');
const SESSION_ID = formatSessionStamp(new Date(), process.pid);
const SESSION_ID = formatSessionStamp(new Date(), String(Math.floor(100000 + Math.random() * 900000)));
let _activeLogKey = null; // remembers (mode + date-or-session) so cache rolls correctly
let _activeLogPath = null;
function getLogFilePath() {
const config = configStore.load();
const mode = (config && config.globalSettings && config.globalSettings.logMode) || 'single';
const mode = _getLogSettings().logMode;
const base = getBaseLogFilePath();
const dir = path.dirname(base);
const ext = path.extname(base);
@ -454,8 +591,7 @@ function getLogFilePath() {
function buildFallbackLogName(dir) {
// Match the active log-mode's naming so the fallback file is consistent with
// what the primary write would have produced.
const config = configStore.load();
const mode = (config && config.globalSettings && config.globalSettings.logMode) || 'single';
const mode = _getLogSettings().logMode;
return path.join(dir, resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode, date: new Date(), sessionId: SESSION_ID }));
}
@ -596,6 +732,7 @@ function _persistFallbackLogPath(workingPath) {
cfg.globalSettings = gs;
configStore.save({ globalSettings: gs }).catch(() => {});
_invalidateUploadLogTargetCache();
_invalidateLogSettings();
safeSend('log-path-auto-updated', { logFilePath: toSave });
} catch (err) {
debugLog(`persist fallback logpath failed: ${err.message}`);
@ -1151,14 +1288,29 @@ function createWindow() {
app.on('child-process-gone', (_event, details) => {
_writeCrashLog('CHILD PROCESS GONE', new Error(details.reason || 'unknown'), details);
debugLog(`CHILD PROCESS GONE: type=${details.type} reason=${details.reason} exitCode=${details.exitCode}`);
if (details && details.type === 'GPU') {
try { const f = _gpuDisableFlagPath(); if (f) fs.writeFileSync(f, new Date().toISOString(), 'utf-8'); } catch {}
}
});
mainWindow.loadFile(path.join(__dirname, 'renderer', 'index.html'));
}
function createTray() {
const iconPath = path.join(__dirname, 'assets', 'app_icon.ico');
tray = new Tray(iconPath);
try {
const candidates = [
path.join(process.resourcesPath || __dirname, 'assets', 'app_icon.ico'),
path.join(__dirname, 'assets', 'app_icon.ico'),
path.join(__dirname, 'assets', 'icon.png')
];
let icon = null;
for (const p of candidates) {
try {
const img = nativeImage.createFromPath(p);
if (img && !img.isEmpty()) { icon = img; break; }
} catch {}
}
tray = new Tray(icon || nativeImage.createEmpty());
tray.setToolTip('Multi-Hoster-Upload');
const contextMenu = Menu.buildFromTemplate([
@ -1171,6 +1323,10 @@ function createTray() {
tray.on('click', () => {
if (mainWindow) { mainWindow.show(); mainWindow.focus(); }
});
} catch (err) {
tray = null;
debugLog(`createTray failed (non-fatal): ${err && err.message ? err.message : err}`);
}
}
function updateTrayTooltip(text) {
@ -1315,6 +1471,7 @@ ipcMain.handle('get-config', () => {
ipcMain.handle('save-config', async (_event, config) => {
await configStore.save(config);
if (config && config.globalSettings) _invalidateLogSettings();
try {
if (config && config.globalSettings && Object.prototype.hasOwnProperty.call(config.globalSettings, 'logVerbose')) {
setLogVerbose(!!config.globalSettings.logVerbose);
@ -1690,6 +1847,7 @@ ipcMain.handle('start-upload', (_event, payload) => {
if (data.state === 'uploading' && data.activeJobs > 0) {
const speedMb = ((Number(data.globalSpeedKbs) || 0) / 1024).toFixed(1);
updateTrayTooltip(`Upload: ${data.activeJobs} aktiv - ${speedMb} MB/s`);
_maybeLogEventLoopDelay(data.activeJobs);
} else {
updateTrayTooltip('Multi-Hoster-Upload');
}
@ -2058,9 +2216,11 @@ ipcMain.handle('clear-history', async () => {
// --- Backup export / import ---
ipcMain.handle('export-backup', async () => {
const _bd = new Date();
const _bdate = `${String(_bd.getDate()).padStart(2, '0')}-${String(_bd.getMonth() + 1).padStart(2, '0')}-${_bd.getFullYear()}`;
const { canceled, filePath } = await dialog.showSaveDialog(mainWindow, {
title: 'Backup exportieren',
defaultPath: `multi-hoster-backup-${new Date().toISOString().slice(0, 10)}.mhu`,
defaultPath: `${_bdate}-multihoster-backup.mhu`,
filters: [
{ name: 'Multi-Hoster Backup (verschlüsselt)', extensions: ['mhu'] },
{ name: 'Multi-Hoster Backup (Klartext JSON)', extensions: ['json'] }
@ -2152,6 +2312,7 @@ ipcMain.handle('import-backup', async (_event, legacyPassword) => {
history: []
};
await configStore._atomicWrite(configStore._serializeForDisk(merged));
_invalidateLogSettings();
return { ok: true, config: configStore.load() };
});
@ -2228,6 +2389,7 @@ ipcMain.handle('app:check-updates', async () => {
});
ipcMain.handle('app:install-update', () => {
try { if (uploadManager) uploadManager.cancel(); } catch {}
installUpdate((progress) => {
safeSend('app:update-progress', progress);
}).catch((err) => {
@ -2286,6 +2448,7 @@ function _preserveDiagSubtree(globalSettings) {
ipcMain.handle('save-global-settings', async (_event, globalSettings) => {
globalSettings = _preserveDiagSubtree(globalSettings);
await configStore.save({ globalSettings });
_invalidateLogSettings();
if (uploadManager) uploadManager.updateSettings(null, globalSettings);
return true;
});
@ -2328,9 +2491,12 @@ ipcMain.on('save-global-settings-sync', (event, globalSettings) => {
const _diskDiag = current.globalSettings && current.globalSettings.diagnostics;
current.globalSettings = globalSettings;
if (_diskDiag) current.globalSettings.diagnostics = _diskDiag;
try { configStore._guardHosters(current, false); } catch {}
_invalidateLogSettings();
const data = configStore._serializeForDisk(current);
const backupPath = configStore.filePath + '.bak';
fs.writeFileSync(tmpPath, data, 'utf-8');
const _fd = fs.openSync(tmpPath, 'w');
try { fs.writeSync(_fd, data); fs.fsyncSync(_fd); } finally { fs.closeSync(_fd); }
if (fs.existsSync(configStore.filePath)) {
// Use try/catch around the read so an AV/lock race doesn't fail the
// whole save just because we couldn't refresh the .bak — the write to
@ -2459,7 +2625,7 @@ function _diagAgentInfo() {
return {
version: app.getVersion(),
port: diag.port || 9110,
bindAddress: diag.bindAddress || '127.0.0.1',
bindAddress: _diagBindHost(diag),
clientCount: diagnosticAgent ? diagnosticAgent.getClientCount() : 0,
lastAccess: diagnosticAgent ? diagnosticAgent.getLastAccess() : null
};
@ -2468,6 +2634,7 @@ function _diagAgentInfo() {
function _buildDiagnosticHandler() {
const collectors = createCollectors({
loadConfig: () => configStore.load(),
loadHistory: () => configStore.loadHistory(),
getAllLogPaths,
support: { sanitizeConfig, collectSecretValues, redactLogText, valueScrub, collectFile, REDACTED },
stats,
@ -2484,19 +2651,42 @@ function _buildDiagnosticHandler() {
};
}
function buildDiagnosticCode(diag, fp) {
function _getSuggestedRemoteHosts() {
const os = require('os');
const payload = { v: 1, port: diag.port || 9110, token: diag.token, label: diag.label || os.hostname() };
if (fp) payload.fp = fp;
return 'mhu1_' + Buffer.from(JSON.stringify(payload)).toString('base64url');
const hosts = [];
try {
for (const entry of Object.values(os.networkInterfaces())) {
for (const net of (entry || [])) {
if (net && net.family === 'IPv4' && !net.internal && net.address) hosts.push(net.address);
}
}
} catch {}
return [...new Set(hosts)];
}
function _safeDiagBindAddress(addr) {
const a = String(addr || '').trim();
if (a === '127.0.0.1' || a === '::1') return a;
function _diagAllowlist(diag) {
return Array.isArray(diag && diag.allowlist) ? diag.allowlist.map((x) => String(x).trim()).filter(Boolean) : [];
}
function _diagBindHost(diag) {
const mode = (diag && diag.bindMode) || 'local';
if (mode === 'network' && _diagAllowlist(diag).length > 0) return '0.0.0.0';
return '127.0.0.1';
}
function _diagPublicHost(diag) {
const explicit = String((diag && diag.publicHost) || '').trim();
if (explicit) return explicit;
if (_diagBindHost(diag) === '127.0.0.1') return '127.0.0.1';
return _getSuggestedRemoteHosts()[0] || '127.0.0.1';
}
function buildDiagnosticCode(diag, fp) {
const payload = { v: 1, h: _diagPublicHost(diag), p: diag.port || 9110, t: diag.token, n: diag.label || require('os').hostname() };
if (fp) { payload.fp = fp; payload.s = 'wss'; }
return 'mhu1_' + Buffer.from(JSON.stringify(payload)).toString('base64url');
}
async function startDiagnosticAgent() {
if (diagnosticAgent) { try { diagnosticAgent.stop(); } catch {} diagnosticAgent = null; }
const config = configStore.load();
@ -2511,7 +2701,8 @@ async function startDiagnosticAgent() {
}
if (!_diagHandler) _diagHandler = _buildDiagnosticHandler();
const host = _safeDiagBindAddress(diag.bindAddress);
const host = _diagBindHost(diag);
const allowlist = _diagAllowlist(diag);
diagnosticAgent = new RemoteServer();
try {
await diagnosticAgent.start({
@ -2519,9 +2710,10 @@ async function startDiagnosticAgent() {
host,
token,
diagnosticMode: true,
allowlist,
onDiagnosticRequest: _diagHandler
});
debugLog(`diagnostics-agent started on ${host}:${diagnosticAgent.getPort()}`);
debugLog(`diagnostics-agent started on ${host}:${diagnosticAgent.getPort()} (allowlist ${allowlist.length})`);
} catch (e) {
debugLog(`diagnostics-agent start failed: ${e.message}`);
diagnosticAgent = null;
@ -2538,7 +2730,11 @@ ipcMain.handle('diagnostics:get-settings', () => {
return {
enabled: !!diag.enabled,
port: diag.port || 9110,
bindAddress: diag.bindAddress || '127.0.0.1',
bindMode: diag.bindMode === 'network' ? 'network' : 'local',
bindAddress: _diagBindHost(diag),
publicHost: diag.publicHost || '',
allowlist: _diagAllowlist(diag),
suggestedHosts: _getSuggestedRemoteHosts(),
label: diag.label || require('os').hostname(),
codeIssuedAt: diag.codeIssuedAt || 0,
code: diag.token ? buildDiagnosticCode(diag) : ''
@ -2552,13 +2748,18 @@ ipcMain.handle('diagnostics:save-settings', async (_e, incoming) => {
...cur,
enabled: !!(incoming && incoming.enabled),
port: (incoming && Number(incoming.port)) || cur.port || 9110,
bindAddress: _safeDiagBindAddress((incoming && incoming.bindAddress) || cur.bindAddress),
bindMode: (incoming && incoming.bindMode === 'network') ? 'network' : 'local',
publicHost: (incoming && incoming.publicHost != null) ? String(incoming.publicHost).trim() : (cur.publicHost || ''),
allowlist: (incoming && Array.isArray(incoming.allowlist))
? incoming.allowlist.map((x) => String(x).trim()).filter(Boolean)
: _diagAllowlist(cur),
label: (incoming && incoming.label != null) ? String(incoming.label) : cur.label
};
next.bindAddress = _diagBindHost(next);
const gs = { ...cfg.globalSettings, diagnostics: next };
await configStore.save({ globalSettings: gs });
await startDiagnosticAgent();
return { ok: true };
return { ok: true, bindAddress: next.bindAddress, allowlistCount: next.allowlist.length };
});
ipcMain.handle('diagnostics:regenerate', async () => {
@ -2577,7 +2778,10 @@ ipcMain.handle('diagnostics:status', () => {
return {
running: !!diagnosticAgent,
port: diagnosticAgent ? diagnosticAgent.getPort() : (diag.port || 9110),
bindAddress: diag.bindAddress || '127.0.0.1',
bindMode: diag.bindMode === 'network' ? 'network' : 'local',
bindAddress: _diagBindHost(diag),
publicHost: _diagPublicHost(diag),
allowlistCount: _diagAllowlist(diag).length,
clientCount: diagnosticAgent ? diagnosticAgent.getClientCount() : 0,
lastAccess: diagnosticAgent ? diagnosticAgent.getLastAccess() : null
};

View File

@ -1,6 +1,6 @@
{
"name": "multi-hoster-uploader",
"version": "3.3.84",
"version": "3.3.108",
"description": "Upload files to doodstream, voe, vidmoly, byse simultaneously",
"main": "main.js",
"scripts": {
@ -33,7 +33,9 @@
"main.js",
"preload.js",
"lib/**/*",
"renderer/**/*"
"renderer/**/*",
"assets/app_icon.ico",
"assets/app_icon.png"
],
"win": {
"target": [

View File

@ -18,6 +18,62 @@ let config = { hosters: {}, hosterSettings: {}, globalSettings: {} };
let hosterSettings = {};
let uploading = false;
let healthCheckRunning = false;
let _rLongTasks = 0, _rLongTaskMax = 0, _rFrameLast = 0, _rFrameWorst = 0, _rFrameCount = 0, _rFrameJank = 0, _rPerfLastLog = 0, _rPerfWindowStart = 0;
function _rElLabel(el) {
try {
if (!el || !el.tagName) return '?';
let s = el.tagName.toLowerCase();
if (el.id) s += '#' + el.id;
else if (el.className && typeof el.className === 'string') { const c = el.className.trim().split(/\s+/)[0]; if (c) s += '.' + c; }
const a = el.getAttribute && (el.getAttribute('data-action') || el.getAttribute('data-tab') || el.getAttribute('aria-label') || el.getAttribute('title'));
if (a) s += `[${String(a).slice(0, 24)}]`;
return s;
} catch { return '?'; }
}
try {
if (window.PerformanceObserver) {
new window.PerformanceObserver((list) => {
for (const e of list.getEntries()) {
_rLongTasks++;
if (e.duration > _rLongTaskMax) _rLongTaskMax = e.duration;
if (e.duration >= 100 && window.api && window.api.debugLog) window.api.debugLog(`renderer-longtask dur=${Math.round(e.duration)}ms`);
}
}).observe({ entryTypes: ['longtask'] });
}
} catch {}
try {
if (window.PerformanceObserver) {
new window.PerformanceObserver((list) => {
for (const e of list.getEntries()) {
const proc = Math.round((e.processingEnd || 0) - (e.processingStart || 0));
if (window.api && window.api.debugLog) window.api.debugLog(`renderer-interaction ${e.name} dur=${Math.round(e.duration)}ms proc=${proc}ms target=${_rElLabel(e.target)}`);
}
}).observe({ type: 'event', durationThreshold: 50, buffered: true });
}
} catch {}
function _rFrameTick(ts) {
if (_rFrameLast) { const d = ts - _rFrameLast; _rFrameCount++; if (d > _rFrameWorst) _rFrameWorst = d; if (d > 33) _rFrameJank++; }
_rFrameLast = ts;
requestAnimationFrame(_rFrameTick);
}
requestAnimationFrame(_rFrameTick);
function _resetRendererPerf() {
_rPerfWindowStart = Date.now();
_rFrameCount = 0; _rFrameJank = 0; _rFrameWorst = 0; _rLongTasks = 0; _rLongTaskMax = 0;
}
function _maybeLogRendererPerf(activeJobs) {
const now = Date.now();
if (!_rPerfWindowStart) _rPerfWindowStart = now;
if (now - _rPerfLastLog < 5000) return;
const winSec = (now - _rPerfWindowStart) / 1000;
const fps = winSec > 0 ? Math.round(_rFrameCount / winSec) : 0;
if (window.api && window.api.debugLog) {
window.api.debugLog(`renderer-perf active=${activeJobs} fps=${fps} jankFrames=${_rFrameJank} worstFrame=${Math.round(_rFrameWorst)}ms longtasks=${_rLongTasks} maxTask=${Math.round(_rLongTaskMax)}ms`);
}
_rPerfLastLog = now;
_resetRendererPerf();
}
let accountStatuses = {}; // { accountId: { status: 'ok'|'warn'|'error'|'checking'|'unchecked', message: '' } }
let editingAccountId = null; // null = adding, string = editing account by ID
let autoHealthCheckEnabled = true;
@ -276,6 +332,7 @@ async function init() {
// --- Tab switching ---
let _historyDirty = false;
let _historyEverLoaded = false;
function _isHistoryTabActive() {
const tab = document.querySelector('.tab.active');
return !!(tab && tab.dataset.view === 'history');
@ -303,8 +360,7 @@ function _isHistoryTabActive() {
const nextView = viewsById[`${tab.dataset.view}-view`];
if (nextView) nextView.classList.add('active');
activeTab = tab;
if (tab.dataset.view === 'history') {
_historyDirty = false;
if (tab.dataset.view === 'history' && (_historyDirty || !_historyEverLoaded)) {
loadHistory();
}
};
@ -1177,7 +1233,7 @@ let _recentRenderQueued = false;
function scheduleRecentRender() {
if (_recentRenderQueued) return;
_recentRenderQueued = true;
requestAnimationFrame(() => { _recentRenderQueued = false; renderRecentUploadsPanel(); });
requestAnimationFrame(() => { _recentRenderQueued = false; renderRecentUploadsPanel(true); });
}
// Toggle the .selected class on existing rows without rebuilding the table.
@ -2539,6 +2595,12 @@ function _handleStatsImpl(data) {
updateStatusBar();
updateStatsPanel();
if (data.state === 'uploading' && (data.activeJobs || 0) > 0) {
_maybeLogRendererPerf(data.activeJobs);
} else {
_resetRendererPerf();
}
// Track run time
if (data.state === 'uploading' || data.state === 'stopping') {
if (!statsStartTime) {
@ -2750,13 +2812,13 @@ const SESSION_FILES_CAP = 2000;
function maybeAddSessionFile(job) {
if (!job) return;
const dt = formatDateTime(new Date());
if (job.status === 'done' && job.result) {
const link = job.result.download_url || job.result.embed_url || '';
if (!link) return;
const dedupKey = `${link}\u0001${job.fileName}\u0001${job.hoster}`;
if (!_sessionFileKeys.has(dedupKey)) {
_sessionFileKeys.add(dedupKey);
const dt = formatDateTime(new Date());
sessionFilesData.push({
date: dt.text,
dateTs: dt.ts,
@ -2768,6 +2830,7 @@ function maybeAddSessionFile(job) {
});
_recentDataVersion++;
_sessionDoneCount++;
_recentPendingAppends++;
// Drop oldest entries past the cap to keep render cost bounded.
// Without this, sessionFilesData grows unbounded across the session
// and every renderRecentUploadsPanel call becomes a megabyte-sized
@ -3216,12 +3279,25 @@ function renderSettings() {
<input type="number" class="hs-input" id="diagPortInput" min="1024" max="65535" value="9110" style="width:100px">
</div>
<div class="settings-row">
<label>Bind-Adresse</label>
<select class="hs-input" id="diagBindInput" style="width:auto">
<option value="127.0.0.1">Nur lokal (Tunnel/VPN) empfohlen</option>
<label>Sichtbarkeit</label>
<select class="hs-input" id="diagBindModeInput" style="width:auto">
<option value="local">Nur lokal (127.0.0.1) Tunnel/VPN</option>
<option value="network">Im Netzwerk (0.0.0.0) Allowlist nötig</option>
</select>
</div>
<div class="settings-row"><span class="hint">Direkte LAN-/Internet-Bindung ist deaktiviert, bis verschlüsselter Transport (wss/TLS) verfügbar ist. Zugriff aus der Ferne läuft über einen SSH- oder VPN-Tunnel zu <code>127.0.0.1</code>.</span></div>
<div class="settings-row">
<label>Adresse für den Code</label>
<input type="text" class="hs-input" id="diagPublicHostInput" placeholder="127.0.0.1 oder Tunnel-/Tailscale-Adresse" style="flex:1">
</div>
<div class="settings-row" id="diagSuggestRow" style="display:none">
<label></label>
<div id="diagSuggestChips" style="display:flex;gap:6px;flex-wrap:wrap"></div>
</div>
<div class="settings-row" id="diagAllowlistRow" style="display:none;align-items:flex-start">
<label>Allowlist (IP/CIDR, eine pro Zeile)</label>
<textarea class="hs-input" id="diagAllowlistInput" rows="3" style="flex:1;font-family:monospace" placeholder="100.64.0.0/10&#10;203.0.113.5"></textarea>
</div>
<div class="settings-row"><span class="hint" id="diagBindHint"></span></div>
<div class="settings-row">
<label>Verbindungs-Code</label>
<input type="text" class="key-input" id="diagCodeInput" value="" readonly style="flex:1" placeholder="(aktivieren zum Erzeugen)">
@ -3360,7 +3436,13 @@ function renderSettings() {
(function wireDiagnostics() {
const enabledEl = document.getElementById('diagEnabledInput');
const portEl = document.getElementById('diagPortInput');
const bindEl = document.getElementById('diagBindInput');
const modeEl = document.getElementById('diagBindModeInput');
const publicHostEl = document.getElementById('diagPublicHostInput');
const allowlistEl = document.getElementById('diagAllowlistInput');
const allowlistRow = document.getElementById('diagAllowlistRow');
const suggestRow = document.getElementById('diagSuggestRow');
const suggestChips = document.getElementById('diagSuggestChips');
const bindHintEl = document.getElementById('diagBindHint');
const codeEl = document.getElementById('diagCodeInput');
const issuedEl = document.getElementById('diagCodeIssued');
const badgeEl = document.getElementById('diagStatusBadge');
@ -3370,13 +3452,40 @@ function renderSettings() {
if (!ts) return '';
try { return 'Code erstellt: ' + new Date(ts).toLocaleString('de-DE'); } catch { return ''; }
};
const parseAllowlist = () => allowlistEl.value.split(/\r?\n/).map((l) => l.trim()).filter(Boolean);
const renderModeUi = (suggestedHosts) => {
const network = modeEl.value === 'network';
allowlistRow.style.display = network ? '' : 'none';
bindHintEl.innerHTML = network
? 'Bindet an <code>0.0.0.0</code>. Nur IPs/CIDRs aus der Allowlist dürfen verbinden (Loopback immer) — zusätzlich zum Token. Über Tailscale: trage deinen Tailnet-Bereich ein (z.B. <code>100.64.0.0/10</code>) und die Tailscale-IP/MagicDNS oben als Code-Adresse. Transport ist plaintext über den Tunnel — Tailscale/WireGuard verschlüsselt.'
: 'Bindet nur an <code>127.0.0.1</code>. Fernzugriff nur über einen Tunnel (z.B. Tailscale/SSH) — die sicherste Variante.';
const hosts = Array.isArray(suggestedHosts) ? suggestedHosts : [];
if (hosts.length) {
suggestRow.style.display = '';
suggestChips.innerHTML = '';
for (const h of hosts) {
const b = document.createElement('button');
b.className = 'btn btn-xs btn-secondary';
b.textContent = h;
b.addEventListener('click', () => { publicHostEl.value = h; save(); });
suggestChips.appendChild(b);
}
} else {
suggestRow.style.display = 'none';
}
};
let lastSuggested = [];
const applySettings = (s) => {
if (!s) return;
enabledEl.checked = !!s.enabled;
portEl.value = s.port || 9110;
bindEl.value = s.bindAddress || '127.0.0.1';
modeEl.value = s.bindMode === 'network' ? 'network' : 'local';
publicHostEl.value = s.publicHost || '';
allowlistEl.value = Array.isArray(s.allowlist) ? s.allowlist.join('\n') : '';
lastSuggested = Array.isArray(s.suggestedHosts) ? s.suggestedHosts : [];
codeEl.value = s.code || '';
issuedEl.textContent = fmtIssued(s.codeIssuedAt);
renderModeUi(lastSuggested);
if (badgeEl) {
badgeEl.textContent = s.enabled ? 'Aktiv' : 'Inaktiv';
badgeEl.className = 'panel-status' + (s.enabled ? ' active' : '');
@ -3388,7 +3497,8 @@ function renderSettings() {
if (!el || !st) return;
if (st.running) {
const last = st.lastAccess ? new Date(st.lastAccess).toLocaleString('de-DE') : '—';
el.textContent = `Aktiv auf ${st.bindAddress}:${st.port}${st.clientCount} Client(s) — Letzter Zugriff: ${last}`;
const scope = st.bindMode === 'network' ? `Netzwerk (Allowlist: ${st.allowlistCount})` : 'nur lokal';
el.textContent = `Aktiv auf ${st.bindAddress}:${st.port} (${scope}) — ${st.clientCount} Client(s) — Letzter Zugriff: ${last}`;
el.style.color = '#10b981';
} else {
el.textContent = 'Nicht aktiv';
@ -3397,10 +3507,17 @@ function renderSettings() {
}).catch(() => {});
};
const save = async () => {
const allowlist = parseAllowlist();
if (enabledEl.checked && modeEl.value === 'network' && allowlist.length === 0) {
if (bindHintEl) { bindHintEl.innerHTML = '<span style="color:#f59e0b">Netzwerkmodus braucht mindestens eine IP/CIDR in der Allowlist — sonst bleibt es fail-closed auf Loopback.</span>'; }
return;
}
await window.api.diagnosticsSaveSettings({
enabled: enabledEl.checked,
port: parseInt(portEl.value, 10) || 9110,
bindAddress: bindEl.value
bindMode: modeEl.value,
publicHost: publicHostEl.value.trim(),
allowlist
});
applySettings(await window.api.diagnosticsGetSettings());
refreshStatus();
@ -3411,7 +3528,9 @@ function renderSettings() {
enabledEl.addEventListener('change', save);
portEl.addEventListener('change', save);
bindEl.addEventListener('change', save);
modeEl.addEventListener('change', () => { renderModeUi(lastSuggested); save(); });
publicHostEl.addEventListener('change', save);
allowlistEl.addEventListener('change', save);
document.getElementById('diagCopyCodeBtn').addEventListener('click', async () => {
if (!codeEl.value) return;
await window.api.copyToClipboard(codeEl.value);
@ -4511,6 +4630,8 @@ function _hideOtpField() {
async function loadHistory() {
const history = await window.api.getHistory();
window._historyForStats = history || [];
_historyEverLoaded = true;
_historyDirty = false;
_invalidateHosterLifetimeCache();
const retSel = document.getElementById('historyRetentionSelect');
if (retSel) retSel.value = (config.globalSettings && config.globalSettings.historyRetention) || 'all';
@ -4616,44 +4737,68 @@ function _buildRecentRowHtml(row) {
// accumulating new uploads (the default case: sort=date desc, rows only grow).
let _recentLastRenderedSig = '';
let _recentLastRenderedLen = 0;
let _recentPendingAppends = 0;
let _recentWorking = [];
let _recentLastRange = { start: -1, end: -1 };
let _recentScrollQueued = false;
function renderRecentUploadsPanel() {
function _onRecentScroll() {
if (_recentScrollQueued) return;
_recentScrollQueued = true;
requestAnimationFrame(() => { _recentScrollQueued = false; _renderRecentVirtualRows(); });
}
function _renderRecentVirtualRows() {
const wrap = document.querySelector('.recent-files-table-wrap');
const tbody = document.getElementById('recentFilesBody');
if (!wrap || !tbody) return;
const total = _recentWorking.length;
if (!total) return;
const scrollTop = wrap.scrollTop;
const viewportHeight = Math.max(wrap.clientHeight, 400);
const startIdx = Math.max(0, Math.floor(scrollTop / VIRTUAL_ROW_HEIGHT) - VIRTUAL_OVERSCAN);
const endIdx = Math.min(total, Math.ceil((scrollTop + viewportHeight) / VIRTUAL_ROW_HEIGHT) + VIRTUAL_OVERSCAN);
if (startIdx === _recentLastRange.start && endIdx === _recentLastRange.end) return;
_recentLastRange = { start: startIdx, end: endIdx };
const topPad = startIdx * VIRTUAL_ROW_HEIGHT;
const bottomPad = Math.max(0, (total - endIdx) * VIRTUAL_ROW_HEIGHT);
const parts = [];
if (topPad > 0) parts.push(`<tr class="virtual-spacer" style="height:${topPad}px"><td colspan="4"></td></tr>`);
for (let i = startIdx; i < endIdx; i++) parts.push(_buildRecentRowHtml(_recentWorking[i]));
if (bottomPad > 0) parts.push(`<tr class="virtual-spacer" style="height:${bottomPad}px"><td colspan="4"></td></tr>`);
tbody.innerHTML = parts.join('');
}
function renderRecentUploadsPanel(appendOnly = false) {
const tbody = document.getElementById('recentFilesBody');
if (!tbody) return;
_recentPendingAppends = 0;
const wrap = tbody.closest('.recent-files-table-wrap');
if (!sessionFilesData.length) {
tbody.innerHTML = '<tr><td colspan="4" class="empty-state">Noch keine Uploads in dieser Session.</td></tr>';
_recentLastRenderedSig = '';
_recentLastRenderedLen = 0;
return;
}
const rows = sortRecentFiles(sessionFilesData);
const sig = `${recentSortState.key}|${recentSortState.direction}`;
const dateDescAppendOnly = sig === 'date|desc'
&& _recentLastRenderedSig === sig
&& rows.length > _recentLastRenderedLen
&& tbody.querySelectorAll('.recent-file-row').length === _recentLastRenderedLen;
const wrap = tbody.closest('.recent-files-table-wrap');
const wasAtTop = !wrap || wrap.scrollTop <= 48;
let wasAppendOnly = false;
if (dateDescAppendOnly) {
const added = rows.length - _recentLastRenderedLen;
let html = '';
for (let i = 0; i < added; i++) html += _buildRecentRowHtml(rows[i]);
tbody.insertAdjacentHTML('afterbegin', html);
wasAppendOnly = true;
_recentWorking = [];
_recentLastRange = { start: -1, end: -1 };
} else {
tbody.innerHTML = rows.map(_buildRecentRowHtml).join('');
const prevLen = _recentWorking.length;
_recentWorking = sortRecentFiles(sessionFilesData);
_recentLastRange = { start: -1, end: -1 };
const sig = `${recentSortState.key}|${recentSortState.direction}`;
if (wrap) {
const added = _recentWorking.length - prevLen;
if (sig === 'date|desc' && wrap.scrollTop <= 48) wrap.scrollTop = 0;
else if (sig === 'date|desc' && added > 0) wrap.scrollTop += added * VIRTUAL_ROW_HEIGHT;
}
_renderRecentVirtualRows();
}
if (wrap && sig === 'date|desc' && wasAtTop) wrap.scrollTop = 0;
_recentLastRenderedSig = sig;
_recentLastRenderedLen = rows.length;
// Event delegation bind once, not per-row
if (!_recentListenersBound) {
_recentListenersBound = true;
if (wrap) {
wrap.addEventListener('scroll', _onRecentScroll, { passive: true });
if (typeof window.ResizeObserver !== 'undefined') new window.ResizeObserver(_onRecentScroll).observe(wrap);
}
tbody.addEventListener('click', (e) => {
const tr = e.target.closest('.recent-file-row');
if (!tr) return;
@ -4692,17 +4837,63 @@ function renderRecentUploadsPanel() {
});
}
// Sort headers only change when the sort state changes — skip on appends.
if (!wasAppendOnly) updateRecentSortHeaders();
updateRecentSortHeaders();
}
const HISTORY_RENDER_CAP = 2000;
let _historyWorking = [];
let _historyLastRange = { start: -1, end: -1 };
let _historyListenersBound = false;
let _historyScrollQueued = false;
function _onHistoryScroll() {
if (_historyScrollQueued) return;
_historyScrollQueued = true;
requestAnimationFrame(() => { _historyScrollQueued = false; _renderHistoryVirtualRows(); });
}
function _renderHistoryVirtualRows() {
const container = document.getElementById('historyContainer');
const tbody = document.getElementById('historyBody');
if (!container || !tbody) return;
const total = _historyWorking.length;
const scrollTop = container.scrollTop;
const viewportHeight = Math.max(container.clientHeight, 600);
const startIdx = Math.max(0, Math.floor(scrollTop / VIRTUAL_ROW_HEIGHT) - VIRTUAL_OVERSCAN);
const endIdx = Math.min(total, Math.ceil((scrollTop + viewportHeight) / VIRTUAL_ROW_HEIGHT) + VIRTUAL_OVERSCAN);
if (startIdx === _historyLastRange.start && endIdx === _historyLastRange.end) return;
_historyLastRange = { start: startIdx, end: endIdx };
const topPad = startIdx * VIRTUAL_ROW_HEIGHT;
const bottomPad = Math.max(0, (total - endIdx) * VIRTUAL_ROW_HEIGHT);
const parts = [];
if (topPad > 0) parts.push(`<tr class="virtual-spacer" style="height:${topPad}px"><td colspan="4"></td></tr>`);
for (let i = startIdx; i < endIdx; i++) {
const row = _historyWorking[i];
const link = row.link || '';
parts.push('<tr class="history-row');
if (row.isError) parts.push(' error');
parts.push('" data-link="');
parts.push(escapeAttr(link));
parts.push(`" style="height:${VIRTUAL_ROW_HEIGHT}px"><td class="col-date">`);
parts.push(escapeHtml(row.date));
parts.push('</td><td class="col-filename">');
parts.push(escapeHtml(row.filename));
parts.push('</td><td class="col-host">');
parts.push(escapeHtml(row.host));
parts.push('</td><td class="col-link">');
parts.push(escapeHtml(link));
parts.push('</td></tr>');
}
if (bottomPad > 0) parts.push(`<tr class="virtual-spacer" style="height:${bottomPad}px"><td colspan="4"></td></tr>`);
tbody.innerHTML = parts.join('');
}
function renderHistoryTable(container) {
if (!container || !historyRowsData.length) {
if (container) container.innerHTML = '<p class="empty-state">Noch keine Uploads.</p>';
const emptyNotice = document.getElementById('historyCapNotice');
if (emptyNotice) emptyNotice.style.display = 'none';
_historyWorking = [];
return;
}
@ -4718,50 +4909,22 @@ function renderHistoryTable(container) {
}
}
const rows = sortHistoryRows(working);
_historyWorking = sortHistoryRows(working);
_historyLastRange = { start: -1, end: -1 };
const headerCell = (key, label) => {
const active = historySortState.key === key;
const dir = active ? (historySortState.direction === 'asc' ? '▲' : '▼') : '↕';
return `<th class="sortable${active ? ' active' : ''}" data-history-sort="${key}">${label}<span class="sort-indicator">${dir}</span></th>`;
};
let html = `<table class="results-table history-table"><thead><tr>
container.innerHTML = `<table class="results-table history-table"><thead><tr>
${headerCell('date', 'Date')}${headerCell('filename', 'Filename')}${headerCell('host', 'Host')}${headerCell('link', 'Link')}
</tr></thead><tbody>`;
</tr></thead><tbody id="historyBody"></tbody></table>`;
const parts = [html];
const len = rows.length;
for (let i = 0; i < len; i++) {
const row = rows[i];
const link = row.link || '';
const date = escapeHtml(row.date);
const filename = escapeHtml(row.filename);
const host = escapeHtml(row.host);
const linkHtml = escapeHtml(link);
const linkAttr = escapeAttr(link);
parts.push('<tr class="history-row');
if (row.isError) parts.push(' error');
parts.push('" data-link="');
parts.push(linkAttr);
parts.push('"><td class="col-date">');
parts.push(date);
parts.push('</td><td class="col-filename">');
parts.push(filename);
parts.push('</td><td class="col-host">');
parts.push(host);
parts.push('</td><td class="col-link">');
parts.push(linkHtml);
parts.push('</td></tr>');
}
parts.push('</tbody></table>');
container.innerHTML = parts.join('');
// Delegated listeners: bind once per render-target instead of once per
// row/header. With a 5000-row history the per-row bind path was a
// 5000-iteration synchronous loop on every Verlauf-tab switch — the
// dominant cause of "tab switching lags" in the user report.
if (!container.dataset.historyListenersBound) {
container.dataset.historyListenersBound = '1';
if (!_historyListenersBound) {
_historyListenersBound = true;
container.addEventListener('scroll', _onHistoryScroll, { passive: true });
if (typeof window.ResizeObserver !== 'undefined') new window.ResizeObserver(_onHistoryScroll).observe(container);
container.addEventListener('click', (e) => {
const th = e.target.closest('th.sortable');
if (th && container.contains(th)) {
@ -4774,6 +4937,7 @@ function renderHistoryTable(container) {
} else {
historySortState.direction = historySortState.direction === 'asc' ? 'desc' : 'asc';
}
container.scrollTop = 0;
renderHistoryTable(container);
return;
}
@ -4784,6 +4948,8 @@ function renderHistoryTable(container) {
}
});
}
_renderHistoryVirtualRows();
}
function sortHistoryRows(rows) {

View File

@ -657,6 +657,7 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
.recent-file-row {
cursor: pointer;
transition: background 0.15s;
height: 28px;
}
.recent-file-row:hover {
background: rgba(255, 255, 255, 0.03);
@ -1231,6 +1232,11 @@ select.hs-input { max-width: none; width: auto; min-width: 140px; }
.results-table th.active, .history-table th.active { color: var(--text); }
.sort-indicator { margin-left: 4px; font-size: 10px; }
.history-table { table-layout: fixed; }
.history-table .col-date { width: 16%; }
.history-table .col-filename { width: 34%; }
.history-table .col-host { width: 12%; }
.history-table .col-link { width: 38%; }
.history-row {
cursor: pointer;
transition: background 0.15s;

View File

@ -1,5 +1,80 @@
# Lessons
## 2026-06-21 — Das eigene Instrument lügt nicht, aber sein Log-Code kann buggen (`queue=undefined`)
**Symptom:** Drei Builds lang jagte ich Read-Bursts (highWaterMark, threadpool), während der WAHRE Treiber
eine 38,5-MB-electron-config.json war, die 137×/73s geklont/geparst/serialisiert wurde (~47% Main-Thread).
Erst das in v3.3.98 eingebaute `config-load/config-serialize`-Log machte es sichtbar — aber mein eigenes
Log-Feld `queue=` las `.length` auf dem pendingQueue-OBJEKT (immer undefined) und hätte mich fast in die
falsche Richtung (pendingQueue statt history) geschickt.
**Root cause:** (1) Ich hatte die config-Persistenz als „instrumentieren, nicht fixen" zurückgestellt (richtig
für die unsichere Migration), aber den Lag-Treiber dort nicht früh genug vermutet. (2) Instrument-Felder selbst
müssen verifiziert werden: `(obj || []).length` auf einem Objekt = undefined, still falsch.
**Regel:** Wenn der User „es laggt unverändert" sagt obwohl die letzte Messung gut aussah, ist der gemessene
Pfad NICHT der Hot-Path — sofort BREITER messen (jeden IPC-Handler, jede periodische Main-Op, Main-Thread-
Longtask-Monitor), nicht den schon-gemessenen Pfad weiter optimieren. Und Instrument-Ausgaben gegen ein
bekanntes Beispiel prüfen (zeigt `queue=` je eine echte Zahl?).
**Wie anwenden:** Bei „Symptom unverändert trotz Fix": Hypothese fallen lassen, Coverage verbreitern. Log-Felder
beim Schreiben mit einem realen Wert gegenchecken, nie blind `(x||[]).length` auf unklar getypten Feldern.
## 2026-06-21 — „Brot finden, nicht Krümel": die EINE Änderung, die alle Kosten killt, schlägt drei sichere Teilfixes
**Symptom:** Fix-Design bot loadShallow (Klon vermeiden) + cache-repopulate + resolution-cache. Adversary zeigte:
loadShallow killt nur den Klon (~10s von 34s), die 38 Serializes (8,4s) + 38 Post-Write-Reparses (15,2s) bleiben,
weil Writes den Cache nullen → loadShallow allein = Krümel.
**Root cause:** Alle drei Kosten (parse+clone+serialize) entstehen daraus, dass history IM Hot-Config liegt.
Nur history RAUS aus der immer-geladenen Datei (eigene electron-history.json) killt alle drei gleichzeitig.
cache-repopulate-Gate feuerte nie (toter Code); resolution-cache hätte stale-Pools → Failover-Regression
(rotation/byse) riskiert = die EINE Sache die Uploads STILL korrumpiert, schlimmer als Lag.
**Regel:** Wenn der User „komplett wegmachen" fordert und mehrere sichere Teilfixes vs. ein riskanterer
Komplettfix zur Wahl stehen: den Komplettfix nehmen, aber RICHTIG absichern (hier: fsync+verify-before-strip,
permanenter .pre-history-split.bak, Migration packaged-only + per-Init nicht in load(), Crash-Window-Fallback,
Test gegen die ECHTE 194MB-Fixture). Teilfixes die den Treiber nur anknabbern NICHT bündeln (verwässert Messung
+ Risiko). Einen Fix der etwas STILL korrumpieren könnte (stale Account-Pools) NIE für Performance einbauen.
**Wie anwenden:** Bei mehreren Fix-Optionen fragen: „welche EINE Änderung entfernt die gemeinsame Wurzel ALLER
Kostenpfade?" — die nehmen und maximal absichern, statt N sichere Teilfixes die je nur einen Pfad treffen.
## 2026-06-21 — Ein-Variablen-Disziplin: nicht zwei Fixes bündeln, wenn einer den anderen maskiert
**Symptom:** Nach dem tp=8-Win wollte ich in EINEM Build A (1MB highWaterMark, Read-Burst) + B (Renderer
chunked rAF Batch-Drain, der 243ms-Longtask) + C-Instrument shippen.
**Root cause / Korrektur (Advisor):** Der Renderer war 14/15 Fenstern gesund; der EINE 243ms-Longtask (W14)
ist laut beiden Agenten DOWNSTREAM des Main-Thread-Read-Bursts (geflutetes IPC). Fix A reduziert diese
Stalls → der Renderer-Longtask verschwindet wahrscheinlich OHNE B. B mitzuliefern (a) verwässert die nächste
Messung (war die Besserung A oder B?) und (b) fasst den Progress-Hot-Path an, der hier schon gebissen hat
(formatDateTime-Burst, ghost-fix).
**Regel:** Wenn Fix A einen vermuteten Symptom-Treiber X reduziert und Fix B genau X behandeln würde —
NUR A shippen, messen, B nur nachziehen wenn X überlebt. Sonst kann das nächste Log nicht sauber attribuieren.
Bei gekoppelten Symptomen ist die Reihenfolge (Upstream-Fix zuerst, dann messen) wichtiger als „alles auf
einmal".
**Wie anwenden:** Vor dem Bündeln fragen: „Maskiert Fix A die Wirkung, die Fix B beheben soll?" Wenn ja →
entkoppeln, A zuerst, eine Variable pro Build.
## 2026-06-21 — Nicht aus EINEM konfundierten Sample eine Ursache behaupten
**Symptom:** Ich wollte dem User sagen „1-Sekunden-Persist-Freeze gefunden" auf Basis von W13 (max=1021ms,
heap→142MB).
**Root cause / Korrektur (Advisor):** W13 ist EIN Sample und konfundiert (hat gleichzeitig FSReqCallback=66)
und das EINZIGE Heap-Spike-Fenster. Die anderen isolierten Maxes (W4 415ms/heap41, W10 852ms/heap18) haben
NIEDRIGEN Heap → sind KEIN 140MB-structuredClone+stringify → eine andere Ursache (account-failed sync load()
nahe Connection-Churn). Eine Behauptung aus einem konfundierten Punkt hätte den falschen Fix priorisiert.
**Regel:** Bei isolierten Spitzen erst die Co-Signale (heap, FSReq, gc, Nachbarfenster) gegenchecken, ob sie
EINE Familie sind. Wenn die Magnitude-Signatur (hier: Heap-Spike) nicht bei allen passt → es sind mehrere
Ursachen. „Instrumentieren + bestätigen", nicht „gefunden", solange nur ein konfundierter Punkt existiert.
**Wie anwenden:** Vor „Ursache X gefunden": gibt es ≥2 unkonfundierte Samples mit derselben Signatur? Wenn
nein → als Hypothese formulieren und messen, nicht als Befund verkaufen.
## 2026-06-21 — Histogram-Korrelation beweist KEINE Kausalrichtung; rss-Mathe als Sanity-Check
**Symptom:** ELD-Spikes korrelierten exakt mit hohem `FSReqCallback` (File-Reads in flight) → ich wollte
sofort ein Read-Concurrency-Semaphore über 5 Dateien bauen.
**Root cause / Korrektur (Advisor):** (1) Korrelation ≠ Kausalität: hohe in-flight-Reads können auch SYMPTOM
sein — ein aus ANDEREM Grund blockierter Loop drained die Read-Completions nicht, also stapeln sie sich im
Snapshot. (2) rss-Mathe widerlegte meine „Read-Buffer ballonen den Speicher"-These: 70 Streams × 256KB ≈
18MB, aber rss schwang ~300MB → das ist Heap-/Objekt-Churn (GC), nicht die Read-Buffer.
**Regel:** Bevor ich auf Basis einer Histogramm-Korrelation einen Multi-File-Refactor baue: (a) Kausalrichtung
mit einem BILLIGEN reversiblen 1-Zeilen-Hebel testen (hier UV_THREADPOOL_SIZE 64→8), (b) die Größenordnung
gegenrechnen (passt die vermutete Quelle zahlenmäßig zur beobachteten Wirkung?), (c) den fehlenden Co-Faktor
(GC) erst MESSEN, bevor ich ihn aus- oder einschließe. Ein Build kann gleichzeitig Kandidaten-Fix UND
Diskriminator sein.
**Wie anwenden:** Bei „X korreliert mit Y, also fixe X": erst fragen „könnte Y → X statt X → Y?" und „passt
die Magnitude?". Wenn nein/unklar → erst der billige reversible Knopf + Messung, dann der teure Refactor.
## 2026-04-21 — DOM-Doppelrender bei Bulk-State-Changes
**Symptom:** User klickt auf "Erneut versuchen" mit 500+ Jobs → App hängt sekundenlang.
**Root cause:** `retrySelectedJobs()` ruft `renderQueueTable + updateQueueActionButtons + updateStatusBar` auf, `startSelectedUpload()` ruft direkt danach genau dieselben Funktionen nochmal auf.
@ -108,3 +183,67 @@
**Symptom:** Single-Instance-Lock (G) über 3 Turns hinweg in Prosa „dem User vorgelegt"; User hat 0× darauf reagiert (stattdessen Hunt re-run). Drohte ein 4. Mal als vage Sorge im Summary aufzutauchen.
**Regel:** Re-Bestätigung dass ein Bug ECHT ist ≠ Zustimmung zu einer Verhaltensänderung. Die offene Frage bei user-facing Changes ist nicht „ist es real" sondern „will der User dieses Verhalten" (hier: startet er je absichtlich 2 Instanzen?). Eine vierte Prosa-Erwähnung ist „ambient worry", keine Entscheidung.
**Wie anwenden:** Nach dem Release EINE AskUserQuestion feuern (user-facing Change + ggf. weitere geparkte Punkte als Multi-Select gebündelt) ODER in EINER Zeile sagen „geparkt bis dein Wort" und aufhören es zu wiederholen. Nie denselben user-facing Flag 3+ Mal als Sorge raisen.
## 2026-06-19 — Security-E2E muss JEDEN Collector-Default-Pfad treiben, nicht nur den Aggregat-Hub (Remote-Diagnostics)
**Kontext:** Read-only Remote-Diagnose gebaut (Agent im App-Prozess + lokales MCP-Gateway, connect-by-code). Redaktion ist die EINZIGE Garantie, dass kein Secret die Box verlässt. Drei Scrub-Ebenen: (1) `sanitizeConfig` redactet CRED_KEY-gekeyte Felder strukturell, (2) `valueScrub` ersetzt bekannte Config-Secret-WERTE per split/join, (3) `redactLogText` pattern-scrubt Secret-SHAPES (Bearer/token=/cookie:/discord-webhook/?key=) in Freitext.
**Bug 1 (E2E-Gate gefangen):** Ein Hoster-zurückgegebenes `token=<opaque>` in einem History-Error-String überlebte — es ist KEIN gespeichertes Config-Credential, also greift value-scrub nicht, und das line-48-Pattern kannte nur `access_token`, nicht bare `token`. Fix: token-Familie (`token`/`auth_token`/`refresh_token`/`session_token` + standalone `Bearer <opaque>`) ins Pattern.
**Bug 2 (Advisor gefangen, NACH grünem E2E):** `get_config_redacted` section:'all' und `get_queue_state {includeJobs:true}` (DEFAULT-Pfad!) liefen nur durch value-scrub, nie pattern-scrub → derselbe opaque-Token-Leak. `redactLogText` direkt über kompaktes JSON laufen zu lassen geht NICHT (die `cookie:`/`[^\n]*`-Patterns fressen über Feldgrenzen → JSON kaputt). Fix: `_deepRedact` = per-String-Leaf-Walk, der redactLogText auf jedes Leaf einzeln anwendet (JSON-safe, da pro Leaf begrenzt). history aus get_config gedroppt (hat eigenen Collector).
**Warum der erste E2E es verfehlte (der Diskriminator):** `server_health` ruft `getQueueState({includeJobs:false})` → kein Job-Error je serialisiert; UND das Fixture nutzte `apiKey=<SECRET>` als Queue-Error → value-scrub fing es eh, auch ohne pattern-scrub. ZWEI Zufälle versteckten den Leak. Der direkte `get_queue_state{includeJobs:true}`-Pfad mit NICHT-Config-Token war nie getrieben.
**Regel:** Bei einem Redaktions-Gate (a) jeden Collector EINZELN mit seinen DEFAULT-Args treiben, nicht nur den Aggregat-Hub, der bequeme Flags setzt; (b) Fixtures MÜSSEN ein NICHT-Config-Secret enthalten (opaque Token, der nur als Shape erkennbar ist), sonst testet man value-scrub und glaubt, pattern-scrub zu testen; (c) ein grüner E2E heißt nicht „dicht" — Advisor/Review über jeden Pfad laufen lassen, der einen Freitext-Fehlerstring serialisiert.
**Wie anwenden:** Denylist-Redaktion kann einen bare opaque String OHNE `key=`/`Bearer`/URL-Kontext NICHT scrubben — das ist inhärent, kein Defekt. Commit-/Release-Notes ehrlich halten („common secret shapes pattern-scrubbed", nicht „kein Secret verlässt je die Box"). Default-Bind 127.0.0.1 erzwingen (`_safeDiagBindAddress`), 0.0.0.0-Option erst wenn wss/TLS existiert — nie plaintext ws:// auf allen Interfaces.
## 2026-06-19 — Ein Diagnose-Tool darf NIE den Prozess einfrieren, den es diagnostiziert (v3.3.85)
**Kontext:** Nach Release v3.3.84 das Diagnose-System "intensiv durchtesten". Drei Werkzeuge gebaut: (1) Live-Integration-Harness, das den ECHTEN Gateway-MCP-Prozess (StdioClientTransport) gegen einen echten in-process Agent fährt und ALLE 14 Tools durchprüft; (2) adversariale Redaktions-/Abuse-Probe; (3) unabhängiger code-reviewer-Audit-Subagent. Parallel laufen lassen.
**Bug 1 (Probe + Audit, REAL DoS):** `read_log` kompilierte den vom Client gelieferten `grep` zu `new RegExp(grep,'i')` und lief synchron über bis zu 1 MB Log-Tail — IM Electron-Main-Prozess. `(a+)+$` gegen eine lange Zeile = katastrophisches Backtracking → ganze App friert ein (empirisch: 8s-Timeout, gekillt). JS-Regex ist synchron und nicht abbrechbar → der einzige sichere Fix ist KEINE User-Regex: grep ist jetzt case-insensitive Substring-Filter mit `|`-Alternation. Provably linear.
**Bug 2 (Probe, Whitelist-Integrität):** Die Op-Tabelle war ein Plain-Object-Literal → `OPS['constructor']`/`['toString']`/`['valueOf']` lösen geerbte Object.prototype-Funktionen auf, bestehen `typeof fn==='function'` und liefern `{ok:true}`. Harmlos (kein Secret/Write), aber Whitelist-Loch. Fix: `typeof op==='string' && Object.prototype.hasOwnProperty.call(OPS,op)`.
**Bug 3 (nur Live-Integration sichtbar):** Der Gateway las die App-Version aus `info.data.version`, der Collector liefert sie aber als `info.data.app.version` — der "connected to vX.Y.Z"-Hinweis war still leer. Unit-Tests mit Stubs fingen das NIE; nur das Fahren des echten Gateway-Prozesses gegen den echten Collector deckte die Shape-Diskrepanz auf.
**Regeln:** (1) Ein read-only Diagnose-Agent, der IM Zielprozess läuft, darf keine vom Client kontrollierte Synchron-Operation mit unbegrenztem Aufwand ausführen (Regex, JSON.parse von Riesen-Payloads, etc.) — sonst DoS der diagnostizierten App. Literal-Match statt Regex; alles clampen. (2) Whitelist NIE als Plain-Object mit `obj[key]`-Lookup — Prototype-Member lecken; Set/Map/null-proto/hasOwnProperty. (3) Eine Live-Integration gegen den ECHTEN Out-of-Process-Consumer findet Shape-/Contract-Mismatches, die Mock-Unit-Tests strukturell nicht sehen — bei jedem Protokoll-Grenzübergang (MCP-Tool↔Collector) mindestens EINEN echten End-to-End-Lauf. (4) Ein grüner Custom-E2E ist kein Freibrief: der Advisor/Audit fand den Queue-Leak NACH grünem E2E, weil das Gate bequeme Flags (includeJobs:false) setzte — jeden Collector mit DEFAULT-Args fahren.
## 2026-06-19 — "Mach es wie <anderes Projekt>" = das Projekt FINDEN und EXAKT mappen, nicht raten (Tailscale/Allowlist, v3.3.86)
**Kontext:** User: "nutzen wir dasselbe wie der downloader mit tailscale was mcp betrifft" + "ka, das was der downloader nutzt, mach dasselbe". Der User kannte die Details NICHT — er wollte 1:1-Replikation eines Schwester-Projekts.
**Vorgehen das funktioniert hat:** (1) Sibling-Projekte gelistet, per Grep nach `tailscale`/`100.64`/`ts.net` gesucht → `Real-Debrid-Downloader/tools/rd-diagnostics-mcp` gefunden (eine fast identische MCP-Ferndiagnose existierte schon). (2) Einen Explore-Subagenten eine PRÄZISE Implementierungs-Map mit file:line + Code-Excerpts erstellen lassen (bind modes, fail-closed allowlist, code-format, bridge, UI, IPC). (3) Das Muster EXAKT repliziert statt zu raten.
**Was der Downloader anders macht (und warum es Tailscale ermöglicht):** Host steckt IM Code (`{v,h,p,t,n,fp?,s?}`), nicht extern. Zwei Bind-Modi: lokal (127.0.0.1) ODER Netzwerk (0.0.0.0) — letzteres NUR mit nicht-leerer **fail-closed IP-Allowlist**: leere Allowlist = nur Loopback; geprüft am ECHTEN socket.remoteAddress (NIE forwarded-Header), `::ffff:`-normalisiert, CIDR-Matching. Tailscale wird NICHT autodetektiert — es ist nur eine der `os.networkInterfaces()`-IPs, erreicht über den Tunnel; Allowlist (auf den Tailnet, z.B. `100.64.0.0/10`) + Token sind das Gate, WireGuard ist die Verschlüsselung.
**Regel:** Bei "mach es wie X": X lokalisieren (grep), die security-kritischen Teile mit einem Subagenten verbatim mappen, dann replizieren. Eine fail-closed Allowlist (Loopback immer erlaubt, leer=loopback-only, real peer IP) ist das richtige Modell für netzwerk-erreichbare read-only Diagnose über einen vertrauten Tunnel — plaintext-Transport ist ok, WENN der Tunnel (Tailscale/WireGuard) verschlüsselt UND die Allowlist+Token den Zugriff gaten. Den HAPPY-Path (allowlisted non-loopback peer über echten 0.0.0.0-Socket) auch LIVE testen, nicht nur per Komposition aus Unit+Wiring.
**Prozess-Stolperstein:** Test NACH dem Feature-Commit hinzugefügt → release_gitea.mjs brach ab ("uncommitted tracked changes"). Vor jedem Release: `git status --porcelain | grep -v '^??'` muss leer sein. Tracked-aber-uncommitted (auch ein nachgereichter Test) blockt den Build.
## 2026-06-21 — "Gefühlt laggy nach Zeit, CPU/RAM normal" = ERST messen, dann in echtem Blink profilen (v3.3.87)
**Kontext:** User: Programm fühlt sich nach langer Laufzeit mit vielen Uploads zäh an, CPU ~40%/8 Kerne, RAM 6/32 GB — beide normal/stabil. Erste Hypothese (Haupt­prozess-Config-I/O skaliert mit wachsender History) war für DIESEN User FALSCH.
**Was es wirklich war (gemessen + profiliert):** `renderRecentUploadsPanel` hatte einen Append-only-Fastpath, gegated auf `rows.length > _recentLastRenderedLen`. `maybeAddSessionFile` capped per push-then-slice (2000→2001→zurück auf 2000). Ab dem Cap ist `rows.length` auf 2000 fixiert → Gate für IMMER false → JEDE Completion fiel in den Full-`innerHTML`-Rebuild von 2000 Zeilen. Blink-Messung (Playwright, table-layout:fixed, gleiche Engine wie Electron): **~80 ms pro Completion** → wiederkehrender 80-ms-Freeze. Fix (append-evict, Gate auf `pendingAppends>0`, Overflow vom DOM-Boden evicten): **80 ms → 7,4 ms** (>10×), DOM bleibt exakt == Daten (Cap/Reihenfolge/keine Dupes), über 5000 Completions verifiziert.
**Regel 1 — Magnituden NICHT raten, LESEN:** „wächst über Zeit" ist eine Annahme über GRÖSSE. Die echte electron-config.json war 52 KB (History 23 Zeilen) — ein einziger `node`-Read killte die ganze Config-I/O-Theorie. Bevor man eine „skaliert-mit-X"-Ursache fixt: X am echten Artefakt messen (Dateigröße, Array-Länge, Job-Count im persistierten State).
**Regel 2 — Im ECHTEN Renderer-Engine profilen, nicht analytisch raten:** jsdom rendert kein Blink-Layout. Playwright (Chromium = Electron-Blink) mit `performance.now()` um (a) Rebuild und (b) erzwungenes Relayout nach Style-Write liefert die Zahl, die entscheidet: 3 ms = unsichtbar, 80 ms = DIE Ursache. Dieselbe Messung ist Fix-Auswahl UND Vorher/Nachher-Verifikation (das Goal verlangt „verifiziere dass behoben" — ein grüner Test beweist Korrektheit, NICHT dass der Lag weg ist).
**Regel 3 — Multi-Agent-Findings gegen primäre Evidenz prüfen (Control-Char-Falsch­positiv):** Der Hunt meldete HIGH-ish einen „_sessionFileKeys delete-key separator mismatch". Beim Versuch ihn zu fixen matchte der Edit-`old_string` NICHT. Char-Code-Dump (`HAS_U0001: True`) zeigte: die Zeile hat ECHTE U+0001-Zeichen — die Read-Tools der Verifier-Agenten rendern Steuerzeichen unsichtbar, sie schlossen fälschlich „keine Separatoren". KEIN Bug. **Wenn ein Fix-`old_string` nicht matcht obwohl Grep ihn zeigt: Char-Codes dumpen, bevor man dem Tool misstraut — die Quelle kann unsichtbar von der Read-Anzeige abweichen.**
**Regel 4 — Den negligible-aber-realen Befund mit Zahl ABLEHNEN, nicht aus dem Bauch:** queueJobs O(N)-Scan pro Render (wächst unbounded, da removeFromQueueOnDone=false UND Folder-Monitor EINEN Batch via addJobs am Leben hält → 500-Cap-Prune feuert nie) — real, aber Blink-gemessen <0,1 ms bei 3000 Jobs. Den riskanten Inkremental-Counter-Refactor mit DIESER Zahl skippen, nicht mit fühlt sich klein an".
**Wie anwenden:** Append-only-Optimierungen, die auf Längenwachstum gaten, brechen still an JEDEM Cap (push-then-slice fixiert die Länge) — stattdessen die Anzahl NEUER Items zählen und am Boden evicten. „Mach es wie die Queue-Tabelle (virtualisieren)" war hier NICHT nötig: die Messung zeigte stehende 2000 Zeilen kosten median 0,4 ms; nur der Rebuild war teuer. Simplest-Fix der die gemessene Ursache trifft schlägt die größere Architektur-Änderung.
## 2026-06-21 — "Audit JEDE zeile" = audit + measure + risk-appropriate DEFER, nicht fix-everything (v3.3.88)
**Kontext:** Nach dem v3.3.87-Lag-Fix Folge-Goal: „schau dir wirklich JEDE zeile an die du geschrieben hast und schau ob es solche probleme gibt o. geben könnte". 18-Agenten-Audit + Eigen-Review jeder Hot-Path-Zeile + Blink-Benchmarks.
**Befund:** Der Audit fand, dass MEIN eigener T1-Fix (config-store cache, 29d1944) eine latente Regression einführte: `load()` macht ein unconditionales `structuredClone` der GANZEN config (inkl. unbounded history) pro Call → write-interleaved loads 2,22,4× LANGSAMER als das alte read+parse (gemessen @8000 Batches: 9,65 ms → 22,96 ms). Skaliert mit historySize. ABER: der echte User hat 8 Batches / 4,8 KB → Mikrosekunden. Negligible.
**Die Falle (Advisor hat geblockt):** Ich wollte es „elegant" fixen mit `history.slice()` (shallow) statt deep-clone. Advisor: STOPP. `load()` ist der gefährlichste Code im Repo (config + credentials; Korruption = Datenverlust), ich war hier schon mal von Cache-Semantik gebissen worden. Und: der Perf-Win und das Risiko sind DIESELBE Münze — der Speedup kommt NUR vom Sharing der Batch-Objekte by-reference, und genau dieses Sharing IST die Silent-Cache-Corruption-Gefahr (hängt an einem globalen Invariant „nichts deep-mutated je eine history-Batch" den ich über zukünftigen Code + jeden getHistory-Consumer nicht erzwingen kann). Es gibt KEINE sichere Version dieses Ansatzes → falsches Werkzeug für safety-kritischen Code. Hardcoded 5 keys in `_cloneConfig` wäre ein zweiter Footgun (zukünftiger top-level key verschwindet still aus jedem load()).
**Regel:** „Audit jede Zeile" heißt JEDE Zeile ANSCHAUEN + die Magnitude MESSEN + eine risiko-angemessene Entscheidung treffen — NICHT jeden geflaggten Befund fixen. Bei einem Audit-Goal ist „ich habe jede Zeile geprüft, jeden Befund als sub-ms bei realistischer History gemessen, den Mechanismus bestätigt aber den Fix als riskante Persistenz-Chirurgie für einen latenten Mikro-Cost eingestuft, also dokumentiere ich ihn statt ihn zu shippen" die VOLLSTÄNDIGE, gründliche Antwort. Jeden geflaggten Punkt unabhängig vom Risiko zu fixen ist keine Gründlichkeit — so wird aus einer Lag-Fix-Session ein Datenverlust-Incident. Nur den EINEN Befund shippen der im echten Szenario beißt (doodstream `_debugLog`: sync statSync+appendFileSync ~815×/Upload auf dem Main-Loop während des Uploads → hinter `logVerbose` gaten, default off, near-zero risk). Den Rest als bewusste Defers mit Messzahlen dokumentieren.
**Wie anwenden:** Wenn ein Goal („JEDE!! JEDE!!!") + ein Stop-Hook Druck erzeugen, immer weiterzuschneiden: das ist genau der Moment, den Advisor VOR dem Edit zu rufen. Magnitude am ECHTEN Artefakt prüfen (der User-Config, nicht @8000-Batches-Hypothese). Persistenz-/Credential-Code nur anfassen wenn der Fix risiko-frei UND der Gewinn real-spürbar ist — sonst dokumentieren und stoppen.
## 2026-06-21 — "Nicht-Persistenz also sicher" ist ein Trugschluss: der Redaktions-Layer ist GENAUSO gefährlich (v3.3.89)
**Kontext:** 3. identische /goal-Re-Fire („JEDE zeile, alles drum-und-dran"). Diesmal die un-auditierte Remote-Diagnostics-Code (v3.3.84/85) zeilenweise auditiert (52 Agenten). 14 von 15 actionable Findings konvergierten auf EINEN Cold-Path-Freeze: `server_health` macht O(historySize) sync-Arbeit pro Request (~67 config-clones + unbounded history-walks; `limit` slict nur den Output). Gemessen 258 ms6,7 s bei großer History → friert die App ein, die es diagnostiziert (verletzt die v3.3.85-Regel).
**Der Trugschluss (Advisor hat geblockt):** Ich begründete „diesmal ist der Fix sicher, weil es Diagnostics-Collectors sind, KEIN Credential-Persistenz-Code wie letzte Runde". FALSCH. `lib/diagnostics-collectors.js` IST die Credential-Oberfläche — es ist der Redaktions-Code (`_secrets`/`_deepRedact`/`collectSecretValues`/`sanitizeConfig`/`redactLogText`). Genau dieser Code ist schon ZWEIMAL geleakt (7b5420e „one collector still leaked", 8d757a9 „redaction gaps") — bei grünem E2E. Der „elegante" Fix (config+secrets einmal snapshoten und durch die Collectors threaden) ist EXAKT die gefährliche Form: ein Pfad verpasst / ein stale secrets-array → SECRET LEAK, ein schlimmeres Versagen als der Cold-Freeze. Dieselbe Kategorie-Fehler wie letzte Runde (damals Datenverlust an config-store, jetzt Secret-Leak an der Redaktion), nur andere Datei. „Nicht Persistenz" hat mich getäuscht.
**Regel:** Die Frage ist nicht „ist es Persistenz?", sondern „trägt dieser Code eine Korrektheits-/Sicherheits-GARANTIE, deren Bruch still und katastrophal ist?" — Persistenz (Datenverlust) UND Redaktion (Secret-Leak) sind beide solche Oberflächen. Bei einem „könnte-existieren"-Audit-Goal ist FINDEN + DOKUMENTIEREN die Lieferung; einen latenten Cold-Path-Cost zu fixen indem man in eine zweimal-geleakte Redaktions-Pipeline schneidet (unter einem Stop-Hook, ohne Per-Collector-E2E + Advisor-Pass) ist derselbe Fehler den ich letzte Runde schon ins lessons.md geschrieben hatte. KONSISTENT anwenden. Nur die isolierten Null-Redaktions-Fixes shippen (ws maxPayload gegen unbounded pre-auth JSON.parse; sendToClient readyState+try-guard gegen uncaughtException). Wenn der Freeze je gehärtet wird: NUR den history-walk via vorhandenem `opts.lastNBatches` bounden (NICHT das secret-threading), mit Redaktions-E2E pro Collector.
**Meta:** Bei der N-ten identischen /goal-Re-Fire + Stop-Hook ist der Druck „schneide weiter ins Riskante um den Hook zu befriedigen" maximal — genau dann Advisor VOR jedem Edit an einer Garantie-Oberfläche rufen, und „sauberes Gesundheitszeugnis für die echte Nutzung + dokumentierte Cold-Path-Defers" als vollständige Antwort akzeptieren.
## 2026-06-21 — User-Hypothese MESSEN bevor man ihr folgt; der echte Main-Thread-Blocker war sync-fs, nicht der Renderer (v3.3.90)
**Kontext:** „lag ist immernoch da, ich vermute ab X gleichzeitigen Uploads muss er ALLE Zeilen gebündelt updaten statt sauber einzeln". 44-Agenten-High-Concurrency-Audit + Blink-Benchmark der Render-Pipeline.
**Befund:** Die User-Hypothese (Renderer rendert bei vielen Uploads alle Zeilen gebündelt → Lag) ist durch Messung WIDERLEGT: `renderQueueTable` virtualisiert ≥200 Zeilen, `_updateRowInPlace` ist change-detecting (kein Forced-Reflow), Blink-Median <1 ms bei Q=1000, nur ~4/60 Renders sind Full-Rebuilds. Der Renderer ist NICHT der Flaschenhals. Der ECHTE Blocker: `lib/clouddrop-upload.js _uploadChunked` las jeden 16-MB-Chunk mit `fs.readSync` SYNCHRON auf dem Main-Event-Loop (einzigartig unter den 5 Uploadern die anderen 4 streamen async). Bei jedem Read ~59 ms SSD / 30100 ms langsame Platte friert der GANZE Main-Loop (alle Progress/IPC/Render/andere-Uploads). Skaliert mit der Zahl paralleler clouddrop-Uploads. Passt exakt auf laggy beim Hochladen, schlimmer mit mehr gleichzeitig". User nutzt clouddrop.
**Fix:** `fs.openSync`/`readSync`/`closeSync` → `fs.promises.open` + `await fh.read` + `await fh.close()`. Byte-Äquivalenz mit Hash-Vergleich über alle Chunk-Grenzfälle verifiziert (volle/partielle/multi-Chunk/1-Byte) BEVOR geshipped — ein Chunk-Read-Bug = korrupter Upload, deshalb Pflicht-Verifikation, nicht „sieht richtig aus". Separater Fix: rotation-retry + suspect-alternate progressCb in upload-manager.js feuerten `_emitProgress` (sync `emit` + frischer Object-Spread) bei JEDEM Stream-Chunk (hunderte/s/Job) — der 250-ms-`lastEmitTime`-Gate des Primary-Path fehlte. Gate gespiegelt (activeEntry-Mutation bleibt ungated für Stats/Speed-Monitor, nur der emit ist gegated).
**Regel:** Wenn der User eine konkrete Mechanik vermutet („er updatet alle Zeilen gebündelt"), die Mechanik MESSEN bevor man sie fixt — nicht der Plausibilität folgen. Die Messung kann die Hypothese widerlegen UND den echten Verursacher woanders aufdecken (hier: nicht Renderer-DOM, sondern sync-fs im Upload-Datapfad). „Laggy bei moderater CPU" (40%/8 Kerne = ein Kern bei 100%) zeigt auf Main-Thread-Sättigung/sync-Blocking, NICHT auf DOM-Amplifikation. Bei Daten-Pfad-Fixes (Upload-Bytes) immer Byte-Äquivalenz beweisen, nicht nur Tests grün.
**Discriminator nicht vergessen:** Mit der echten User-Config (parallelCount 2×5 Hoster ≈10 gleichzeitig) sind „100 gleichzeitig" nur erreichbar wenn die Parallel-Counts hochgedreht wurden — sonst sind „100" die QUEUE-Größe, nicht concurrent. Nach dem Ship dem User die Unterscheidungsfrage stellen (Lag clouddrop-spezifisch? Parallel-Counts erhöht?), statt blind Sieg zu erklären — bei echter High-Concurrency bräuchte es ein Concurrency-Cap / Worker-Prozess, keinen Mikro-Fix.
## 2026-06-21 — Zwei lebende Hypothesen mit GEGENSÄTZLICHEN Fixes: instrumentieren statt per Elimination refactoren (v3.3.91)
**Kontext:** Discriminator beantwortet — Lag ist TRUE high-concurrency (User fährt 50+ gleichzeitig, Counts hochgedreht), nicht clouddrop. Ich wollte einen Mess-Workflow starten, um „inherent TLS → Worker" zu belegen.
**Die Falle (Advisor hat geblockt):** Der Workflow hätte die JS-Kosten (schon weitgehend als billig gemessen) nur RE-bestätigt und dann „TLS → Worker" per ELIMINATION geschlossen — derselbe Renderer-Rate-Fehler eine Ebene höher. Den credential-tragenden Upload-Core (throttle/rotation/abort/progress) auf Eliminations-Schluss umzubauen ist genau „measure-before-build" verletzt. ZWEI Hypothesen leben und brauchen GEGENSÄTZLICHE Fixes: (A) Main-Thread CPU-blockiert (TLS/crypto/sync) → Event-Loop stallt → Cap/Worker helfen; (B) Main-Thread fein aber IO-STARVED (libuv-Threadpool/Sockets) → Loop bleibt responsiv, Uploads stauen nur → Worker sind VERSCHWENDET, Config fixt es. Ich konnte im Sandbox die echte 50-fach-TLS-Last nicht messen → also hätte JEDER Sandbox-Bench die falsche Antwort per Elimination geliefert.
**Regel:** Wenn zwei Hypothesen gegensätzliche, teure/schwer-reversible Fixes implizieren UND du die entscheidende Größe im Sandbox nicht messen kannst — baue das MESSINSTRUMENT in die echte App, nicht den Fix. Hier: `perf_hooks.monitorEventLoopDelay` im Main-Prozess, geloggt während Uploads (reine Zahlen → keine Redaktions-Oberfläche). Hohe mean/p99 → CPU-blockiert → Worker gerechtfertigt; niedrige Delay während Uploads stauen → IO-bound → Worker verschwendet, Threadpool/Sockets ist der Hebel. Die Zahl entscheidet die ganze Architektur und blockiert nicht. Den Worker/Child-Process-Refactor NIE off-sandbox per Elimination shippen — erst die echte-App-Zahl + explizites User-OK (hart reversibel, fasst Credentials/Abort/Rotation an).
**Billigster konkreter Verdächtiger zuerst (reversibel, kein Refactor):** `UV_THREADPOOL_SIZE` Default 4 — alle Uploader speisen undici aus `fs.createReadStream` + DNS getaddrinfo durch denselben Pool → 50 concurrent vs 4 Threads = harter Cliff bei kleiner Connection-Zahl = exakt „ab X connections". Auf 64 (erste Zeile vor require('electron'), libuv liest beim Lazy-Init; Threads on-demand → 64-Max kostet nichts wenn ungenutzt). EINE Env-Var testet die Hypothese mit null Risiko. Windows-Eigenheit dabei gefunden: getaddrinfo wird vom Windows-DNS-Client-Service serialisiert → die DNS-Hälfte des Cliffs ist auf Windows maskiert (fs-Read-Hälfte profitiert trotzdem) — weiterer Grund, warum nur die echte-App-ELD-Zahl zählt, nicht der Sandbox-Bench.
## 2026-06-21 — Das REGIME erfragen bevor man misst/fixt; die Lag-Knoten war eine verschwendete Intl-Format pro Progress-Event (v3.3.93)
**Kontext:** User liefert echte Daten: „25 connections okay, 50+61 laggt, EVTL wenn die uploadenden Zeilen nicht im Bild sind." Ich wollte sofort meine „non-virtual reflow"-Theorie benchmarken/fixen.
**Die Falle (Advisor hat geblockt — ZUM DRITTEN MAL die Regime-Falle):** Meine Theorie ruhte auf zwei UNBESTÄTIGTEN Annahmen — (1) Queue <200 (non-virtual), (2) Dateiname-Sort. Beide für einen User mit 50-61 concurrent wahrscheinlich FALSCH. Nach M=10 (falsches Regime) und Renderer-cleared-dann-doch-nicht wäre das der dritte Regime-Fehler gewesen: synthetischer Bench auf angenommenem Regime". Advisor: ERST die zwei Fakten vom User holen (Queue-Größe? Geklickte Sort-Spalte?) sie entscheiden, OB die Theorie überhaupt gilt. Antwort: Queue 200-1000 (VIRTUELL off-screen Zeilen NICHT im DOM reflow-Theorie tot) + Sort nach Fortschritt/Speed (dynamisch). Das lenkte auf den PER-EVENT-Pfad statt den Render-Pfad.
**Befund (gemessen, nicht geraten):** `maybeAddSessionFile(job)` berechnete `formatDateTime(new Date())` UNBEDINGT ganz oben — VOR dem `status==='done'`-Check, der für alles andere früh returnt. formatDateTime macht ZWEI Intl-Locale-Formate (toLocaleDateString+toLocaleTimeString) = ~83µs/Call gemessen. Läuft bei JEDEM Progress-Event (onUploadProgressBatch loopt den M-Item-Batch → handleProgress → maybeAddSessionFile) = 10×M/s, und WIRFT es weg für alle nicht-done-Events. Skaliert exakt mit M (250/s @25 → 610/s @61) und feuert in BURSTS: jeder Batch = M Calls back-to-back = synchroner Main-Thread-Block ~2,4ms@25 → ~5ms@61 alle 100ms → sprengt das 16ms-Frame-Budget → Scroll-Stutter. Per-Event, NICHT per-Render → scroll-unabhängig → erklärt „Lag wenn aktive Zeilen off-screen" exakt. DAS war der 25→50-Cliff.
**Fix:** `const dt = formatDateTime(new Date())` in den `if (!_sessionFileKeys.has(dedupKey))`-Block verschoben → läuft 1× pro echt-neuem fertigen Upload statt pro Progress-Tick. Faithful Blink-Bench am BESTÄTIGTEN Regime (Q=500 virtuell, Progress-Sort, Scrolling, M=25/50/61, OLD vs FIXED): Per-Batch 1,7/3,2/4,1ms (OLD, M-skalierend) → 0/0/0ms (FIXED, flach). Frame-P95 7,3→4,2ms @61. Render/Scroll-Pfad selbst flach ~2,5ms über alle M → KEIN zweiter Knoten dort.
**Regel:** Bei perzeptuellem Lag IMMER zuerst das REGIME erfragen (Datenmenge, aktive Konfiguration wie Sort-Spalte), bevor man benchmarkt oder fixt — eine plausible Mechanik für das FALSCHE Regime zu messen ist exakt der M=10-Fehler. Wenn der User eine konkrete Beobachtung liefert („wenn off-screen"), ist scroll-UNABHÄNGIG (per-event) vs scroll-abhängig (per-render) der Schlüssel-Diskriminator. Und: verschwendete Arbeit auf dem heißesten Pfad (Intl/new Date/Regex/DOM-Query UNBEDINGT berechnet, dann verworfen) ist ein klassischer M-skalierender Lag-Knoten — `formatDateTime` immer hinter den Guard schieben der das Ergebnis tatsächlich nutzt. Prozess-Grenze beachten: Renderer-Jank ≠ Main-Prozess; das Main-ELD-Log sieht Renderer-Lag NICHT.

View File

@ -1,87 +1,527 @@
# Queue-Persistenz Bug: fertige Dateien tauchen nach Neustart wieder auf
# v3.3.108 — session log filename: 6-digit uniqueness suffix
## Symptom
User: 300 Dateien, 100 übrig, Programm schließen + öffnen → manchmal sind bereits
fertig hochgeladene Dateien wieder in der Liste.
User: append a generated 6-digit number to the session log filename, e.g.
26-06-2026-mdu-session-06-02-847581.log. Dropping seconds/pid in v3.3.107 reintroduced same-minute
collision risk on a fast close/reopen.
- formatSessionStamp(date, rand) appends `-${rand}` when rand is supplied (number or string), else unchanged.
- main.js stamps SESSION_ID = formatSessionStamp(new Date(), String(Math.floor(100000 + Math.random()*900000))).
- stripModeStampFromFileName newSessionRe gains an optional `(?:-\d+)?` so the suffix strips back to the base.
- Tests: stamp-with-rand (string + number), strip-with-suffix. 411 pass.
Shipped: gitea v3.3.108 (updater latest.yml verified) + GitHub mirror (lib/log-mode.js, main.js, package.json,
tests/log-mode.test.js only; tag repointed to the sanitized mirror commit, NOT the gitea history commit).
## Root Cause (verifiziert im Code)
- **RC-1 (Persist-Starvation, code-confirmed):** `persistQueueStateSoon()` setzt bei
jedem Progress-Event den Timer per `clearTimeout` zurück; Delay während Upload war
10000ms. Progress-Events feuern öfter als alle 10s → Timer feuert NIE während eines
aktiven Uploads. Der Disk-Snapshot bleibt auf dem Stand VOR Upload-Start stehen
(alle Jobs `preview`).
- **RC-2 (unzuverlässiger Close-Flush):** beforeunload-Sync-Flush existiert
(app.js:4605) und fängt den sauberen Close ab. Bei hartem Kill / Crash / OS-Kill
läuft er nicht → der stale Snapshot bleibt liegen.
- **RC-3 (Dedup-Asymmetrie):** `_autoDeduplicateFromLog` droppt beim Start nur Jobs
mit Status `done`. Die Ghosts aus dem stale Snapshot stehen aber als `preview` da
→ werden NICHT gedroppt → fertige Dateien erscheinen erneut.
---
## Fix (mechanismus-unabhängig, vom Kern auf)
- [x] **FIX A — Timestamp-gated Dedup (Kern-Fix, durable):** Beim Start jeden restored
Job droppen, dessen file+hoster im Log mit `ts >= floor(savedAt)` steht — egal ob
`preview` oder `done`. Fängt Ghosts auch nach hartem Kill (hängt vom Log ab, nicht
vom Snapshot). `lib/queue-dedup.js` additiver 3. Param `savedAt`; `buildPersistedQueueState`
stempelt `savedAt`; `restoreQueueStateFromConfig` merkt `_restoredSnapshotSavedAt`;
Log-Zeile → `ts` geparst; `_autoDeduplicateFromLog` reicht savedAt durch.
- [x] **FIX B — Throttle mit max-wait:** `lib/throttle-timer.js` (neu). Upload: delay 500
+ maxWait 20000 → Snapshot alle ~20s statt nie. Idle: reine Debounce. Fallback-Shim
honoriert maxWait (kein stilles Starvation-Reintro).
- [x] **FIX C — Close-Write-Härtung:** `save-global-settings-sync` renameSync-Retry bei
EBUSY/EPERM/EACCES + pid-unique tmp + tmp-cleanup. Startup-Sweep `_sweepOrphanConfigTmps`
räumt verwaiste `<config>.<pid>.tmp` toter PIDs (gegen Orphan-Akkumulation).
- [x] **Seam-Extraktion (Advisor #2):** `lib/upload-log.js` (neu) — `formatUploadLogLine`
+ `parseUploadLogLine` aus main.js gezogen; Test fährt den ECHTEN Writer→Reader→Gate-
Vertrag (kein Mirror) → fängt künftige Format-/Epoch-Brüche.
# v3.3.107 — session log filename template → DD-MM-YYYY-mdu-session-HH-MM
## Tests
- [x] `tests/throttle-timer.test.js`: Starvation ohne maxWait → 0 Fires; mit maxWait →
periodische Fires; last-write-wins (distinct fn); flushSync/cancel.
- [x] `tests/queue-dedup.test.js`: ts>=savedAt→DROP; ts<savedAtKEEP; same-secondDROP;
max-ts; Multi-Hoster Teilabschluss (reale Bug-Form); ohne savedAt/ohne ts→Legacy.
- [x] `tests/upload-log.test.js`: realer Writer→Reader-Roundtrip + Seam-Drop/Keep.
- [x] 334/334 grün, ESLint clean, Smoke-Boot identisch zu Baseline (kein Regress).
User: change the session log filename from fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log to
DD-MM-YYYY-mdu-session-HH-MM.log (hour-minute, no seconds/pid). lib/log-mode.js:
- formatSessionStamp(date) now returns `${DD}-${MM}-${YYYY}-mdu-session-${HH}-${MM}` (pid arg dropped; main.js
still passes process.pid, harmlessly ignored).
- resolveLogFileName session branch returns `${sid}${ext}` (the stamp is the full app-defined stem, baseName
ignored — single/daily still use baseName 'fileuploader').
- stripModeStampFromFileName recognizes the new format (^DD-MM-YYYY-mdu-session-HH-MM(.ext)$) and resets to the
default 'fileuploader' base (the new format embeds no base); the old daily + old-session strip regexes stay
for backward-compat with any persisted old paths. The compounding round-trip stays idempotent.
Tests updated (formatSessionStamp, resolveLogFileName session, strip new-format, idempotency). 410 pass.
## Review
- **Adversariale Multi-Agent-Review (4 Dimensionen, 15 Findings):** 14 refuted (meist
"ist korrekt"-Bestätigungen, Kommentar-Drift, Test-Härtungs-Vorschläge). 1 confirmed
(LOW): pid-unique tmp konnte bei Hard-Kill zwischen write und rename verwaisen → mit
Startup-Sweep behoben. Stale-Kommentare (queue-dedup Header + _autoDeduplicateFromLog)
auf die Zwei-Regel-Logik korrigiert.
- **Was bewiesen ist:** Komponenten-Logik (Unit-Tests inkl. realer Format-Seam),
Code-getraceter Wiring-Pfad, adversariale Gegenprüfung. Der Fix ist
MECHANISMUS-UNABHÄNGIG: greift egal ob der stale Snapshot von Starvation, einem
Mid-Upload-Close-Race ODER einem Hard-Kill kommt.
- **Ehrliche Einschränkung:** KEIN Live-Repro mit echtem byse-Key (Key unter anderem
Windows-Profil verschlüsselt, nicht entschlüsselbar). Symptom tritt nur auf bei
Close WÄHREND aktivem Upload oder Hard-Kill — ein sauberer Idle-Close war schon
vorher korrekt.
---
## Runde 2 — "noch intensiver" (v3.3.81)
- **Echte ausführbare Tests** für bisher nur logisch abgedeckte Pfade: `tests/orphan-tmp.test.js`
(Sweep-Entscheidung, extrahiert nach `lib/orphan-tmp.js`), config-store `pendingQueue`+`savedAt`
Roundtrip, `tests/queue-persistence-scenario.test.js` (exakte 300/100-Bug-Form + multi-hoster),
`tests/queue-dedup-property.test.js` (3000+500 Fuzz-Iterationen gegen die formale Invariante).
- **Breite adversariale Bug-Jagd** übers GANZE Subsystem: lief tooling-bedingt teils kaputt
(bug-analyzer ohne File-Tools, Socket-Fehler) → 3 unverifizierte Hypothesen SELBST am Code
geprüft:
- #2 (gedroppte done-Jobs reappear via buildQueuePreview) → REFUTED: `_completedUploadKeys`
(full path) == buildQueuePreview-Key; nach Gate räumt `syncSelectedFilesFromQueue` selectedFiles.
- #3 (done-File bleibt in selectedFiles → re-preview) → **REAL (Advisor-Catch) & gefixt.** Meine
erste Abweisung war zu schnell: `syncSelectedFilesFromQueue` läuft NICHT beim Mid-Upload-Close.
Bei `removeFromQueueOnDone=ON` werden fertige Jobs aus queueJobs entfernt, bleiben aber in
selectedFiles; `updateUploadView`→`buildQueuePreview` (Startup, Zeile 990) re-materialisiert sie
als Preview-Ghost NACH dem Gate → sticky. Fix: `completedSelectionKeys` (queue-dedup.js) seedet
beim Start `_completedUploadKeys` (full-path, log-basiert/hard-kill-durabel, gleiche Ambiguity-
Guard) → buildQueuePreview überspringt fertige (file|hoster)-Paare. Nur relevant bei
removeFromQueueOnDone=ON (Default OFF).
- #1 (basename-Kollision droppt PENDING Datei = Lost Work) → **REAL & gefixt.** FIX A's ts-Regel
keyt auf basename, Restore-Collapse auf full path → zwei gleichnamige Dateien aus verschiedenen
Ordnern an denselben Hoster: die geloggte droppte fälschlich auch die andere PENDING. **Ambiguity-
Guard** in queue-dedup.js: ts-Regel wird unterdrückt, wenn ein basename|hoster-Key auf mehrere
DISTINKTE Pfade zeigt (done-Regel unberührt). Fail-safe: schlimmstenfalls überlebt ein
sichtbarer Ghost, NIE stiller Datenverlust.
- **Un-gehuntete Bereiche selbst abgeklopft:** DST/Clock-Skew → Fehler nur in SICHERER Richtung
(Ghost bleibt, kein Lost Work), inhärente Grenze von Sekunden-Lokalzeit-Logs. Log-Discovery
readdir-Filter `startsWith(base)&&endsWith(ext)` fängt single/daily/session — keine verpassten
Log-Files. 353/353 grün, ESLint clean, 3× Suite ohne Flake.
# v3.3.106 — intermittent white-screen on startup (RDP/VM GPU) + export filename
Two asks. (A) White screen: user sometimes gets a PURE-WHITE window on start (no error banner, NOT even the
static menu bar) on a 12-vCPU Windows VM over RemoteDesktop. Workflow wn2k04x5t (2 agents + adversarial verify)
nailed it by one airtight deduction: the BrowserWindow backgroundColor is DARK (#16181c, main.js:1228), so
"white" can NEVER be an un-painted/loading/failed state — those all show DARK. Pure-white + no menu bar + no
banner + SILENT eliminates every DOM/init/CSS/load mode (each leaves the dark styled shell, unstyled-black-on-
white menu text, or the red init().catch banner) → the ONLY match is a GPU/compositor surface failure on the
RDP virtual display adapter. Confirmed: NO disableHardwareAcceleration / disable-gpu / appendSwitch ANYWHERE,
and child-process-gone (GPU) is log-only (matches the silent symptom) while render-process-gone pops a dialog.
Renderer has ZERO WebGL/canvas/video (audited) → software compositing costs ~nothing and doesn't undo the perf
work; a webContents.reload() doesn't disrupt uploads (uploadManager lives in main, torn down only on quit).
Watchdog REJECTED: the GPU mode lets init complete, so an init-complete signal wouldn't detect the white screen.
SHIPPED v3.3.106 (main.js — adversary's safe subset):
- app.disableHardwareAcceleration() at module top (before app.whenReady) GATED on RDP (process.env.SESSIONNAME
matches /^RDP/) OR a persisted gpu-disabled.flag (in userData). Zero-regression for local/console users.
- Auto-heal: on a GPU child-process-gone, write gpu-disabled.flag → next launch disables HW accel even if the
RDP gate missed (covers VM-via-console / bad virtual GPU). Self-healing after at most one white screen.
- Kept the child-process-gone/render-process-gone/did-fail-load instrumentation to CONFIRM on the server log
(caveat: root cause is the standard RDP-GPU bet, not yet confirmed on the affected machine — next white-start
log shows CHILD PROCESS GONE type=GPU = confirmed).
- (B) export-backup defaultPath: multi-hoster-backup-YYYY-MM-DD.mhu → DD-MM-YYYY-multihoster-backup.mhu.
409 tests pass, clean boot (guard inert on non-RDP dev machine).
DEFERRED (perf polish, workflow w5o2rpffx design ready): history JSONL + batch-start residual.
---
# v3.3.105 — URGENT data-safety: config write fsync + account-wipe guard
User report: after a server CRASHED during upload (NOT the v3.3.104 update — the other server updated fine and
kept its accounts), the accounts/credentials were gone. Root-cause chain (in code): config writes were atomic
(tmp+rename) but had NO fsync — a hard crash can leave electron-config.json truncated/unflushed → on restart
load() reads the corrupt/empty file, falls to .bak, and if that's also bad returns empty DEFAULTS → the next
settings/queue save persists EMPTY hosters → accounts permanently wiped (and the async _atomicWrite blindly
copyFileSync'd the live → .bak, so an empty live could clobber a good .bak).
SHIPPED v3.3.105 (lib/config-store.js + main.js — data-safety, no behavior change):
- fsync before rename in BOTH write paths: _atomicWrite (openSync+writeSync+fsyncSync+closeSync, then
guarded-.bak + rename) and main.js save-global-settings-sync (openSync+writeSync+fsyncSync+closeSync). A hard
crash can no longer leave a truncated config.
- _atomicWrite .bak is now GUARDED: read the live file and only refresh .bak if it's non-trivial (trim>2) —
an empty/truncated live can never clobber a good .bak (matches what the sync-save already did).
- WIPE-GUARD (_guardHosters): in save()/saveRotationCursors()/the sync-save, when the write does NOT
intentionally set hosters (config.hosters absent) AND the resulting hosters are all-empty, recover the
hosters from disk (_recoverHostersFromDisk tries live → .bak → .pre-history-split.bak) instead of persisting
the wipe. An EXPLICIT save({hosters:{}}) (user deleted all) is still allowed (hostersIntentional=true).
- load() gained a 3rd fallback tier: .pre-history-split.bak (the permanent v3.3.99 snapshot with accounts) so
load() itself recovers after corruption.
2 new tests (post-wipe valid-empty live + .bak → guard restores; explicit empty NOT blocked). 409 tests pass.
RECOVERY for the affected server: %APPDATA%\multi-hoster-uploader\electron-config.json.pre-history-split.bak
(or .bak) → copy over electron-config.json with the app closed.
DEFERRED (perf polish, workflow w5o2rpffx designs ready): history JSONL (append-only, kills per-batch 185MB
rewrite + first-open parse via loadHistoryRecent tail-read + meta sidecar) and the batch-start one-time
render/231ms residual. Do these AFTER the data-safety fix is confirmed stable.
---
# v3.3.104 — virtualize the Recent-uploads panel (the last non-virtual table)
v3.3.103 log (real 2464-job, 4-hoster batch → 95 concurrent): the statSync fix HELD (batch-start main spike
336→231ms with 10× more jobs), and the whole 90s ramp to 95 active was PRISTINE (mean ~11ms, fps=32,
longtasks=0). Residual: rapidly clicking tabs DURING the 95-active upload → renderer-longtask 210-221ms
(proc=0ms = layout). Cause: the Recent-uploads panel (renderRecentUploadsPanel) rendered ALL sessionFilesData
rows (≤2000) into the DOM non-virtualized — the exact analog of the History table pre-v3.3.102. Switching to
that view laid out ~2000 rows.
Workflow w23318hm0 hit transient 529 overload (no cached results); did the fix directly using the proven
History-virtualization template + Playwright empirical verification (stronger than agent review for layout).
SHIPPED v3.3.104 (renderer/app.js + styles.css):
- Virtualized renderRecentUploadsPanel mirroring History/_renderVirtualRows: tbody#recentFilesBody gets only
~visible rows + top/bottom spacer <tr> (VIRTUAL_ROW_HEIGHT=28, OVERSCAN=10). Scroll handler (_onRecentScroll
rAF-coalesced) + ResizeObserver on .recent-files-table-wrap (doubles as show-trigger). _recentWorking holds
the sorted set. DROPPED the insertAdjacentHTML append-only fast path (a ~40-row window re-render is cheap);
every render re-renders the visible window. Scroll-position preserved on prepend (date|desc: scrollTop=0 at
top, else += added*ROW_HEIGHT).
- SELECTION SAFE: _buildRecentRowHtml already stamps `selected` from selectedRecentIds.has(row.order) per row,
so off-screen-selected rows render selected when scrolled in; selectedRecentIds stays the source of truth;
shift-select already uses _recentSortCache (not the DOM). applyRecentSelectionClasses toggling only visible
rows is correct.
- styles.css: .recent-file-row { height: 28px } so the virtualization math is exact (table already had
table-layout:fixed, so no column-jump fix needed unlike History).
- Empty-state guard: _renderRecentVirtualRows returns early when total=0 so it never wipes the "Noch keine
Uploads" message.
- PLAYWRIGHT-VERIFIED @2000 rows (bounded container): show-cost 118ms→2.4ms, DOM stays 29-39 rows, scroll maps
correctly (row1500→window@1486), scrollHeight exact (56014≈56000), off-screen-selected renders with class,
row height exactly 28. 407 tests pass, clean boot.
Every large table is now virtualized (Queue, History, Recent). DEFERRED still (one-time/minor): batch-start
~500ms first-render + residual 231ms main spike for 2464 jobs (one-time per batch); first-get-history-after-
batch parse-cache/JSONL.
---
# v3.3.103 — kill the batch-start 336ms main stall (synchronous statSync storm)
v3.3.102 log (real 224-job batch): History virtualization CONFIRMED (no get-history on tab switch),
steady-state pristine (fps=32, ELD 11.8ms). Residual = a ~6s BATCH-START spin-up burst: main ELD
max=336ms @cpu=0%core + renderer-longtasks 196-391ms; settles to clean by +6s. Workflow w3bzkumo8
(4 agents + adversarial verify) CORRECTED my hypothesis:
- My "uncapped renderer progress-drain" theory was WRONG: handleProgress only mutates JS + SCHEDULES
coalesced renders (rAF/200ms); render is already one-per-frame; main coalesces to ~50 latest-per-job/100ms.
Chunking the drain fixes nothing. ADVERSARY FOUND IT WOULD REGRESS: rAF throttles to ~0 when the window is
minimized (the common background-uploader state) → unbounded _pBuf backlog AND deferred persistQueueStateSoon
(last line of _handleProgressImpl) → terminal 'done' lost on close = the queue-persistence-ghost-fix class.
DEFERRED/REJECTED as sketched.
- REAL cause (cpu=0%core = blocked on I/O): synchronous fs.statSync storm in UploadManager.startBatch dedup
loop (lib/upload-manager.js:363-377): up to DEDUP_CHUNK=200 fs.statSync in ONE tick before yielding. 200 ×
~1.68ms (measured on the VM) = the exact 336ms. Plus a per-job statSync (428). On a disk already saturated
by the 1MB read-ahead.
SHIPPED v3.3.103 (lib/upload-manager.js — adversary's zero-risk headline fix, but the more thorough async form):
- Dedup loop: dedup synchronously (cheap Map ops), then stat the unique files in PARALLEL via
`await Promise.all(toStat.map(f => fs.promises.stat(f)))` per chunk → stats run on the libuv threadpool,
main thread NEVER blocks. Preserves the exact results-Map shape {name,size,results:[]} + dedup semantics
(size=0 on failure). The 336ms sync block → 0 main-thread block.
- Per-job statSync (428) → `await fs.promises.stat` (in an async fn before the first real await; cachedResult
fast-path already skips it for ~all jobs — consistency only).
- Tests: updated the fs.statSync mocks in upload-manager.test.js (2 sites) + suspect-reject-alternates.test.js
to also mock fs.promises.stat (returns the same fake sizes). 407 tests pass, clean boot.
The renderer-longtasks (391/280/299ms) during the burst were (medium-confidence) the user's OWN tab clicks
landing while the main thread was stalled — fixing the main stall frees IPC so those clicks stay responsive.
DEFERRED still (only if needed): first-get-history-after-batch parse-cache/JSONL; the renderer chunked drain
ONLY if ever needed for interaction-responsiveness AND gated on a high-water-mark sync drain + a
document.hidden setTimeout fallback (never rAF-only).
---
# v3.3.102 — virtualize the History table (the last tab-switch layout cost)
v3.3.101's gate killed the get-history PARSE on tab switch, but the v3.3.101 log showed a RESIDUAL: tab
clicks still 216ms with a ~197ms `renderer-longtask` and NO get-history (gate worked). proc=0ms → pure
browser layout, not JS. Cause: `.view{display:none}`→`.active{display:flex}` + the History table builds up
to 2000 `<tr>` NON-virtualized (renderHistoryTable), so showing the view lays out 2000 rows (~197ms on the
RDP VM). The queue was already virtualized; history was the only non-virtual large table.
MEASURED with Playwright (real DOM, this machine; VM ≈1.7×):
- current 2000 rows auto-layout: 118ms ; content-visibility+fixed: 117ms (USELESS — rows still laid out)
- cap 300: 16ms (but rejected: history rows are per-file×hoster, a single 1280-file batch ≈3840 rows, so a
small cap would HIDE a recent batch's links)
- virtualize (40 visible of 2000): 2.3ms ✓
Verified the virtualization end-to-end @6000 rows: showCost 1.1ms, DOM stays 32-42 rows, scroll maps
correctly (top=row0/mid=row2990/bottom=row5999), scrollHeight exact, columns STABLE (table-layout:fixed),
rows update on scroll.
SHIPPED v3.3.102 (renderer/app.js + styles.css):
- Virtualized renderHistoryTable mirroring the queue's _renderVirtualRows: header always rendered, tbody#historyBody
gets only visible rows + top/bottom spacer `<tr>` (VIRTUAL_ROW_HEIGHT=28, OVERSCAN=10). Scroll handler
(_onHistoryScroll, rAF-coalesced) + ResizeObserver on #historyContainer — the ResizeObserver doubles as the
show-trigger (hidden 0×0 container → visible size → re-render at correct height). Sort resets scrollTop=0 +
re-renders. Click delegation (copy-link / sort) unchanged. _historyWorking holds the sorted working set.
- styles.css: `.history-table{table-layout:fixed}` + scoped col widths (16/34/12/38%) so columns don't jump as
rows scroll in/out (does NOT touch the shared .col-* used by the queue). Measured: content-visibility was a
no-op, so NOT used.
All rows stay scrollable (no UX loss); show cost ~100× lower. 407 tests pass, clean boot. Playwright-verified.
DEFERRED still (only if needed): the FIRST get-history after a new batch parses 185MB once (~450ms) — needs
the ConfigStore parse-cache (+185MB RAM, guarded) or JSONL. appendHistory still rewrites 185MB per batch-done
(JSONL fixes that). The ~625ms batch-start spin-up + 107ms debug-log residuals.
---
# v3.3.101 — History-tab lag: gate the unconditional reload + fix the diagnostics history regression
v3.3.100's interaction instrument named the residual exactly: EVERY slow click was `button.tab`/`nav.tab-bar`
(200840ms), each coupled to `ipc get-history wall=150-200ms sync` + `main-longtask blocked=242-289ms
lastIpc=get-history` + `renderer-longtask 200-248ms`. Uploads themselves pristine (ELD mean 11.7ms; the only
spikes line up with the get-history tab-switches). Workflow wgxr06myb (4 agents + adversarial verify):
- get-history fires ONLY entering the History tab (not every tab) — but the handler called loadHistory()
UNCONDITIONALLY (app.js:362-365) despite tracking `_historyDirty` and never checking it. Each call:
synchronous readFileSync+JSON.parse of the ~185MB / 30000-entry electron-history.json (no cache), ships all
30000 over IPC, renderer flattens ~120000 row objects then .slice(-2000) for the DOM (DOM already capped 2000).
- Adversary safe subset = STEP 1 ALONE (gate the load, dirty-coverage verified complete: every append routes
batch-done→appendHistory + upload-batch-done→handleBatchDone sets _historyDirty=true). Zero risk.
SHIPPED v3.3.101 (STEP 1 + a regression fix, both safe):
- renderer/app.js: gate `if (tab.dataset.view==='history' && (_historyDirty || !_historyEverLoaded)) loadHistory()`;
added `_historyEverLoaded`, set both flags inside loadHistory() AFTER the await succeeds (retry on failure).
→ REPEAT History tab-switches (no new uploads) now do ZERO ipc/parse/flatten = instant. (Honest limit: the
FIRST History open after a new batch still parses 185MB once ~450ms — needs the parse-cache, see below.)
- lib/diagnostics-collectors.js + main.js: getHistory now reads loadHistory() not load().history — fixes a
CORRECTNESS regression I introduced in v3.3.99 (migrated mode → load().history is [] → remote diagnostics
reported totalBatches:0 despite 30000 real batches). Backward-compatible fallback kept. +2 regression tests.
407 tests pass, clean boot.
DEFERRED (adversary-flagged, by design — do only if the next log/user still shows pain):
- STEP 2(1) ConfigStore parse-cache for history (mtime+size key, invalidate BOTH _writeHistoryFileAtomic AND
_writeHistoryFileDurable, slice-before-push for 'all'-retention same-ref aliasing). Makes first-after-upload
switch instant + appendHistory read-half free, but ADDS ~185MB resident in main (NOT a relocation — adversary
corrected the design's false RAM claim).
- STEP 2(2) slice get-history to last-N-batches: REGRESSION VECTOR (breaks browse/sort-all 30000), needs a
net-new paging/search-in-main IPC + UI that doesn't exist. Defer.
- JSONL append-only storage: the ONLY thing that kills appendHistory's 185MB-rewrite-per-batch-done AND the
parse entirely (tail-readable). On-disk format migration → own careful build.
- Two minor independent residuals from the sweep: ~625ms batch-start spin-up (synchronous 22-job build/prime
burst in one tick) + 107ms debug-log block mid-batch. Separate, low priority.
---
# v3.3.100 — close the LAST measurement gap: renderer interaction timing (switches/clicks)
User asked "haben wir wirklich ALLES gemessen, auch switches/wechsel?". Audit: main-side was already
fully covered (IPC wrapper ≥50ms on every handler, main-longtask >100ms with lastIpc, config instrument);
account switchAccount is a trivial sync Map-set + the rotation work is async (can't block) → already covered.
The REAL gap was RENDERER-side: renderer-perf was upload-gated (idle clicks unmeasured) and only aggregate
(no per-interaction latency, no element attribution). Closed it (renderer/app.js, additive, self-silencing):
- Event Timing API observer (`type:'event', durationThreshold:50, buffered`) → `renderer-interaction <type>
dur=Xms proc=Yms target=<el>` for EVERY UI interaction ≥50ms (switch/sort-header/tab/button), always-on,
names the element (id/class/data-action/aria-label). The direct "click→reaction" latency.
- Idle renderer-longtask logging: any longtask ≥100ms logged immediately (`renderer-longtask dur=Xms`),
not just during uploads.
405 tests pass, clean boot. Now EVERY action — main or renderer, idle or under load — names itself if slow.
---
# v3.3.99 — THE KILL: 38.5MB config-thrash → history split out of the hot config + full instrumentation
THE ROOT CAUSE (from v3.3.98's instrument, the real "bread"): electron-config.json was **38.5MB** and got
loaded/cloned/serialized **137× in 73s** on the main thread (140-592ms each) = **~47% main-thread occupancy**
→ that IS the 1-2s button lag. The 1MB read-ahead was irrelevant against it. NOT the queue (`queue=undefined`
was a LOGGING BUG: read `.length` on the pendingQueue OBJECT); the bulk is HISTORY — each batch-done appended
the full per-file result list (`summary.files` w/ per-hoster URLs), 75 batches, default historyRetention='all'
never prunes → unbounded. (5-agent workflow wz2g4bwka + adversarial verify; bench fixture confirmed 185MB =
100MB history/30000 entries.)
Why writes drove the storm: save-global-settings (queue-persist) did 2 loads + 1 serialize per call, and
_atomicWrite NULLS the cache → next load is a full 38.5MB reparse. Even cache HITS structuredCloned 38.5MB.
Per-job upload path makes ZERO config calls (selectUploadAuth/rotation take config by param) — pending=1280
is NOT the driver.
THE FIX (history split — kills clone AND serialize AND reparse at once; advisor-gated, adversarially verified):
- History moved to its OWN file **electron-history.json** (lib/config-store.js). _migrateHistory() runs ONCE
at init (packaged only), fail-safe: write history.json + fsync + verify count BEFORE the config is ever
allowed to drop history, keep a permanent `electron-config.json.pre-history-split.bak`. _loadImpl returns
`history:[]` when migrated → cached result is tiny → clones cheap; config file shrinks to ~KB on the first
save() → reparse cheap; _serializeForDisk writes ~KB → serialize cheap. loadHistory/appendHistory/
pruneHistory/clearHistory redirected to history.json (own write-queue, no-clobber guard); legacy config
path kept as fallback when migration fails. get-history/export-history go through loadHistory.
- REAL 194MB-FIXTURE VALIDATION: migrate 1.5s (1×), all 30000 entries preserved + .bak; load() 631ms cold
(1×) → **0.1ms** after strip; save() strips config **185MB→2.1KB**; loadHistory() still 30000. RESULT:
data-preserved=true, hotpath-fast=true.
- DROPPED per advisor (one-variable + risk): loadShallow (moot after split), Fix#2 cache-repopulate (dead
gate), Fix#4 resolution-cache (stale-cache → rotation/byse failover-regression class — the one thing that
could SILENTLY corrupt uploads). Fix#3 (this split) was the only complete fix.
"MEASURE EVERYTHING" instrumentation (user demand) — all additive, threshold-gated, MHU_PERF=0 to disable:
- IPC handler wrapper (monkey-patch ipcMain.handle/.on) → `ipc <channel> wall=Xms sync=Yms` ≥50ms = the
button-press→response latency, hardened so a logging throw can never break IPC (Promise.resolve(p).finally).
- Main-process long-task drift monitor (setInterval 100ms) → `main-longtask blocked=Xms lastIpc=… gc=… gcMax=…`
for any single main-thread turn >100ms (catches GC, fs scans, serialize the IPC wrapper structurally can't see).
- config-store: caller attribution `via=<stack>` + `wqDepth=` on config-load/config-serialize lines; FIXED the
`queue=` logging bug (now reads pendingQueue.queueJobs.length).
405 tests pass (9 new migration tests covering preserve-count, round-trip, save()-never-loses-history,
crash-window fallback, idempotency). Clean Electron boot (9s, no errors). No repo pollution (migration packaged-only).
NEXT LOG must show: config-load/config-serialize wall= drop to single digits (or vanish), main-longtask rare,
ipc lines name any residual. If a residual remains it's pendingQueue (own follow-up, not a regression).
---
# v3.3.98 — read-burst absorption (1MB hwm) + persist/load instrument; B (renderer) DEFERRED
v3.3.97 (threadpool 64→8) was a DECISIVE win: mean ELD 200ms→~11ms at 70 active (18×), renderer healthy
14/15 windows. User: "ganz flüssig isses noch nicht". A 5-agent ultracode workflow + adversarial verify
localized the RESIDUAL to TWO distinct, measured spike sources (full data: subagents output wjskjo1xk):
1. READ-BURSTS (tail W13/14/15, 15:18:53-19:04): FSReqCallback 66/70/46 vs threadpool=8 (~8.75× queue
depth), SimpleWriteWrap collapses to 7/4/24, mean climbs 12.9→30.3→41.9ms. GC EXCLUDED (gcMax ≤27ms
always). The FSReq↔SimpleWrite inversion at stable active=70/pending=1287 proves reads are CAUSAL, not
a symptom of a block elsewhere.
2. SYNC CONFIG PERSIST (suspected): save()→load() reparses the WHOLE electron-config.json (1287-job
pendingQueue nested in globalSettings + full history) on every persist because _atomicWrite nulls the
cache; _serializeForDisk JSON.stringify(...,null,2) of all of it. W13's single 1021ms max with heap→142MB
fits a big synchronous structuredClone+stringify. CAVEAT (advisor): W13 is ONE confounded sample (also
FSReq=66) and the ONLY heap-spike window; W4(415ms,heap41) & W10(852ms,heap18) are LOW-heap → NOT persist
clones → likely the SECONDARY suspect: account-failed's synchronous configStore.load() per failure near
connection churn (W6 teardown had doodstream connect-timeouts). So: INSTRUMENT, don't claim "found a 1s
freeze".
SHIPPED v3.3.98 (one-variable discipline — advisor cut B to keep the next measurement clean):
- A: highWaterMark 256KB→1MB in all 5 streaming read loops (hosters.js:291, doodstream:342, voe:245,
vidmoly:190 CHUNK_SIZE consts; clouddrop:108 inline — NOT clouddrop:12's 16MB server chunk). Keep tp=8.
Deepens per-stream read-ahead 0.43s→~1.7s (absorbs threadpool-queue latency so writes don't starve),
4× fewer read completions + allocs. Zero multipart byte-risk (Content-Length=preamble+fileSize+epilogue,
independent of chunk size). REVERSIBLE PROBE; read-semaphore held in reserve (trigger: FSReq still ~70 +
writes starved + mean elevated after 1MB).
- C-instrument (BROADENED per advisor): config-store.js times load() (full reparse, incl. account-failed
path) AND _commit serialize; logs `config-load wall=Xms cache=hit/miss hist=N queue=M` and
`config-serialize wall=Xms bytes=Y hist=N queue=M` when ≥20ms (perfLog hook set in main.js via
configStore.setPerfLog→logInfo). load() split into wrapper + _loadImpl. 397 tests pass.
- B (renderer chunked rAF batch drain, app.js:188-193 — the 243ms longtask at W14) DEFERRED: renderer was
healthy 14/15 windows and the one longtask is DOWNSTREAM of the main-thread read-burst flooding IPC.
Fix A should make it self-heal. Bundling B would confound attribution + touches the progress hot path
that bit before (formatDateTime burst, ghost-fix). Add B next round ONLY if renderer still janks after A.
NEXT LOG answers 3 things cleanly: (1) did A kill the read-bursts (FSReq per-window + tail mean drop)?
(2) is the persist/load actually heavy (new config-load/config-serialize lines + their wall/queue/hist)?
(3) did the renderer self-heal from A alone (longtasks back to 0)? Then decide: persist refactor for v3.3.99
(queue-out-of-config OR cache-repopulation — latter lower-risk but renderer's incoming globalSettings isn't
default-merged like load() produces, so confirm merge-equivalence first), and/or B, and/or read-semaphore.
---
# v3.3.97 — DECISIVE ELD finding: file-read phase-flip + threadpool 64→8 + GC instrument
The v3.3.96 `eventloop-delay` logs gave the decisive signal. At CONSTANT active-count, the system flips
between two regimes:
- HEALTHY (ELD ~11ms, rss 268308MB): `SimpleWriteWrap ≈ active`, `FSReqCallback ≈ 01` (write/network-bound)
- BLOCKED (ELD 49217ms, rss 540610MB): `FSReqCallback ≈ active` (6271 file reads in flight), `SimpleWriteWrap ≈ 04`
ELD spike + rss balloon both track `FSReqCallback` → file-read path through the libuv threadpool, NOT crypto,
NOT renderer, NOT GC-alone. All 5 uploaders read identically (256KB createReadStream); byse/dood/voe run
through the GENERIC uploadFile in hosters.js (no dedicated module).
Advisor caveats baked into the build (do NOT skip on re-measure):
1. Causation UNPROVEN — high FSReqCallback could be a SYMPTOM (blocked loop can't drain read-completions).
2. rss math kills "read buffers ballooned": 70×256KB ≈ 18MB, but rss swings ~300MB → heap/object churn (GC).
3. Cheapest discriminator already wired: `UV_THREADPOOL_SIZE` 64→8 (1 line, reversible, NOT an upload cap;
8×256KB reads ≈ 100MB/s ≫ 41MB/s aggregate). Suspect tp=64 made it WORSE (removed read-serialization).
Shipped v3.3.97 = candidate-fix + discriminator in one build:
- main.js:1 `UV_THREADPOOL_SIZE` 64→8.
- ELD line now also logs `heap=`(heapUsed) `ext=`(external) `ab=`(arrayBuffers) `gc=`/`gcTotal=`/`gcMax=`ms
(PerformanceObserver entryTypes:['gc'], reset per window).
DECISION RULE for the next user log:
- ELD drops with tp=8 → read over-parallelism confirmed → keep 8 or productionize a DEDICATED read-semaphore.
- ELD high + gcTotal/gcMax align with spikes → heap churn → hunt the allocator (semaphore would be wasted).
- ELD high + gc flat → causation reversed (symptom) → pivot.
WAIT for the next `eventloop-delay` log before any read-path refactor. NO upload cap (user rejected it).
---
# v3.3.94 — comprehensive measurement build (user: "mach alles messen was man messen kann")
Localization so far (each step EMPIRICAL, not by elimination — advisor caught the elimination-leap):
- Renderer queue render PROVEN cheap: loaded the REAL app.js in headless Chromium (Playwright) with a mocked
window.api, populated Q=1000 / 61 active / progress sort, drove the real onUploadProgressBatch +
renderQueueTable + scroll → ALL <0.5ms. (Caveat: component cost, not frame rate.)
- User CONFIRMED the discriminator: a full 1000-row queue scrolls SMOOTH when idle, ruckelt ONLY while ~6170
uploads are active → the lag is driven by the active uploads (main-process / system load), not the table.
- Screenshot: 70 connections, 1413 files, 41.3 MB/s, "write ECONNRESET". ECONNRESET is already classified
transient (upload-manager _isTransientNetworkError line 171 → retried, not account-fatal) — it's the
SIGNATURE of oversubscription (servers RST the excess connections). Same root cause as the lag.
Immediate user lever (already exists): Settings → Uploads → "Globale parallele Uploads" (parallelUploadCount,
global semaphore, default 0=off). Capping total concurrent uploads (~20) should fix lag AND ECONNRESET AND
likely keep throughput (bandwidth-limited at 41 MB/s; reset connections waste bandwidth on retries).
SHIPPED measurement (all additive, zero upload-behavior change) to pinpoint CPU-vs-IO vs renderer from the
user's REAL 70-connection run:
- main.js ELD line now also logs: cpu=X%core (process.cpuUsage delta / wall, >100% = multi-core),
rss=YMB, active-by-hoster={dood:.., voe:.., ...} (per-hoster live connection distribution → shows which
hoster is oversubscribed), transient-errs=N (cumulative ECONNRESET-class on the primary path), pending=M.
- lib/upload-manager.js: getDiagnostics() {activeByHoster, transientErrors, pending, active}; activeEntry
now carries hoster; _transientErrorTotal++ in the primary catch when _isTransientNetworkError.
- renderer/app.js: PerformanceObserver('longtask') + a rAF frame-time monitor → logs every 5s WHILE
uploading: `renderer-perf active=N fps=X jankFrames=Y worstFrame=Zms longtasks=W maxTask=Vms`. This is the
DIRECT renderer ground truth (the component-timing harness couldn't capture real frame rate). Low fps /
high jankFrames / longtasks → renderer IS blocked; ~60fps + no jank while it still feels laggy → it's the
main-process/system, and the cpu=/eld= numbers in the same log say CPU-bound (→ workers/cap) vs IO-bound.
Both logs land in the normal debug log (logInfo / window.api.debugLog). 397/397 tests, eslint clean.
NEXT: user runs the 70-load on v3.3.94, shares the `eventloop-delay` + `renderer-perf` log lines (or connects
diagnostics). Those two lines together localize it definitively. Do NOT build workers/cap before that.
---
# v3.3.93 — THE renderer lag knot FOUND + FIXED + MEASURED: formatDateTime per progress event
User gave the decisive data: "25 connections okay, 50+61 laggt, EVTL wenn die uploadenden Zeilen nicht im
Bild sind." Two regime facts (asked, not assumed — advisor caught the assume-the-regime trap a 3rd time):
queue = 2001000 rows (VIRTUAL mode) + sort = clicked PROGRESS/SPEED (dynamic). This killed the non-virtual
reflow theory (off-screen rows aren't in the DOM when virtual) AND pointed at the per-event path.
ROOT CAUSE (renderer process, NOT main — the v3.3.91 ELD log can't see this): `maybeAddSessionFile(job)`
computed `const dt = formatDateTime(new Date())` UNCONDITIONALLY at the top, before the `status==='done'`
check that early-returns for everything else. formatDateTime does TWO Intl locale formats
(toLocaleDateString + toLocaleTimeString) = ~83µs/call MEASURED. It runs on EVERY progress event
(onUploadProgressBatch loops the M-item batch → handleProgress → _handleProgressImpl → maybeAddSessionFile),
i.e. 10×M/sec, and THROWS IT AWAY for all non-done events (the overwhelming majority while uploading).
- Scales exactly with M (active count): 250/sec at M=25 → 610/sec at M=61.
- Bursts: each progress batch runs M calls back-to-back = a SYNCHRONOUS main-thread block of
~2.4ms (M=25) → ~5ms (M=61) every 100ms, on top of render+sort → blows the 16ms frame budget → scroll
stutter. Scroll-independent (per-event, not per-render) → matches "lag when actives off-screen" exactly.
This is the 25→50 cliff.
FIX: move `const dt = formatDateTime(new Date())` inside the `if (!_sessionFileKeys.has(dedupKey))` block, so
it runs ONCE per genuinely-new completed upload, never per progress tick.
VERIFIED (faithful Blink benchmark at the CONFIRMED regime: Q=500 virtual, dynamic progress sort, scrolling,
M=25/50/61, OLD vs FIXED): per-batch cost 1.7/3.2/4.1 ms (OLD, scales with M) → 0.0/0.0/0.0 ms (FIXED, flat).
Frame P95 7.3→4.2 ms at M=61. M-scaling ELIMINATED. The render/scroll path itself is flat ~2.5ms median
across all M → NO second knot there. updateStatusBar/StatsPanel = one cached O(Q) arithmetic pass (cheap);
updateQueueActionButtons = O(selection) (cheap). No other Intl/Date on any per-event/per-frame hot path
(2582 = job-log modal, 4587 = History view — both on-demand/cold). 397/397 tests pass, eslint clean.
NOTE: the v3.3.91/92 main-process event-loop-delay instrument is for the OTHER (CPU-vs-IO) hypothesis and is
a separate process — keep it; it still answers whether the main thread also saturates at 50+ TLS streams.
---
# High-concurrency lag audit (v3.3.90) — "lag ist immernoch da, ich vermute ab X gleichzeitig muss er alle Zeilen gebündelt updaten"
Method: 44-agent high-concurrency audit of the full upload→IPC→render path + Blink benchmark of the
renderer queue table (Playwright/Chromium = same Blink engine), targeting the user's NEW hypothesis:
"with ~100 concurrent uploads the renderer has to update ALL rows bundled rather than cleanly per-row."
## The user's hypothesis is MEASURED-REFUTED — the renderer is NOT the bottleneck.
Blink benchmark over scenarios Q=150..1000, M=10 active, 60 ticks each:
- renderQueueTable virtualizes at ≥200 rows; <200 = change-detecting in-place update.
- _updateRowInPlace is change-detecting (no forced reflow, no layout reads).
- median render <1 ms at Q=1000; only ~4/60 renders are full rebuilds even with progress-crossing sorts.
- progress is coalesced main-side (_progressByJob Map keyed by jobId + 100ms flush → one batch sized by
active-job count, ~10/sec); renderer iterates the batch with cheap per-row handleProgress.
DOM amplification is ruled out by measurement. "Laggy at ~40% CPU / 8 cores" = ONE core at 100% =
main-thread saturation / synchronous blocking, not DOM.
## SHIPPED (v3.3.90) — the two real main-thread blockers, both behavior-preserving
1. lib/clouddrop-upload.js `_uploadChunked`: was reading each 16 MB chunk with `fs.readSync` SYNCHRONOUSLY
on the main event loop — unique among the 5 uploaders (the other 4 stream async). Each read blocks the
WHOLE loop (~59 ms SSD, 30100 ms slow disk) → freezes all progress/IPC/render/other-uploads, scaling
with the number of concurrent clouddrop uploads. Fits "laggy when uploading, worse with more concurrent."
User uses clouddrop. Fix: `fs.openSync`/`readSync`/`closeSync` → `fs.promises.open` + `await fh.read` +
`await fh.close()`. Byte-equivalence verified by SHA-256 over all chunk-boundary cases (full chunk,
partial last chunk, 2/3/4-chunk, single byte) before shipping — a chunk-read bug = corrupt upload.
2. lib/upload-manager.js rotation-retry (944) + suspect-alternate (1075) progressCb: both called
`_emitProgress` (a synchronous `emit('progress')` + fresh object spread) on EVERY stream chunk
(hundreds/sec per job) — they were missing the 250 ms `lastEmitTime` gate that the primary path (631)
has. With many concurrent uploads in rotation/suspect mode that's real main-thread emit amplification.
Mirrored the gate exactly: activeEntry mutation stays UNGATED (stats/speed-monitor stay fresh), only the
emit is throttled to 4/sec. Behavior-preserving.
397/397 tests pass, eslint clean (1 pre-existing unrelated warning at line 554).
## DROPPED (advisor: measured fine, don't chase perception)
- Lowering the virtual-row threshold below 200: the Blink benchmark shows <200 in-place updates are already
sub-ms; no change warranted.
## DISCRIMINATOR ANSWERED (user, 2026-06-21)
(a) Lag NOT clouddrop-specific — other hosters. (b) Parallel counts RAISED deliberately (10+).
(c) 50+ uploading SIMULTANEOUSLY active. → This is the TRUE high-concurrency main-thread-funnel branch,
NOT clouddrop. v3.3.90 stands but does not target this user's case.
## v3.3.91 — instrument first, don't refactor the upload core off elimination-reasoning
Advisor reframe: two LIVE hypotheses need OPPOSITE fixes — (A) main thread CPU-blocked (TLS/crypto/sync) →
event loop stalls → a cap/workers help; (B) main thread fine but IO-STARVED (libuv threadpool/sockets) →
loop stays responsive, uploads just queue → workers are WASTED, config fixes it. A worker/child-process
upload refactor touches throttle/rotation/abort/progress/credentials and is hard to reverse — DO NOT ship it
off sandbox elimination. One measurement splits the hypotheses and must run in the REAL app.
SHIPPED (both reversible, zero upload-core refactor):
1. main.js: `perf_hooks.monitorEventLoopDelay({resolution:10})` enabled at startup; logged via logInfo every
~5 s WHILE uploading (state==='uploading' && activeJobs>0) as
`eventloop-delay active=N mean=..ms p99=..ms max=..ms stddev=..ms threadpool=..`. Pure numbers, no secret
→ does NOT touch the redaction surface. This is the GROUND TRUTH: high mean/p99 → CPU-blocking → workers
justified; low delay while uploads stall → IO-bound → workers wasted, threadpool/sockets is the fix.
2. main.js (first statement, before require('electron')): `UV_THREADPOOL_SIZE = env || '64'`. Default is 4;
every async uploader feeds undici from fs.createReadStream (+ clouddrop fh.read) and DNS getaddrinfo goes
through the same pool → 50 concurrent vs 4 threads = reads/DNS serialize 4-at-a-time = a hard cliff at a
small connection count = the "ab X connections" symptom. Threads are created lazily on demand → 64-max
costs nothing if unused (zero-risk, reversible). The advisor's prescribed one-env-var hypothesis test.
CAVEAT (honest): synthetic sandbox benches could NOT confirm the threadpool is the bottleneck — pbkdf2 is
CPU-core-bound (masks pool size); DNS .invalid returns instantly; real-RTT DNS showed NO pool benefit because
WINDOWS serializes getaddrinfo via the OS DNS Client service (so on Windows the DNS half of the cliff is
masked by the resolver, though the fs-read half still benefits). This is exactly why the ELD number must come
from the user's real load, not the sandbox. Per-uploader undici Agent audit: clouddrop has a shared
module-level Agent (connections:50); doodstream/voe/vidmoly use the global dispatcher (pooled per origin, NO
per-call agent explosion) — so no agent fix needed.
## v3.3.92 — make the single measurement decisive + breadth audit of un-checked main.js hot paths
Enriched the ELD log line with `process.getActiveResourcesInfo()` as a compact type-histogram:
`eventloop-delay active=N mean/p99/max/stddev ms threadpool=64 resources=K {TCPSocketWrap:50,FSReqCallback:4,...}`.
Now ONE run splits all three readings in a single line: high mean/p99 → CPU-blocked (workers/cap);
low delay + many TCP/FS/GetAddrInfo resources → IO-bound queueing (threadpool/sockets, NOT workers);
low delay + few resources → not saturated (lag elsewhere / perception). Pure numbers, no redaction surface.
Breadth audit this round (4th /goal re-fire, code I wrote, NOT re-measuring cleared render/persist):
- main.js logging (debug/rot/upload): all buffered + ASYNC fs.appendFile (write-guard flag, 500ms timer,
setImmediate re-flush). Sync appendFileSync ONLY in crash/signal/exit handlers (correct there). CLEAN.
- main.js progress coalescing (_progressByJob Map + 100ms batch → one upload-progress-batch via safeSend):
non-terminal = Map.set (keeps latest/job); gated upstream to 4/sec/job. CLEAN at N=50.
- _appendJobLog: capped in-memory ring buffer (Map, FIFO-evict). CLEAN.
- All 5 uploaders: doodstream/voe/vidmoly/clouddrop-simple stream via async createReadStream + for-await +
async throttle.consume; clouddrop-chunked now async fh.read. NONE block the main loop per chunk. CLEAN
(clouddrop's old readSync was the unique outlier, fixed v3.3.90).
## NEXT (gated on the real-app ELD number + user's explicit nod)
User runs their 50-concurrent load once; the enriched `eventloop-delay` log line decides:
- mean/p99 HIGH (tenshundreds ms) → CPU-blocked → propose worker_threads/child-process upload pool OR a
smart concurrency cap (WITH the user's nod — it's hard to reverse and touches credentials/abort/rotation).
- delay LOW while it still lags → IO-bound → threadpool bump already addresses it; if not, look at socket
caps / undici Agent connection limits / per-origin pooling, NOT workers.
Do NOT build the worker refactor before this number exists.

View File

@ -17,6 +17,7 @@ function createStore() {
// We override by setting filePath directly
store = new ConfigStore(fakeApp);
store.filePath = path.join(tmpDir, 'electron-config.json');
store.historyPath = path.join(tmpDir, 'electron-history.json');
return store;
}
@ -253,6 +254,32 @@ describe('ConfigStore', () => {
assert.equal(config.globalSettings.alwaysOnTop, true);
});
it('load() returns independent clones — mutating one result must not leak into the cache', () => {
store.load(); // warm the cache
const a = store.load();
a.globalSettings.alwaysOnTop = true;
a.hosters['voe.sx'].push({ id: 'mutant' });
a.history.push({ id: 'ghost' });
const b = store.load();
assert.equal(b.globalSettings.alwaysOnTop, false, 'mutating a prior load() result must not corrupt the cache');
assert.equal(b.hosters['voe.sx'].length, 0);
assert.equal(b.history.length, 0);
});
it('load() reflects an external file change (mtime/size cache invalidation)', () => {
store.load(); // warm cache on the no-file defaults
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: true } }), 'utf-8');
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'an external write must invalidate the cache');
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: false } }), 'utf-8');
assert.equal(store.load().globalSettings.alwaysOnTop, false, 'a second external write must be seen too');
});
it('save() invalidates the cache so the next load() sees the new value', async () => {
assert.equal(store.load().globalSettings.alwaysOnTop, false);
await store.save({ globalSettings: { alwaysOnTop: true } });
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'load() after save() must reflect the write');
});
it('backup recovery when main file is corrupted', () => {
// Write valid config first
fs.writeFileSync(store.filePath, JSON.stringify({
@ -266,4 +293,117 @@ describe('ConfigStore', () => {
const config = store.load();
assert.equal(config.hosters['doodstream.com'][0].apiKey, 'from-backup');
});
it('wipe-guard: a settings-only save recovers accounts from .bak when the live config validly has none', async () => {
// Post-wipe state: live config parses fine but has empty hosters; a backup still holds the accounts.
fs.writeFileSync(store.filePath, JSON.stringify({ hosters: {}, hosterSettings: {}, globalSettings: {}, history: [] }), 'utf-8');
fs.writeFileSync(store.filePath + '.bak', JSON.stringify({
hosters: { 'voe.sx': [{ id: 'v1', authType: 'api', apiKey: 'survive-key' }] },
hosterSettings: {}, globalSettings: {}, history: []
}), 'utf-8');
await store.save({ globalSettings: { alwaysOnTop: true } });
const cfg = store.load();
assert.ok(cfg.hosters['voe.sx'] && cfg.hosters['voe.sx'].length === 1, 'guard must restore accounts from .bak, not persist the wipe');
assert.equal(cfg.hosters['voe.sx'][0].apiKey, 'survive-key');
assert.equal(cfg.globalSettings.alwaysOnTop, true);
});
it('wipe-guard: an explicit save({hosters:{}}) (user deleted all) is NOT blocked', async () => {
await store.save({ hosters: { 'doodstream.com': [{ id: 'd1', authType: 'api', apiKey: 'k' }] } });
await store.save({ hosters: {} });
const cfg = store.load();
assert.equal((cfg.hosters['doodstream.com'] || []).length, 0, 'an intentional hosters write must be allowed to empty them');
});
});
describe('ConfigStore history split (electron-history.json)', () => {
let dir;
let s;
function makeStore() {
const st = new ConfigStore({ isPackaged: false, getPath: () => dir });
st.filePath = path.join(dir, 'electron-config.json');
st.historyPath = path.join(dir, 'electron-history.json');
return st;
}
function writeConfigWithHistory(n) {
const history = [];
for (let i = 0; i < n; i++) history.push({ id: `batch-${i}`, timestamp: 1750000000000 + i, total: 3, files: [{ name: `f${i}.mkv` }] });
fs.writeFileSync(path.join(dir, 'electron-config.json'), JSON.stringify({
hosters: { 'byse.sx': [{ id: 'a1', authType: 'api', apiKey: 'k' }] },
hosterSettings: {}, globalSettings: { historyRetention: 'all' }, history
}), 'utf-8');
}
beforeEach(() => { dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cfg-hist-')); s = makeStore(); });
afterEach(() => { fs.rmSync(dir, { recursive: true, force: true }); });
it('migration moves history into electron-history.json, preserving every entry', () => {
writeConfigWithHistory(50);
s._migrateHistory();
assert.equal(s._historyMigrated, true);
assert.ok(fs.existsSync(s.historyPath));
const hist = JSON.parse(fs.readFileSync(s.historyPath, 'utf-8'));
assert.equal(hist.length, 50);
assert.equal(hist[0].id, 'batch-0');
assert.equal(hist[49].id, 'batch-49');
assert.ok(fs.existsSync(s.filePath + '.pre-history-split.bak'), 'a permanent pre-split backup is kept');
});
it('after migration load() excludes history (cheap hot path) but loadHistory() returns the real data', () => {
writeConfigWithHistory(30);
s._migrateHistory();
assert.deepEqual(s.load().history, [], 'history is not carried in the always-loaded config');
assert.equal(s.loadHistory().length, 30);
});
it('appendHistory writes to history.json; the next config write strips stale history from the config file', async () => {
writeConfigWithHistory(10);
s._migrateHistory();
await s.appendHistory({ id: 'new-batch', timestamp: 1750000099999, total: 1, files: [{ name: 'x.mkv' }] });
assert.equal(s.loadHistory().length, 11, 'append goes to history.json');
await s.save({ globalSettings: { alwaysOnTop: true } });
const onDisk = JSON.parse(fs.readFileSync(s.filePath, 'utf-8'));
assert.ok(!onDisk.history || onDisk.history.length === 0, 'a config write strips stale history from the config file');
assert.equal(s.loadHistory().length, 11, 'history.json is unaffected by the config write');
});
it('save({globalSettings}) after migration NEVER loses history (data-loss invariant)', async () => {
writeConfigWithHistory(40);
s._migrateHistory();
await s.save({ globalSettings: { alwaysOnTop: true } });
assert.equal(s.loadHistory().length, 40, 'a settings write must not touch history');
assert.equal(s.load().globalSettings.alwaysOnTop, true);
});
it('clearHistory empties history.json only', async () => {
writeConfigWithHistory(20);
s._migrateHistory();
await s.clearHistory();
assert.equal(s.loadHistory().length, 0);
});
it('migration is idempotent — re-running with history.json present does not re-derive or clobber', () => {
writeConfigWithHistory(15);
s._migrateHistory();
const after = makeStore();
after._migrateHistory();
assert.equal(after._historyMigrated, true);
assert.equal(after.loadHistory().length, 15);
});
it('crash-window fallback: not migrated + no history.json → loadHistory reads config.history', () => {
writeConfigWithHistory(7);
assert.equal(s._historyMigrated, false);
assert.equal(s.loadHistory().length, 7, 'legacy path still serves history if migration never ran');
});
it('pruneHistory trims history.json and persists the retention setting', async () => {
writeConfigWithHistory(12);
s._migrateHistory();
const res = await s.pruneHistory('all', { dryRun: false });
assert.equal(s.loadHistory().length, 12);
assert.ok(res.keptBatches === 12);
});
});

View File

@ -30,6 +30,17 @@ test('agent rejects unknown ops and any write/exec-shaped op', () => {
}
});
test('agent rejects inherited Object.prototype members (no whitelist bypass via the prototype chain)', () => {
const agent = createAgent(stubCollectors());
for (const proto of ['constructor', 'toString', 'valueOf', 'hasOwnProperty', 'isPrototypeOf', 'toLocaleString']) {
const r = agent.handle(proto, {});
assert.equal(r.ok, false, `${proto} (inherited) must NOT be treated as an op`);
}
for (const bad of [null, undefined, 42, {}, ['read_log']]) {
assert.equal(agent.handle(bad, {}).ok, false, `non-string op ${JSON.stringify(bad)} must be rejected`);
}
});
test('agent maps each whitelisted op to its collector and is read-only only', () => {
const stub = stubCollectors();
const agent = createAgent(stub);

View File

@ -53,6 +53,32 @@ test('getConfigRedacted strips password/apiKey/token/webhookUrl and value-scrubs
assert.ok(!json.includes('WBHOOKSECRETTOKEN'), 'webhook secret must be redacted');
});
test('getHistory reads loadHistory (migrated mode: loadConfig().history is empty)', () => {
const c = createCollectors({
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [] }),
loadHistory: () => [
{ timestamp: '2026-01-01T00:00:00.000Z', files: [{ name: 'a.mkv', results: [{ hoster: 'voe.sx', status: 'done', url: 'https://voe.sx/a' }] }] },
{ timestamp: '2026-01-02T00:00:00.000Z', files: [{ name: 'b.mkv', results: [{ hoster: 'byse.sx', status: 'done', url: 'https://byse.sx/b' }] }] }
],
getAllLogPaths: () => ({ logDir: os.tmpdir() }),
support, stats,
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
});
const out = c.getHistory({ limit: 10 });
assert.equal(out.totalBatches, 2, 'must report real history from loadHistory, not the empty load().history');
assert.equal(out.returned, 2);
});
test('getHistory falls back to loadConfig().history when loadHistory is absent (legacy mode)', () => {
const c = createCollectors({
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [{ timestamp: '2026-01-01T00:00:00.000Z', files: [] }] }),
getAllLogPaths: () => ({ logDir: os.tmpdir() }),
support, stats,
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
});
assert.equal(c.getHistory({ limit: 10 }).totalBatches, 1, 'legacy path reads load().history when loadHistory not injected');
});
test('readLog redacts a planted token and a Bearer line; doodstream is NOT readable; unknown name rejected', () => {
const { collectors } = makeFixture();
const dbg = collectors.readLog({ name: 'debug', tailKb: 64 });
@ -63,6 +89,30 @@ test('readLog redacts a planted token and a Bearer line; doodstream is NOT reada
assert.equal(collectors.readLog({ name: 'crash' }).name, 'crash');
});
test('readLog grep is case-insensitive substring with | alternation, and is ReDoS-safe', () => {
const { paths } = makeFixture();
const fs2 = require('fs');
fs2.writeFileSync(paths.debug, ['ERROR upload failed', 'info all good', 'WARN timeout hit', 'a'.repeat(120) + '! catastrophic bait'].join('\n'));
const { collectors } = (() => {
const support2 = require('../lib/support-bundle');
const stats2 = require('../lib/stats');
const c = require('../lib/diagnostics-collectors').createCollectors({
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [] }),
getAllLogPaths: () => paths, support: support2, stats: stats2,
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
});
return { collectors: c };
})();
const alt = collectors.readLog({ name: 'debug', grep: 'error|timeout' });
assert.equal(alt.matchedLines, 2, 'matches the ERROR and timeout lines case-insensitively');
assert.ok(alt.content.includes('ERROR upload failed') && alt.content.includes('WARN timeout hit'));
assert.ok(!alt.content.includes('info all good'), 'non-matching line excluded');
const t0 = Date.now();
const redos = collectors.readLog({ name: 'debug', grep: '(a+)+$' });
assert.ok(Date.now() - t0 < 1000, 'catastrophic-looking grep must return promptly (literal substring, no backtracking)');
assert.equal(redos.matchedLines, 0, '"(a+)+$" is treated as a literal substring, matching nothing here');
});
test('getQueueState flags stale=true for the persisted snapshot and counts by status', () => {
const { collectors } = makeFixture();
const q = collectors.getQueueState({});

View File

@ -1,10 +1,20 @@
const { test } = require('node:test');
const assert = require('node:assert');
const os = require('os');
const WebSocket = require('ws');
const RemoteServer = require('../lib/remote-server');
const TOKEN = 'a'.repeat(64);
function firstLanIpv4() {
for (const entry of Object.values(os.networkInterfaces())) {
for (const net of (entry || [])) {
if (net && net.family === 'IPv4' && !net.internal && net.address) return net.address;
}
}
return null;
}
function startAgent(onDiagnosticRequest, extra) {
const srv = new RemoteServer();
return srv.start({ port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true, onDiagnosticRequest, ...(extra || {}) })
@ -56,6 +66,44 @@ test('a diagnostic client NEVER triggers the screen-capture window', async () =>
ws.close(); agent.stop();
});
test('allowlist gate (wiring): a non-loopback peer is closed 4005 when not allowlisted (fail-closed)', () => {
const srv = new RemoteServer();
const closeCodeFor = (remoteAddress, allowlist) => {
srv._config = { allowlist, token: TOKEN, diagnosticMode: true };
let closed = null;
srv._handleConnection({ close: (c) => { closed = c; }, on: () => {} }, { socket: { remoteAddress } });
return closed;
};
assert.equal(closeCodeFor('100.64.0.9', []), 4005, 'empty allowlist => non-loopback rejected (fail-closed)');
assert.equal(closeCodeFor('203.0.113.5', ['100.64.0.0/10']), 4005, 'peer outside the allowlist CIDR rejected');
});
test('a loopback diagnostic client connects even with a non-matching allowlist (loopback is always allowed)', async () => {
const agent = await startAgent(() => {}, { allowlist: ['100.64.0.0/10'] });
const ws = connect(agent.getPort());
await new Promise((r) => ws.on('open', r));
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
const ok = await once(ws, 'auth-ok');
assert.ok(ok.clientId);
ws.close(); agent.stop();
});
test('network bind (0.0.0.0): an allowlisted non-loopback peer connects over a real socket (the Tailscale path)', async (t) => {
const lan = firstLanIpv4();
if (!lan) { t.skip('no non-internal IPv4 interface available'); return; }
const agent = await startAgent(() => {}, { host: '0.0.0.0', allowlist: [lan] });
const port = agent.getPort();
const ws = new WebSocket(`ws://${lan}:${port}`);
try {
await new Promise((resolve, reject) => { ws.on('open', resolve); ws.on('error', reject); });
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
const ok = await once(ws, 'auth-ok');
assert.ok(ok.clientId, 'allowlisted LAN peer authed over the 0.0.0.0 bind');
} finally {
ws.close(); agent.stop();
}
});
test('wrong token is rejected and the ip is locked out after 5 attempts', async () => {
const agent = await startAgent(() => {});
const port = agent.getPort();

View File

@ -0,0 +1,52 @@
const { test } = require('node:test');
const assert = require('node:assert');
const { normalizeIp, isLoopbackIp, matchIpRule, evaluateClientAllowed } = require('../lib/ip-allowlist');
test('normalizeIp strips ::ffff: and lowercases', () => {
assert.equal(normalizeIp('::ffff:100.64.0.5'), '100.64.0.5');
assert.equal(normalizeIp('::FFFF:127.0.0.1'), '127.0.0.1');
assert.equal(normalizeIp(' 100.64.0.5 '), '100.64.0.5');
});
test('loopback is always allowed, even with a non-matching allowlist', () => {
for (const ip of ['127.0.0.1', '::1', '::ffff:127.0.0.1', '', 'localhost', '127.5.5.5']) {
assert.equal(evaluateClientAllowed(ip, ['203.0.113.5']), true, `${ip} loopback`);
}
});
test('fail-closed: empty allowlist rejects every non-loopback peer', () => {
for (const ip of ['100.64.0.5', '203.0.113.5', '10.0.0.2', '::ffff:192.168.1.9']) {
assert.equal(evaluateClientAllowed(ip, []), false, `${ip} must be rejected with empty allowlist`);
}
});
test('exact IP allow + reject', () => {
assert.equal(evaluateClientAllowed('203.0.113.5', ['203.0.113.5']), true);
assert.equal(evaluateClientAllowed('203.0.113.6', ['203.0.113.5']), false);
});
test('CIDR matching incl. the Tailscale CGNAT range 100.64.0.0/10', () => {
assert.equal(evaluateClientAllowed('100.64.0.5', ['100.64.0.0/10']), true);
assert.equal(evaluateClientAllowed('100.127.255.254', ['100.64.0.0/10']), true);
assert.equal(evaluateClientAllowed('100.128.0.1', ['100.64.0.0/10']), false, 'just outside the /10');
assert.equal(evaluateClientAllowed('::ffff:100.64.0.5', ['100.64.0.0/10']), true, 'mapped v4 in CIDR');
assert.equal(evaluateClientAllowed('10.0.0.5', ['10.0.0.0/24']), true);
assert.equal(evaluateClientAllowed('10.0.1.5', ['10.0.0.0/24']), false);
});
test('wildcard rules allow everything', () => {
assert.equal(evaluateClientAllowed('8.8.8.8', ['*']), true);
assert.equal(evaluateClientAllowed('8.8.8.8', ['0.0.0.0/0']), true);
});
test('matchIpRule rejects malformed rules and out-of-range octets', () => {
assert.equal(matchIpRule('1.2.3.4', 'not-an-ip'), false);
assert.equal(matchIpRule('1.2.3.4', '1.2.3.0/33'), false);
assert.equal(matchIpRule('1.2.3.999', '1.2.3.0/24'), false);
});
test('isLoopbackIp recognizes loopback forms', () => {
assert.equal(isLoopbackIp('127.0.0.1'), true);
assert.equal(isLoopbackIp('::1'), true);
assert.equal(isLoopbackIp('100.64.0.1'), false);
});

View File

@ -57,13 +57,23 @@ test('resolveLogFileName: daily mode → fileuploader-YYYY-MM-DD.log', () => {
);
});
test('resolveLogFileName: session mode → fileuploader-session-<id>.log', () => {
test('resolveLogFileName: session mode → <sessionId>.log (baseName ignored)', () => {
assert.equal(
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session', sessionId: '2026-05-28_22-44-52-12345' }),
'fileuploader-session-2026-05-28_22-44-52-12345.log'
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session', sessionId: '26-05-2026-mdu-session-22-44' }),
'26-05-2026-mdu-session-22-44.log'
);
});
test('formatSessionStamp: DD-MM-YYYY-mdu-session-HH-MM', () => {
const { formatSessionStamp } = require('../lib/log-mode');
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36)), '26-06-2026-mdu-session-06-02');
});
test('formatSessionStamp: appends a 6-digit suffix when a rand is supplied', () => {
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36), '847581'), '26-06-2026-mdu-session-06-02-847581');
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36), 847581), '26-06-2026-mdu-session-06-02-847581');
});
test('resolveLogFileName: session mode with missing sessionId falls back to single (never emits malformed name)', () => {
assert.equal(
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session' }),
@ -102,12 +112,17 @@ test('stripModeStampFromFileName: strips a session-stamp suffix (with and withou
);
});
test('stripModeStampFromFileName: new DD-MM-YYYY-mdu-session-HH-MM resets to the default base', () => {
assert.equal(stripModeStampFromFileName('26-06-2026-mdu-session-06-02.log'), 'fileuploader.log');
assert.equal(stripModeStampFromFileName('26-06-2026-mdu-session-06-02-847581.log'), 'fileuploader.log');
});
test('regression: resolveLogFileName(stripModeStampFromFileName(...)) is idempotent — persisting then re-resolving never compounds stamps', () => {
// This is the exact bug shape: persist the resolved path, then on next call
// re-resolve from the saved base — must produce the same file, not a doubled
// session-stamped one. The fix is the strip; this test guards against
// regressing _persistFallbackLogPath into the 3.3.35 bug.
const sessionId = '2026-06-03_18-16-20-8132';
const sessionId = '03-06-2026-mdu-session-18-16';
const dailyDate = new Date(2026, 5, 3);
for (const mode of ['daily', 'session']) {
const date = mode === 'daily' ? dailyDate : new Date();
@ -129,12 +144,7 @@ test('formatDateStamp: zero-pads month and day', () => {
assert.equal(formatDateStamp(new Date(2026, 11, 31)), '2026-12-31');
});
test('formatSessionStamp: produces YYYY-MM-DD_HH-MM-SS-pid', () => {
const d = new Date(2026, 4, 28, 7, 9, 5);
assert.equal(formatSessionStamp(d, 12345), '2026-05-28_07-09-05-12345');
});
test('formatSessionStamp: omits the pid suffix when none provided', () => {
const d = new Date(2026, 4, 28, 22, 44, 52);
assert.equal(formatSessionStamp(d), '2026-05-28_22-44-52');
test('formatSessionStamp: DD-MM-YYYY-mdu-session-HH-MM (no seconds/pid)', () => {
assert.equal(formatSessionStamp(new Date(2026, 4, 28, 7, 9, 5)), '28-05-2026-mdu-session-07-09');
assert.equal(formatSessionStamp(new Date(2026, 4, 28, 22, 44, 52)), '28-05-2026-mdu-session-22-44');
});

View File

@ -44,6 +44,37 @@ test('redactLogText leaves benign "token" prose alone', () => {
assert.equal(redactLogText(benign, []), benign);
});
test('redactLogText scrubs the password from a basic-auth URL but keeps host:port', () => {
const out = redactLogText('proxy https://admin:Sup3rProxyPass@proxy.internal:8080/path', []);
assert.ok(!out.includes('Sup3rProxyPass'), 'basic-auth password must be redacted');
assert.ok(out.includes('proxy.internal:8080'), 'host:port preserved');
assert.ok(out.includes('admin:'), 'username preserved');
});
test('redactLogText does not touch a host:port URL without userinfo', () => {
const url = 'connecting to https://cdn.voe.sx:8080/upload now';
assert.equal(redactLogText(url, []), url);
});
test('redactLogText scrubs Basic auth, JWTs and bare session= values (defense in depth)', () => {
const cases = [
{ line: 'Authorization: Basic dXNlcjpwYXNzd29yZDEyMw==', secret: 'dXNlcjpwYXNzd29yZDEyMw' },
{ line: 'jwt eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N', secret: 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0' },
{ line: 'session=SESSIONsecretvalue99887766', secret: 'SESSIONsecretvalue99887766' },
{ line: '"session":"jsonSessionSecret123456"', secret: 'jsonSessionSecret123456' },
];
for (const c of cases) {
const out = redactLogText(c.line, []);
assert.ok(!out.includes(c.secret), `must redact: ${c.line} -> ${out}`);
assert.ok(out.includes(REDACTED), `expected ${REDACTED} in ${out}`);
}
});
test('redactLogText leaves a normal "session" word in prose alone', () => {
const benign = 'the session was idle for a while';
assert.equal(redactLogText(benign, []), benign);
});
test('sanitizeConfig does not mutate input', () => {
const input = { hosters: { 'voe.sx': [{ password: 'secret' }] } };
const clone = JSON.parse(JSON.stringify(input));

View File

@ -26,14 +26,20 @@ describe('suspect-reject alternate accounts', () => {
fileProbe.probeFileHead = (...a) => mockProbe(...a);
const fs = require('fs');
const origStatSync = fs.statSync;
fs.statSync = function (p) {
if (typeof p === 'string' && p.startsWith('/test/')) {
const fakeSize = (p) => {
const m = /-(\d+)gb/i.exec(p);
return { size: (m ? parseInt(m[1], 10) : 3) * 1024 * 1024 * 1024 };
}
};
const origStatSync = fs.statSync;
fs.statSync = function (p) {
if (typeof p === 'string' && p.startsWith('/test/')) return fakeSize(p);
return origStatSync.call(this, p);
};
const origStat = fs.promises.stat;
fs.promises.stat = async function (p) {
if (typeof p === 'string' && p.startsWith('/test/')) return fakeSize(p);
return origStat.call(this, p);
};
UploadManager = require('../lib/upload-manager');
});

View File

@ -33,7 +33,7 @@ describe('UploadManager', () => {
hosters.uploadFile = mockUploadFile;
hosters.prefetchBaseline = async () => null;
// Mock fs.statSync for test file paths
// Mock fs.statSync + fs.promises.stat for test file paths
const fs = require('fs');
const origStatSync = fs.statSync;
fs.statSync = function(p) {
@ -42,6 +42,13 @@ describe('UploadManager', () => {
}
return origStatSync.call(this, p);
};
const origStat = fs.promises.stat;
fs.promises.stat = async function(p) {
if (typeof p === 'string' && p.startsWith('/test/')) {
return { size: fakeFileSize };
}
return origStat.call(this, p);
};
UploadManager = require('../lib/upload-manager');
});
@ -331,10 +338,10 @@ describe('UploadManager', () => {
});
it('file not found produces descriptive error', async () => {
// Override fs.statSync to throw ENOENT for a specific path
// Override fs.promises.stat to throw ENOENT for a specific path
const fs = require('fs');
const origStat = fs.statSync;
fs.statSync = function(p) {
const origStat = fs.promises.stat;
fs.promises.stat = async function(p) {
if (p === '/test/deleted.mp4') throw Object.assign(new Error('ENOENT'), { code: 'ENOENT' });
return origStat.call(this, p);
};
@ -347,7 +354,7 @@ describe('UploadManager', () => {
{ file: '/test/deleted.mp4', hoster: 'doodstream.com', apiKey: 'key1' }
]);
fs.statSync = origStat;
fs.promises.stat = origStat;
assert.ok(errors.some(e => e.includes('nicht gefunden')), `expected "nicht gefunden" error, got: ${errors.join(', ')}`);
});