Compare commits

...

4 Commits

Author SHA1 Message Date
Administrator
c37c8e3906 release: v3.3.87 2026-06-21 01:49:23 +02:00
Administrator
9cc8fee02c docs(todo,lessons): long-run lag root cause was the recent-panel append-gate cliff (measured 80ms->7ms)
Records the verified root cause + the dismissed-with-evidence findings:
- _sessionFileKeys "separator mismatch" = FALSE POSITIVE (real U+0001 chars, verifier Read rendered them invisibly)
- queueJobs O(N) per-render scan = real but Blink-measured <0.1ms at 3000 jobs -> skipped
- standing 2000-row relayout = median 0.4ms -> no virtualization needed
- doodstream sync _debugLog = constant freeze, deferred to a separate change

Lesson: measure magnitudes at the real artifact before fixing a "scales-with-X" cause;
profile in real Blink (Playwright) not jsdom; verify multi-agent findings against primary
evidence (char-code dump caught the control-char false positive).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 01:47:59 +02:00
Administrator
0f9096be3c perf(renderer): keep recent-uploads panel append-only past the cap (kill ~80ms per-completion freeze)
The recent-uploads panel had a cheap append-only fast path, but it was gated on
`rows.length > _recentLastRenderedLen`. maybeAddSessionFile caps sessionFilesData
by push-then-slice (2000 -> 2001 -> sliced back to 2000), so once a session
produces more than SESSION_FILES_CAP rows the length is pinned at the cap and the
gate is false forever. Every subsequent completion then fell through to the full
`tbody.innerHTML = rows.map(...).join('')` rebuild of all ~2000 rows.

The cap is per (link x file x hoster), so with 4-5 selected hosters the 2000 cap
is hit at only ~400-500 distinct files — very reachable in a long folder-monitor
session. Profiled in Chromium (same Blink engine as Electron, table-layout:fixed):
the full 2000-row rebuild costs ~80ms and ran on EVERY completion past the cap — a
repeating ~80ms main-thread freeze. That is the "fine on a fresh start, gets laggy
after many uploads while CPU (~40%) and RAM (~6GB, stable) stay normal" symptom: a
render-thread stall, not CPU saturation or a memory leak.

Fix: track newly-pushed rows in _recentPendingAppends (incremented in
maybeAddSessionFile, consumed every render) and gate the fast path on
`pendingAppends > 0` instead of length growth, so it survives the cap. Prepend the
new rows, then evict the same overflow count from the DOM bottom (oldest rows,
which is where the date-desc view places the front-of-array entries the cap slices
off). DOM work is O(added) again. The fast path is gated behind an explicit
`appendOnly` flag passed only by scheduleRecentRender's rAF, so selection / delete
/ clear / sort / batch-done renders stay full rebuilds and cannot wrong-evict or
double-prepend.

Verified in Blink over a simulated 5000-completion session (3/frame, far past the
cap): per-frame render 80ms -> median 7.4ms (>10x), and the DOM stays exactly equal
to the data (cap held at 2000, newest-on-top, oldest evicted, zero duplicates).
397/397 tests pass, eslint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 01:47:59 +02:00
Administrator
29d1944328 perf(config): cache parsed config + lean serialize + drop load() from log flush (long-run lag)
A 13-agent hunt pinned the "wird mit der Zeit laggy" symptom (CPU/RAM normal, UI
sluggish after many uploads) to the main process re-doing config I/O that scales
with the ever-growing history, stalling the synchronous main event loop so the
renderer's IPC round-trips feel laggy. The renderer render path was already
optimized (virtualized queue, capped panels) — confirmed clean.

This commit lands the two contained fixes (T1 + T3); the queue-persist rewrite (T2)
follows separately.

config-store (T1):
- load() now has an in-memory cache keyed on the file mtime+size. The processed
  config (merged + credential-decrypted) is re-read/re-parsed/re-DPAPI-decrypted
  ONLY when the file actually changes; our writes invalidate it, external edits
  change mtime/size so the cache misses. Eliminates a full disk read + JSON.parse of
  the whole growing history + per-credential decrypt on the vast majority of the ~38
  load() call sites. load() always returns a structuredClone so callers can mutate
  freely without corrupting the cache.
- _serializeForDisk clones ONLY the hosters subtree (the only thing encryptCredentials
  touches) instead of JSON.parse(JSON.stringify(whole config)) — no more deep-cloning
  an 8 MB history on every write.
- _atomicWrite refreshes the .bak with a raw fs.copyFileSync instead of
  read + JSON.parse + write (it was re-parsing the full config a 2nd time per write).

main.js (T3):
- The log-flush paths resolved the log file via configStore.load() ~8x/second during
  uploads (re-reading + cloning the whole config just to read logMode/logFilePath).
  Cache those two strings in module scope, invalidate on the settings-save handlers.

Verified: 26 config-store tests (incl. new cache-correctness: independent clones,
external-change invalidation, save invalidation) + full 394-test suite green, lint 0
errors. Benchmark (8000-batch / 4.6 MB history): log flush no longer calls load() at
all; the remaining per-write history serialize is what T2 removes from the hot path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 00:49:55 +02:00
8 changed files with 185 additions and 70 deletions

View File

@ -54,6 +54,7 @@ const nodeGlobals = {
URLSearchParams: 'readonly', URLSearchParams: 'readonly',
fetch: 'readonly', fetch: 'readonly',
crypto: 'readonly', crypto: 'readonly',
structuredClone: 'readonly',
}; };
export default [ export default [

View File

@ -185,6 +185,8 @@ class ConfigStore {
: path.join(__dirname, '..'); : path.join(__dirname, '..');
this.filePath = path.join(dir, 'electron-config.json'); this.filePath = path.join(dir, 'electron-config.json');
this._writeQueue = Promise.resolve(); // Serializes all writes to prevent race conditions this._writeQueue = Promise.resolve(); // Serializes all writes to prevent race conditions
this._cache = null;
this._cacheKey = '';
// Migrate config from old location if current doesn't exist // Migrate config from old location if current doesn't exist
if (!fs.existsSync(this.filePath) && app && app.isPackaged) { if (!fs.existsSync(this.filePath) && app && app.isPackaged) {
@ -221,8 +223,29 @@ class ConfigStore {
return JSON.parse(raw); return JSON.parse(raw);
} }
_clone(obj) {
try { return structuredClone(obj); }
catch { return JSON.parse(JSON.stringify(obj)); }
}
load() { load() {
try { try {
// In-memory cache keyed on the file's mtime+size. The processed config
// (merged + credential-decrypted) is reparsed/re-decrypted from disk ONLY
// when the file actually changes. Our own writes refresh the cache (see
// _commit), and an external edit changes mtime/size so the cache misses
// and we reread. Without this, every one of the ~38 main.js load() call
// sites (incl. the per-500ms log-flush path) re-read disk + JSON.parse the
// whole growing history + DPAPI-decrypt every credential — the dominant
// long-running main-thread drag. load() always returns a CLONE so callers
// can mutate the result without corrupting the cache.
let stat = null;
try { stat = fs.statSync(this.filePath); } catch {}
const statKey = stat ? `${stat.mtimeMs}:${stat.size}` : '';
if (stat && this._cache && this._cacheKey === statKey) {
return this._clone(this._cache);
}
let data = null; let data = null;
// Try main config // Try main config
try { data = this._readAndParse(this.filePath); } catch {} try { data = this._readAndParse(this.filePath); } catch {}
@ -309,7 +332,11 @@ class ConfigStore {
// Decrypt credentials stored with safeStorage so the rest of the app // Decrypt credentials stored with safeStorage so the rest of the app
// keeps working with plaintext in memory. // keeps working with plaintext in memory.
secretStore.decryptCredentials(result); secretStore.decryptCredentials(result);
return result; if (stat) {
this._cache = result;
this._cacheKey = statKey;
}
return this._clone(result);
} catch { } catch {
const fresh = JSON.parse(JSON.stringify(DEFAULTS)); const fresh = JSON.parse(JSON.stringify(DEFAULTS));
fresh.globalSettings.logMode = normalizeLogMode(fresh.globalSettings); fresh.globalSettings.logMode = normalizeLogMode(fresh.globalSettings);
@ -317,12 +344,19 @@ class ConfigStore {
} }
} }
// Deep-clone a config and encrypt its credential fields. Never mutate the // Encrypt credential fields without mutating the caller's plaintext object.
// caller's object — the rest of the app holds plaintext references. // Only `hosters` carries credentials, so we clone ONLY that subtree — the rest
// (history, globalSettings, …) is referenced read-only into the stringified
// object. Deep-cloning the whole config here (incl. an ever-growing history)
// on every write was a primary long-running main-thread stall.
_serializeForDisk(config) { _serializeForDisk(config) {
const clone = JSON.parse(JSON.stringify(config)); const hosters = this._clone(config.hosters || {});
secretStore.encryptCredentials(clone); secretStore.encryptCredentials({ hosters });
return JSON.stringify(clone, null, 2); return JSON.stringify({ ...config, hosters }, null, 2);
}
_commit(config) {
return this._atomicWrite(this._serializeForDisk(config));
} }
_enqueueWrite(fn) { _enqueueWrite(fn) {
@ -336,7 +370,7 @@ class ConfigStore {
if (config.hosters) current.hosters = config.hosters; if (config.hosters) current.hosters = config.hosters;
if (config.hosterSettings) current.hosterSettings = config.hosterSettings; if (config.hosterSettings) current.hosterSettings = config.hosterSettings;
if (config.globalSettings) current.globalSettings = config.globalSettings; if (config.globalSettings) current.globalSettings = config.globalSettings;
return this._atomicWrite(this._serializeForDisk(current)); return this._commit(current);
}); });
} }
@ -352,25 +386,24 @@ class ConfigStore {
fs.writeFile(tmpPath, data, 'utf-8', (err) => { fs.writeFile(tmpPath, data, 'utf-8', (err) => {
if (err) return reject(err); if (err) return reject(err);
try { try {
// Refresh .bak from the previous live file. Wrapped in try/catch // Refresh .bak from the previous live file with a raw byte copy —
// so an AV/indexer briefly locking the file doesn't fail the whole // no read+JSON.parse+write. The live file was itself written through
// save — the rename to the live path is the part that matters, // this atomic path, so re-validating it by parsing the whole (growing)
// a stale .bak is preferable to losing the new write entirely. // config on every write was pure waste. Wrapped in try/catch so an
// AV/indexer briefly locking the file doesn't fail the save — the
// rename to the live path is the part that matters.
try { try {
if (fs.existsSync(this.filePath)) { if (fs.existsSync(this.filePath)) {
const existing = fs.readFileSync(this.filePath, 'utf-8'); fs.copyFileSync(this.filePath, backupPath);
if (existing && existing.trim().length > 2) {
let isValid = false;
try {
const parsed = JSON.parse(existing);
isValid = parsed && typeof parsed === 'object' && (parsed.hosters || parsed.hosterSettings || parsed.globalSettings);
} catch {}
if (isValid) fs.writeFileSync(backupPath, existing, 'utf-8');
}
} }
} catch {} } catch {}
fs.renameSync(tmpPath, this.filePath); fs.renameSync(tmpPath, this.filePath);
} catch (e) { return reject(e); } } catch (e) { return reject(e); }
// Invalidate the read cache: the next load() re-reads + re-merges the
// freshly-written file (the on-disk format is sparse — load() fills
// defaults — so we must NOT serve a pre-merge in-memory object).
this._cache = null;
this._cacheKey = '';
resolve(); resolve();
}); });
}); });
@ -382,7 +415,7 @@ class ConfigStore {
config.history.push(entry); config.history.push(entry);
const retention = (config.globalSettings && config.globalSettings.historyRetention) || 'all'; const retention = (config.globalSettings && config.globalSettings.historyRetention) || 'all';
config.history = applyHistoryRetention(config.history, retention, Date.now()); config.history = applyHistoryRetention(config.history, retention, Date.now());
return this._atomicWrite(this._serializeForDisk(config)); return this._commit(config);
}); });
} }
@ -402,7 +435,7 @@ class ConfigStore {
if (dryRun) return result; if (dryRun) return result;
config.history = pruned; config.history = pruned;
if (config.globalSettings) config.globalSettings.historyRetention = String(retention || 'all'); if (config.globalSettings) config.globalSettings.historyRetention = String(retention || 'all');
return this._atomicWrite(this._serializeForDisk(config)).then(() => result); return this._commit(config).then(() => result);
}); });
} }
@ -410,7 +443,7 @@ class ConfigStore {
return this._enqueueWrite(() => { return this._enqueueWrite(() => {
const config = this.load(); const config = this.load();
config.history = []; config.history = [];
return this._atomicWrite(this._serializeForDisk(config)); return this._commit(config);
}); });
} }
@ -418,7 +451,7 @@ class ConfigStore {
return this._enqueueWrite(() => { return this._enqueueWrite(() => {
const config = this.load(); const config = this.load();
config.rotationCursors = (cursors && typeof cursors === 'object' && !Array.isArray(cursors)) ? cursors : {}; config.rotationCursors = (cursors && typeof cursors === 'object' && !Array.isArray(cursors)) ? cursors : {};
return this._atomicWrite(this._serializeForDisk(config)); return this._commit(config);
}); });
} }
} }

34
main.js
View File

@ -414,11 +414,26 @@ function getDefaultLogFilePath() {
return path.join(__dirname, 'fileuploader.log'); return path.join(__dirname, 'fileuploader.log');
} }
// The log flush paths resolve the log file ~8x/second during uploads. Going
// through configStore.load() there meant re-reading + cloning the whole config
// (incl. an 8 MB+ history) on every flush — a major long-running main-thread
// drag. logFilePath/logMode change only when the user saves settings, so cache
// the two strings and invalidate on those saves (see _invalidateLogSettings).
let _cachedLogSettings = null;
function _getLogSettings() {
if (!_cachedLogSettings) {
const gs = (configStore.load() || {}).globalSettings || {};
_cachedLogSettings = {
logFilePath: String(gs.logFilePath || '').trim(),
logMode: gs.logMode || 'single'
};
}
return _cachedLogSettings;
}
function _invalidateLogSettings() { _cachedLogSettings = null; }
function getBaseLogFilePath() { function getBaseLogFilePath() {
const config = configStore.load(); const customPath = _getLogSettings().logFilePath;
const customPath = config && config.globalSettings
? String(config.globalSettings.logFilePath || '').trim()
: '';
return customPath || getDefaultLogFilePath(); return customPath || getDefaultLogFilePath();
} }
@ -433,8 +448,7 @@ let _activeLogKey = null; // remembers (mode + date-or-session) so cache rolls
let _activeLogPath = null; let _activeLogPath = null;
function getLogFilePath() { function getLogFilePath() {
const config = configStore.load(); const mode = _getLogSettings().logMode;
const mode = (config && config.globalSettings && config.globalSettings.logMode) || 'single';
const base = getBaseLogFilePath(); const base = getBaseLogFilePath();
const dir = path.dirname(base); const dir = path.dirname(base);
const ext = path.extname(base); const ext = path.extname(base);
@ -454,8 +468,7 @@ function getLogFilePath() {
function buildFallbackLogName(dir) { function buildFallbackLogName(dir) {
// Match the active log-mode's naming so the fallback file is consistent with // Match the active log-mode's naming so the fallback file is consistent with
// what the primary write would have produced. // what the primary write would have produced.
const config = configStore.load(); const mode = _getLogSettings().logMode;
const mode = (config && config.globalSettings && config.globalSettings.logMode) || 'single';
return path.join(dir, resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode, date: new Date(), sessionId: SESSION_ID })); return path.join(dir, resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode, date: new Date(), sessionId: SESSION_ID }));
} }
@ -596,6 +609,7 @@ function _persistFallbackLogPath(workingPath) {
cfg.globalSettings = gs; cfg.globalSettings = gs;
configStore.save({ globalSettings: gs }).catch(() => {}); configStore.save({ globalSettings: gs }).catch(() => {});
_invalidateUploadLogTargetCache(); _invalidateUploadLogTargetCache();
_invalidateLogSettings();
safeSend('log-path-auto-updated', { logFilePath: toSave }); safeSend('log-path-auto-updated', { logFilePath: toSave });
} catch (err) { } catch (err) {
debugLog(`persist fallback logpath failed: ${err.message}`); debugLog(`persist fallback logpath failed: ${err.message}`);
@ -1315,6 +1329,7 @@ ipcMain.handle('get-config', () => {
ipcMain.handle('save-config', async (_event, config) => { ipcMain.handle('save-config', async (_event, config) => {
await configStore.save(config); await configStore.save(config);
if (config && config.globalSettings) _invalidateLogSettings();
try { try {
if (config && config.globalSettings && Object.prototype.hasOwnProperty.call(config.globalSettings, 'logVerbose')) { if (config && config.globalSettings && Object.prototype.hasOwnProperty.call(config.globalSettings, 'logVerbose')) {
setLogVerbose(!!config.globalSettings.logVerbose); setLogVerbose(!!config.globalSettings.logVerbose);
@ -2152,6 +2167,7 @@ ipcMain.handle('import-backup', async (_event, legacyPassword) => {
history: [] history: []
}; };
await configStore._atomicWrite(configStore._serializeForDisk(merged)); await configStore._atomicWrite(configStore._serializeForDisk(merged));
_invalidateLogSettings();
return { ok: true, config: configStore.load() }; return { ok: true, config: configStore.load() };
}); });
@ -2286,6 +2302,7 @@ function _preserveDiagSubtree(globalSettings) {
ipcMain.handle('save-global-settings', async (_event, globalSettings) => { ipcMain.handle('save-global-settings', async (_event, globalSettings) => {
globalSettings = _preserveDiagSubtree(globalSettings); globalSettings = _preserveDiagSubtree(globalSettings);
await configStore.save({ globalSettings }); await configStore.save({ globalSettings });
_invalidateLogSettings();
if (uploadManager) uploadManager.updateSettings(null, globalSettings); if (uploadManager) uploadManager.updateSettings(null, globalSettings);
return true; return true;
}); });
@ -2328,6 +2345,7 @@ ipcMain.on('save-global-settings-sync', (event, globalSettings) => {
const _diskDiag = current.globalSettings && current.globalSettings.diagnostics; const _diskDiag = current.globalSettings && current.globalSettings.diagnostics;
current.globalSettings = globalSettings; current.globalSettings = globalSettings;
if (_diskDiag) current.globalSettings.diagnostics = _diskDiag; if (_diskDiag) current.globalSettings.diagnostics = _diskDiag;
_invalidateLogSettings();
const data = configStore._serializeForDisk(current); const data = configStore._serializeForDisk(current);
const backupPath = configStore.filePath + '.bak'; const backupPath = configStore.filePath + '.bak';
fs.writeFileSync(tmpPath, data, 'utf-8'); fs.writeFileSync(tmpPath, data, 'utf-8');

View File

@ -1,6 +1,6 @@
{ {
"name": "multi-hoster-uploader", "name": "multi-hoster-uploader",
"version": "3.3.86", "version": "3.3.87",
"description": "Upload files to doodstream, voe, vidmoly, byse simultaneously", "description": "Upload files to doodstream, voe, vidmoly, byse simultaneously",
"main": "main.js", "main": "main.js",
"scripts": { "scripts": {

View File

@ -1177,7 +1177,7 @@ let _recentRenderQueued = false;
function scheduleRecentRender() { function scheduleRecentRender() {
if (_recentRenderQueued) return; if (_recentRenderQueued) return;
_recentRenderQueued = true; _recentRenderQueued = true;
requestAnimationFrame(() => { _recentRenderQueued = false; renderRecentUploadsPanel(); }); requestAnimationFrame(() => { _recentRenderQueued = false; renderRecentUploadsPanel(true); });
} }
// Toggle the .selected class on existing rows without rebuilding the table. // Toggle the .selected class on existing rows without rebuilding the table.
@ -2768,6 +2768,7 @@ function maybeAddSessionFile(job) {
}); });
_recentDataVersion++; _recentDataVersion++;
_sessionDoneCount++; _sessionDoneCount++;
_recentPendingAppends++;
// Drop oldest entries past the cap to keep render cost bounded. // Drop oldest entries past the cap to keep render cost bounded.
// Without this, sessionFilesData grows unbounded across the session // Without this, sessionFilesData grows unbounded across the session
// and every renderRecentUploadsPanel call becomes a megabyte-sized // and every renderRecentUploadsPanel call becomes a megabyte-sized
@ -4672,10 +4673,13 @@ function _buildRecentRowHtml(row) {
// accumulating new uploads (the default case: sort=date desc, rows only grow). // accumulating new uploads (the default case: sort=date desc, rows only grow).
let _recentLastRenderedSig = ''; let _recentLastRenderedSig = '';
let _recentLastRenderedLen = 0; let _recentLastRenderedLen = 0;
let _recentPendingAppends = 0;
function renderRecentUploadsPanel() { function renderRecentUploadsPanel(appendOnly = false) {
const tbody = document.getElementById('recentFilesBody'); const tbody = document.getElementById('recentFilesBody');
if (!tbody) return; if (!tbody) return;
const pendingAppends = _recentPendingAppends;
_recentPendingAppends = 0;
if (!sessionFilesData.length) { if (!sessionFilesData.length) {
tbody.innerHTML = '<tr><td colspan="4" class="empty-state">Noch keine Uploads in dieser Session.</td></tr>'; tbody.innerHTML = '<tr><td colspan="4" class="empty-state">Noch keine Uploads in dieser Session.</td></tr>';
_recentLastRenderedSig = ''; _recentLastRenderedSig = '';
@ -4685,9 +4689,10 @@ function renderRecentUploadsPanel() {
const rows = sortRecentFiles(sessionFilesData); const rows = sortRecentFiles(sessionFilesData);
const sig = `${recentSortState.key}|${recentSortState.direction}`; const sig = `${recentSortState.key}|${recentSortState.direction}`;
const dateDescAppendOnly = sig === 'date|desc' const dateDescAppendOnly = appendOnly
&& pendingAppends > 0
&& sig === 'date|desc'
&& _recentLastRenderedSig === sig && _recentLastRenderedSig === sig
&& rows.length > _recentLastRenderedLen
&& tbody.querySelectorAll('.recent-file-row').length === _recentLastRenderedLen; && tbody.querySelectorAll('.recent-file-row').length === _recentLastRenderedLen;
const wrap = tbody.closest('.recent-files-table-wrap'); const wrap = tbody.closest('.recent-files-table-wrap');
@ -4695,10 +4700,17 @@ function renderRecentUploadsPanel() {
let wasAppendOnly = false; let wasAppendOnly = false;
if (dateDescAppendOnly) { if (dateDescAppendOnly) {
const added = rows.length - _recentLastRenderedLen; const added = Math.min(pendingAppends, rows.length);
let html = ''; let html = '';
for (let i = 0; i < added; i++) html += _buildRecentRowHtml(rows[i]); for (let i = 0; i < added; i++) html += _buildRecentRowHtml(rows[i]);
tbody.insertAdjacentHTML('afterbegin', html); tbody.insertAdjacentHTML('afterbegin', html);
let evict = (_recentLastRenderedLen + added) - rows.length;
while (evict > 0) {
const last = tbody.lastElementChild;
if (!last) break;
last.remove();
evict--;
}
wasAppendOnly = true; wasAppendOnly = true;
} else { } else {
tbody.innerHTML = rows.map(_buildRecentRowHtml).join(''); tbody.innerHTML = rows.map(_buildRecentRowHtml).join('');

View File

@ -130,3 +130,12 @@
**Was der Downloader anders macht (und warum es Tailscale ermöglicht):** Host steckt IM Code (`{v,h,p,t,n,fp?,s?}`), nicht extern. Zwei Bind-Modi: lokal (127.0.0.1) ODER Netzwerk (0.0.0.0) — letzteres NUR mit nicht-leerer **fail-closed IP-Allowlist**: leere Allowlist = nur Loopback; geprüft am ECHTEN socket.remoteAddress (NIE forwarded-Header), `::ffff:`-normalisiert, CIDR-Matching. Tailscale wird NICHT autodetektiert — es ist nur eine der `os.networkInterfaces()`-IPs, erreicht über den Tunnel; Allowlist (auf den Tailnet, z.B. `100.64.0.0/10`) + Token sind das Gate, WireGuard ist die Verschlüsselung. **Was der Downloader anders macht (und warum es Tailscale ermöglicht):** Host steckt IM Code (`{v,h,p,t,n,fp?,s?}`), nicht extern. Zwei Bind-Modi: lokal (127.0.0.1) ODER Netzwerk (0.0.0.0) — letzteres NUR mit nicht-leerer **fail-closed IP-Allowlist**: leere Allowlist = nur Loopback; geprüft am ECHTEN socket.remoteAddress (NIE forwarded-Header), `::ffff:`-normalisiert, CIDR-Matching. Tailscale wird NICHT autodetektiert — es ist nur eine der `os.networkInterfaces()`-IPs, erreicht über den Tunnel; Allowlist (auf den Tailnet, z.B. `100.64.0.0/10`) + Token sind das Gate, WireGuard ist die Verschlüsselung.
**Regel:** Bei "mach es wie X": X lokalisieren (grep), die security-kritischen Teile mit einem Subagenten verbatim mappen, dann replizieren. Eine fail-closed Allowlist (Loopback immer erlaubt, leer=loopback-only, real peer IP) ist das richtige Modell für netzwerk-erreichbare read-only Diagnose über einen vertrauten Tunnel — plaintext-Transport ist ok, WENN der Tunnel (Tailscale/WireGuard) verschlüsselt UND die Allowlist+Token den Zugriff gaten. Den HAPPY-Path (allowlisted non-loopback peer über echten 0.0.0.0-Socket) auch LIVE testen, nicht nur per Komposition aus Unit+Wiring. **Regel:** Bei "mach es wie X": X lokalisieren (grep), die security-kritischen Teile mit einem Subagenten verbatim mappen, dann replizieren. Eine fail-closed Allowlist (Loopback immer erlaubt, leer=loopback-only, real peer IP) ist das richtige Modell für netzwerk-erreichbare read-only Diagnose über einen vertrauten Tunnel — plaintext-Transport ist ok, WENN der Tunnel (Tailscale/WireGuard) verschlüsselt UND die Allowlist+Token den Zugriff gaten. Den HAPPY-Path (allowlisted non-loopback peer über echten 0.0.0.0-Socket) auch LIVE testen, nicht nur per Komposition aus Unit+Wiring.
**Prozess-Stolperstein:** Test NACH dem Feature-Commit hinzugefügt → release_gitea.mjs brach ab ("uncommitted tracked changes"). Vor jedem Release: `git status --porcelain | grep -v '^??'` muss leer sein. Tracked-aber-uncommitted (auch ein nachgereichter Test) blockt den Build. **Prozess-Stolperstein:** Test NACH dem Feature-Commit hinzugefügt → release_gitea.mjs brach ab ("uncommitted tracked changes"). Vor jedem Release: `git status --porcelain | grep -v '^??'` muss leer sein. Tracked-aber-uncommitted (auch ein nachgereichter Test) blockt den Build.
## 2026-06-21 — "Gefühlt laggy nach Zeit, CPU/RAM normal" = ERST messen, dann in echtem Blink profilen (v3.3.87)
**Kontext:** User: Programm fühlt sich nach langer Laufzeit mit vielen Uploads zäh an, CPU ~40%/8 Kerne, RAM 6/32 GB — beide normal/stabil. Erste Hypothese (Haupt­prozess-Config-I/O skaliert mit wachsender History) war für DIESEN User FALSCH.
**Was es wirklich war (gemessen + profiliert):** `renderRecentUploadsPanel` hatte einen Append-only-Fastpath, gegated auf `rows.length > _recentLastRenderedLen`. `maybeAddSessionFile` capped per push-then-slice (2000→2001→zurück auf 2000). Ab dem Cap ist `rows.length` auf 2000 fixiert → Gate für IMMER false → JEDE Completion fiel in den Full-`innerHTML`-Rebuild von 2000 Zeilen. Blink-Messung (Playwright, table-layout:fixed, gleiche Engine wie Electron): **~80 ms pro Completion** → wiederkehrender 80-ms-Freeze. Fix (append-evict, Gate auf `pendingAppends>0`, Overflow vom DOM-Boden evicten): **80 ms → 7,4 ms** (>10×), DOM bleibt exakt == Daten (Cap/Reihenfolge/keine Dupes), über 5000 Completions verifiziert.
**Regel 1 — Magnituden NICHT raten, LESEN:** „wächst über Zeit" ist eine Annahme über GRÖSSE. Die echte electron-config.json war 52 KB (History 23 Zeilen) — ein einziger `node`-Read killte die ganze Config-I/O-Theorie. Bevor man eine „skaliert-mit-X"-Ursache fixt: X am echten Artefakt messen (Dateigröße, Array-Länge, Job-Count im persistierten State).
**Regel 2 — Im ECHTEN Renderer-Engine profilen, nicht analytisch raten:** jsdom rendert kein Blink-Layout. Playwright (Chromium = Electron-Blink) mit `performance.now()` um (a) Rebuild und (b) erzwungenes Relayout nach Style-Write liefert die Zahl, die entscheidet: 3 ms = unsichtbar, 80 ms = DIE Ursache. Dieselbe Messung ist Fix-Auswahl UND Vorher/Nachher-Verifikation (das Goal verlangt „verifiziere dass behoben" — ein grüner Test beweist Korrektheit, NICHT dass der Lag weg ist).
**Regel 3 — Multi-Agent-Findings gegen primäre Evidenz prüfen (Control-Char-Falsch­positiv):** Der Hunt meldete HIGH-ish einen „_sessionFileKeys delete-key separator mismatch". Beim Versuch ihn zu fixen matchte der Edit-`old_string` NICHT. Char-Code-Dump (`HAS_U0001: True`) zeigte: die Zeile hat ECHTE U+0001-Zeichen — die Read-Tools der Verifier-Agenten rendern Steuerzeichen unsichtbar, sie schlossen fälschlich „keine Separatoren". KEIN Bug. **Wenn ein Fix-`old_string` nicht matcht obwohl Grep ihn zeigt: Char-Codes dumpen, bevor man dem Tool misstraut — die Quelle kann unsichtbar von der Read-Anzeige abweichen.**
**Regel 4 — Den negligible-aber-realen Befund mit Zahl ABLEHNEN, nicht aus dem Bauch:** queueJobs O(N)-Scan pro Render (wächst unbounded, da removeFromQueueOnDone=false UND Folder-Monitor EINEN Batch via addJobs am Leben hält → 500-Cap-Prune feuert nie) — real, aber Blink-gemessen <0,1 ms bei 3000 Jobs. Den riskanten Inkremental-Counter-Refactor mit DIESER Zahl skippen, nicht mit fühlt sich klein an".
**Wie anwenden:** Append-only-Optimierungen, die auf Längenwachstum gaten, brechen still an JEDEM Cap (push-then-slice fixiert die Länge) — stattdessen die Anzahl NEUER Items zählen und am Boden evicten. „Mach es wie die Queue-Tabelle (virtualisieren)" war hier NICHT nötig: die Messung zeigte stehende 2000 Zeilen kosten median 0,4 ms; nur der Rebuild war teuer. Simplest-Fix der die gemessene Ursache trifft schlägt die größere Architektur-Änderung.

View File

@ -1,36 +1,52 @@
# Tailscale / network-bind diagnostics — match the downloader (rd-diagnostics-mcp) # Long-running lag — root cause + fix (symptom: UI laggy over time, CPU 40%/RAM 6GB both normal)
Goal: make the MHU read-only diagnostics reachable like the Real-Debrid-Downloader does over ## What "laggy after time, low CPU, stable RAM" actually was (MEASURED, not assumed)
Tailscale — host embedded in the connection code, two bind modes (local / network), and a First instinct (main-process config I/O scaling with history) was WRONG for this user: the
fail-closed IP allowlist as the access gate. No Tailscale auto-detection (the downloader has none); real config is tiny (history 23 rows / 4.8 KB, total 52 KB, queue ~153 jobs). Measured with a
Tailscale is just one of the offered interface IPs reached over the tunnel, gated by the allowlist. one-off node read of the live electron-config.json. So serialize-cost theories were dead on arrival.
## Plan A 13→18-agent leak hunt + adversarial verify + a Playwright/Blink microbenchmark found the
- [ ] lib/ip-allowlist.js — fail-closed allowlist (normalizeIp/::ffff:, isLoopback, ipv4ToInt, matchIpRule exact+CIDR+wildcard, evaluateClientAllowed: loopback always, empty=loopback-only). + unit tests. 真 cause:
- [ ] remote-server.js — accept config.allowlist; reject non-allowlisted peers at connection (close 4005). + protocol test.
- [ ] config-store.js — diagnostics subtree: bindMode ('local'), publicHost (''), allowlist ([]).
- [ ] main.js — bindMode->host (local=127.0.0.1, network=0.0.0.0, network requires non-empty allowlist); buildDiagnosticCode with host (h); getSuggestedRemoteHosts (os.networkInterfaces); pass allowlist; IPC save-settings/status.
- [ ] gateway/code.js — decode h/p/t/n/fp/s (tolerant of old port/token/label). + test.
- [ ] gateway/index.js — connect_server takes host from the code; host arg optional override.
- [ ] renderer/app.js — bind-mode selector, publicHost input + suggested-host chips, allowlist textarea (network), network-requires-allowlist validation.
- [ ] docs/remote-diagnostics-setup.md — network mode + allowlist + Tailscale (set allowlist to your tailnet, e.g. 100.64.0.0/10).
- [ ] Tests: ip-allowlist unit, remote-server allowlist protocol, gateway decode, integration (network bind + allowlist), adversarial fail-closed (empty allowlist rejects non-loopback).
- [ ] Release v3.3.86 (gitea + github mirror).
## Review (done) ### ROOT CAUSE (confirmed + profiled): recent-uploads panel append-only path defeats itself at the cap
All steps implemented and verified. lib/ip-allowlist.js (fail-closed, ::ffff:, CIDR incl. - `renderRecentUploadsPanel` had a cheap append-only fast path gated on `rows.length > _recentLastRenderedLen`.
100.64.0.0/10) + 8 unit tests. remote-server.js rejects non-allowlisted peers (close 4005), - `maybeAddSessionFile` caps `sessionFilesData` by push-then-slice (2000 → 2001 → sliced back to 2000).
opt-in via config.allowlist (existing remote-control unaffected) + 2 protocol tests (fail-closed - So once past SESSION_FILES_CAP, `rows.length` is pinned at 2000 → the gate is FALSE forever →
wiring + loopback-always-allowed). Code now carries the host (mhu1_{v,h,p,t,n,fp?,s?}); gateway EVERY completion fell through to `tbody.innerHTML = rows.map(...).join('')` — a full ~2000-row
decode is tolerant of the legacy long keys; connect_server takes the host from the code (host arg rebuild. Cap is per (link × file × hoster), so 45 hosters hit 2000 at only ~400500 files.
optional override) — proven end-to-end by the integration harness connecting with NO host arg. - Blink measurement (table-layout:fixed, same engine as Electron): full 2000-row rebuild = **~80 ms**
Renderer: bind-mode selector + public-host input + suggested-host chips + allowlist textarea + on EVERY completion past the cap. At several completions/sec that is a repeating ~80 ms main-thread
network-requires-allowlist validation. Docs rewritten for Tailscale (set allowlist to the tailnet, freeze → exactly "fine fresh, gets laggy after many uploads, CPU/RAM fine."
put the Tailscale IP/MagicDNS in the code address). 393 app tests + 9 gateway tests + e2e +
integration + adversarial all green, lint 0 errors.
## Security model shift ## Fix (shipped) — append-evict, keeps the panel append-only past the cap
v3.3.85 hard-locked loopback. This change replaces that with the downloader's model: network bind - Track newly-pushed rows in `_recentPendingAppends` (incremented in maybeAddSessionFile), consumed
(0.0.0.0) is allowed ONLY with a non-empty fail-closed IP allowlist (empty => loopback only). The every render. Gate the fast path on `pendingAppends > 0` (not length-delta) so it survives the cap.
allowlist (real socket peer, ::ffff: normalized, CIDR) + token are the gate; the tunnel - Prepend the new rows, then evict the same overflow count from the DOM bottom (oldest, = data front
(Tailscale/WireGuard) is the confidentiality layer. Plaintext ws:// — document the trust boundary. eviction in date-desc) to honour the cap. DOM work back to O(added).
- Gate behind an explicit `appendOnly` flag passed ONLY by scheduleRecentRender's rAF, so selection/
delete/clear/sort/batch-done renders stay full + correct (no wrong-row eviction, no double-prepend).
- VERIFIED in Blink over a simulated 5000-completion session: per-frame render **80 ms → median 7.4 ms**
(>10×), DOM stays exactly == data (cap held, newest-on-top, oldest evicted, ZERO duplicates).
## Also shipped earlier (29d1944) — defensible, but NOT the cause for this user
- T1 ConfigStore in-memory cache (mtime/size keyed) + lean _serializeForDisk (clones only hosters) +
copyFileSync .bak. T3 cache logMode/logFilePath (drop load() from the 500 ms log flush).
## Investigated and DISMISSED with evidence
- `_sessionFileKeys` "delete-key separator mismatch" (workflow flagged it HIGH-ish): FALSE POSITIVE.
Line uses REAL U+0001 chars (char-code dump: `HAS_U0001: True`); the verifier agents' Read rendered
the control chars invisibly and wrongly concluded "no separators". Keys match at runtime. No leak.
- queueJobs O(N) per-render scans (grows unbounded since removeFromQueueOnDone=false AND folder-monitor
keeps ONE batch alive via addJobs so the 500-cap prune never fires): REAL but Blink-measured at
**<0.1 ms even at 3000 jobs** imperceptible. Incremental-counter refactor NOT worth the risk. Skipped.
- "Continuous standing relayout of 2000 rows": Blink median 0.4 ms regardless of row count; its spikes
were caused by the rebuild (same root cause). Not a standing cost → no need to virtualize/lower cap.
## Deferred (separate, not this symptom)
- doodstream-upload.js `_debugLog`: ungated SYNC fs.appendFileSync + statSync ~1020×/upload on the
main loop. CONSTANT cost (does not grow over a session), only when doodstream is active. Real freeze
contributor but NOT the reported progressive lag — convert to buffered-async like main.js, separately.
## Verification summary
- 397/397 tests pass, eslint 0 errors.
- Blink before/after: recent-panel render 80 ms → 7.4 ms; correctness asserted (cap/order/no-dupes).

View File

@ -253,6 +253,32 @@ describe('ConfigStore', () => {
assert.equal(config.globalSettings.alwaysOnTop, true); assert.equal(config.globalSettings.alwaysOnTop, true);
}); });
it('load() returns independent clones — mutating one result must not leak into the cache', () => {
store.load(); // warm the cache
const a = store.load();
a.globalSettings.alwaysOnTop = true;
a.hosters['voe.sx'].push({ id: 'mutant' });
a.history.push({ id: 'ghost' });
const b = store.load();
assert.equal(b.globalSettings.alwaysOnTop, false, 'mutating a prior load() result must not corrupt the cache');
assert.equal(b.hosters['voe.sx'].length, 0);
assert.equal(b.history.length, 0);
});
it('load() reflects an external file change (mtime/size cache invalidation)', () => {
store.load(); // warm cache on the no-file defaults
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: true } }), 'utf-8');
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'an external write must invalidate the cache');
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: false } }), 'utf-8');
assert.equal(store.load().globalSettings.alwaysOnTop, false, 'a second external write must be seen too');
});
it('save() invalidates the cache so the next load() sees the new value', async () => {
assert.equal(store.load().globalSettings.alwaysOnTop, false);
await store.save({ globalSettings: { alwaysOnTop: true } });
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'load() after save() must reflect the write');
});
it('backup recovery when main file is corrupted', () => { it('backup recovery when main file is corrupted', () => {
// Write valid config first // Write valid config first
fs.writeFileSync(store.filePath, JSON.stringify({ fs.writeFileSync(store.filePath, JSON.stringify({