Captures the two redaction leaks from the remote-diagnostics build: a hoster-
returned opaque token surviving because value-scrub only covers stored config
secrets, and the get_config_redacted/get_queue_state default paths skipping
pattern-scrub entirely. The discriminator the first E2E missed, plus the rule:
exercise each collector with its DEFAULT args and seed fixtures with a NON-config
secret so you test pattern-scrub, not value-scrub by accident.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_queue_state with includeJobs:true (the DEFAULT path) scrubbed the job list
with value-scrub only, so an opaque token a hoster returns inside a job error
(token=... that is NOT one of the user's stored credentials) survived in the
response. Same leak class already fixed for get_config_redacted and
server_health, still open on the queue collector's default path.
The first end-to-end gate missed it on two coincidences: server_health calls
getQueueState with includeJobs:false (no job error ever serialized), and the
fixture's queue error used a value that WAS a config secret (so value-scrub
caught it anyway). The direct get_queue_state{includeJobs:true} path with a
non-config token was never exercised.
- getQueueState job list and getRotationState now go through _deepRedact
(per-leaf pattern + value scrub), matching the other collectors.
- e2e-verify.mjs now plants a non-config token in a queue job error and asserts
both get_queue_state{includeJobs:true} and the default-args call leak nothing.
- Added a main-suite regression test for the default includeJobs path.
386 app tests + 9 gateway tests + e2e gate pass; lint 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the connect-by-code side of remote diagnostics and closes two real
secret-leak vectors that an end-to-end gateway<->agent test surfaced.
Gateway (gateway/, local stdio MCP, Claude connects once):
- 14 read-only tools (server_health hub, read_log, list_logs, list_errors,
get_queue_state, get_history, get_config_redacted, get_system_info,
get_rotation_state, get_app_events + connect/disconnect/list/current).
- The HOST is always supplied by the operator, never taken from the code.
- TLS fingerprint pinning is enforced in the socket 'open' handler BEFORE the
token is sent (wss opt-in); plain ws is loopback-only.
- registry.json (holds bearer tokens) is gitignored; only an empty example ships.
Security hardening (gates every off-box payload):
- redactLogText now scrubs opaque bearer/token-family secrets that are NOT
stored config credentials (e.g. a session token a hoster returns inside an
error string): bare token/auth_token/refresh_token/session_token + standalone
"Bearer <opaque>". Benign "token bucket" prose is left intact.
- get_config_redacted deep-redacts every string leaf (JSON-safe, per-leaf, so
the cookie/sess line patterns can't gobble across a compact-JSON field) and
drops the history subtree (served by get_history with its own per-error
redaction). This plugs leaks via globalSettings.pendingQueue[].error etc.
Bind-address safety:
- _safeDiagBindAddress() forces the diagnostic agent to 127.0.0.1/::1; the
0.0.0.0 UI option is removed. Direct LAN/Internet bind stays disabled until
encrypted transport (wss) exists — remote access goes through an SSH/VPN
tunnel to loopback. (Never plaintext ws:// on all interfaces.)
Tests: end-to-end gateway<->agent gate (connect -> server_health/read_log/
get_config_redacted, asserts zero secret leakage, rejects doodstream log, path
traversal and write ops); + redaction regression tests in the main suite.
385 app tests + 9 gateway tests pass; lint 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the server/app half of a remote-diagnostics system so Claude (via a local
MCP gateway, added separately) can read a server's full state to diagnose
problems: logs, errors, queue/app state, redacted config, system health. All
read-only; security review folded in as hard requirements.
Architecture (design-verified):
- A SECOND, independent RemoteServer instance (diagnosticMode) on its own port
(default 9110, bind 127.0.0.1) with NO capture/input callbacks, so screen
capture and sendInputEvent are structurally unreachable from the diagnostic
path. The existing screen-share server (port 9100) is byte-for-byte unchanged.
- lib/remote-server.js: a `host` bind option, a post-auth `diag-request` branch
that delegates to onDiagnosticRequest and replies with a reqId-correlated
`diag-response`, a `diagnosticMode` guard (capture never spawns), a
timing-safe token compare (length-guarded), and getLastAccess().
- lib/diagnostics-agent.js: handle(op,args) enforces a HARDCODED read-only op
whitelist as the sole authority — no write/exec op, no run_health_check.
- lib/diagnostics-collectors.js: pure, dependency-injected collectors
(server_health one-shot hub, read_log, list_errors, get_queue_state,
get_history, get_config_redacted, get_system_info, list_logs, get_app_events,
get_rotation_state, get_health) reusing support-bundle + stats.
Security (all mandatory, implemented):
- Redaction gates every off-box payload. support-bundle now adds webhookUrl +
diagToken to CRED_KEYS, and exports redactLogText (value-scrub of live secret
strings + pattern-scrub of Discord webhooks / Bearer / api_key= / cookies /
sess ids) + valueScrub + collectSecretValues. read_log takes a logical NAME
(no path traversal); doodstream-debug.log is excluded from the readable set
(it logs live api-key-bearing HTML). grep is length-capped (ReDoS guard).
- diagnostics config subtree (enabled/port/token/label/codeIssuedAt/bindAddress)
defaults OFF, bind 127.0.0.1; deep-merge makes it migration-free. The
server-owned token is preserved against renderer clobber in both
save-global-settings handlers; the diagnostics:* IPC is the only mutator.
- app.requestSingleInstanceLock() (also the approved fix#6) so a relaunch can't
EADDRINUSE-kill the agent and two instances can't clobber the config.
main.js: buildDiagnosticCode (mhu1_ base64url, no host embedded), start/stop +
auto-start-on-launch + stop-on-quit, the four diagnostics:* IPC handlers.
renderer: a "Diagnose-Zugriff" settings subtab (toggle, port, bind-address,
copyable connection code + regenerate, status, read-only security note).
382 tests pass (incl. 11 new: collectors+redaction, agent whitelist, and a live
RemoteServer protocol test proving auth->diag-response correlation, that a
diagnostic client never triggers the capture window, and brute-force lockout).
Lint clean (0 errors). Smoke boots identically to baseline. The standalone MCP
gateway package + operator setup docs land in a follow-up commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three lessons from the second adversarial bug hunt:
- A dedup/skip key has more re-add paths than "in-place vs rebuild" suggests; enumerate every .add site and every selectedFiles re-add, cross-tabulate which clears the key. retrySelectedJobs (retry-of-done) and the folder-monitor branch were the two missed paths.
- Per-cell (file|hoster) deletion needs its own persisted suppression set with the OPPOSITE lifecycle to the completed-key (set on delete, cleared on re-add); overloading the completed-key would break re-upload-after-delete.
- Re-confirming a user-facing change is "real" is not the user consenting to the behavior change; force the decision via one AskUserQuestion or park it, do not re-raise it as ambient worry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 36-agent adversarial sweep over the queue-persistence context surfaced three
NEW, reachable defects (all renderer-only) that the v3.3.80-82 work missed:
#1 retrySelectedJobs (the reuploadBtn / "erneut hochladen" path) is a THIRD
re-add path that never cleared the completed-dedup guard. _completedUploadKeys
gains file|hoster when a job reaches done; retry of a done row reset the job to
pending and re-added its path to selectedFiles but kept the key. On a restart
before the re-upload re-completes, restoreQueueStateFromConfig re-seeds the key
and buildQueuePreview then refuses to recreate the row, so the deliberate
re-upload silently vanished (lost work). Retry now clears the exact file|hoster
keys it re-activates (per key, NOT per path, so a sibling hosters completed
ghost is not resurrected).
#7 The folder-monitor pre-selected-hosters branch re-adds a path and calls
buildQueuePreview WITHOUT going through applyHosterSelection, so the v3.3.82
key-clear never ran. With removeFromQueueOnDone ON, a re-encoded / re-dropped
file whose row was auto-removed was suppressed forever (silent no-upload that
even survived a restart). The branch now clears the dedup keys for the freshly
re-added paths, matching the manual modal flow.
#4 Deleting one hosters row of a multi-hoster file was silently undone. A
manually deleted non-completed file|hoster pair is recreated by buildQueuePreview
the next time it runs (adding another file, or a folder-monitor drop), because
the recreate guard only consulted _completedUploadKeys, never the per-cell
deletion. Result: the file got uploaded to a hoster the user explicitly
removed, burning quota and publishing an unwanted link. Introduces
_suppressedPreviewKeys: a manual delete of a non-done job whose file stays
selected (pinned by a sibling job) suppresses that exact file|hoster from
re-creation; the suppression is persisted in pendingQueue.suppressedKeys and
re-seeded on restore (mirroring completedKeys), and is cleared whenever the
user deliberately re-adds the file (modal or folder monitor) so a real re-add
still works.
The clear-on-re-add logic for both sets is unified into clearDedupKeysForPaths()
and reused by applyHosterSelection and the folder-monitor branch.
Held for a separate decision/round (surfaced to the user): single-instance lock,
the sync-close write-sequence guard (its ghost symptom is masked by the existing
microtask-ordered removal + restore-time ts-gate; its real residual is rotation/
history integrity), and the upload-log timezone/same-basename edges (need a log
format change). 362 tests pass, lint clean, smoke boots identically to baseline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Captures the v3.3.82 near-miss the advisor caught: persisting _completedUploadKeys
risked silently no-op'ing a re-upload IF any retry path rebuilt jobs via
buildQueuePreview. Both retry paths (retrySelectedJobs, _retryFailedFromBuckets)
mutate jobs in place and are immune; only fresh modal re-adds go through the
preview and clear the key. Rule: read every 'erneut/retry/reupload' handler
before touching a skip-guard, decide mutate-vs-rebuild per handler.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The completed-upload dedup guard (_completedUploadKeys, "file|hoster") is the
single source of truth that keeps a finished job from being re-materialised as a
"Bereit" preview by buildQueuePreview(). Three independent paths leaked ghosts
back into the queue:
A) removeJobFromIndex() unconditionally DELETED the dedup key. The
removeFromQueueOnDone auto-remove path (handleProgress), the batch-done sweep,
and the terminal-job prune all call removeJobFromIndex on a *finished* job, so
the guard handleProgress had just added was immediately wiped and
buildQueuePreview re-created the job as a preview both mid-session and after
restart. removeJobFromIndex now takes keepCompletedKey; the three auto-removal
call sites pass true (keep the guard), while the manual-delete sites keep the
old behaviour (drop the guard so the user can re-add the file).
D/E) The dedup guard lived only in memory, so every restart began with an empty
set and buildQueuePreview rebuilt ghosts from the persisted selectedFiles.
buildPersistedQueueState() now serialises the keys whose file is still in the
snapshot (completedKeys) and restoreQueueStateFromConfig() re-seeds them. To
keep deliberate re-uploads working, applyHosterSelection() clears the
persisted key for every freshly (re-)added file: re-adding through the hoster
modal is the explicit "upload this again" signal. Both retry paths
(retrySelectedJobs, _retryFailedFromBuckets) mutate the existing job in place
instead of relying on buildQueuePreview, so they are unaffected.
H) buildQueuePreview() excluded error jobs from its existingKeys set, so a
file+hoster that already had an 'error' row got a second 'preview' row stacked
beside it. Error jobs now count as existing.
B/C) parseUploadLogLine() took the filename from a fixed field index and trimmed
it. A pipe inside the link shifted the field (entry lost from the log-based
dedup) and trimming broke matching against the untrimmed OS basename used as
the queue-job key. The filename is now the last non-empty field and is no
longer trimmed, so log lines with pipes in the URL and leading-space filenames
both match end-to-end.
362 tests pass, lint clean on all touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up hardening on the v3.3.80 queue-persistence fix after a deeper adversarial
sweep + reviewer pass over the WHOLE subsystem (not just the diff). Two real defects:
1. Basename-collision lost work (regression introduced by v3.3.80 FIX A).
restoreQueueStateFromConfig collapses jobs on the FULL path while the ts-gate keys
on basename|hoster. Two genuinely different files with the same basename queued to
the same hoster from different folders therefore share a gate key: if one was logged
after savedAt, the ts-rule dropped BOTH — silently losing the still-pending one.
Pre-FIX-A only 'done' jobs were dropped, so a pending file was never at risk.
Fix: an ambiguity guard in partitionRestoredJobsByLog — the ts-rule is suppressed
when a basename|hoster key maps to more than one distinct file path (the log records
only basenames, so it can't say which physical file completed). The done-in-log rule
is unchanged. Fails safe: worst case a visible ghost survives, never silent data loss.
2. selectedFiles re-materialization with removeFromQueueOnDone=ON (second mechanism,
independent of the stale snapshot). When that setting is on, a completed job is
stripped from queueJobs but its path stays in selectedFiles (syncSelectedFilesFromQueue
only runs at batch-done, never on a mid-upload close). On restart the ts-gate operates
on queueJobs and never sees it, then the startup updateUploadView -> buildQueuePreview
re-creates it as a preview ghost AFTER the gate ran, and it re-persists with a fresh
savedAt — sticky. Fix: completedSelectionKeys() seeds _completedUploadKeys (the set
buildQueuePreview already consults) from the log at startup, keyed on full path, with
the same ambiguity guard. Log-based so it survives a hard kill, consistent with FIX A.
Also extracts the orphan-tmp sweep decision into lib/orphan-tmp.js (was untested inline
code in main.js; behavior-preserving) and adds executable coverage for the paths that
were previously only argued from logic: orphan-tmp sweep, config-store pendingQueue+savedAt
round-trip, an end-to-end scenario in the exact user-reported shape (300 queued / ~200
finished mid-session), and a 3000+500-iteration property fuzz of the gate invariant
including the lost-work guarantee. 359/359 green, ESLint clean, smoke-boot unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closing the app (especially during an active upload or on a hard kill) and
reopening sometimes left already-uploaded files sitting in the queue as if
still pending. Root cause is three layers stacked:
1. Persist-starvation: persistQueueStateSoon() reset a 10s debounce on every
progress event, so during an upload the on-disk queue snapshot was never
rewritten and stayed frozen at the pre-upload state (all jobs "preview").
2. The beforeunload sync flush only covers a clean close; a hard kill / crash
leaves that stale snapshot on disk.
3. Startup auto-dedup only dropped jobs with status "done". The completed
files were stored as "preview" in the stale snapshot, so they survived and
reappeared.
Fix (mechanism-independent — holds whether the stale snapshot came from
starvation, a mid-upload close race, or a hard kill):
FIX A (core, durable): the upload log is the source of truth. Each persisted
snapshot is now stamped with savedAt; on restart any restored job whose newest
matching log entry is timestamped at/after floor(savedAt) is dropped regardless
of status — it provably completed after the snapshot, so a "preview" row for it
is a ghost. lib/queue-dedup.js gains an additive 3rd savedAt param; without
savedAt or without log timestamps it behaves exactly as before (the 5 canary
tests stay green, so intentional re-uploads of older files still survive).
FIX B: new lib/throttle-timer.js with a max-wait. During uploads the snapshot
is now written at most ~20s into a continuous progress burst instead of never;
idle stays a pure debounce. The fallback shim honors max-wait too, so a missing
library can never silently reintroduce the starvation.
FIX C: the synchronous close-write retries renameSync on EBUSY/EPERM/EACCES and
uses a pid-unique tmp (de-conflicts it from config-store._atomicWrite's fixed
.tmp). A startup sweep reclaims orphaned <config>.<pid>.tmp files left by a hard
kill between write and rename.
lib/upload-log.js extracts formatUploadLogLine + parseUploadLogLine from main.js
so the real writer -> reader -> gate seam is unit-tested (a future log-format or
epoch-basis change can no longer pass green while breaking the fix).
Verified: 334/334 tests green (incl. throttle fake-clock starvation/maxWait,
ts-gate multi-hoster partial-completion, and the real-format seam tests), ESLint
clean, smoke-boot identical to baseline, and an adversarial multi-agent review
(15 findings, 14 refuted, 1 low — the tmp orphan, now swept) on the diff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v3.3.78 rejected upload-server hosts ending in the bare ".filemoon" label so the
lookup would retry for a valid server. In practice byse is currently returning a
truncated host for its WHOLE pool, so the rejection found nothing to fall back to and
turned every byse upload into a slow "Kein Upload-Server erhalten: OK" (6x2.5s server
retry per file) instead of the fast ENOTFOUND fail it had before. That is a worse UX
with zero upside while byse is down, so revert to the v3.3.77 behavior.
The premise (byse returns a MIX of good and truncated hosts, skip the bad ones) is
unverified; the evidence is byse returning truncated hosts for everything. And the
correct upload domain is NOT safely determinable from outside: a swarm of filemoon-brand
TLDs resolve (filemoon.eu/.me/.xyz/.art/.nl, kerapoxy.cc, moonmov.pro, ...) but the ones
probed are parked/squatted (parklogic landers, shared catch-all certs), and byse's real
response is only visible with a valid key. Guessing a TLD would point uploads at a parking
page or a third party, so we do not.
lib/hosters.js is now byte-identical to v3.3.77. The v3.3.77 fixes (5xx/ECONNRESET treated
as transient -> no failover cascade/blacklist; the /api/file/list path fix) are kept.
ENOTFOUND remains transient, so byse's outage fails clean (retry same account, no cascade)
with the actual unresolvable host shown in the message. Suite 311/311.
When byse restores its server pool (returns complete hostnames), uploads resume
automatically with no client change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the v3.3.78 byse lesson: a truncated upload host (sNNNN.filemoon, no TLD)
must not be "repaired" by guessing a TLD — every resolving candidate was a parked /
squatter domain (parklogic PTR, shared catch-all cert). Verify PTR + cert SAN before
trusting a domain; discard the obviously-broken value and let the existing
retry/cache machinery find a valid one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In v3.3.77 the byse uploads started failing with "getaddrinfo ENOTFOUND
s1065.filemoon" / "s1070.filemoon" / "s1075.filemoon". The hostname has no TLD,
so DNS cannot resolve it. Root cause: byse's GET /upload/server is INTERMITTENTLY
handing back a truncated upload-server host — "sNNNN.filemoon" with the TLD dropped
(consistent with byse's ongoing filemoon migration; their own docs already expose an
old_domain/new_domain embed switch). Some uploads still work because byse returns the
complete host on those; the failing ones got truncated. byse did not change the API
contract — this is malformed data from their server pool.
The correct upload domain is NOT externally verifiable: every TLD that resolves for
these server IDs is a parked/squatter domain (filemoon.art -> ParkLogic "lander" PTR;
filemoon.nl -> a shared catch-all cert for kaobei.cc/babesex.xyz/...). Appending a TLD
would point uploads at a parking page (or a third party) — so we do NOT guess one.
Instead, treat an obviously-truncated host as "no valid server": normalizeAbsoluteUrl
now returns null for any host ending in the bare ".filemoon" label (never a real TLD).
extractUploadServerUrl then yields nothing for that response, so getUploadServer falls
through to its existing machinery — it retries the lookup (SERVER_RETRY_ATTEMPTS) to get
a different server and, crucially, returns the last-known-good server from
LAST_UPLOAD_SERVERS once one has been cached. So after any single complete response, the
truncated ones transparently reuse the good server instead of uploading into a DNS void.
If byse's whole pool is truncating (cold cache), the lookup fails CLEAN as hosterTransient
(no cascade, no blacklist — the v3.3.77 behavior), and the next batch tries again.
This composes with v3.3.77: ENOTFOUND was already transient (retry same account, no
failover); this stops the unresolvable host from ever reaching the POST in the first place
when a working server is available.
Tests: extractUploadServerUrl rejects https://sNNNN.filemoon / bare sNNNN.filemoon /
https://filemoon and keeps a complete host (https://s1065.filemoon.sx/...) untouched.
Suite 313/313.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the v3.3.77 byse lesson: an unclassified HTTP 502/gateway error cascaded
through the failover chain and blacklisted every account; the fix is an explicit
transientNetwork flag at the throw site (authoritative over message-keyword
heuristics) + symmetric server-lookup hardening, and throw-site tagging must be
tested end-to-end, not just flag-injected in a mock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A single byse.sx gateway hiccup (HTTP 502 "<!doctype html>..." or a mid-upload
ECONNRESET) was cascading through the whole failover chain and blacklisting every
account for the batch. Root cause: the upload-POST throw sites threw PLAIN errors
with no classification, so a 502 was treated as a GENERIC error -> mark-failed ->
emit('account-failed') -> failover to the next account (which hits the same
gateway) -> repeat until the chain is exhausted, then poison sibling files in the
batch via the blacklist. The screenshots showed exactly this: "Primär ... Fallback
#3" all failing with the same 502. byse did NOT change their API (the upload
contract still matches their docs and was live-probed: GET /upload/server -> POST
{key}+file -> files[0].filecode); these are transient infrastructure failures.
A 5xx gateway error is not an account fault: every account hits the same gateway,
so failing over is pointless and blacklisting is harmful. Fail open instead — retry
the SAME account and, if byse stays down, fail the file cleanly without touching the
account or the rotation cursor.
- upload-manager: _isTransientNetworkError() now honors an explicit err.transientNetwork
flag (checked before the empty-message guard) and, as a defensive fallback, matches
/HTTP 5\d\d/, Bad Gateway, Service Unavailable, Gateway Time-out. The flag is made
authoritative in _isFileRejectedError and _shouldSkipRetryOnAccountError (both return
early when it is set) so a 5xx whose HTML body happens to contain a rejection/auth
keyword can never be mis-binned as file/account. The post-rotation retry loop also
breaks on a transient error (parity with the primary loop) to avoid burning the
retry budget re-uploading on a fallback.
- hosters: the upload POST throw sites tag err.transientNetwork when statusCode >= 500
(non-JSON body and non-2xx-with-JSON) and when the 2xx status-envelope carries
status:500; 401/403/429 stay PLAIN so they remain account errors. The server-lookup
path (apiGet/getUploadServer) is hardened symmetrically: a 5xx there tags
transientNetwork and getUploadServer preserves the flag onto its wrapped error, so
the heuristic shouldRetryServerLookup() can no longer be defeated by a 5xx body that
contains an auth keyword.
- hosters: fixed a separate real bug surfaced during investigation — _fetchByseFileList
built the wrong URL https://api.byse.sx/api/file/list (the /api/ prefix is correct
only for doodstream's doodapi.co host; byse already carries the api. subdomain).
Live-probed: /api/file/list 302-redirects to the docs page, /file/list returns 200
JSON. The wrong path made the byse async-recovery poll silently dead (always []),
removing the safety net that reclaims a large file that registered despite a 502.
Now https://api.byse.sx/file/list.
ECONNRESET was already transient and is unchanged. doodstream's 2xx empty-form stays
hosterTransient (one attempt, no re-upload). vidmoly/clouddrop use their own uploaders
and are unaffected; doodstream/voe apiKey uploads share uploadFile and correctly
benefit from the same 5xx-is-infra logic.
Deferred follow-up: poll-first-on-5xx dedup (route a 5xx through the now-working
recovery poll before retrying, to reclaim a registered-but-502 file instead of
re-uploading). It does not add new dupe risk vs the prior cascade and a pure 502
means the backend was never reached, so retry-same-account is dupe-safe for it.
Investigated and reviewed by two multi-agent workflows (4-lens investigation with a
synthesized fix spec; 4-lens adversarial review with per-finding verification): no
API drift, 0 confirmed defects. Tests: classifier units (flag-above-message-guard,
5xx-transient, 4xx-stay-account), an end-to-end 502/503/500-envelope tag through
uploadFile, and a 502-retries-same-account-no-cascade integration case. Suite 311/311.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The byse suspect size-memo (softened in v3.3.73 to arm only after two
confirmed suspect rejections on the same account and only for strictly
larger files) still occasionally pre-skips a primary account with
"Bekanntes Größen-Limit auf diesem Account", which the user finds annoying
when they would rather just let every file attempt the upload for real.
Add a per-hoster opt-out: a new "Größen-Limit merken" checkbox (Accounts tab,
default ON so existing behavior is preserved for everyone) that, when
unchecked, disables the memo-based pre-skip entirely for that hoster.
- config-store.js / upload-manager.js DEFAULT_SETTINGS: sizeMemoEnabled: true.
- upload-manager.js _suspectMemoBlocks(): returns false up front when the
hoster has sizeMemoEnabled === false. That single guard covers BOTH consult
sites — the pre-job block (skips the guaranteed-fail upload) and the
alternate-account skip — since both route through _suspectMemoBlocks. The
memo is still recorded; it is simply never allowed to block. This moves the
flow toward "fail open": with the toggle off, every file always gets a real
attempt on its account.
- The genuine suspect-reject alternates walk is gated on err.suspectReject,
not the memo, so disabling the memo does not weaken real failover — a file
the account actually rejects still rolls over to the next account.
- renderer/app.js: checkbox data-hs="sizeMemoEnabled", checked when
hs.sizeMemoEnabled !== false; persisted by the existing generic checkbox
path in saveHosterSettingsFromDom(). The running manager picks it up on the
next batch (fresh UploadManager) and immediately on save (updateSettings).
Tests: suspect-reject-alternates gains a disabled-memo case proving the larger
third file still gets a real primary attempt (key1 tried 3x, no
suspect-memo-skip event); config-store gains a default-true + persist-false
case. Full suite 305/305. Reviewed by a 4-lens adversarial workflow
(completeness / composition / persistence-ui / conventions): 0 findings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the v3.3.74→v3.3.75 lesson: a fresh-per-call round-robin picker
silently no-ops under the folder-monitor drip-feed (each add-jobs-to-batch
call rebuilt the cursor from 0 → account 1 every time). Distribution state
that must be fair across multiple call entry points has to outlive the calls,
and across sessions when the goal is a rolling quota.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v3.3.74 created a fresh account picker inside every buildUploadTasks /
buildUploadTasksFromJobs call, so its in-memory rotation index reset to 0
on each call. That is correct for a one-shot batch (drag-drop N files at
once distributes fine), but it silently no-ops in exactly the pattern the
byse 80-uploads/30-days quota cares about: the folder monitor feeds new
files into a *running* batch one detection at a time via add-jobs-to-batch
(renderer -> add-jobs-to-batch IPC), and per-detection autoStart likewise
fires start-upload per file. Each of those calls rebuilt the picker from
index 0, so account 1 won every single time and the secondary accounts
never received anything.
The rotation cursor now lives outside the picker and survives across calls
and across app restarts (the quota spans a rolling 30-day window, so a
session-only counter would still starve secondaries for users who restart
between small uploads):
- account-rotation.js: createAccountPicker() accepts a seed { indices } map,
resumes the per-hoster cursor from it, and exposes indices() (the advanced
cursors) + dirty() (whether any rotation actually happened this call). The
cursor is a monotonic counter taken mod the current enabled-account count,
so it keeps wrapping correctly even if an account is later enabled/disabled.
- config-store.js: new top-level rotationCursors map (added to DEFAULTS, read
back in load() which otherwise reconstructs the result and would drop
unknown keys) plus a saveRotationCursors() method. The existing
read-modify-write save() preserves it across unrelated settings/credential
saves; secret-store never touches it.
- main.js: a module-level _rotationCursors is the authoritative source of
truth (seeded once from disk on first use, updated synchronously per batch),
with config as restart-survival backing. makeAccountPicker() seeds the
picker from it; persistRotation() folds the advance back and flushes to
config only when dirty. Authoritative in-memory state also closes the
disk-read race two rapid batches would otherwise hit. Both task builders
now receive the picker instead of constructing their own.
Composition with failover is unchanged and verified: the picker only chooses
each file's *initial* account and is never called on the failover path. The
pre-job-swap reroutes a task only when that task's own account is in
_failedAccounts, so a dead account's jobs reroute while healthy accounts keep
their rotation share. No double-advance of the cursor.
Tests: account-rotation gains seeded-resume, drip-feed-across-pickers (the
regression), dirty()-semantics, carry-forward, and count-shrink-wrap cases;
config-store gains rotationCursors default, save round-trip, no-clobber, and
credentials-undisturbed cases. Full suite 303/303.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
byse.sx requires each account to upload >=80 videos in the last 30 days
(measured across a rolling 3-month window) or it goes dormant. With a
single primary account doing all the work, the other configured accounts
decay and eventually get suspended. This adds an opt-in "Accounts rotieren"
toggle per hoster that round-robins files across every enabled account that
has credentials: file 1 -> account 1, file 2 -> account 2, file 3 ->
account 3, then wraps. Off by default, so existing single-account behavior
is unchanged.
Mechanics:
- New lib/account-rotation.js: createAccountPicker() returns a stateful
pick(hoster) closure holding a per-hoster round-robin index. It only
rotates when rotateAccounts === true AND more than one usable account
exists; otherwise it returns the first enabled account (the old primary).
enabledAccountsFor() filters disabled + credential-less accounts while
preserving configured order, so a disabled account is simply skipped in
the cycle rather than leaving a gap.
- main.js: both buildUploadTasks() and buildUploadTasksFromJobs() now build
one picker per call and use pick(hoster) instead of getPrimaryAccount(),
which is now removed (dead code). A fresh picker per batch means each
upload session starts the cycle at account 1, matching "die erste Datei
auf Account eins".
- config-store.js: rotateAccounts: false added to HOSTER_SETTINGS_DEFAULTS.
- renderer/app.js: "Accounts rotieren" checkbox in the per-hoster upload
settings (Accounts tab). Persisted by the existing generic checkbox path
in saveHosterSettingsFromDom().
Composes with failover: rotation only chooses the *initial* account per
file. On a hard account failure the existing failover (_failedAccounts +
pre-job account swap) reroutes just that account's jobs; the healthy
accounts keep their rotation share.
Single enabled account (or rotation off) = no behavior change.
Tests: tests/account-rotation.test.js (10 cases) covers off=primary,
on=round-robin+wrap, single-account no-op, skip disabled, skip no-creds,
null when none usable, per-hoster index independence, byse-only rotation,
100-file 50/50 split, and the enabledAccountsFor filter. Full suite 294/294.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-batch size memo previously armed on a SINGLE confirmed suspect rejection
and blocked every same-or-larger file on that account (fileSize >= memoSize).
A single spurious byse "Not video file format" at e.g. 1.3GB therefore pre-failed
the whole series of ~1.3GB files in that batch with "Bekanntes Größen-Limit auf
diesem Account", even when byse could actually take that size.
Now the memo:
- arms only after the 2nd confirmed rejection on the same account (count >= 2),
so a lone byse aussetzer no longer short-circuits anything; and
- blocks only STRICTLY LARGER files (fileSize > memoSize), so a same-size file
always still gets one real attempt.
_suspectSizeMemo now stores { size: smallest-rejected, count } instead of a bare
number. Still per-batch (cleared at batch start, never re-primed across batches).
Updated the two memo tests to the new semantics (added a size-aware statSync mock
so a strictly-larger file can be exercised). Full suite 284 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three changes, investigated and adversarially reviewed via multi-agent workflow.
1. Queue maximize bug: the virtual-scrolled upload queue derived its visible-row
window from #queueContainer.clientHeight but only recomputed it on 'scroll'.
Maximizing the window enlarges the container without a scroll event, so rows
below the old viewport stayed blank until you scrolled. Fix: a ResizeObserver
on #queueContainer reusing the existing rAF-coalesced _onQueueScroll. Strictly
stronger than a window 'resize' listener — it also covers the hidden->visible
view-switch case (container 0 -> real height). No feedback loop (container box
is flex-sized, not content-sized); early-returns for <200-row queues.
2. Einstellungen tab restructured from 4 dense stacked collapsible panels into
horizontal sub-tabs: Allgemein / Automatik / Logs & Diagnose / Fernsteuerung /
Backup. All sub-pages render into the DOM at once (only the active one is
shown), so every element id stays present and saveSettings keeps working. The
cluttered "Allgemein" block is split across Allgemein + Automatik + Logs.
Per-hoster settings already moved to Accounts in v3.3.69.
3. saveSettings hardened to be null-safe (elTxt/elChk/elInt helpers): when a
settings element is absent from the DOM it now keeps the current config value
instead of collapsing to a default. Behavior is identical when elements are
present; this is insurance against a future dropped id silently overwriting a
real setting (e.g. webhook URL, folder-monitor hoster pre-selection).
Verified: 43 sub-tab ids unique, all 26 saveSettings ids present, lint clean,
boot clean, 284 tests green, 0 confirmed defects from a 3-lens adversarial review.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The settings section was nested inside the collapsible cards body, so you had to
expand a hoster group first before you could open its upload settings. Moved the
section out to the group level (sibling of the cards body), so the
"Upload-Einstellungen" toggle is always visible directly under each hoster header
— collapsed or not — for every hoster. Styled as a full-width row matching the
group header. Group-cards toggle and the settings toggle are now fully independent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of "4 accounts configured, rotation only ever uses 1-3": byse
returns HTTP 200 + per-file status "Not video file format" for valid MKVs
above an account's size tier (proven from the live rotation log: account
4ha0 accepted 1158 files up to 2.62 GB and deterministically rejected all
6 files >= 2.81 GB with that status, each costing a full ~20 min upload).
The parser tagged this fileRejected, which (a) skipped the 30s recovery
poll that was originally BUILT for this misleading status (guard regression
from "byse skips 30s recovery poll on explicit reject") and (b) made the
upload manager skip rotation entirely - so the remaining accounts were
never tried and the files failed forever. Account selection is
primary-first failover, so a later account only ever runs when every
earlier one is hard-failed or disabled, which matches the reported
"account 4 only gets used when I disable the others".
The fix, layered:
- parseByseResult flags "Not video file format" as err.suspectReject
(alongside fileRejected). Genuine rejections (Duplicate, too small,
account-level disk-full) behave exactly as before.
- uploadFile runs the byse recovery poll again for suspect rejections
(the file often registers asynchronously despite the status), but skips
it when the caller's file probe says the upload is genuinely not a video
(new opts.probeIsVideoLike, threaded from the upload manager's probe).
Poll helpers now swallow an abort during their sleep and return null so
a cancelled poll surfaces the original error instead of a bare
AbortError.
- upload-manager: on a suspect rejection of a probe-verified video, the
file gets ONE attempt on each remaining pool account (new accountPools
from main.js, refreshed on save-config) - WITHOUT blacklisting the
rejecting account, which keeps working for smaller files. A per-batch
size memo (hoster:account -> smallest rejected size) plus a known-good
account preference prevent re-uploading every following oversized file
through the whole chain: the second file skips known-limited accounts
outright and goes straight to the account that took the first one.
Alternates that fail with a genuine account error (quota/ban/full) are
marked failed in-batch so no later file burns a multi-GB upload on them;
deliberately no account-failed emit there, as repointing the hoster-wide
override would reroute normal-sized files away from a healthy primary.
- Rotation hardening from the adversarial review: the rotated-to retry
loop now fast-breaks on file-rejected/hoster-transient errors instead of
burning maxAttempts full uploads, and a file-class error on a rotated
account no longer blacklists that account (it fails only the file).
Cancelling mid-alternates or mid-rotation now records the job as
aborted instead of error (this also stops the batch webhook from firing
on fully user-cancelled batches). New upload-failure rot-logs are
guarded against logging user aborts as failures; rotation retries are
now visible in the rotation log at all (previously failures on a
rotated-to account were never logged, which is why account 2's death
was invisible in the support log).
- file-probe: the mpeg-ts signature required only a single leading 0x47
byte, so GIFs and "G"-prefixed text classified as video and would have
qualified junk for pool-wide alternate uploads; it now requires TS
sync-byte periodicity (0x47 at offsets 0/188/376) and the probe head
read grew from 64 to 512 bytes to make that check possible. GIF gets
its own non-video signature.
Tests: 9 new/reworked across suspect-reject-alternates.test.js (alternate
walk, no blacklist, memo short-circuit, account-error marking, abort
labeling, probe gating) and byse-reject-recovery.test.js (suspect polls +
recovers, empty poll throws suspectReject, Duplicate still fast-fails,
probe-confirmed non-video skips the poll) plus a TS/GIF probe regression
test. Full suite 276 green.
Declutters the Einstellungen tab: the per-hoster panels (Retries, Max Speed,
Parallel Uploads, Restart-below, Interval, Max Size, log-to-file) are no longer
listed there. Each hoster group in the Accounts tab now has a collapsible
"Upload-Einstellungen" section with exactly those controls, so everything about
a hoster lives in one place. The Einstellungen tab shows a short pointer to where
they moved.
Per-hoster edits in Accounts save via a dedicated lightweight handler
(scheduleHosterSettingsSave → saveHosterSettings) that touches only
hosterSettings — it deliberately does NOT route through the global saveSettings,
so tweaking a hoster slider can't re-run the folder-monitor start/stop or clobber
global fields. The global saveSettings still reads .hs-input[data-hoster] wherever
they live, and global inputs (which have no data-hoster) are unaffected. The new
inputs drop the .settings-autosave class and bind via delegation on the persistent
accounts container so they survive re-renders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a custom HTML menu bar above the tab bar, replicating the Real-Debrid
Downloader's look: click-to-open triggers, hover-switch between open menus,
click-outside / Escape to close, right-flyout submenus.
- Datei: Dateien hinzufügen, Ordner hinzufügen, Sicherung submenu
(Export/Import backup), Neustart, Beenden.
- Einstellungen: "Einstellungen öffnen" (jumps to the settings tab) plus an
inline live grid with the max-parallel-uploads spinner and a speed-limit
checkbox + MB/s spinner, both two-way-synced with the Settings tab fields
via saveGlobalSettings (parallelUploadCount / globalMaxSpeedKbs).
- Hilfe: Log-Ordner öffnen, Diagnose-Paket exportieren, Suche Aktualisierungen.
Wiring reuses existing handlers (addFiles/addFolder buttons, doBackupExport/
Import, openLogFolder, createSupportBundle, checkForUpdate) and two new tiny
IPC handlers app:restart (app.relaunch+quit) / app:quit. initMenuBar() is
wrapped in try/catch in setupListeners so a menu fault can never break core
app init. CSS maps the downloader's menu styles onto this app's theme vars.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a bottom-right button under the accounts list that expands every hoster
group at once (and collapses them again — the label flips based on current
state). Reuses the existing _hosterGroupOpenMemory so the per-group open/closed
state stays consistent with manual header clicks. The footer is hidden when no
accounts exist. Pure DOM toggle, no re-render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Upload-Verlauf tab took 30s+ to open on large, never-cleared histories.
Measured on the live config: 18.9 MB config / 41 batches / 53,403 rendered rows.
Two independent problems, fixed together:
1. Render side (the 30s killer): renderHistoryTable built ALL rows into one
innerHTML. Now the build is capped to the newest 2,000 rows (HISTORY_RENDER_CAP)
after slicing the chronological tail, then sorted for display. A notice shows
"Zeige neueste 2000 von N" so nothing looks deleted; full history stays on disk
and "Verlauf exportieren" still emits everything (export reads loadHistory()).
2. Storage side: history grew unbounded because appendHistory only ever pushed.
New globalSettings.historyRetention ('all' default, non-destructive) with
policies: 7d / 30d / 90d (time-based) and 1000 / 100 (newest-uploads-based).
appendHistory now prunes after each batch; a new prune-history IPC re-applies
the policy immediately when the user changes it, gated behind a confirm() that
shows the exact removal count (dry-run first).
Prune model keeps whole batches (atomic): count policies accumulate rendered rows
from the newest batch backward and always keep >= the newest batch even if it
alone exceeds the target; time policies keep batches whose timestamp is missing
or unparseable (old entries have none) so ambiguous data is never silently dropped.
New retention dropdown lives in the Verlauf header. 9 unit tests cover the prune
edge cases incl. a realistic 41-batch fixture. Full suite: 272 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
