User report: in session mode the upload-log lines split across two files — the
first few before the auto-persist fallback fired, the rest after into a path
with the session-stamp DOUBLED.
Root cause in main.js _persistFallbackLogPath:
1. The strip was gated on the legacy `sessionLog` boolean, which 3.3.35 retired
in favour of `logMode`. So in session/daily mode the gate was false and the
resolved path got persisted with its stamp intact.
2. Even when the gate triggered, its regex matched only the daily YYYY-MM-DD
suffix, not the session "session-YYYY-MM-DD_HH-MM-SS-pid" suffix.
The next getLogFilePath() call read that saved path as the "base", treated the
already-stamped filename as the base name, and re-applied another stamp on top.
First flush hit the original session file; everything after hit a doubly-
stamped one — exactly the symptom (top file: 2 lines, bottom file: the rest).
- lib/log-mode.js: new pure stripModeStampFromFileName helper that removes both
the daily and the session suffix patterns. Anchored to $, no nested
quantifiers (linear).
- main.js: gate on logMode (not sessionLog) and call the helper for daily AND
session, so logFilePath always persists as a bare base.
- Tests: 4 new — strip behaviour + an idempotence regression that locks in
"resolve → strip → resolve = same path" so this can't silently come back. 204/200.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a third choice next to the existing single-file and per-day modes: a new
log file is created at every app start (process boot) and used until the app is
closed. A close → reopen of the app starts a new session, hence a new file.
File pattern: fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log.
The boolean sessionLog field — misnamed: it actually toggled daily mode — is
replaced by a logMode enum: "single" | "daily" | "session". The misnomer made
the migration the trap to watch: existing users with sessionLog:true must land
on "daily", NOT "session". normalizeLogMode handles this and is unit-tested.
- lib/log-mode.js (new, pure, dual CJS/window export): normalizeLogMode +
resolveLogFileName + format helpers. No fs, no Date.now() at call time.
- config-store.js: normalize at the single load() boundary so downstream
readers consume logMode only. logMode is deliberately NOT seeded in DEFAULTS
(would beat the legacy migration after merge).
- main.js: stamp SESSION_ID once at process start (with pid hedge against
same-second restart collisions); getLogFilePath and buildFallbackLogName
switch on mode via the lib. _resolveUploadLogTarget cache key is now just
the primary path, which already encodes mode/date/session — self-invalidates.
- renderer: <select> with three German labels replaces the old checkbox;
saveSettings writes logMode; index.html loads the lib so window.LogMode is
available in renderSettings.
- Tests: 14 log-mode tests (incl. legacy-migration regression), 3 config-store
tests (defaults, legacy migration, round-trip all three values). 200/200.
End-to-end simulated locally: two launches → two distinct session files; PID
hedge produces distinct names even within the same second.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Live diagnosis against the real account: it holds 90,548 files. The recovery
fetched only page 1 of /api/file/list without forcing order, so a just-uploaded
file could be missed if the default sort isn't newest-first. Add
sort=created&order=desc — verified to return the account's newest uploads first —
so the codeless-result recovery reliably finds the file regardless of account
size.
Diagnosis also confirmed the fix's premise: the API key is valid, /api/upload/server
returns a working node even on a lapsed-premium account, and uploads DO land
server-side (many Burn Notice episodes uploaded in the failure window are present).
So the API path returns the filecode directly and sidesteps the intermittent
web-form empty-result; entering the API key makes uploads succeed. 183/183.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Direct improvement to the login/web path (no API key needed): we were POSTing a
stale field set — sess_id + utype=reg + file — but doodstream's CURRENT upload
form dropped `utype` and added file_title / fakefilepc / submit_btn. Submitting
an incomplete/stale field set can make the CDN node accept the bytes but skip
the registration step (→ the empty result form with no fn). Now we parse the
live upload form (already fetched in _getUploadServer) and replicate ALL its
non-file fields faithfully — exactly what the browser submits — while keeping
sess_id (the fresh node token) and utype as a harmless compatibility extra.
- _parseUploadFormFields(html): pull every named input/button from the upload
form, excluding the file input (streamed separately). Adapts to whatever
fields doodstream uses now rather than hardcoding.
- upload() builds the multipart from those fields; minimal known-good fallback
if the form wasn't parsed.
- Tests: real-form extraction (incl. file-input exclusion) + no-form safety. 183/183.
Low regression risk (superset of the previously-working fields). Whether it
resolves the large-file empty-form is for the server run; the API path
(3.3.31/32) remains the reliable route when a key is available/derivable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the gap between the unit-tested parseDoodstreamResult and the real
uploadFile orchestration. Mocks the undici transport (reassign undici.request +
refresh the hosters cache; mock.module needs an experimental flag npm test
doesn't pass) and global fetch, then drives the full doodstream API path against
the doc-verified response shapes:
- filecode returned directly in result[0].filecode → used.
- codeless 2xx → recovered by polling file/list and name-matching the title.
- codeless + file never appears → throws with err.hosterTransient=true (so the
account is not blacklisted).
Verified live this session: doodapi.co returns {"status":400,"msg":"Invalid
key"} for a bad key, so validation/list logic keys off status correctly.
Also makes the recovery poll count/delay tunable via __test.DOODSTREAM_POLL
(same 12 × 2.5 s defaults — non-behavioral) so the exhaustion test runs in ms.
Full suite 181/181.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backstop for the API path: if the doodapi upload POST returns no filecode (the
same backend registration hiccup that empties the web form), poll
doodapi.co/api/file/list for a newly-appeared file whose normalized title
matches what we uploaded, and claim its code — instead of failing the upload.
This is the exact recovery byse already uses in this file for the identical
symptom (large MKV, server-side "OK" but empty immediate response, file shows up
in the account shortly after). Doodstream is the same XFileSharing family with
the same doodapi-style API, and it directly addresses the user's observation
that the same file often succeeds on a second run.
- _fetchDoodstreamFileList / _resolveDoodstreamUploadByName: list via
/api/file/list?key=&per_page=200, baseline-diff + exact normalized-title match
(never "take the only new one", so parallel uploads can't claim each other's
files), 12 polls × 2.5s.
- uploadFile snapshots a doodstream baseline before upload and polls after a
codeless result, before the hosterTransient throw.
Verified solo: doodapi.co is reachable and returns {"status":400,"msg":"Invalid
key"} for a bad key, so the validation/list path keys off status correctly.
178/178. The real large-file run on the server is the final confirmation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The user uploads with username/password (login), so 3.3.30's "use API when a
key is configured" did nothing for them — and the web-form upload keeps failing
with empty forms on large files. Fix the LOGIN path itself: after logging in,
pull the account's API key out of the logged-in session and upload via the
official doodapi API (which returns result[0].filecode directly, no empty form).
The user keeps using login and configures nothing.
How the key is derived without knowing doodstream's (cookie-gated, unseen)
settings DOM: brute-force candidate extraction + API validation.
- DoodstreamUploader.deriveApiKey(): fetch the logged-in settings page
(?op=my_account / /settings), pull every plausible long token from form-field
values + element contents (ranked: tokens near an "api" mention first), and
validate each against doodapi.co/api/account/info — only the account's real
key returns status 200. A wrong guess is therefore harmless (fails validation
→ web fallback). Logs the raw settings HTML when nothing validates, so the
scrape can be refined from a real capture if doodstream's markup differs.
- upload-manager: doodstream login-path now resolves the key ONCE per batch
(cached by accountId; '' = tried-none) and routes to the API when found, else
the existing web-form upload. Keyless accounts: one extra probe-login per
batch, then unchanged.
- Tests: candidate extraction (value/textarea/api_key shapes, api-context
ranking), validate-then-pick, null→web-fallback, preset short-circuit. 178/178.
If derivation works the login path now uploads via the API. It does NOT change
doodstream's backend; the server run confirms. Falls back safely if no key.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The new API upload path POSTs to the same cloudatacdn.com nodes as the web
path, so it can hit the same backend flake — a 2xx response with no filecode, or
a transient "no servers available" from /api/upload/server (now that the stale
fallback node is gone). hosters.uploadFile threw GENERIC errors for both, which
the upload-manager would treat as an account failure → mark-failed →
pre-job-swap-blocked on the next batch: the exact symptom 3.3.29 fixed for the
web path, reintroduced via the unprotected API path.
Tag both API-path analogs of the empty form as err.hosterTransient=true:
- codeless 2xx ("lieferte keine file_code-Antwort") — bytes accepted, no link.
- transient "no upload server" (shouldRetryServerLookup true: no-servers/busy/
try-again) — but NOT genuine auth failures (invalid key/unauthorized), which
stay classified as account errors.
The upload-manager checks _isHosterTransientError (flag-based) before the
account-error classifier, so both now fail the file without blacklisting the
account. Consumption side already covered by the 3.3.29 regression test. 173/173.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of the recurring "kein Filecode — Server gab leeren Link zurueck":
the web-session upload flow gets the filecode back inside an XFileSharing HTML
form, and on long/large uploads that form comes back empty (no fn). Verified
research: doodstream's server-side file-registration callback times out under
large-file load, so the upload "succeeds" (bytes sent, HTTP 200) but no filecode
is minted — and because registration failed, the file is NOT in the file list
either, so polling can't recover it. The web path also rides a per-page-load
sess_id token that ages over the multi-minute upload.
The official doodapi.co JSON API has no such failure mode for result retrieval:
the upload response returns result[0].filecode directly, and it authenticates
with a persistent api_key (no aging sess_id). Git history confirms the API was
doodstream's ORIGINAL upload path (initial commit); web login was added later
only "as an alternative to API key" — so preferring the key restores the
intended primary path rather than fighting a deliberate choice.
- lib/account-auth.js (new, pure, unit-tested): selectUploadAuth() prefers the
doodstream API key over username/password; all other hosters unchanged.
- main.js buildTaskFromAccount delegates to it → a doodstream account with an
apiKey now routes through hosters.uploadFile (doodapi API) instead of the web
uploader; keyless accounts keep using web login.
- hosters.js: drop the stale hardcoded fallback node from the doodstream API
config (same dead tr1128ve host removed from the web path) so a failed server
lookup throws cleanly instead of uploading into a dead end.
- Tests: 8 routing cases (doodstream key-preference, keyless fallback, voe
unaffected, authType=api, null-safety). Full suite 173/173.
This eliminates the empty-form failure mode for result retrieval when a key is
configured. It does NOT change doodstream's backend — whether the large-file
timeout recurs (now as a structured JSON error, not a silent empty form) is for
the server run to confirm. Requires a doodstream API key on the account.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "kein Filecode — Server gab leeren Link zurueck" error was treated as a
generic upload failure → after retries exhausted, the manager called mark-failed
and added the account to _failedAccounts → next batch re-primed with
primedFailed=1 → pre-job-swap-blocked because no fallback override exists for a
single-account hoster. One server-side flake permanently poisoned the session.
It's not an account problem — same account + same file works on a later try.
This is a doodstream-backend processing flake (empty CDN form, no fn / no st),
the same class as a transient network error: don't blacklist, just fail this
file cleanly.
- doodstream-upload.js: tag the empty-form throw with err.hosterTransient=true
(explicit flag, primary signal — matches the err.accountError / err.fileRejected
pattern already used elsewhere).
- upload-manager.js: new _isHosterTransientError classifier (flag first, message
regex as defensive fallback). In the retry loop: break on first hit (server
flake won't clear in 3 s, re-uploading the file 4× is pure bandwidth waste).
Post-loop: dedicated branch that emits the final error WITHOUT blacklisting
the account — same shape as the existing transient-network branch.
- Tests: classifier unit tests (flag path, regex path, negatives) + regression
test that proves the account is NOT added to _failedAccounts and mark-failed
does NOT fire. Drops the hoster-transient test from ~19 s to ~1.5 ms,
confirming the in-loop fast-break works.
We now fail fast on this error class instead of retrying — the next-batch
manual retry is the recovery path, and the account stays usable for it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reproduced from a real saved config: pendingQueue held 4 'preview' jobs (one
file across 4 hosters); the queue saved + restored correctly. But
_autoDeduplicateFromLog (runs at init after restore) removed jobs whose
fileName|hoster appeared ANYWHERE in the lifetime fileuploader.log, regardless
of status — so all 4 pending previews were deleted and the queue showed the
empty "Dateien hierhin ziehen" state. Looked update-specific only because the
server restarts on update; a plain restart did the same.
- New lib/queue-dedup.js (pure, dual CJS/window export like queue-prune.js):
partitionRestoredJobsByLog drops ONLY 'done' jobs that match the log. Pending
(preview/queued) and failed (error/aborted) jobs always survive — they're
intentional queued work (often a deliberate re-upload of a previously
uploaded file). Manual importUploadLog stays separate/explicit.
- renderer wires it in; index.html loads the module before app.js.
- Tests: 5 cases incl. the exact reproduced scenario (4 previews all in log ->
0 removed). Full suite 162/162.
Verified against the user's real electron-config.json + fileuploader.log: old
logic removed 4/4 (empty queue), new logic removes 0/4 (queue preserved).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
After 3.3.26 fixed the filecode parsing, the remaining intermittent failure is
a generic "fetch failed" — a transient network error on one of the requests
around the multi-minute upload. Can't tell from one log line whether it's the
server-discovery GET or the post-upload result-submit, so harden both:
- _fetch (the native-fetch chokepoint for discovery, redirects, result-submit):
retry up to 3x with short backoff on a thrown network error, each attempt
bounded by a 20s timeout (Node fetch has none by default). Caller aborts are
not retried. The big file upload (undici) is retried at the upload-manager
level, not here.
- result-submit is now best-effort: if it still fails after retries but we
already hold the filecode from the CDN response, return that instead of
discarding a completed upload.
- label the undici upload-POST error with phase + MB sent + node, preserving the
original message so transient classification still matches.
- eslint: add AbortSignal to globals.
- Tests: _fetch transient-retry path (10 doodstream tests total).
"fetch failed" is already classified transient by upload-manager, so this is
additive resilience; next logs will show if anything still slips through.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 3.3.25 diagnostics captured the live upload page: doodstream moved the
upload server from a `srv_url` JS variable into the multipart form's action,
e.g. action="https://xxx.cloudatacdn.com/upload/01?SESSID", with a per-page
session token in the query that matches the page's hidden sess_id input. The
old parser found neither and fell through to the stale hardcoded node, which
returns an empty filecode.
- Parse the upload server from the form action (matched via the /upload/ path),
un-escaping & in the query string.
- Refresh this.sessId from the SAME page (only on action match) so the
multipart sess_id field matches the node URL's token; login-time and node
tokens otherwise diverge. Keep the existing sessId if the input is absent.
- Keep the legacy ?op=upload_server JSON and srv_url paths as fallbacks; the
fail-fast throw from 3.3.25 stays as the last resort.
- Tests: form-action parse, sess_id refresh, & un-escape (9 total).
Whether this fully resolves the uploads is for the next server logs to confirm;
both the node and sess_id fixes are individually correct.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Real root cause from the 3.3.24 diagnostics: the failing upload used CDN
"tr1128ve.cloudatacdn.com/upload/01" — character-for-character the hardcoded
last-resort fallback in _getUploadServer(). The CDN form came back with only
op=upload_result and NO fn/NO st, i.e. the bytes went into a stale node that
returns an empty form. So _getUploadServer can no longer extract the current
upload server (Doodstream likely changed the upload_server response/format) and
silently fell back to a dead node — wasting ~90s/95MB per attempt.
- Remove the silent hardcoded-node fallback; throw a clear error when discovery
fails so the upload fails instantly instead of 90s later with a cryptic msg.
- Embed the raw upload_server response (status, content-type, body) and
upload-page URL hints in the error AND debug log, to pin the format change.
- Tests: getUploadServer JSON path, srv_url HTML fallback, and the no-silent-
fallback throw (asserts the hardcoded node never leaks into the error).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "upload_result Seite hat keinen filecode" error fired with no actionable
detail when Doodstream's CDN returned an empty filecode (fn). Root cause is
server-side: the page structure is unchanged, the link is just missing —
Doodstream's backend refused the file (copyright/hash match, duplicate, size,
quota). XFileSharing reports the reason in the `st` field, which we ignored.
- Surface `st`: non-OK status now throws "Doodstream lehnt Datei ab (Status: …)".
- Enrich the generic error with st, fn-state, and the CDN node for diagnosis.
- Fix debug-log path: wrote to __dirname/.. which is read-only (app.asar) in
packaged builds, so production captured zero traces. Now uses Electron's
writable userData dir, with repo-root fallback for tests/plain node.
- Add tests/doodstream-upload.test.js (4 tests) pinning the parse/error paths.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New per-hoster setting "Links in Log schreiben" (logToFile, default
on). When unchecked for a hoster, that hoster's successful upload
links are no longer written to fileuploader.log — other hosters keep
logging independently.
- lib/config-store.js: logToFile: true added to HOSTER_SETTINGS_DEFAULTS;
merge-on-load gives every hoster the key (old configs included).
- renderer/app.js: checkbox per hoster panel + collection loop now
handles type=checkbox (boolean) alongside the numeric fields. The
autosave bind already special-cased checkboxes (change event).
- lib/log-policy.js (new): hosterLogToFileEnabled() — pure, opt-out
semantics. Only an explicit logToFile===false disables; missing/
malformed/non-true values all default ON so links are never
silently dropped.
- main.js: shouldLogHosterToFile() reads the LIVE uploadManager
.hosterSettings (so a mid-batch toggle takes effect at once), falls
back to persisted config, then to enabled. Guards appendUploadLog
in the done handler; skipped writes get a debugLog line.
Tests: 8 log-policy (defaults, opt-out, per-hoster independence,
malformed input) + 2 config-store (default true, persisted false
survives reload). 147/147 green, eslint clean.
The microtask-coalesce path from 3.3.1 (queueMicrotask + Set so 500
finishing jobs become one queueJobs.filter pass instead of 500) lived
inline in renderer/app.js. Pulled out into lib/coalesced-set.js with
an injectable scheduler so a Node test can drive timing without
async waits.
API: makeCoalescedSet({ apply, scheduler? }) returns
add(id) — queue an id for the next batch
drainSync() — flush synchronously (used by beforeunload)
pendingSize() — diagnostics
isScheduled() — diagnostics
Renderer rewires the previous _pendingDoneRemovalIds + manual
queueMicrotask plumbing to the new helper. Optional-chained: if the
script fails to load, a slower per-event filter runs as fallback.
Coverage:
- multiple adds same tick → 1 apply, all ids deduped
- duplicate ids deduped
- batches between flushes stay independent
- add after flush re-schedules
- drainSync flushes synchronously, queued microtask becomes a no-op
- empty drainSync is a no-op
- throwing apply doesn't lock out subsequent batches
- default scheduler (queueMicrotask) runs eventually
- 5000-id burst still coalesces to 1 apply
137/137 green.
3.3.2 fixed fileuploader.log unbounded growth, but three siblings kept
growing without limit:
- upload-debug.log (verbose, every IPC + progress event log line)
- account-rotation.log (every rotation decision)
- doodstream-debug.log (per-hoster trace from lib/doodstream-upload.js)
A multi-month dev install or a heavy production user could fill the
log dir with multi-GB files and slow every appendFile.
Wire all three through the same lib/log-rotation.js helper:
- upload-debug.log → 25 MB cap, 2 numbered backups (~75 MB worst)
- account-rotation.log → 10 MB cap, 2 numbered backups (~30 MB worst)
- doodstream-debug.log → 10 MB cap, 1 numbered backup (~20 MB worst)
The rotation check runs once per flush call (each is debounced or
already a once-per-event path), so the statSync overhead is
microscopic. _flushDebugLog passes a noop logger to avoid recursing
into itself; _flushRotLog and _debugLog (doodstream) use the normal
debugLog so any rotation surprises end up in upload-debug.log.
126/126 tests still green.
The account-rotation while-loop entered with a signal.aborted /
stopAfterActive check (line 681) but then awaited _sleep(800) on
line 690 (waiting for main.js to resolve the next fallback) without
re-checking on the way out. If the user cancelled during that 800 ms
window the loop kept going — resolved the override, set up new
credentials, fired retrying-event, started a fresh attempt loop —
before _executeUpload's own signal handling finally noticed the
abort. Cancellation latency could therefore stretch by an extra
attempt's worth of work per still-spinning hoster.
One-line fix: add the same `if (signal.aborted || this.stopAfterActive)
break` after the await. Found by deep-audit MED-5.
126/126 tests still green; the fix is a guard on an already-tested
flow, no test infrastructure exists for cancel-during-rotation
specifically (would need fake-timer + mocked override-resolution).
When a hoster server replies with a body that JSON-parses to a
non-object (literal "null", a bare string, a number, a top-level
array), uploadFile's downstream code crashed:
payload.msg → TypeError on null
payload.status → TypeError on null
config.parseResult() → TypeError inside parseDoodstreamResult
(payload.result) and parseByseResult
(payload.files / payload.result)
The user saw a confusing "Cannot read properties of null" instead of
a useful "server returned no JSON object". Found by deep-audit pass.
Fix in three places:
1. uploadFile (lib/hosters.js): after JSON.parse, normalise non-object
payloads to {}. Subsequent `payload.X` accesses then return
undefined and the existing fallback paths handle the empty case.
2. parseDoodstreamResult: defensive `payload && payload.result` so
direct callers (tests, hypothetical future callers) get the same
guarantee instead of relying on uploadFile to have normalised.
3. parseByseResult: same `payload || typeof payload !== 'object'`
short-circuit at entry, plus null-checks on `f` (the first files
entry) so a server returning [null] in files doesn't crash either.
Tests: 7 new unit tests covering null/undefined/string/number/array
payloads, malformed files entries, the fileRejected/accountError
classification (regression-pinning the 3.1.4 phrasing tweaks), and
the valid-filecode happy path. 126/126 green.
Two related fixes from the deep-audit pass:
HIGH-2: save-global-settings-sync used `try {} catch {}` and always
returned `event.returnValue = true`, so a disk-full / AV-lock /
permissions failure looked like success to the renderer's beforeunload
chain. The user closes the app, comes back, settings are gone —
without any indication. Now the catch sets returnValue=false and
debugLogs the error message, and the bak refresh is in its own
nested try so a transient lock there doesn't fail the whole save.
MED-4: lib/config-store.js _atomicWrite had the same TOCTOU on the
.bak refresh — fs.existsSync(...) then fs.readFileSync(...) without
guarding the read. Wrapped the read+write of the backup in its own
try/catch: a stale .bak is preferable to dropping the new write
entirely just because Windows Defender briefly locked the file mid-
operation. The rename of tmp → live still throws on real failure,
which is what the outer reject is for.
119/119 tests still green; both fixes are defensive guards on
already-tested write paths.
The dynamic-key sort throttle (3.3.0) used an inline ad-hoc cache
object with a Date.now() comparison. Pull it out into a clean
generic-purpose makeThrottledCache helper that takes the TTL and an
optional clock function so tests can drive time without sleeping.
Same dual-environment loader (CommonJS for tests, window global for
the renderer via index.html script tag) as queue-prune.
API: get(sig, input) / set(sig, input, value) / clear() / peek().
sig + input identity must both match for a hit. Inputs are compared
by reference (===), exactly what sortQueueJobs needs to invalidate
on a fresh queueJobs array (e.g. backup import).
Coverage:
- empty cache → undefined
- within TTL → cached value
- past TTL → miss (boundary at refreshMs)
- different signature → miss
- different input identity → miss (even with same content)
- overwrite refreshes timestamp
- clear empties everything
- peek reports age + signature for diagnostics
- invalid TTL throws (negative, NaN, non-number)
- TTL=0 means every call misses (immediate expiry)
- default clock works (Date.now)
- large arrays tracked by identity, not value
Renderer rewires _dynamicSortCache to the new helper with a fallback
no-op shim if window.ThrottledCache failed to load. 119/119 green.
handleBatchDone's terminal-job auto-cap (introduced in 3.3.0) lived
inline as a manual two-pass loop over queueJobs. Pull the algorithm
into lib/queue-prune.js as pure pruneOldestTerminalJobs(jobs, limit)
that returns { kept, dropped } so the caller can clean up its index/
selection in one go. Same single implementation backs runtime and
tests via dual-environment loader (CommonJS module.exports for Node
tests, window.QueuePrune global for the renderer via index.html
script tag).
Coverage:
- Empty / null / non-array input → no-op
- All-non-terminal → no-op (regardless of limit)
- Terminal count ≤ limit → no-op
- Terminal count > limit → drops oldest by insertion order
- Mixed queue: non-terminals always kept, only terminals dropped
- limit=0 → drops every terminal
- Negative / NaN / Infinity limits → safe no-op
- Malformed entries (null, missing status) handled without throwing
- Large-queue stress (5000 done jobs) keeps newest 500
- TERMINAL_STATUSES set covers exactly done/skipped/error/aborted
Renderer uses window.QueuePrune?. so a failed script load just
disables the prune rather than crashing every batch-done. 107/107
tests green.
The fileuploader.log rotation introduced in 3.3.2 lived inline in
main.js — fine for the runtime path, but it required electron's `app`
to even reach the function under test. Pull the rotation logic into
lib/log-rotation.js (pure fs/path, no electron deps) and cover it
properly:
- ENOENT (file missing) → no-op
- Below cap → no-op
- Over cap → live → .1, returns true
- Existing backups shift up: .1 → .2, .2 → .3
- At maxBackups limit → oldest dropped, others shift, live becomes .1
- Idempotent: rotating twice keeps the chain consistent
- maxBackups=1: never grows past .1
- Invalid maxBytes (0/negative/NaN) → safe no-op
- Provided debug callback receives a "rotated" message
- File without extension still rotates correctly
main.js now imports `maybeRotateLogFile` and calls it directly. 97/97
tests pass.
Before: a non-transient / non-file-rejected / non-account-specific
error (e.g. "VOE Upload: <any generic message>") would burn the full
retries-per-account budget on the primary before the rotation logic
even kicked in. On retries=5 that's "Retry 2/5 Primär", "Retry 3/5
Primär", … all on the same broken account before the fallback gets
a shot.
Now:
- main.js pre-resolves the next fallback for every hoster at batch-
start (stored in _accountOverrides via the existing session cache +
primeOverrides path). Pre-job-swap still ignores it until the
primary is actually marked failed, so jobs still begin on primary.
- upload-manager.js: in the retry loop's generic error branch,
_hasPendingOverride() checks whether a usable fallback is ready.
If yes and the error is NOT transient (transient = network glitch
= retry same acc), break out to rotation. Marks primary failed,
rotates to acc2, retries there.
- Result: for a 2-account hoster, worst case is 1 attempt on primary
+ retries-per-account on fallback, instead of N × 2. Transient
network errors (ENOTFOUND / ECONNRESET / socket hang up) keep the
old "retry same account" semantics because the network is the
issue, not the account.
- Single-account hosters: unchanged. No pending override = classic
retry-on-same-account until exhausted.
3 new tests pin: generic + override → rotate on attempt 1; transient
+ override → stay on same acc; no override → classic retry. 87/87
green.
Two related visibility improvements.
1. Status cell now shows which account the job is running on:
"Upload · Primär", "Retry 2/3 · Fallback #1: <error>", etc.
- _emitProgress passes task.accountId in every progress event
- renderer maps accountId → position in config.hosters[hoster] and
renders "Primär" for index 0 and "Fallback #N" for the rest
- Applies to uploading/getting-server/retrying (static states like
done/error already tell their own story)
2. Right-click on a job → "Log anzeigen" opens a modal with the full
per-job trail: every rot-log entry tagged with that job's jobId
plus every non-uploading progress transition. Replaces the need to
grep account-rotation.log for a single filename.
- UploadManager: all 13 job-scoped _rotLog calls now carry jobId
- main.js: _jobLogCollector Map<jobId, Array<entry>> with 200-entry
ring buffer per job; cleared on each new start-upload (fresh
batch = fresh log). addJobs mid-batch keeps history.
- New IPC 'get-job-log' returns the array; preload.js exposes
window.api.getJobLog(jobId)
- renderer: modal card + context-menu item "Log anzeigen";
entries formatted as "[HH:MM:SS.mmm] [event] k=v k=v"; copy-to-
clipboard button
Previously: clicking "Erneut versuchen" after a batch had already
finished spawned a fresh UploadManager with empty _failedAccounts and
_accountOverrides. The first retry then burned the full retry budget
on the account we already knew was dead (e.g. disk-space-full byse
account) before rotation kicked in again — same problem we fixed for
within-batch flow but for across-batch flow.
- main.js: two module-level maps (_sessionFailedAccounts,
_sessionAccountOverrides) cache rotation state across batches in the
same app session. Populated on account-failed and on both
switchAccount paths (event-driven + save-config re-resolve).
- lib/upload-manager.js: startBatch(tasks, opts) accepts
primeFailedAccounts + primeOverrides. State is still cleared first
(legacy behaviour for callers without opts), then re-primed from the
passed session state. batch-start rot-log entry reports how many
entries were primed for diagnostics.
- Tests: prime priority is honored (pre-job-swap fires on first
attempt, no fast-fail, no upload to acc1); back-compat for callers
that don't pass opts.
App restart remains the reset signal — matches the "neuer Tag, acc1
hat vielleicht wieder Platz" expectation.
Three small, unrelated reliability improvements bundled:
1. lib/hosters.js (_resolveByseUploadByName): drop the "only one new
file → claim it" fallback. Under parallel byse uploads, job A's
poller could claim job B's newly-uploaded file and return the wrong
URL. Now requires exact normalized name match. Trade-off: a few
false negatives if byse rewrites the filename beyond our
normalizer, but parallel correctness wins.
2. tests/upload-manager.test.js: pin the transient-network classifier
behaviour with 2 new tests covering common transient strings
(ENOTFOUND, ECONNRESET, socket hang up, fetch failed, EAI_AGAIN…)
and verifying real account-level / file-rejected errors are NOT
misclassified as transient. Baseline stays clean: 82/82 green.
3. main.js: log process.memoryUsage() snapshot at batch-start and
batch-done. One line each — harmless in the happy path, gives us
the data points needed to spot long-session RSS/heap growth across
batches without DevTools instrumentation.
Three related gaps closed so one full byse account stops wasting
attempts on every subsequent job and later-added accounts get picked
up without an app restart.
1. Pre-job-swap moved BEHIND the semaphore acquire. At scale (500 jobs
/ 1 slot) every worker was checking _failedAccounts at spawn time
before the first upload had even tried — so none of them saw the
failed state. Now each worker re-checks right before its first
upload attempt.
2. save-config IPC handler re-resolves fallbacks for any account that
is already in _failedAccounts but has no override set. Previously
account-failed only fired once per account, so a config change
after the first mark-failed was silently ignored and the batch
stayed stuck on the dead account until the app restarted.
3. UploadManager exposes getFailedAccountKeys() and getOverride(hoster)
so main.js can drive the late re-resolve without poking private
fields.
4 new tests: pre-job-swap after semaphore, getters contract, fresh
manager resets learned state, late-added fallback is honored by
subsequent jobs. 80/80 green.
Byse rejects uploads with status like "not enough disk space on your
account" when the account's storage is exhausted. The parser was
flagging every non-OK status as err.fileRejected=true, and the upload-
manager classifier additionally matched the generic "lehnte Datei ab"
prefix as file-rejected. Result: rotation was skipped on a full account
and every subsequent file failed on the same dead account.
- hosters.js: byse parser now distinguishes account-level phrases
(disk space / storage / quota / insufficient / account full) and sets
err.accountError=true for those. File-specific failures (Duplicate,
wrong format, size) keep err.fileRejected=true.
- upload-manager.js: _isFileRejectedError no longer matches the generic
"lehnte Datei ab" prefix and short-circuits when err.accountError is
true. _shouldSkipRetryOnAccountError honors the flag and has added
regex patterns as a safety net.
- Tests: 5 new unit tests covering disk-space/account-level/duplicate
and the accountError-wins-over-fileRejected precedence.
User reported uploads appearing on the byse dashboard (2+ GB MKV,
Server #262, status OK) even though the app marked them failed. The
byse API sometimes replies with msg=OK + files:[{filecode:"",
status:"Not video file format"}] — a misleading response where the
file is actually being accepted and gets its filecode assigned
asynchronously.
- Before the upload POST, snapshot the current /api/file/list to
know what was already there.
- If parseByseResult returns an empty filecode (or throws a
fileRejected error), poll /api/file/list up to 15 × 2s looking
for a new file_code matching the uploaded filename (case/punct
normalized, extension stripped).
- If matched, return the real download/embed URLs and let the
upload complete as successful. Only throw the parser's error
if polling also finds nothing.
The log revealed byse's true response shape for rejected files:
{ msg: 'OK', status: 200, files: [{ filecode: '', status: 'Not video file format' }] }
HTTP 200 + msg=OK made the old code treat it as 'success but no
file_code'. The real error ('Not video file format') was buried in
files[0].status. parseByseResult now surfaces that with a dedicated
err.fileRejected flag so the rotation layer can distinguish
file-specific vs account-specific failures.
Rotation behavior:
- file-rejected errors: no retries, no account blacklist, no
rotation. The same file is going to get the same verdict on
any account, so skip straight to 'error' status and keep the
account available for other files in the batch.
- network errors (already handled): no account blacklist either.
- everything else: unchanged (retry then rotate).
Also added pattern matches for common rejections (Duplicate, File
too small/large, Unsupported format, etc.) so other hosters'
per-file errors get the same treatment.
Two bugs visible in the user's rotation log:
1. 'error=OK' for byse.sx — the server returned a payload with
msg='OK' and no file_code anywhere we recognized. Our generic
uploadFile threw the bare 'OK' as the error message, which is
useless and misleading. Now when we see an ok-ish msg without
the expected file_code we throw a descriptive error that
includes the first ~400 bytes of the payload so the next time
it happens we can see what's actually being returned (API
changed, new field name, etc.).
2. 'getaddrinfo ENOTFOUND s1055.filemoon' was marking accounts as
permanently failed, blacklisting BOTH byse accounts within the
same batch even though neither was the actual problem — filemoon
(byse's storage backend) briefly had a DNS blip. Added
_isTransientNetworkError() covering DNS/ECONNRESET/ETIMEDOUT/etc.
When all retries on an account exhaust with a transient error,
we now fail just that file and emit 'skip-rotation-transient'
instead of adding the account to _failedAccounts. Other files
in the same batch still get a fresh try on the same account.
When multiple jobs run in parallel on the same hoster and the primary
account starts failing, the first job marks it failed + triggers
rotation. The second job's retries then also exhaust on the same
(already-failed) primary — but the old while-condition
`!_failedAccounts.has(...)` short-circuited the whole rotation loop
for anything already marked, so the second job went straight to
final-error even though a resolved override was sitting right there.
Now the loop always checks for an available override; it only skips
the mark-failed + emit step if the account was already marked by a
concurrent job. Fixed visible symptom: first job rotates A→B, every
other job in the same batch that hit A got final-error instead of
also switching to B.
Also extended fast-fail patterns to include 429 (Too many requests),
CSRF-Token / 'Bist du eingeloggt' — both were showing up as the
primary failure mode in real uploads and were wasting 5 retries
each.
Three related improvements that landed together while wiring up the
rotation log infrastructure:
- Fast-fail classifier: errors that clearly indicate the account
itself is the problem (rate limit, quota, banned/suspended, auth
failure, 401/403, 'Kein Upload-Server' from delivery-node etc.)
now skip the remaining retries and go straight to rotation. No
more waiting 5 × 3s between retries just to end up rotating
anyway. Emits a 'fast-fail' rot-log event so the shortcut is
visible.
- Settings: 'Öffnen' button next to the log-file-path input reveals
the active log file (or its directory if nothing's written yet)
in the OS file manager, so users don't have to remember paths.
- rotLog() writes the rotation log synchronously. Only a handful
of events fire per batch; the 500ms flush batching was saving
nothing and made the file look empty when users checked right
after an event. (The main debug log still uses the batched async
path — that one is high-volume.)
To trace whether the fallback chain actually engages during real uploads,
every rotation decision now emits a structured 'rot-log' event from the
upload-manager. main.js persists each event to a new account-rotation.log
(same directory as fileuploader.log; falls back to Desktop then userData)
and also mirrors it into the main debug log with a [ROT] prefix for
single-file grepping.
Logged events:
- batch-start (clears _failedAccounts / _accountOverrides)
- pre-job-swap / pre-job-swap-blocked (job picks override before first try)
- retries-exhausted / mark-failed (enters rotation loop)
- rotate (switched to new account, retry starting)
- rotation-end (no override / override already failed)
- final-error (all accounts exhausted)
- switchAccount (main resolved the next fallback)
The renderer shows a toast on 'rotate', 'rotation-end' and 'final-error'
so fallback behavior is visible live instead of buried in logs.
Captured the real browser upload POST and compared to our request.
Two corrections:
- The file field is named 'file', not 'file_0'. The XFS-indexed
naming was a bad guess — the current transit accepts only 'file'.
- The form also needs 'to_json=1' (forces JSON response instead of
an HTML redirect page, matching what the browser submits) and
'fld_id=0' (destination folder, 0 = root). Dropped upload_type,
srv_tmp_url, utype — those were XFS remnants and aren't part of
the current server's contract.
- Response shape is now { status: 'OK', file_code, msg } instead of
the older { files: [...] } / { result: ... } XFS variants; the
parser handles all three plus carries the server's msg forward
on explicit rejections.
The transit server runs nginx-upload-progress and requires an
X-Progress-ID query parameter on the POST URL to finalize the
upload session. Without it the server accepts all bytes but never
sends the response — matches the reported 99%-stuck behavior. The
browser appends it automatically before submit; we now do the same.
Upload stalled at 99% because we were sending vidmoly.me cookies to
*.vmwesa.online (transit server rejects them silently). Browsers never
send those cross-origin. Now we omit the Cookie header and match the
Origin/Referer the browser uses. Also added the full classic XFS field
set (upload_type, sess_id, srv_tmp_url, utype) in the order the
server's handler expects.
The SPA redesign killed the old XFS form POST at / with op=login.
The new flow is a JSON POST to /api/auth/login that returns a
vidmoly_session HttpOnly cookie, which is what /api/upload/config
actually authenticates against.
After login we also probe /api/upload/config once to fail fast if
the session was issued but not actually valid for uploads.
The old /my HTML check failed because it couldn't distinguish an XFS
session from a full SPA session. Since /api/upload/config is what the
upload actually needs, probe it directly after login — 200 JSON with
sess_id/upload_url means we're good, anything else means we're out.
The Vidmoly SPA redesign removed the /?op=upload HTML form — the old
regex-scrape of hidden inputs no longer works. The site now exposes
GET /api/upload/config which returns { sess_id, upload_url } plus the
allowed extensions. Rewrote getUploadParams() to use that endpoint;
the rest of the multipart upload flow (sess_id + utype + file_0) is
the same classic XFS shape.
Three fixes bundled:
- Vidmoly redesign broke login: the old check required either the
'login' or 'xfsts' cookie, but the new site sets different cookie
names. Now we verify by fetching /?op=my_account and looking for
logged-in markers (Logout / My Account / My Files) in the body
instead of relying on specific cookie names.
- retrySelectedJobs left the stale uploadId in _jobIndexByUploadId
when resetting a job. A late 'aborted'/'error' event from the
original (cancelled) upload could route back to the reset job
and overwrite its 'preview' state. Now the old uploadId is
removed from the index and marked in _deletedJobIds so those
stragglers get dropped.
- toggleAccount did two IPC round-trips (saveConfig + getConfig) on
every enable/disable click, plus four re-renders (Accounts,
HosterSummary, HosterModal, Settings). Rapid clicks felt laggy.
The getConfig refetch is redundant since we mutated the flag in
place, and HosterModal/Settings don't depend on account enabled
state. Click now renders immediately and the save runs async.
