The dynamic-key sort throttle (3.3.0) used an inline ad-hoc cache
object with a Date.now() comparison. Pull it out into a clean
generic-purpose makeThrottledCache helper that takes the TTL and an
optional clock function so tests can drive time without sleeping.
Same dual-environment loader (CommonJS for tests, window global for
the renderer via index.html script tag) as queue-prune.
API: get(sig, input) / set(sig, input, value) / clear() / peek().
sig + input identity must both match for a hit. Inputs are compared
by reference (===), exactly what sortQueueJobs needs to invalidate
on a fresh queueJobs array (e.g. backup import).
Coverage:
- empty cache → undefined
- within TTL → cached value
- past TTL → miss (boundary at refreshMs)
- different signature → miss
- different input identity → miss (even with same content)
- overwrite refreshes timestamp
- clear empties everything
- peek reports age + signature for diagnostics
- invalid TTL throws (negative, NaN, non-number)
- TTL=0 means every call misses (immediate expiry)
- default clock works (Date.now)
- large arrays tracked by identity, not value
Renderer rewires _dynamicSortCache to the new helper with a fallback
no-op shim if window.ThrottledCache failed to load. 119/119 green.
handleBatchDone's terminal-job auto-cap (introduced in 3.3.0) lived
inline as a manual two-pass loop over queueJobs. Pull the algorithm
into lib/queue-prune.js as pure pruneOldestTerminalJobs(jobs, limit)
that returns { kept, dropped } so the caller can clean up its index/
selection in one go. Same single implementation backs runtime and
tests via dual-environment loader (CommonJS module.exports for Node
tests, window.QueuePrune global for the renderer via index.html
script tag).
Coverage:
- Empty / null / non-array input → no-op
- All-non-terminal → no-op (regardless of limit)
- Terminal count ≤ limit → no-op
- Terminal count > limit → drops oldest by insertion order
- Mixed queue: non-terminals always kept, only terminals dropped
- limit=0 → drops every terminal
- Negative / NaN / Infinity limits → safe no-op
- Malformed entries (null, missing status) handled without throwing
- Large-queue stress (5000 done jobs) keeps newest 500
- TERMINAL_STATUSES set covers exactly done/skipped/error/aborted
Renderer uses window.QueuePrune?. so a failed script load just
disables the prune rather than crashing every batch-done. 107/107
tests green.
_sessionTrackedJobs and _sessionDoneJobs accumulated jobIds across
the whole session — IDs of jobs already removed from queueJobs (by
removeFromQueueOnDone or the auto-cap that lands in handleBatchDone)
stayed in those sets forever. ~50 bytes/entry × hundreds of batches
× many jobs/batch = small but real growth over a multi-day session.
At batch-done, walk the sets and drop any ID that's no longer present
in queueJobs. _completedUploadKeys is intentionally kept — it's the
dedup against re-queueing the same file across batches and would
break that contract if pruned.
The prune is a single pass per batch-done (rare event) and only
happens when the sets aren't already empty. 97/97 tests still green.
The .queue-row rule had transition: background 0.15s applied
unconditionally. Every status flip (queued → getting-server →
uploading → done) on every visible row therefore animated for 150 ms,
and with 30+ rows changing state in close succession the compositor
ran overlapping tweens that ate paint time during heavy upload bursts.
Move the transition into the :hover rule. Hover-enter and hover-leave
keep their smooth fade — that's where transitions actually help the
user. Status changes now snap to the new background colour instantly,
which is what the queue table really wants: it conveys progress, not
animation.
No JS change. 97/97 tests still green.
The fileuploader.log rotation introduced in 3.3.2 lived inline in
main.js — fine for the runtime path, but it required electron's `app`
to even reach the function under test. Pull the rotation logic into
lib/log-rotation.js (pure fs/path, no electron deps) and cover it
properly:
- ENOENT (file missing) → no-op
- Below cap → no-op
- Over cap → live → .1, returns true
- Existing backups shift up: .1 → .2, .2 → .3
- At maxBackups limit → oldest dropped, others shift, live becomes .1
- Idempotent: rotating twice keeps the chain consistent
- maxBackups=1: never grows past .1
- Invalid maxBytes (0/negative/NaN) → safe no-op
- Provided debug callback receives a "rotated" message
- File without extension still rotates correctly
main.js now imports `maybeRotateLogFile` and calls it directly. 97/97
tests pass.
applyQueueSelectionClasses + applyRecentSelectionClasses ran
tbody.querySelectorAll('.queue-row') / ('.recent-file-row') on every
click. querySelectorAll always walks the tree and returns a fresh
static NodeList. With 200 visible queue rows + frequent click/drag
selections that's a measurable per-click cost.
Switch to getElementsByClassName: returns a live HTMLCollection that
the engine memoizes and updates incrementally as nodes are
inserted/removed. First call still walks once; subsequent calls are
near-free reads. Iteration uses a plain index loop because
HTMLCollection is array-like, not iterable in older runtimes (it is
in modern Chromium, but the index loop is also marginally faster).
No behaviour change. 87/87 still green.
The per-job log collector was only cleared at start-upload — across a
long session with many add-jobs-to-running-batch interactions (no new
start-upload), the Map grew unbounded. At ~5000 tracked jobs that's
1 MB × 5 = 5 MB+ of stale history hanging around in the main process,
bigger as ring buffers fill.
Add a cap: when a new jobId would push size past 1000, evict the
oldest entry (Map iteration order is insertion order per spec). 1000
× 200 entries/job × ~100 B/entry ≈ 20 MB worst case, properly bounded
no matter how long the session runs. Per-job ring buffer (200 entries)
unchanged; only the count of tracked jobs is now capped.
The "Log anzeigen" modal still works for any job in the most-recent
1000 — older jobs return an empty array, which the renderer already
displays as "Keine Log-Einträge".
A long-running install can otherwise grow the upload log into the
gigabyte range, eating disk and slowing every appendFile. Add a
size-checked rotation right before each flush:
- statSync the resolved log target (cheap, ENOENT skips silently).
- If size exceeds 50 MB, drop the oldest backup (.3), shift .2→.3
and .1→.2, then rename the live file to .1 and let appendFile
create a fresh primary on the next call.
- Max 3 backups (~200 MB worst case, bounded). debugLog records
each rotation for diagnostics.
- Pure additive: skips when file is small or doesn't exist; no
effect on the daily-log mode (already date-rotated).
handleProgress on a 'done' event with removeFromQueueOnDone=true was
calling queueJobs.filter() once per event. With 500 parallel jobs all
finishing at roughly the same time, that's 500 × O(N) = O(N²) work
synchronously on the IPC handler thread — visible as a brief UI freeze
when a big batch completes.
Coalesce into one microtask: removeJobFromIndex + selection cleanup
stay synchronous (so subsequent lookups see the right state), but the
array rewrite is deferred to a single filter against a Set of all
ids that came in this tick. JS microtask runs after the sync IPC
batch, so within one batch-of-events we get one filter pass instead
of N.
beforeunload drains the pending set synchronously before persisting
so removeFromQueueOnDone=true users don't see jobs reappear after
restart that they expected to be gone.
Bundles four findings from a stability audit plus the missing-log bug
the user reported.
1. main.js _flushUploadLog: ENOENT after the log file's directory got
deleted mid-session was swallowed; the buffer was cleared before
appendFile so entries were silently lost and the cached target kept
pointing at the dead path. Now: mkdirSync(recursive) before every
flush idempotently recreates a missing dir; on any append error we
invalidate the cache, prepend the chunk back to the buffer and
schedule a retry. Survives "user dragged the log folder into the
trash and didn't notice".
2. renderer/app.js queueJobs auto-prune: with the default
removeFromQueueOnDone=false the queue grew forever. Past ~5000
entries every render became O(N) on a perpetually-growing N and
the user saw progressive scroll/tab lag. Cap the in-queue
terminal jobs (done/skipped/error/aborted) at 500 most-recent on
each batch-done; oldest get pruned with their index entries.
3. sortQueueJobs dynamic-key throttle: status/speed/progress/size
sorts ran a full O(N log N) sort on every progress tick. Added a
200ms-window cache for the dynamic-key path so the sort is reused
within the same UI_UPDATE_INTERVAL — invisibly small reorder lag,
massive cost savings at 5000+ jobs.
4. renderHistoryTable delegated listeners: every Verlauf-tab switch
was binding one click listener per row (5000 listeners on a
long-history user) and rebuilding the entire <tbody> innerHTML.
Single delegated tbody listener covers both row-click (copy link)
and th-click (sort), bound once per container via dataset flag.
5. sessionFilesData (recent-files panel) cap at 2000 entries with
matching _sessionFileKeys cleanup using the existing separator.
Stops the lower-panel innerHTML write from inflating to multiple
MB on long sessions.
87/87 tests still green.
After importing a backup or restoring the queue at startup, queueJobs
is reassigned to a fresh array. The sort-cache keyed its hit on
"key|direction|length" — identical across a replacement with the same
job count. Result: renderQueueTable kept getting the cached sorted
array, which held references to the DISCARDED job objects (frozen at
status='preview'). Uploads ran perfectly in the background, the
status bar updated from stats events, but every row stayed "Bereit"
with "..." as size. The user had to poke `_queueSortCache={sig:'',…}`
in DevTools to unstick it.
Include the jobs array identity (jobsRef) in the cache check. A
replacement of queueJobs → different reference → cache miss → fresh
sort with the real current objects. O(1) identity check, no CPU cost
on the common case (same array, mutated jobs).
Before: a non-transient / non-file-rejected / non-account-specific
error (e.g. "VOE Upload: <any generic message>") would burn the full
retries-per-account budget on the primary before the rotation logic
even kicked in. On retries=5 that's "Retry 2/5 Primär", "Retry 3/5
Primär", … all on the same broken account before the fallback gets
a shot.
Now:
- main.js pre-resolves the next fallback for every hoster at batch-
start (stored in _accountOverrides via the existing session cache +
primeOverrides path). Pre-job-swap still ignores it until the
primary is actually marked failed, so jobs still begin on primary.
- upload-manager.js: in the retry loop's generic error branch,
_hasPendingOverride() checks whether a usable fallback is ready.
If yes and the error is NOT transient (transient = network glitch
= retry same acc), break out to rotation. Marks primary failed,
rotates to acc2, retries there.
- Result: for a 2-account hoster, worst case is 1 attempt on primary
+ retries-per-account on fallback, instead of N × 2. Transient
network errors (ENOTFOUND / ECONNRESET / socket hang up) keep the
old "retry same account" semantics because the network is the
issue, not the account.
- Single-account hosters: unchanged. No pending override = classic
retry-on-same-account until exhausted.
3 new tests pin: generic + override → rotate on attempt 1; transient
+ override → stay on same acc; no override → classic retry. 87/87
green.
buildPersistedQueueState persisted every non-aborted job's status
as-is. At restart no upload manager exists, so a serialized 'queued'
or 'uploading' job showed up as "Wartet" / "Upload" even though
nothing was actually running — next to a "Bereit" job for the same
file on a different hoster (screenshot the user sent).
Collapse every non-terminal state (queued, getting-server, uploading,
retrying, aborted) to 'preview' during persistence. Terminal states
(done, error, skipped) survive as-is so the user keeps their history
/ error messages. Also clears error/result when collapsing so the
restored preview row doesn't carry stale failure text.
Two related visibility improvements.
1. Status cell now shows which account the job is running on:
"Upload · Primär", "Retry 2/3 · Fallback #1: <error>", etc.
- _emitProgress passes task.accountId in every progress event
- renderer maps accountId → position in config.hosters[hoster] and
renders "Primär" for index 0 and "Fallback #N" for the rest
- Applies to uploading/getting-server/retrying (static states like
done/error already tell their own story)
2. Right-click on a job → "Log anzeigen" opens a modal with the full
per-job trail: every rot-log entry tagged with that job's jobId
plus every non-uploading progress transition. Replaces the need to
grep account-rotation.log for a single filename.
- UploadManager: all 13 job-scoped _rotLog calls now carry jobId
- main.js: _jobLogCollector Map<jobId, Array<entry>> with 200-entry
ring buffer per job; cleared on each new start-upload (fresh
batch = fresh log). addJobs mid-batch keeps history.
- New IPC 'get-job-log' returns the array; preload.js exposes
window.api.getJobLog(jobId)
- renderer: modal card + context-menu item "Log anzeigen";
entries formatted as "[HH:MM:SS.mmm] [event] k=v k=v"; copy-to-
clipboard button
Status "Fehlgeschlagen" alone forced the user to dig into
account-rotation.log to understand why a job failed. For error /
retrying / skipped statuses, append the (shortened, whitespace-
collapsed) error message — same approach already in place for v2.
Caps at 100 chars so the cell stays readable.
Previously: clicking "Erneut versuchen" after a batch had already
finished spawned a fresh UploadManager with empty _failedAccounts and
_accountOverrides. The first retry then burned the full retry budget
on the account we already knew was dead (e.g. disk-space-full byse
account) before rotation kicked in again — same problem we fixed for
within-batch flow but for across-batch flow.
- main.js: two module-level maps (_sessionFailedAccounts,
_sessionAccountOverrides) cache rotation state across batches in the
same app session. Populated on account-failed and on both
switchAccount paths (event-driven + save-config re-resolve).
- lib/upload-manager.js: startBatch(tasks, opts) accepts
primeFailedAccounts + primeOverrides. State is still cleared first
(legacy behaviour for callers without opts), then re-primed from the
passed session state. batch-start rot-log entry reports how many
entries were primed for diagnostics.
- Tests: prime priority is honored (pre-job-swap fires on first
attempt, no fast-fail, no upload to acc1); back-compat for callers
that don't pass opts.
App restart remains the reset signal — matches the "neuer Tag, acc1
hat vielleicht wieder Platz" expectation.
Three small, unrelated reliability improvements bundled:
1. lib/hosters.js (_resolveByseUploadByName): drop the "only one new
file → claim it" fallback. Under parallel byse uploads, job A's
poller could claim job B's newly-uploaded file and return the wrong
URL. Now requires exact normalized name match. Trade-off: a few
false negatives if byse rewrites the filename beyond our
normalizer, but parallel correctness wins.
2. tests/upload-manager.test.js: pin the transient-network classifier
behaviour with 2 new tests covering common transient strings
(ENOTFOUND, ECONNRESET, socket hang up, fetch failed, EAI_AGAIN…)
and verifying real account-level / file-rejected errors are NOT
misclassified as transient. Baseline stays clean: 82/82 green.
3. main.js: log process.memoryUsage() snapshot at batch-start and
batch-done. One line each — harmless in the happy path, gives us
the data points needed to spot long-session RSS/heap growth across
batches without DevTools instrumentation.
Non-uploading progress events (queued/getting-server/retrying/done/
error/aborted/skipped) were firing renderQueueTable +
updateQueueActionButtons + updateStatusBar + updateStatsPanel
synchronously on EVERY event. At batch start, 500 jobs going
preview→queued→getting-server within milliseconds meant ~2000 sync DOM
updates — visible jank on large batches.
New scheduleStatusChangeUpdate() uses requestAnimationFrame to coalesce
the four-helper call into at most one run per frame (~60 Hz). Functional
result is identical; the user just sees smooth flips instead of a
briefly frozen renderer.
The uploading-progress throttle (200ms) is unchanged since those events
are much more frequent and the user doesn't need 60 Hz upload-byte
updates.
At 500+ queued jobs these lines bloated upload-debug.log with
megabyte-sized entries per batch-start and added visible latency to
the IPC handler. Log sizes only.
Three related gaps closed so one full byse account stops wasting
attempts on every subsequent job and later-added accounts get picked
up without an app restart.
1. Pre-job-swap moved BEHIND the semaphore acquire. At scale (500 jobs
/ 1 slot) every worker was checking _failedAccounts at spawn time
before the first upload had even tried — so none of them saw the
failed state. Now each worker re-checks right before its first
upload attempt.
2. save-config IPC handler re-resolves fallbacks for any account that
is already in _failedAccounts but has no override set. Previously
account-failed only fired once per account, so a config change
after the first mark-failed was silently ignored and the batch
stayed stuck on the dead account until the app restarted.
3. UploadManager exposes getFailedAccountKeys() and getOverride(hoster)
so main.js can drive the late re-resolve without poking private
fields.
4 new tests: pre-job-swap after semaphore, getters contract, fresh
manager resets learned state, late-added fallback is honored by
subsequent jobs. 80/80 green.
Byse rejects uploads with status like "not enough disk space on your
account" when the account's storage is exhausted. The parser was
flagging every non-OK status as err.fileRejected=true, and the upload-
manager classifier additionally matched the generic "lehnte Datei ab"
prefix as file-rejected. Result: rotation was skipped on a full account
and every subsequent file failed on the same dead account.
- hosters.js: byse parser now distinguishes account-level phrases
(disk space / storage / quota / insufficient / account full) and sets
err.accountError=true for those. File-specific failures (Duplicate,
wrong format, size) keep err.fileRejected=true.
- upload-manager.js: _isFileRejectedError no longer matches the generic
"lehnte Datei ab" prefix and short-circuits when err.accountError is
true. _shouldSkipRetryOnAccountError honors the flag and has added
regex patterns as a safety net.
- Tests: 5 new unit tests covering disk-space/account-level/duplicate
and the accountError-wins-over-fileRejected precedence.
retrySelectedJobs() was calling renderQueueTable + updateQueueActionButtons
+ updateStatusBar and then immediately awaiting startSelectedUpload(),
which runs the exact same trio right after. At 500+ failed jobs the
double render/sort/button-refresh freezes the UI for several seconds
after clicking "Erneut versuchen".
Drop the outer render trio — startSelectedUpload's one is enough. The
inner call sees the freshly-mutated job state in the same tick, so the
visible result is identical with half the work.
User reported uploads appearing on the byse dashboard (2+ GB MKV,
Server #262, status OK) even though the app marked them failed. The
byse API sometimes replies with msg=OK + files:[{filecode:"",
status:"Not video file format"}] — a misleading response where the
file is actually being accepted and gets its filecode assigned
asynchronously.
- Before the upload POST, snapshot the current /api/file/list to
know what was already there.
- If parseByseResult returns an empty filecode (or throws a
fileRejected error), poll /api/file/list up to 15 × 2s looking
for a new file_code matching the uploaded filename (case/punct
normalized, extension stripped).
- If matched, return the real download/embed URLs and let the
upload complete as successful. Only throw the parser's error
if polling also finds nothing.
The log revealed byse's true response shape for rejected files:
{ msg: 'OK', status: 200, files: [{ filecode: '', status: 'Not video file format' }] }
HTTP 200 + msg=OK made the old code treat it as 'success but no
file_code'. The real error ('Not video file format') was buried in
files[0].status. parseByseResult now surfaces that with a dedicated
err.fileRejected flag so the rotation layer can distinguish
file-specific vs account-specific failures.
Rotation behavior:
- file-rejected errors: no retries, no account blacklist, no
rotation. The same file is going to get the same verdict on
any account, so skip straight to 'error' status and keep the
account available for other files in the batch.
- network errors (already handled): no account blacklist either.
- everything else: unchanged (retry then rotate).
Also added pattern matches for common rejections (Duplicate, File
too small/large, Unsupported format, etc.) so other hosters'
per-file errors get the same treatment.
Two bugs visible in the user's rotation log:
1. 'error=OK' for byse.sx — the server returned a payload with
msg='OK' and no file_code anywhere we recognized. Our generic
uploadFile threw the bare 'OK' as the error message, which is
useless and misleading. Now when we see an ok-ish msg without
the expected file_code we throw a descriptive error that
includes the first ~400 bytes of the payload so the next time
it happens we can see what's actually being returned (API
changed, new field name, etc.).
2. 'getaddrinfo ENOTFOUND s1055.filemoon' was marking accounts as
permanently failed, blacklisting BOTH byse accounts within the
same batch even though neither was the actual problem — filemoon
(byse's storage backend) briefly had a DNS blip. Added
_isTransientNetworkError() covering DNS/ECONNRESET/ETIMEDOUT/etc.
When all retries on an account exhaust with a transient error,
we now fail just that file and emit 'skip-rotation-transient'
instead of adding the account to _failedAccounts. Other files
in the same batch still get a fresh try on the same account.
