User reported uploads appearing on the byse dashboard (2+ GB MKV,
Server #262, status OK) even though the app marked them failed. The
byse API sometimes replies with msg=OK + files:[{filecode:"",
status:"Not video file format"}] — a misleading response where the
file is actually being accepted and gets its filecode assigned
asynchronously.
- Before the upload POST, snapshot the current /api/file/list to
know what was already there.
- If parseByseResult returns an empty filecode (or throws a
fileRejected error), poll /api/file/list up to 15 × 2s looking
for a new file_code matching the uploaded filename (case/punct
normalized, extension stripped).
- If matched, return the real download/embed URLs and let the
upload complete as successful. Only throw the parser's error
if polling also finds nothing.
The log revealed byse's true response shape for rejected files:
{ msg: 'OK', status: 200, files: [{ filecode: '', status: 'Not video file format' }] }
HTTP 200 + msg=OK made the old code treat it as 'success but no
file_code'. The real error ('Not video file format') was buried in
files[0].status. parseByseResult now surfaces that with a dedicated
err.fileRejected flag so the rotation layer can distinguish
file-specific vs account-specific failures.
Rotation behavior:
- file-rejected errors: no retries, no account blacklist, no
rotation. The same file is going to get the same verdict on
any account, so skip straight to 'error' status and keep the
account available for other files in the batch.
- network errors (already handled): no account blacklist either.
- everything else: unchanged (retry then rotate).
Also added pattern matches for common rejections (Duplicate, File
too small/large, Unsupported format, etc.) so other hosters'
per-file errors get the same treatment.
Two bugs visible in the user's rotation log:
1. 'error=OK' for byse.sx — the server returned a payload with
msg='OK' and no file_code anywhere we recognized. Our generic
uploadFile threw the bare 'OK' as the error message, which is
useless and misleading. Now when we see an ok-ish msg without
the expected file_code we throw a descriptive error that
includes the first ~400 bytes of the payload so the next time
it happens we can see what's actually being returned (API
changed, new field name, etc.).
2. 'getaddrinfo ENOTFOUND s1055.filemoon' was marking accounts as
permanently failed, blacklisting BOTH byse accounts within the
same batch even though neither was the actual problem — filemoon
(byse's storage backend) briefly had a DNS blip. Added
_isTransientNetworkError() covering DNS/ECONNRESET/ETIMEDOUT/etc.
When all retries on an account exhaust with a transient error,
we now fail just that file and emit 'skip-rotation-transient'
instead of adding the account to _failedAccounts. Other files
in the same batch still get a fresh try on the same account.
Four user-visible lag sources tracked down from a wider audit:
- Tab click was running three full querySelectorAll walks per click
(remove active from all tabs, all views, find new tab). Replaced
with delegated listener on the tab bar plus cached node maps;
tab switching is now O(1) and a no-op when clicking the active tab.
- saveSettings awaited saveHosterSettings + saveGlobalSettings
serially and then re-fetched the full config from main. With
autosave firing on every keystroke this added 100–200ms of IPC
stall per input change. The two saves now run in parallel and the
post-save getConfig refetch is gone — we know the new state.
- showContextMenu rebuilt hosterCounts (queueJobs.forEach) on every
right-click. Replaced with a length-keyed cache; right-click on a
5000-job queue no longer pauses while counting.
- Recent-panel shift-click was querying every .recent-file-row in
the DOM and re-parsing data-order. Reuses _recentSortCache.result
instead, O(visible) vs O(N).
When the configured log path isn't writable and we fall back to
Desktop/userData, the working fallback now gets saved into
globalSettings.logFilePath automatically. Benefits:
- Next session writes directly to the known-working path, no
fallback ladder, no recurring toast warning.
- The Settings input reflects the actual path in use, so users
don't stay confused about where their uploads are being logged.
- Live update via IPC — if the Settings view is currently open,
the input value updates without needing a view switch.
Daily-log mode is handled: we strip the -YYYY-MM-DD suffix before
persisting so tomorrow's auto-rotation doesn't double-date the
filename.
alert() in Electron halts the renderer main thread until the user
clicks OK — the upload table, status bar and progress all freeze.
During a 170-file batch the dialog popped up mid-upload and froze
everything for however long the user took to dismiss it (which is
why stats updates lagged to one every 3-5s instead of the usual 1s
cadence).
Replaced with the same showCopyToast used elsewhere, with an 8s
duration so the message is still readable. showCopyToast now accepts
an optional durationMs argument.
When multiple jobs run in parallel on the same hoster and the primary
account starts failing, the first job marks it failed + triggers
rotation. The second job's retries then also exhaust on the same
(already-failed) primary — but the old while-condition
`!_failedAccounts.has(...)` short-circuited the whole rotation loop
for anything already marked, so the second job went straight to
final-error even though a resolved override was sitting right there.
Now the loop always checks for an available override; it only skips
the mark-failed + emit step if the account was already marked by a
concurrent job. Fixed visible symptom: first job rotates A→B, every
other job in the same batch that hit A got final-error instead of
also switching to B.
Also extended fast-fail patterns to include 429 (Too many requests),
CSRF-Token / 'Bist du eingeloggt' — both were showing up as the
primary failure mode in real uploads and were wasting 5 retries
each.
Three related improvements that landed together while wiring up the
rotation log infrastructure:
- Fast-fail classifier: errors that clearly indicate the account
itself is the problem (rate limit, quota, banned/suspended, auth
failure, 401/403, 'Kein Upload-Server' from delivery-node etc.)
now skip the remaining retries and go straight to rotation. No
more waiting 5 × 3s between retries just to end up rotating
anyway. Emits a 'fast-fail' rot-log event so the shortcut is
visible.
- Settings: 'Öffnen' button next to the log-file-path input reveals
the active log file (or its directory if nothing's written yet)
in the OS file manager, so users don't have to remember paths.
- rotLog() writes the rotation log synchronously. Only a handful
of events fire per batch; the 500ms flush batching was saving
nothing and made the file look empty when users checked right
after an event. (The main debug log still uses the batched async
path — that one is high-volume.)
To trace whether the fallback chain actually engages during real uploads,
every rotation decision now emits a structured 'rot-log' event from the
upload-manager. main.js persists each event to a new account-rotation.log
(same directory as fileuploader.log; falls back to Desktop then userData)
and also mirrors it into the main debug log with a [ROT] prefix for
single-file grepping.
Logged events:
- batch-start (clears _failedAccounts / _accountOverrides)
- pre-job-swap / pre-job-swap-blocked (job picks override before first try)
- retries-exhausted / mark-failed (enters rotation loop)
- rotate (switched to new account, retry starting)
- rotation-end (no override / override already failed)
- final-error (all accounts exhausted)
- switchAccount (main resolved the next fallback)
The renderer shows a toast on 'rotate', 'rotation-end' and 'final-error'
so fallback behavior is visible live instead of buried in logs.
The Accounts view rebuilt the whole list on every enable/disable/
check/reorder. Each render destroyed and recreated four click
listeners plus five drag listeners per card (20 accounts = 180
listeners cycled per click), then ran an IPC getConfig round-trip
on top. Typing-fast enable/disable toggles felt sludgy.
- Single delegated click handler on the accounts container.
- Single delegated set of drag/drop handlers (one per event type,
not per card).
- Listeners are bound once on first render, never rebound.
- updateAccountCard(accountId) swaps just the one affected card's
DOM node when its state changes. toggleAccount / checkSingleAccount
use that instead of calling renderAccounts.
- Drag-and-drop reorder moves the DOM node in place and re-renders
only the priority badges of the affected group — no container
rebuild, no getConfig refetch.
Captured the real browser upload POST and compared to our request.
Two corrections:
- The file field is named 'file', not 'file_0'. The XFS-indexed
naming was a bad guess — the current transit accepts only 'file'.
- The form also needs 'to_json=1' (forces JSON response instead of
an HTML redirect page, matching what the browser submits) and
'fld_id=0' (destination folder, 0 = root). Dropped upload_type,
srv_tmp_url, utype — those were XFS remnants and aren't part of
the current server's contract.
- Response shape is now { status: 'OK', file_code, msg } instead of
the older { files: [...] } / { result: ... } XFS variants; the
parser handles all three plus carries the server's msg forward
on explicit rejections.
The transit server runs nginx-upload-progress and requires an
X-Progress-ID query parameter on the POST URL to finalize the
upload session. Without it the server accepts all bytes but never
sends the response — matches the reported 99%-stuck behavior. The
browser appends it automatically before submit; we now do the same.
Upload stalled at 99% because we were sending vidmoly.me cookies to
*.vmwesa.online (transit server rejects them silently). Browsers never
send those cross-origin. Now we omit the Cookie header and match the
Origin/Referer the browser uses. Also added the full classic XFS field
set (upload_type, sess_id, srv_tmp_url, utype) in the order the
server's handler expects.
The SPA redesign killed the old XFS form POST at / with op=login.
The new flow is a JSON POST to /api/auth/login that returns a
vidmoly_session HttpOnly cookie, which is what /api/upload/config
actually authenticates against.
After login we also probe /api/upload/config once to fail fast if
the session was issued but not actually valid for uploads.
The old /my HTML check failed because it couldn't distinguish an XFS
session from a full SPA session. Since /api/upload/config is what the
upload actually needs, probe it directly after login — 200 JSON with
sess_id/upload_url means we're good, anything else means we're out.
The Vidmoly SPA redesign removed the /?op=upload HTML form — the old
regex-scrape of hidden inputs no longer works. The site now exposes
GET /api/upload/config which returns { sess_id, upload_url } plus the
allowed extensions. Rewrote getUploadParams() to use that endpoint;
the rest of the multipart upload flow (sess_id + utype + file_0) is
the same classic XFS shape.
Three fixes bundled:
- Vidmoly redesign broke login: the old check required either the
'login' or 'xfsts' cookie, but the new site sets different cookie
names. Now we verify by fetching /?op=my_account and looking for
logged-in markers (Logout / My Account / My Files) in the body
instead of relying on specific cookie names.
- retrySelectedJobs left the stale uploadId in _jobIndexByUploadId
when resetting a job. A late 'aborted'/'error' event from the
original (cancelled) upload could route back to the reset job
and overwrite its 'preview' state. Now the old uploadId is
removed from the index and marked in _deletedJobIds so those
stragglers get dropped.
- toggleAccount did two IPC round-trips (saveConfig + getConfig) on
every enable/disable click, plus four re-renders (Accounts,
HosterSummary, HosterModal, Settings). Rapid clicks felt laggy.
The getConfig refetch is redundant since we mutated the flag in
place, and HosterModal/Settings don't depend on account enabled
state. Click now renders immediately and the save runs async.
Three state bugs found during audit:
1. _failedAccounts / _accountOverrides survived across batches. A
rate-limited account from batch 1 stayed permanently blacklisted
for the rest of the app session, so batch 2 skipped straight to
the fallback even after the original recovered. Now cleared in
startBatch so each run evaluates accounts fresh.
2. Account rotation was one level deep. With three accounts [A,B,C]
on the same hoster and A + B both failing, the job errored out
— C was never tried. The fallback-retry was a single if-block.
Replaced with a while-loop that keeps asking main for the next
override and rotating until every account is exhausted.
3. Queue sort cache included 'size' as a static key, but bytesTotal
goes 0 → actual when previews resolve. A queue sorted by size
during preview would cache the all-zeros order and never update.
Removed size from _STATIC_SORT_KEYS — it now re-sorts per render
like status/speed/progress.
Hot path on large table rebuilds — every text cell runs through one
of these. Switching from 4 chained .replace() calls to a single regex
with a lookup map is ~3× faster. At 5000 rows × 4 fields per rebuild,
80k → 20k regex operations.
Last round of targeted wins:
- upload-manager progress callback was allocating a fresh
{ jobId, speedKbs, bytesUploaded } object on every fs stream chunk
(hundreds of times per second per active job). Now a single entry
is created at job start and mutated in place — zero allocations
on the steady-state progress tick.
- upload-manager stats timer's two separate activeJobs.values()
scans (globalSpeedKbs + inProgressBytes) merged into one pass.
- clouddrop-upload.js reuses a single Buffer.allocUnsafe(chunkSize)
across all chunks, taking subarray() only for the tail chunk.
A 1 GB upload no longer allocates 64× 16 MB = 1 GB of short-lived
buffers — real GC relief during many-file batches.
- _resolveUploadLogTarget is now cached; the fallback ladder runs
once per session (or when the user changes the log path / daily-log
date rolls), not on every 500ms flush.
- renderRecentUploadsPanel skips updateRecentSortHeaders on the
append-only fast path — sort state hasn't changed, headers don't
need recomputing.
Three more targeted wins:
- loadHistory() was called unconditionally on every handleBatchDone,
doing an IPC roundtrip + full history-table rebuild even when the
user is on the Upload tab and can't see it. Now it sets a dirty
flag and the actual refresh is deferred until the user switches
to the Verlauf tab. On a fresh tab click it always runs.
- renderRecentUploadsPanel append-only fast path: when the sort is
'date desc' (the default) and the dataset only grew, the panel
inserts the new rows at the top via insertAdjacentHTML instead
of rebuilding the 5000-row tbody from scratch. Length shrinks or
sort-change still trigger a full rebuild.
- handleBatchDone's removeFromQueueOnDone cleanup now does one pass
(build keep-list + detach from index together) instead of two
separate filter() scans over queueJobs.
Four more wins targeting batch-heavy paths:
- updateQueueActionButtons replaced three O(n) queueJobs.some() scans
with a single O(|selection|) pass over selectedJobIds, using the
existing _jobIndexById map. Selection change cost on a 1000-job
queue drops from ~3000 comparisons to |selection|.
- applySummaryResults built a (fileName+hoster)→job Map once per call
instead of running queueJobs.find() per result. Big batches
(hundreds of files × multiple hosters) no longer scale O(n²).
- addPathsToQueue and the folder-monitor auto-queue path built their
dedup Set up front instead of running .find() per incoming path.
Picking a folder with thousands of files now dedups in O(n+m)
instead of O(n×m).
- appendUploadLog became async + buffered like debugLog. A burst of
20 files completing within a second becomes one fs.appendFile
instead of 20 fs.appendFileSync that each blocked the main event
loop. Fallback ladder (primary → Desktop → userData) is preserved;
pending buffer flushes synchronously on before-quit.
Three more rounds of lag removal aimed at heavy upload sessions:
- main-process debugLog() was doing fs.appendFileSync on every call
and was firing hundreds of times per second during busy uploads
(progress transitions, unhandled rejection traces, folder-monitor
events). Replaced with an in-memory buffer flushed every 500ms via
async appendFile — the main event loop is no longer blocked per
line. Buffered entries flush synchronously on before-quit.
- the renderer's 'RX upload-progress' / 'RX upload-stats' listeners
were emitting one IPC roundtrip per event. For 20 concurrent jobs
that's 80 IPC messages/sec just for logging. They now skip the
debug call on the hot 'uploading' tick and only log transitions.
- _onQueueScroll now coalesces scroll events via requestAnimationFrame
so a fast trackpad fling triggers one virtual render per frame
instead of one per wheel event.
- maybeAddSessionFile switched from O(n) sessionFilesData.some() dedup
to an O(1) Set lookup keyed on (link, filename, host). Adding 1000
results to an already-populated panel drops from ~500ms to <5ms.
