fix(rotation): treat byse "disk space" as account-level, not file-rejected
Byse rejects uploads with status like "not enough disk space on your account" when the account's storage is exhausted. The parser was flagging every non-OK status as err.fileRejected=true, and the upload- manager classifier additionally matched the generic "lehnte Datei ab" prefix as file-rejected. Result: rotation was skipped on a full account and every subsequent file failed on the same dead account. - hosters.js: byse parser now distinguishes account-level phrases (disk space / storage / quota / insufficient / account full) and sets err.accountError=true for those. File-specific failures (Duplicate, wrong format, size) keep err.fileRejected=true. - upload-manager.js: _isFileRejectedError no longer matches the generic "lehnte Datei ab" prefix and short-circuits when err.accountError is true. _shouldSkipRetryOnAccountError honors the flag and has added regex patterns as a safety net. - Tests: 5 new unit tests covering disk-space/account-level/duplicate and the accountError-wins-over-fileRejected precedence.
This commit is contained in:
Administrator
parent
e3a785d4a7
commit
17e9a419b2
@ -221,8 +221,15 @@ function parseByseResult(payload) {
|
||||
}
|
||||
|
||||
if (!file_code && perFileError) {
|
||||
// Distinguish account-level from file-level failure. "not enough disk
|
||||
// space", "quota exceeded", "storage full" etc. mean the ACCOUNT is
|
||||
// exhausted — every further file on the same account will hit the same
|
||||
// wall, so we must rotate. File-specific rejections (Duplicate, wrong
|
||||
// format, too small/large) ARE per-file and rotation is pointless.
|
||||
const accountLevel = /(not enough (disk )?(space|storage)|insufficient (disk )?space|disk (space )?full|storage (exhausted|full|voll|limit)|quota (exceeded|voll|überschritten)|account (full|voll|suspended|banned))/i.test(perFileError);
|
||||
const err = new Error(`Byse lehnte Datei ab: ${perFileError}`);
|
||||
err.fileRejected = true;
|
||||
if (accountLevel) err.accountError = true;
|
||||
else err.fileRejected = true;
|
||||
throw err;
|
||||
}
|
||||
|
||||
|
||||
@ -59,12 +59,17 @@ class UploadManager extends EventEmitter {
|
||||
// File-specific rejections from the hoster: the same file will get rejected
|
||||
// on any account, so rotation is pointless. Matches the `err.fileRejected`
|
||||
// flag set by parsers plus known rejection phrases.
|
||||
// NOTE: We deliberately do NOT match the generic "lehnte Datei ab" prefix
|
||||
// here — that phrase is used by the Byse parser for both file- AND
|
||||
// account-level errors. Account-level ones set err.accountError instead,
|
||||
// which takes priority in _shouldSkipRetryOnAccountError.
|
||||
_isFileRejectedError(err) {
|
||||
if (!err) return false;
|
||||
if (err.accountError === true) return false; // explicit account-level wins
|
||||
if (err.fileRejected === true) return true;
|
||||
if (!err.message) return false;
|
||||
const m = String(err.message);
|
||||
return /(Not video file format|Duplicate|Datei zu (klein|gross|groß)|File too (small|large)|Invalid file|Unsupported format|lehnte Datei ab)/i.test(m);
|
||||
return /(Not video file format|Duplicate|Datei zu (klein|gross|groß)|File too (small|large)|Invalid file|Unsupported format)/i.test(m);
|
||||
}
|
||||
|
||||
// Transient network errors — the account is fine, the network or the
|
||||
@ -100,7 +105,10 @@ class UploadManager extends EventEmitter {
|
||||
// account. Keeps single runs fast when an account is rate-limited, banned,
|
||||
// or out of quota.
|
||||
_shouldSkipRetryOnAccountError(err) {
|
||||
if (!err || !err.message) return false;
|
||||
if (!err) return false;
|
||||
// Explicit account-level flag from hoster parsers — highest priority.
|
||||
if (err.accountError === true) return true;
|
||||
if (!err.message) return false;
|
||||
const m = String(err.message);
|
||||
const PATTERNS = [
|
||||
/Kein Upload-Server/i,
|
||||
@ -123,7 +131,13 @@ class UploadManager extends EventEmitter {
|
||||
/CSRF[- ]?Token nicht gefunden/i,
|
||||
/CSRF[- ]?token not found/i,
|
||||
/Bist du eingeloggt/i,
|
||||
/not logged in/i
|
||||
/not logged in/i,
|
||||
// Storage exhaustion — account is full. Rotate instead of hammering it.
|
||||
/not enough (disk )?(space|storage)/i,
|
||||
/insufficient (disk )?space/i,
|
||||
/disk (space )?full/i,
|
||||
/storage (exhausted|full|voll|limit)/i,
|
||||
/account (full|voll)/i
|
||||
];
|
||||
return PATTERNS.some(p => p.test(m));
|
||||
}
|
||||
|
||||
@ -6,6 +6,15 @@
|
||||
**Regel:** Wenn ein Click-Handler `await anotherHandler()` aufruft und der innere Handler seinen eigenen kompletten Render-Zyklus hat, NIEMALS noch einen davor. Einmal ist genug — der folgende innere Render sieht die frischen State-Mutationen ohnehin.
|
||||
**Wie anwenden:** Vor jeder `await fn()`-Folge in einem Handler prüfen: macht `fn` schon `renderQueueTable()`? Wenn ja, äußere Render-Calls löschen.
|
||||
|
||||
## 2026-04-21 — Error-Klassifikation: fileRejected vs accountError
|
||||
**Symptom:** Voller Byse-Account wurde nicht rotiert — `skip-rotation-file-rejected` geloggt für jede Datei.
|
||||
**Root cause:** Generisches Match auf Prefix-String (`"lehnte Datei ab"`) klassifizierte ALLE Byse-Errors als file-level, inklusive Account-voll-Meldungen.
|
||||
**Regel:** Hoster-Parser setzen den **spezifischen Flag** (`fileRejected` ODER `accountError`), nicht beide nie. Classifier matcht **konkrete Phrasen** (Duplicate, Not video format, …), niemals generische Wrapper-Strings die für mehrere Fehlerarten benutzt werden.
|
||||
**Wie anwenden:**
|
||||
- Bei neuen Hostern: per-status-Klassifikation bereits im Parser, nicht erst im Upload-Manager.
|
||||
- Classifier-Regexes auf Rejection-Kernphrasen, nicht auf UI-Prefix.
|
||||
- Defensive: `accountError === true` gewinnt immer gegen `fileRejected` — Account-Rotation ist weniger schlimm als endlose Fails auf einem toten Account.
|
||||
|
||||
## 2026-04-21 — Keine fake Build-ETAs
|
||||
**Symptom:** User wartet 5+ min auf Tauri-Build den ich mit "1-2min" angekündigt habe.
|
||||
**Regel:** Tauri-Release-Builds brauchen real 3-6 min (Rust + NSIS + MSI). Keine Zeitangabe oder ehrlich "kann 3-6min dauern" schreiben.
|
||||
|
||||
@ -1,22 +1,30 @@
|
||||
# Retry-Hang bei 500-600 failed jobs
|
||||
# Rotation-Bug: Byse "not enough disk space" wird als file-rejected klassifiziert
|
||||
|
||||
## Problem
|
||||
User hat 500-600 Uploads, davon viele `error`. Markiert alle, klickt "Erneut versuchen" → App friert mehrere Sekunden ein.
|
||||
Log zeigt: Byse-Account ist voll ("not enough disk space on your account"), aber das System klassifiziert den Fehler als **file-rejected** und rotiert deshalb NICHT zum Fallback-Account. Jede nächste Datei landet beim selben vollen Account → endlose Fails.
|
||||
|
||||
## Root Cause (renderer/app.js)
|
||||
`retrySelectedJobs()` (Z.1952) → mutiert 500 Jobs → `renderQueueTable() + updateQueueActionButtons() + updateStatusBar()` (Z.1988-1990) → dann `await startSelectedUpload()` (Z.1992) → das mutiert SIE ERNEUT (Z.1724-1732) → rendert NOCHMAL (Z.1733-1735).
|
||||
## Root Cause
|
||||
- `lib/hosters.js:223-227` — Byse-Parser setzt `err.fileRejected = true` für JEDEN status-String der nicht `ok/success/done` ist.
|
||||
- `lib/upload-manager.js:67` — `_isFileRejectedError` regex matcht generisch `"lehnte Datei ab"` → gilt für ALLE Byse-Errors unabhängig vom eigentlichen Grund.
|
||||
- Upload-manager flow: `_isFileRejectedError` → break retry → `skip-rotation-file-rejected` → return. Kein `mark-failed`, kein Fallback-Resolve. Account bleibt aktiv für die nächste Datei.
|
||||
|
||||
**Doppelter DOM-Zyklus + doppelte Mutation bei 500 Jobs = hängt-Gefühl.**
|
||||
## Fix
|
||||
- [x] `lib/hosters.js`: Byse-Parser erkennt account-level phrases (disk space / storage / quota / insufficient / account full) → setzt `err.accountError = true` statt `fileRejected`.
|
||||
- [x] `lib/upload-manager.js` — `_isFileRejectedError`: generischen `lehnte Datei ab` Match entfernt. Explicit: `accountError === true` → früher out (ist NICHT file-rejected).
|
||||
- [x] `lib/upload-manager.js` — `_shouldSkipRetryOnAccountError`: honoriert `err.accountError === true` Flag. Patterns erweitert um disk-space/storage/quota/account-voll Phrasen (Safety-Net falls Flag mal fehlt).
|
||||
- [x] `tests/upload-manager.test.js`: 5 neue Tests für die Klassifikation (disk-space ist account-level; Duplicate bleibt file-rejected; accountError gewinnt gegen fileRejected).
|
||||
- [x] `npm test` — 76/76 grün.
|
||||
- [ ] Release als 3.1.4 (auf User-OK).
|
||||
|
||||
Zusätzlicher Multiplikator: main.js:1066 `JSON.stringify(files)` in `debugLog` — auch wenn `files` leer ist. Außerdem `buildUploadTasksFromJobs` loopt synchron 500x im main-Thread.
|
||||
|
||||
## Plan
|
||||
- [x] Entferne doppelten Render in `retrySelectedJobs` — `startSelectedUpload` macht das ohnehin in einem Rutsch.
|
||||
- [x] Lesson dokumentieren in `tasks/lessons.md`.
|
||||
- [ ] (optional, später) main.js:1066 — debugLog nicht mit JSON.stringify(files/hosters), nur counts loggen.
|
||||
- [ ] (optional, später) Progress-Event-Flood während laufendem Upload: batch via rAF statt sync updateQueueActionButtons+updateStatusBar+updateStatsPanel.
|
||||
## Expected Behavior nach Fix
|
||||
Log-Pattern ab Fix:
|
||||
```
|
||||
[retries-exhausted] hoster=byse.sx ... lastError=Byse lehnte Datei ab: 0:0:0:not enough disk space...
|
||||
[mark-failed] hoster=byse.sx accountId=byse.sx-1773722669098-qc45
|
||||
[switchAccount] hoster=byse.sx → fallback byse.sx-XXXXX
|
||||
[rotate] hoster=byse.sx → nächster Account
|
||||
```
|
||||
Bei Fast-Fail (über `_shouldSkipRetryOnAccountError`) entfällt der 5×3s Retry-Wait → rotation setzt sofort ein.
|
||||
|
||||
## Review
|
||||
Fix entfernt eine komplette Render-Pass-Runde vor dem Aufruf von `startSelectedUpload`. Bei 500+ Jobs halbiert das die Sync-Work im Click-Handler (Sort, Virtual-Render, Button-Update, StatusBar-Update). `selectedJobIds` bleibt korrekt befüllt weil die Selektion VOR `startSelectedUpload` gesetzt wird. `persistQueueStateSoon` ist debounced, deshalb kein Zusammenspielproblem.
|
||||
|
||||
Funktional identisch: Die Jobs durchlaufen am Ende dieselbe State-Kaskade (reset → queued → getting-server → uploading), nur mit einem statt zwei Render-Zyklen dazwischen.
|
||||
Zwei-Schichten-Ansatz: Byse-Parser setzt explicit `accountError` Flag (richtige Stelle weil der Parser den status-String direkt sieht), Upload-Manager honoriert den Flag und hat parallel Regex-Safety-Net. Test deckt beide Pfade ab.
|
||||
|
||||
@ -501,4 +501,57 @@ describe('UploadManager', () => {
|
||||
assert.ok('elapsed' in stat);
|
||||
assert.ok('activeJobs' in stat);
|
||||
});
|
||||
|
||||
describe('error classification', () => {
|
||||
it('treats "not enough disk space" as account-level, not file-rejected', () => {
|
||||
const mgr = new UploadManager({});
|
||||
// Shape matches what lib/hosters.js attaches for byse account-storage-full
|
||||
const err = new Error('Byse lehnte Datei ab: 0:0:0:not enough disk space on your account');
|
||||
err.accountError = true;
|
||||
assert.equal(mgr._isFileRejectedError(err), false,
|
||||
'account-level error must NOT be classified as file-rejected');
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(err), true,
|
||||
'account-storage-full must trigger account rotation');
|
||||
});
|
||||
|
||||
it('classifies disk-space errors by message alone (safety net)', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const err = new Error('Byse lehnte Datei ab: not enough disk space');
|
||||
// No flag set — regex alone must catch it.
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(err), true);
|
||||
assert.equal(mgr._isFileRejectedError(err), false,
|
||||
'must not match generic "lehnte Datei ab" as file-rejected');
|
||||
});
|
||||
|
||||
it('keeps true file rejections as file-rejected', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const err = new Error('Byse lehnte Datei ab: Duplicate');
|
||||
err.fileRejected = true;
|
||||
assert.equal(mgr._isFileRejectedError(err), true);
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(err), false);
|
||||
});
|
||||
|
||||
it('file-rejected regex still matches known phrases without flag', () => {
|
||||
const mgr = new UploadManager({});
|
||||
for (const msg of [
|
||||
'Not video file format',
|
||||
'Duplicate',
|
||||
'Datei zu klein',
|
||||
'File too large',
|
||||
'Invalid file'
|
||||
]) {
|
||||
assert.equal(mgr._isFileRejectedError(new Error(msg)), true, `should match: ${msg}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('accountError flag beats fileRejected if both set (defensive)', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const err = new Error('weird');
|
||||
err.fileRejected = true;
|
||||
err.accountError = true;
|
||||
assert.equal(mgr._isFileRejectedError(err), false,
|
||||
'account-level always wins — rotation must happen');
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(err), true);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
Loading…
Reference in New Issue
Block a user