fix(upload): classify doodstream empty-form as hoster-transient (don't kill account)
The "kein Filecode — Server gab leeren Link zurueck" error was treated as a generic upload failure → after retries exhausted, the manager called mark-failed and added the account to _failedAccounts → next batch re-primed with primedFailed=1 → pre-job-swap-blocked because no fallback override exists for a single-account hoster. One server-side flake permanently poisoned the session. It's not an account problem — same account + same file works on a later try. This is a doodstream-backend processing flake (empty CDN form, no fn / no st), the same class as a transient network error: don't blacklist, just fail this file cleanly. - doodstream-upload.js: tag the empty-form throw with err.hosterTransient=true (explicit flag, primary signal — matches the err.accountError / err.fileRejected pattern already used elsewhere). - upload-manager.js: new _isHosterTransientError classifier (flag first, message regex as defensive fallback). In the retry loop: break on first hit (server flake won't clear in 3 s, re-uploading the file 4× is pure bandwidth waste). Post-loop: dedicated branch that emits the final error WITHOUT blacklisting the account — same shape as the existing transient-network branch. - Tests: classifier unit tests (flag path, regex path, negatives) + regression test that proves the account is NOT added to _failedAccounts and mark-failed does NOT fire. Drops the hoster-transient test from ~19 s to ~1.5 ms, confirming the in-loop fast-break works. We now fail fast on this error class instead of retrying — the next-batch manual retry is the recovery path, and the account stays usable for it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
f0f1564322
commit
166b04c526
@ -486,7 +486,16 @@ class DoodstreamUploader {
|
|||||||
if (st && st !== 'OK') {
|
if (st && st !== 'OK') {
|
||||||
throw new Error(`Doodstream lehnt Datei ab (Server-Status: ${st}). CDN=${node}`);
|
throw new Error(`Doodstream lehnt Datei ab (Server-Status: ${st}). CDN=${node}`);
|
||||||
}
|
}
|
||||||
throw new Error(`Doodstream Upload: kein Filecode — Server gab leeren Link zurueck (st=${st || '?'}, fn=${fnInfo}, CDN=${node}). CDN-Antwort: ${(resText || '').slice(0, 200)}`);
|
// Empty form (no fn, no st) is a doodstream-side processing flake — same
|
||||||
|
// account + same file works on a later attempt. Tag it explicitly so the
|
||||||
|
// upload-manager classifies this as a hoster-transient error and does NOT
|
||||||
|
// blacklist the account (otherwise one of these flakes poisons the whole
|
||||||
|
// session and later batches hit `pre-job-swap-blocked` for no fault of
|
||||||
|
// the account). The flag is the primary signal; the message text is a
|
||||||
|
// belt-and-suspenders regex fallback in the classifier.
|
||||||
|
const emptyLinkErr = new Error(`Doodstream Upload: kein Filecode — Server gab leeren Link zurueck (st=${st || '?'}, fn=${fnInfo}, CDN=${node}). CDN-Antwort: ${(resText || '').slice(0, 200)}`);
|
||||||
|
emptyLinkErr.hosterTransient = true;
|
||||||
|
throw emptyLinkErr;
|
||||||
}
|
}
|
||||||
|
|
||||||
// 4. Fallback: follow form action as-is (for non-XFS forms)
|
// 4. Fallback: follow form action as-is (for non-XFS forms)
|
||||||
|
|||||||
@ -97,6 +97,22 @@ class UploadManager extends EventEmitter {
|
|||||||
return /(Not video file format|Duplicate|Datei zu (klein|gross|groß)|File too (small|large)|Invalid file|Unsupported format)/i.test(m);
|
return /(Not video file format|Duplicate|Datei zu (klein|gross|groß)|File too (small|large)|Invalid file|Unsupported format)/i.test(m);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Hoster-side transient flake — the hoster's backend accepted the upload but
|
||||||
|
// returned a malformed/empty result (e.g. doodstream CDN form with no fn/no
|
||||||
|
// st). Same account + same file works on a later attempt; this is NOT an
|
||||||
|
// account problem. Treated exactly like a transient network error: skip
|
||||||
|
// remaining in-batch retries (the flake won't clear in 3s and a re-upload of
|
||||||
|
// 95 MB is expensive), don't blacklist the account, fail this file cleanly.
|
||||||
|
// The user's next manual retry — or a later batch — can use the same account.
|
||||||
|
_isHosterTransientError(err) {
|
||||||
|
if (!err) return false;
|
||||||
|
if (err.hosterTransient === true) return true; // explicit flag — primary
|
||||||
|
if (!err.message) return false;
|
||||||
|
// Defensive fallback: catch the same class of error if it bubbles up
|
||||||
|
// wrapped (e.g. through a different code path) without the flag set.
|
||||||
|
return /Server gab leeren Link zurueck|kein Filecode/i.test(String(err.message));
|
||||||
|
}
|
||||||
|
|
||||||
// Transient network errors — the account is fine, the network or the
|
// Transient network errors — the account is fine, the network or the
|
||||||
// hoster's own backend hiccuped. Retrying on the SAME account is the right
|
// hoster's own backend hiccuped. Retrying on the SAME account is the right
|
||||||
// move; marking it failed would wrongly poison the fallback chain. If all
|
// move; marking it failed would wrongly poison the fallback chain. If all
|
||||||
@ -599,6 +615,17 @@ class UploadManager extends EventEmitter {
|
|||||||
// mind. Break out immediately; the outer file-rejected branch then
|
// mind. Break out immediately; the outer file-rejected branch then
|
||||||
// records the final error without burning through 5 × 3s retries.
|
// records the final error without burning through 5 × 3s retries.
|
||||||
if (this._isFileRejectedError(err)) break;
|
if (this._isFileRejectedError(err)) break;
|
||||||
|
// Hoster-side transient flake (e.g. doodstream empty CDN form). Server
|
||||||
|
// flake won't clear in 3s and re-uploading the whole file 4× is pure
|
||||||
|
// bandwidth waste; bail out of the retry loop so the post-loop branch
|
||||||
|
// can fail this file without blacklisting the account.
|
||||||
|
if (this._isHosterTransientError(err)) {
|
||||||
|
this._rotLog('hoster-transient', {
|
||||||
|
jobId, hoster: task.hoster, fileName, accountId: task.accountId,
|
||||||
|
attempt, error: err && err.message ? err.message : String(err)
|
||||||
|
});
|
||||||
|
break;
|
||||||
|
}
|
||||||
// Account-specific errors — don't waste retries on the same account,
|
// Account-specific errors — don't waste retries on the same account,
|
||||||
// jump straight to rotation.
|
// jump straight to rotation.
|
||||||
if (this._shouldSkipRetryOnAccountError(err)) {
|
if (this._shouldSkipRetryOnAccountError(err)) {
|
||||||
@ -664,6 +691,20 @@ class UploadManager extends EventEmitter {
|
|||||||
recordFinalResult('error', { error });
|
recordFinalResult('error', { error });
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
// Hoster-side transient flake → identical handling to network-transient:
|
||||||
|
// the account is fine, don't blacklist it, just fail this file. Critical
|
||||||
|
// to keep the account usable across batches — otherwise one empty-form
|
||||||
|
// response poisons every subsequent batch with `pre-job-swap-blocked`.
|
||||||
|
if (this._isHosterTransientError(lastError)) {
|
||||||
|
this._rotLog('skip-rotation-hoster-transient', {
|
||||||
|
jobId, hoster: task.hoster, fileName, accountId: task.accountId,
|
||||||
|
lastError: lastError ? lastError.message : null
|
||||||
|
});
|
||||||
|
const error = lastError.message || 'Hoster-Backend lieferte leeres Ergebnis';
|
||||||
|
emitFinalStatus('error', { error });
|
||||||
|
recordFinalResult('error', { error });
|
||||||
|
return;
|
||||||
|
}
|
||||||
// If the reason for failure was a transient network error we do NOT
|
// If the reason for failure was a transient network error we do NOT
|
||||||
// blacklist the account. Other jobs on the same account in this batch
|
// blacklist the account. Other jobs on the same account in this batch
|
||||||
// can still try fresh. This file just errors out for now.
|
// can still try fresh. This file just errors out for now.
|
||||||
|
|||||||
@ -835,6 +835,73 @@ describe('UploadManager', () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('hoster-transient flag is recognised (primary path)', () => {
|
||||||
|
const mgr = new UploadManager({});
|
||||||
|
const err = new Error('whatever');
|
||||||
|
err.hosterTransient = true;
|
||||||
|
assert.equal(mgr._isHosterTransientError(err), true);
|
||||||
|
// Must not be confused with other classes.
|
||||||
|
assert.equal(mgr._isFileRejectedError(err), false);
|
||||||
|
assert.equal(mgr._isTransientNetworkError(err), false);
|
||||||
|
assert.equal(mgr._shouldSkipRetryOnAccountError(err), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('hoster-transient regex fallback catches wrapped doodstream empty-form errors', () => {
|
||||||
|
const mgr = new UploadManager({});
|
||||||
|
const cases = [
|
||||||
|
'Doodstream Upload: kein Filecode — Server gab leeren Link zurueck (st=?, fn=fehlt/leer ...)',
|
||||||
|
'wrapper: Server gab leeren Link zurueck while parsing'
|
||||||
|
];
|
||||||
|
for (const msg of cases) {
|
||||||
|
assert.equal(mgr._isHosterTransientError(new Error(msg)), true, `should match: ${msg}`);
|
||||||
|
}
|
||||||
|
// Plain network and account errors must NOT match the hoster-transient class.
|
||||||
|
const negatives = [
|
||||||
|
'fetch failed',
|
||||||
|
'getaddrinfo ENOTFOUND',
|
||||||
|
'HTTP 429',
|
||||||
|
'quota exceeded',
|
||||||
|
'Byse lehnte Datei ab: Duplicate'
|
||||||
|
];
|
||||||
|
for (const msg of negatives) {
|
||||||
|
assert.equal(mgr._isHosterTransientError(new Error(msg)), false, `must NOT match: ${msg}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('regression: hoster-transient does NOT blacklist the account (account stays usable across batches)', async () => {
|
||||||
|
// Simulate doodstream-upload throwing the tagged empty-form error.
|
||||||
|
mockUploadFile.mock.mockImplementation(async () => {
|
||||||
|
const err = new Error('Doodstream Upload: kein Filecode — Server gab leeren Link zurueck (st=?, fn=fehlt/leer ...)');
|
||||||
|
err.hosterTransient = true;
|
||||||
|
throw err;
|
||||||
|
});
|
||||||
|
|
||||||
|
const mgr = new UploadManager(
|
||||||
|
{ 'doodstream.com': { retries: 3, parallelCount: 1, maxSpeedKbs: 0, restartBelowKbs: 0, timeIntervalSec: 0, maxSizeMb: 0 } }
|
||||||
|
);
|
||||||
|
const rotEvents = [];
|
||||||
|
mgr.on('rot-log', (e) => rotEvents.push(e));
|
||||||
|
|
||||||
|
// No username/password so the manager routes through the mocked
|
||||||
|
// hosters.uploadFile (instead of DoodstreamUploader directly).
|
||||||
|
await mgr.startBatch([
|
||||||
|
{ file: '/test/Arrested.Development.mkv', hoster: 'doodstream.com', apiKey: 'acc1-key', accountId: 'acc1' }
|
||||||
|
]);
|
||||||
|
|
||||||
|
const events = rotEvents.map(e => e.event);
|
||||||
|
// Must NOT poison the account — that's the entire point of this fix.
|
||||||
|
assert.equal(mgr._failedAccounts.size, 0, `account must NOT be blacklisted; _failedAccounts=${JSON.stringify(mgr.getFailedAccountKeys())}`);
|
||||||
|
assert.ok(!events.includes('mark-failed'), `must NOT mark-failed for hoster-transient; got: ${events.join(',')}`);
|
||||||
|
// The in-loop fast-break and the post-loop classification must both fire.
|
||||||
|
assert.ok(events.includes('hoster-transient'),
|
||||||
|
`expected hoster-transient (in-loop break, no wasted retries); got: ${events.join(',')}`);
|
||||||
|
assert.ok(events.includes('skip-rotation-hoster-transient'),
|
||||||
|
`expected skip-rotation-hoster-transient (post-loop branch); got: ${events.join(',')}`);
|
||||||
|
// And the retry loop must NOT burn the full retries=3 -> only 1 attempt on this account.
|
||||||
|
assert.equal(mockUploadFile.mock.calls.length, 1,
|
||||||
|
`must fail fast on hoster-transient, not re-upload the file 4× wasting bandwidth; got ${mockUploadFile.mock.calls.length} calls`);
|
||||||
|
});
|
||||||
|
|
||||||
it('late-resolved override is honored by subsequent jobs (simulates mid-batch config add)', async () => {
|
it('late-resolved override is honored by subsequent jobs (simulates mid-batch config add)', async () => {
|
||||||
// Only acc1 throws; acc2 succeeds.
|
// Only acc1 throws; acc2 succeeds.
|
||||||
mockUploadFile.mock.mockImplementation(async (hoster, filePath, apiKey, onProgress) => {
|
mockUploadFile.mock.mockImplementation(async (hoster, filePath, apiKey, onProgress) => {
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user