DLC-Import Hang gefixt: kein sync-FS Log-I/O mehr pro Link

Symptom: Nutzer zieht DLC mit vielen Paketen rein, App haengt 1-2 min.

Ursache: addPackages() rief logPackageForItem() pro Link auf. Jede
dieser Calls triggerte ~10 synchrone FS-Operationen:
  - ensurePackageLog: mkdirSync + existsSync
  - ensureItemLog: mkdirSync + existsSync + writeFileSync (first-time)
    + 2× appendFileSync (first-time header)
  - logPackage + writeItemLogEvent (appendFileSync, batched)

Bei einer DLC mit 60 Paketen × 25 Links = 1500 Items → ~9.000-15.000
sync FS-Calls. Auf langsamen Disks / Netzwerk-Shares: 60-120 Sekunden
Event-Loop-Blockade. UI eingefroren.

Fix: per-Item-Logs waehrend Bulk-Add nicht mehr initialisieren. Sie
werden lazy beim ersten echten Lifecycle-Event (Download-Start, Fehler)
angelegt. Stattdessen EINE zusammengefasste "Links registriert (N)"
Zeile ins Package-Log pro Paket — bei >50 Links mit gekuerzter
Vorschau (erste 20 + "+N more") damit die Log-Zeile nicht riesig wird.

Neuer Test "bulk-adds large DLC containers without initializing per-item
logs" verifiziert: 1500 Items werden in <5s hinzugefuegt (lokal unter
300ms), keine Item-Log-Dateien entstehen, pro Paket existiert genau ein
Package-Log.
This commit is contained in:
Sucukdeluxe 2026-04-21 21:36:19 +02:00
parent 56656dfef9
commit 75036edbd1
2 changed files with 83 additions and 7 deletions

View File

@ -2746,6 +2746,11 @@ export class DownloadManager extends EventEmitter {
linkCount: links.length
});
// Collect per-link summary for ONE batched package-log entry after the
// loop, instead of ~10 sync FS calls per link. A DLC with many packages
// was freezing the UI for 1-2 minutes because logPackageForItem called
// ensurePackageLog + ensureItemLog + appendFileSync×multiple per link.
const registeredLinkSummary: string[] = [];
for (let linkIdx = 0; linkIdx < links.length; linkIdx += 1) {
const link = links[linkIdx];
const itemId = uuidv4();
@ -2772,13 +2777,7 @@ export class DownloadManager extends EventEmitter {
updatedAt: nowMs()
};
this.assignItemTargetPath(item, path.join(outputDir, fileName));
this.logPackageForItem(item, "INFO", "Link registriert", {
index: linkIdx + 1,
totalLinks: links.length,
url: link,
hintedName: hintName || "",
initialTargetPath: item.targetPath
});
registeredLinkSummary.push(`#${linkIdx + 1} ${fileName} <- ${link}`);
packageEntry.itemIds.push(itemId);
this.session.items[itemId] = item;
this.itemCount += 1;
@ -2795,6 +2794,22 @@ export class DownloadManager extends EventEmitter {
addedLinks += 1;
}
// One batched log entry per package instead of one per link.
// Item-logs are left uninitialized here — they'll be lazily created
// the first time the item actually gets a real lifecycle event
// (download start, error, etc.). For very large packages (>50 links)
// we only log the first 20 + a "... +N more" suffix so the single log
// line doesn't grow into hundreds of KB.
if (registeredLinkSummary.length > 0) {
const PREVIEW = 20;
const linksField = registeredLinkSummary.length <= 50
? registeredLinkSummary.join(" | ")
: `${registeredLinkSummary.slice(0, PREVIEW).join(" | ")} | ... +${registeredLinkSummary.length - PREVIEW} more`;
this.logPackageForPackage(packageEntry, "INFO", `Links registriert (${registeredLinkSummary.length})`, {
links: linksField
});
}
this.session.packages[packageId] = packageEntry;
this.session.packageOrder.push(packageId);
addedPackages += 1;

View File

@ -12,6 +12,7 @@ import { defaultSettings } from "../src/main/constants";
import { parseDebridLinkApiKeys } from "../src/shared/debrid-link-keys";
import { getProviderUsageDayKey } from "../src/shared/provider-daily-limits";
import { getItemLogPath, initItemLogs, shutdownItemLogs } from "../src/main/item-log";
import { initPackageLogs, shutdownPackageLogs } from "../src/main/package-log";
import { createStoragePaths, emptySession } from "../src/main/storage";
import { primeDebridLinkRuntimeCooldownForTests, resetDebridLinkRuntimeStateForTests } from "../src/main/debrid";
import { getRenameLogPath, initRenameLog, shutdownRenameLog } from "../src/main/rename-log";
@ -155,6 +156,7 @@ afterEach(async () => {
globalThis.fetch = originalFetch;
resetDebridLinkRuntimeStateForTests();
shutdownItemLogs();
shutdownPackageLogs();
shutdownRenameLog();
for (const dir of tempDirs.splice(0)) {
await removeDirWithRetries(dir);
@ -10314,4 +10316,63 @@ describe("download manager", () => {
await once(server, "close");
}
}, 30000);
it("bulk-adds large DLC containers without initializing per-item logs (avoids 1-2 min sync-FS freeze)", () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "rd-bulk-add-"));
tempDirs.push(root);
const stateDir = path.join(root, "state");
fs.mkdirSync(stateDir, { recursive: true });
initPackageLogs(stateDir);
initItemLogs(stateDir);
const manager = new DownloadManager(
{
...defaultSettings(),
token: "rd-token",
outputDir: path.join(root, "downloads"),
extractDir: path.join(root, "extract"),
autoExtract: false,
autoReconnect: false
},
emptySession(),
createStoragePaths(stateDir)
);
// 60 packages with 25 links each = 1500 items. This was freezing the UI
// for 1-2 min on slower filesystems because every item triggered
// ensurePackageLog + ensureItemLog + multiple sync appendFileSync calls.
const packages = Array.from({ length: 60 }, (_, pkgIdx) => ({
name: `bulk-pkg-${pkgIdx}`,
links: Array.from({ length: 25 }, (_, linkIdx) => `https://dummy/bulk-${pkgIdx}-${linkIdx}.rar`)
}));
const tStart = Date.now();
const result = manager.addPackages(packages);
const elapsedMs = Date.now() - tStart;
expect(result.addedPackages).toBe(60);
expect(result.addedLinks).toBe(1500);
// Hard cap: on any reasonable CI box this should complete well under 5 s.
// Before the fix, the same workload produced thousands of sync-FS writes
// and took 60-120 s even on fast local disks.
expect(elapsedMs).toBeLessThan(5000);
// No per-item log files should have been created — they're only
// initialized lazily when an item gets a real lifecycle event later.
// Item log files are named item_<id>.txt.
const itemLogsDir = path.join(stateDir, "item-logs");
const itemLogFiles = fs.existsSync(itemLogsDir)
? fs.readdirSync(itemLogsDir).filter((f) => f.startsWith("item_") && f.endsWith(".txt"))
: [];
expect(itemLogFiles.length).toBe(0);
// One package log per package. Package log file names are package_<id>.txt.
// The "Links registriert" entry is appended async (batched flush every
// ~250ms), so we don't assert content here — just that each package has
// been initialized with the startup block (ensurePackageLog wrote the
// "Paket-Log Start" header synchronously).
const packageLogsDir = path.join(stateDir, "package-logs");
const pkgLogFiles = fs.readdirSync(packageLogsDir).filter((f) => f.startsWith("package_") && f.endsWith(".txt"));
expect(pkgLogFiles.length).toBe(60);
});
});