Compare commits
No commits in common. "master" and "v3.3.58" have entirely different histories.
3
.gitignore
vendored
3
.gitignore
vendored
@ -6,10 +6,7 @@ __pycache__/
|
||||
electron-config.json
|
||||
electron-config.json.bak
|
||||
electron-config.json.tmp
|
||||
electron-config.json.pre-history-split.bak
|
||||
electron-config.pre-import-*.json
|
||||
electron-history.json
|
||||
electron-history.json.tmp
|
||||
*.log
|
||||
debug.log
|
||||
fileuploader.log
|
||||
|
||||
@ -1,148 +0,0 @@
|
||||
# Remote Diagnostics Setup (read-only)
|
||||
|
||||
This guide explains how to let Claude Code run **read-only** diagnostics against a
|
||||
Multi-Hoster-Uploader instance running on a remote Windows server, through the local
|
||||
**stdio MCP gateway** in `gateway/`.
|
||||
|
||||
The gateway is an MCP server to Claude and a WebSocket client to a diagnostic agent inside the
|
||||
app. It never touches the screen, never injects input, and never writes anything on the server.
|
||||
|
||||
---
|
||||
|
||||
## 1. Enable diagnostics on the server and copy the code
|
||||
|
||||
On the **server** (the machine running the app):
|
||||
|
||||
1. Open the app's **Settings**.
|
||||
2. Enable **"Diagnose-Zugriff"**.
|
||||
3. Copy the connection **code**. It looks like `mhu1_<base64url...>`.
|
||||
|
||||
The code carries the **host**, **port** and a one-time auth **token** (`mhu1_<base64url{v,h,p,t,n}>`).
|
||||
The bridge dials the host from the code, so you usually just hand over the code. Treat the code as a
|
||||
**secret**: anyone with the code and network reach to the agent can read diagnostics.
|
||||
|
||||
**Two visibility modes** (Settings → Diagnose-Zugriff → Sichtbarkeit):
|
||||
|
||||
- **Nur lokal** (default): the agent binds to `127.0.0.1`. Reach it through a tunnel (next step).
|
||||
- **Im Netzwerk**: the agent binds to `0.0.0.0` but is gated by a **fail-closed IP allowlist** — only
|
||||
source IPs/CIDRs you list may connect (loopback is always allowed), *in addition to* the token. An
|
||||
empty allowlist means loopback only. This is the mode to use with **Tailscale**: set the allowlist
|
||||
to your tailnet (e.g. `100.64.0.0/10`) and put the server's Tailscale IP / MagicDNS name into the
|
||||
code address — then the bridge connects straight over the tailnet, no SSH forward needed.
|
||||
|
||||
**Transport note:** the agent speaks **plaintext `ws://`**. The token and the diagnostic data are
|
||||
*not* encrypted on the wire by the agent itself — confidentiality comes from the **tunnel**
|
||||
(Tailscale/WireGuard/SSH) you reach it through. In network mode the IP allowlist + token are the
|
||||
access gate; **only bind to the network behind a private tunnel you trust** (a tailnet, a VPN, or a
|
||||
trusted LAN). (A future build may add `wss`/TLS with cert pinning via an `fp` field in the code; it
|
||||
is not active today.)
|
||||
|
||||
---
|
||||
|
||||
## 2. Reach the agent over a tunnel
|
||||
|
||||
The agent's default port is `9110`. Pick the path that matches your setup.
|
||||
|
||||
### Tailscale (recommended for many servers)
|
||||
|
||||
Put every server and your gateway machine on the same tailnet. On each server, set **Sichtbarkeit =
|
||||
Im Netzwerk**, allowlist your tailnet (`100.64.0.0/10`, or the specific Tailscale IPs you'll connect
|
||||
from), and set the **code address** to that server's Tailscale IP or MagicDNS name. The bridge then
|
||||
connects straight to `<tailscale-name>:9110` — WireGuard (Tailscale) encrypts the transport, and the
|
||||
allowlist + token gate access. No SSH forward, no per-session tunnel command.
|
||||
|
||||
### SSH local port-forward (keep the agent loopback-only)
|
||||
|
||||
With **Sichtbarkeit = Nur lokal**:
|
||||
|
||||
```
|
||||
ssh -L 9110:127.0.0.1:9110 user@server
|
||||
```
|
||||
|
||||
Leave that session open. Now `127.0.0.1:9110` on **your** machine is forwarded to the server's
|
||||
loopback. The code address is `127.0.0.1` (your local end of the tunnel).
|
||||
|
||||
### WireGuard (manual, alternative)
|
||||
|
||||
Bring up a WireGuard tunnel and either bind the agent to the network with the peer's WG IP in the
|
||||
allowlist, or forward loopback over the tunnel and connect to `127.0.0.1`.
|
||||
|
||||
> In **Nur lokal** mode the agent is unreachable except through a tunnel to loopback. In **Im
|
||||
> Netzwerk** mode the fail-closed IP allowlist (plus the token) is the access gate — only bind to
|
||||
> the network behind a tunnel/VPN you trust (a tailnet, WireGuard, or a trusted LAN). The transport
|
||||
> is plaintext; the tunnel is what encrypts it.
|
||||
|
||||
---
|
||||
|
||||
## 3. Install the gateway and register it with Claude (one time)
|
||||
|
||||
On the machine running Claude Code:
|
||||
|
||||
```
|
||||
cd gateway
|
||||
npm install
|
||||
```
|
||||
|
||||
Register the gateway as an MCP server (one time):
|
||||
|
||||
```
|
||||
claude mcp add --transport stdio mhu-diag -- node "C:\Users\ploet\Desktop\Claude Projekte\multi-hoster-uploader\gateway\index.js"
|
||||
```
|
||||
|
||||
Adjust the absolute path if your checkout lives elsewhere.
|
||||
|
||||
---
|
||||
|
||||
## 4. Usage
|
||||
|
||||
With the tunnel up, tell Claude:
|
||||
|
||||
```
|
||||
server prod-3 at 127.0.0.1, code mhu1_<...>
|
||||
```
|
||||
|
||||
Claude will:
|
||||
|
||||
1. call `connect_server(code:"mhu1_...", host:"127.0.0.1")`,
|
||||
2. immediately read the app version via `get_system_info`,
|
||||
3. remember the server under its label in `registry.json`.
|
||||
|
||||
Next time you can just say:
|
||||
|
||||
```
|
||||
connect_server(label:"prod-3")
|
||||
```
|
||||
|
||||
with no code — the gateway dials it from the registry. Then ask Claude "what's wrong?" — it will
|
||||
typically start with **`server_health`**, the one-shot hub that aggregates errors, queue, rotation
|
||||
and system info in a single call. Other read-only tools: `read_log`, `list_logs`, `list_errors`,
|
||||
`get_queue_state`, `get_history`, `get_config_redacted`, `get_rotation_state`, `get_app_events`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Failure → meaning
|
||||
|
||||
When a connect or a request fails, the gateway returns a human-readable cause. Quick reference:
|
||||
|
||||
| Symptom / signal | What it means / what to do |
|
||||
| -------------------------------------- | ----------------------------------------------------------------------- |
|
||||
| `ECONNREFUSED` | app not running / wrong port / inbound firewall closed |
|
||||
| `ETIMEDOUT` | firewall DROP / NAT not forwarded / tunnel down |
|
||||
| close code **4001** | auth timeout, retry |
|
||||
| close code **4002** | stale or rotated code — re-copy the current code from the server |
|
||||
| close code **4003** | brute-force lockout, wait 60s |
|
||||
| connected but no `auth-ok` | old app version without the diagnostic agent — update that server |
|
||||
| wss fingerprint mismatch | reserved for the future TLS mode (not active today) — re-copy the code |
|
||||
|
||||
If you tunnel and still get `ECONNREFUSED`, check that the SSH session is up and that the agent is
|
||||
actually listening on `127.0.0.1:9110` on the server (Diagnose-Zugriff enabled).
|
||||
|
||||
---
|
||||
|
||||
## 6. What this is — and is not
|
||||
|
||||
- It is **READ-ONLY**. It reads logs, errors, queue/history, redacted config, rotation and system
|
||||
info. **No screen capture. No input injection. No writes or config changes.**
|
||||
- The **code is a secret** (it carries the auth token). Don't paste it into public channels.
|
||||
- `gateway/registry.json` stores tokens for remembered servers and is **git-ignored** — never
|
||||
commit it.
|
||||
@ -1,208 +0,0 @@
|
||||
# V2.0.1 Account Manager und Startup Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Einen atomaren Ein-Klick-Account-Flow, einen deterministischen White-Screen-sicheren Start und eine funktionierende Update-Brücke von 3.3.108 auf die sichtbare Version 2.0.1 liefern.
|
||||
|
||||
**Architecture:** Browserunabhängige Kernlogik kapselt Validate-then-Commit und Release-Versionsauflösung und wird vom bestehenden Renderer beziehungsweise Updater verwendet. Der Electron-Main-Prozess erzwingt Software-Rendering vor Ready und zeigt das Hauptfenster erst nach `ready-to-show`.
|
||||
|
||||
**Tech Stack:** Electron 41, Node.js 24, `node:test`, electron-builder 26, Gitea Releases, GitHub Releases.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Keine Code-Kommentare, XML-Dokumentation, TODOs oder Platzhalter hinzufügen.
|
||||
- Sichtbare Produktversion, UI, Build und öffentlicher GitHub-Tag sind exakt `2.0.1` beziehungsweise `v2.0.1`.
|
||||
- Der private Gitea-Transport-Tag für diese Version ist exakt `v3.3.109`; der Release-Titel ist exakt `Multi-Hoster-Upload v2.0.1`.
|
||||
- Fehlgeschlagene, abgebrochene, veraltete oder OTP-erfordernde Prüfungen persistieren keinen Account.
|
||||
- Release nur aus frischer Positivliste, ohne interne KI-/Task-/Log-/Backup-/Testdaten und nach Quell-, Build-, Archiv- und Secret-Prüfung.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Startup-Renderer absichern
|
||||
|
||||
**Files:**
|
||||
- Create: `lib/startup-renderer.js`
|
||||
- Modify: `main.js`
|
||||
- Create: `tests/startup-renderer.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `configureStartupRenderer(app)` deaktiviert Hardwarebeschleunigung genau einmal vor Ready.
|
||||
- Produces: `createWindow()` erstellt das Hauptfenster unsichtbar und zeigt es auf `ready-to-show`.
|
||||
|
||||
- [ ] **Step 1: Failing Tests schreiben**
|
||||
|
||||
```js
|
||||
test('configureStartupRenderer disables hardware acceleration', () => {
|
||||
let calls = 0;
|
||||
configureStartupRenderer({ disableHardwareAcceleration() { calls++; } });
|
||||
assert.equal(calls, 1);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: RED verifizieren**
|
||||
|
||||
Run: `node --test tests/startup-renderer.test.js`
|
||||
|
||||
Expected: FAIL, weil `lib/startup-renderer.js` noch fehlt.
|
||||
|
||||
- [ ] **Step 3: Minimale Implementierung schreiben**
|
||||
|
||||
```js
|
||||
function configureStartupRenderer(app) {
|
||||
app.disableHardwareAcceleration();
|
||||
}
|
||||
|
||||
module.exports = { configureStartupRenderer };
|
||||
```
|
||||
|
||||
`main.js` ruft die Funktion unmittelbar nach dem Electron-Import auf, entfernt das heuristische RDP-/Flag-Gate, setzt `show: false`, registriert `ready-to-show` vor `loadFile` und behandelt einen abgelehnten Load.
|
||||
|
||||
- [ ] **Step 4: GREEN und Startup-Smoke verifizieren**
|
||||
|
||||
Run: `node --test tests/startup-renderer.test.js`
|
||||
|
||||
Run: `$env:RUN_UI_SMOKE='1'; node tests/ui-smoke.js`
|
||||
|
||||
- [ ] **Step 5: Commit erstellen**
|
||||
|
||||
```powershell
|
||||
git add lib/startup-renderer.js main.js tests/startup-renderer.test.js
|
||||
git commit -m "fix(startup): make renderer initialization deterministic"
|
||||
```
|
||||
|
||||
### Task 2: Account in einem Lauf prüfen und anlegen
|
||||
|
||||
**Files:**
|
||||
- Create: `renderer/account-submit.js`
|
||||
- Modify: `renderer/index.html`
|
||||
- Modify: `renderer/app.js`
|
||||
- Replace: `tests/validate-credentials.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `submitValidatedAccount({ validate, commit, isCurrent })` mit den Resultaten `committed`, `rejected`, `otp_required`, `stale` und `error`.
|
||||
- Consumes: bestehendes `window.api.validateCredentials` und `window.api.saveConfig`.
|
||||
|
||||
- [ ] **Step 1: Failing Tests für Ein-Klick- und Fehlerpfade schreiben**
|
||||
|
||||
```js
|
||||
test('ok validates and commits exactly once in one submission', async () => {
|
||||
let commits = 0;
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => ({ status: 'ok' }),
|
||||
commit: async () => { commits++; },
|
||||
isCurrent: () => true
|
||||
});
|
||||
assert.equal(result.status, 'committed');
|
||||
assert.equal(commits, 1);
|
||||
});
|
||||
```
|
||||
|
||||
Zusätzliche Tests decken `warn`, `error`, `skipped`, Throw, OTP, stale vor Commit und Save-Fehler ab.
|
||||
|
||||
- [ ] **Step 2: RED verifizieren**
|
||||
|
||||
Run: `node --test tests/validate-credentials.test.js`
|
||||
|
||||
Expected: FAIL, weil `submitValidatedAccount` noch fehlt.
|
||||
|
||||
- [ ] **Step 3: Submit-Core und Renderer-Integration implementieren**
|
||||
|
||||
Der Submit-Core validiert, prüft `isCurrent`, committet ausschließlich `ok|warn` und gibt strukturierte Ergebnisse zurück. `saveAccount()` hält Busy bis zum Ende, vergleicht Session und Credential-Snapshot, speichert einen kopierten Kandidaten und übernimmt ihn erst nach erfolgreichem IPC. Der Button bleibt bis zum Schließen deaktiviert.
|
||||
|
||||
- [ ] **Step 4: GREEN und UI-Vertrag verifizieren**
|
||||
|
||||
Run: `node --test tests/validate-credentials.test.js`
|
||||
|
||||
Run: `node --test tests/*.test.js`
|
||||
|
||||
Run: `$env:RUN_UI_SMOKE='1'; node tests/ui-smoke.js`
|
||||
|
||||
- [ ] **Step 5: Commit erstellen**
|
||||
|
||||
```powershell
|
||||
git add renderer/account-submit.js renderer/index.html renderer/app.js tests/validate-credentials.test.js
|
||||
git commit -m "feat(accounts): validate and save in one action"
|
||||
```
|
||||
|
||||
### Task 3: Produktversion und Updater-Brücke implementieren
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/updater.js`
|
||||
- Modify: `scripts/release_gitea.mjs`
|
||||
- Modify: `package.json`
|
||||
- Modify: `package-lock.json`
|
||||
- Create: `tests/updater-version.test.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `resolveReleaseVersion(release)` liest zuerst eine semantische Version aus `release.name`, dann aus `tag_name`.
|
||||
- Produces: Release-CLI `node scripts/release_gitea.mjs 2.0.1 --transport-tag v3.3.109 <notes>`.
|
||||
|
||||
- [ ] **Step 1: Failing Tests für Release-Auflösung und Bridge schreiben**
|
||||
|
||||
```js
|
||||
test('bridge title resolves product version instead of transport tag', () => {
|
||||
assert.equal(resolveReleaseVersion({ name: 'Multi-Hoster-Upload v2.0.1', tag_name: 'v3.3.109' }), '2.0.1');
|
||||
assert.equal(isNewer('2.0.1', '2.0.1'), false);
|
||||
assert.equal(isNewer('2.0.2', '2.0.1'), true);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: RED verifizieren**
|
||||
|
||||
Run: `node --test tests/updater-version.test.js`
|
||||
|
||||
Expected: FAIL, weil `resolveReleaseVersion` noch fehlt.
|
||||
|
||||
- [ ] **Step 3: Produkt-/Transport-Trennung implementieren**
|
||||
|
||||
`checkForUpdate()` verwendet die aufgelöste Produktversion für UI und Vergleich. Der Transport-Tag bleibt im Ergebnis diagnostizierbar. Das Release-Skript validiert `--transport-tag`, baut `2.0.1`, taggt `v3.3.109`, benennt den privaten Release `Multi-Hoster-Upload v2.0.1` und erzeugt `latest.yml` mit `2.0.1`.
|
||||
|
||||
- [ ] **Step 4: Version auf 2.0.1 setzen und GREEN verifizieren**
|
||||
|
||||
Run: `npm version 2.0.1 --no-git-tag-version`
|
||||
|
||||
Run: `node --test tests/updater-version.test.js`
|
||||
|
||||
Run: `npm test`
|
||||
|
||||
- [ ] **Step 5: Commit erstellen**
|
||||
|
||||
```powershell
|
||||
git add lib/updater.js scripts/release_gitea.mjs package.json package-lock.json tests/updater-version.test.js
|
||||
git commit -m "feat(updater): bridge 3.3.108 clients to v2.0.1"
|
||||
```
|
||||
|
||||
### Task 4: Build, Laufzeit und Veröffentlichung verifizieren
|
||||
|
||||
**Files:**
|
||||
- Modify: `tasks/todo.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Tasks 1 bis 3.
|
||||
- Produces: geprüfte private Bridge und kuratierten öffentlichen Release v2.0.1.
|
||||
|
||||
- [ ] **Step 1: Vollständige lokale Gates ausführen**
|
||||
|
||||
Run: `npm test`
|
||||
|
||||
Run: `npx eslint .`
|
||||
|
||||
Run: `npm audit --omit=dev --json`
|
||||
|
||||
Run: `npm run release:win`
|
||||
|
||||
- [ ] **Step 2: Build und Archive prüfen**
|
||||
|
||||
Portable, Setup, Blockmap und `latest.yml` werden vollständig entpackt, auf Version, Struktur, Hashes, private Endpunkte, Credentials und interne Dateien geprüft. Gitleaks und TruffleHog laufen über Quell-Positivliste und entpackte Artefakte.
|
||||
|
||||
- [ ] **Step 3: Update-Brücke isoliert testen**
|
||||
|
||||
Ein unveränderter 3.3.108-Updater muss den privaten Transport-Tag `v3.3.109` als neuer erkennen. Der gebaute 2.0.1-Updater muss denselben Release-Titel als installiert erkennen. Setup und Portable werden separat gestartet; UI, Account-Modal und sauberer Teardown werden geprüft.
|
||||
|
||||
- [ ] **Step 4: Private und öffentliche Releases veröffentlichen**
|
||||
|
||||
Der private Gitea-Release nutzt Transport-Tag `v3.3.109` und Titel `Multi-Hoster-Upload v2.0.1`. Der öffentliche GitHub-Release wird aus einem neuen Positivlisten-Ordner als `v2.0.1` mit ausschließlich benötigtem Source und vier Release-Artefakten erstellt.
|
||||
|
||||
- [ ] **Step 5: Veröffentlichte Downloads erneut verifizieren**
|
||||
|
||||
Alle öffentlichen und privaten Assets werden neu heruntergeladen, byte- und hashverglichen, `latest.yml` wird gegen das Setup geprüft und der veröffentlichte Build wird erneut gestartet. Erst danach wird `tasks/todo.md` mit den exakten Belegen aktualisiert.
|
||||
@ -1,25 +0,0 @@
|
||||
# V2.0.1 Account Manager und Startup
|
||||
|
||||
## Ziel
|
||||
|
||||
Multi-Hoster-Upload wird als sichtbare Produktversion `2.0.1` ausgeliefert. Das Hinzufügen und Bearbeiten eines Accounts validiert und speichert in einem Klick. Ein fehlgeschlagener, abgebrochener oder veralteter Check verändert weder die persistierte noch die im Renderer gehaltene Account-Konfiguration. Der intermittierende weiße Startzustand wird durch einen deterministischen Software-Renderer und ein erst nach erfolgreichem Laden sichtbares Hauptfenster verhindert.
|
||||
|
||||
## Account-Flow
|
||||
|
||||
Der Button lautet im Anlegefall `Prüfen und anlegen`, im Bearbeitungsfall `Prüfen und speichern`. Der bestehende Credential-Check bleibt die einzige fachliche Validierung. `ok` und `warn` führen unmittelbar zum Save, `error`, `skipped`, IPC-Fehler und `otp_required` nicht. OTP zeigt das Eingabefeld; der nächste Klick validiert und speichert in einem Lauf.
|
||||
|
||||
Ein einziger Busy-Zustand umfasst Check und Save. Vor dem Save werden Modal-Session, Hoster, Account-ID und Credential-Snapshot erneut abgeglichen. Hoster- oder Credential-Änderung, Schließen oder erneutes Öffnen invalidieren die laufende Operation. Der Kandidat wird auf einer Kopie der Hoster-Konfiguration aufgebaut. Erst nach erfolgreichem `saveConfig` ersetzt er den Renderer-State. Bei Save-Fehler bleibt das Modal offen und der bisherige State unverändert.
|
||||
|
||||
## Startup
|
||||
|
||||
Die App benötigt keine GPU-beschleunigten Canvas-, WebGL- oder Video-Oberflächen. Hardwarebeschleunigung wird deshalb bei jedem Start vor `app.whenReady()` deaktiviert, unabhängig von `SESSIONNAME` und einem späteren GPU-Crash. Das Hauptfenster startet mit `show: false`, wird auf `ready-to-show` eingeblendet und wertet das Promise von `loadFile` aus. Load- und Renderer-Fehler bleiben sichtbar diagnostizierbar.
|
||||
|
||||
## Versions- und Update-Brücke
|
||||
|
||||
`package.json`, UI, Installer-Metadaten und öffentliche GitHub-Version lauten `2.0.1` beziehungsweise `v2.0.1`. Der ausgelieferte 3.3.108-Client vergleicht den privaten Gitea-Tag numerisch und würde einen Tag `v2.0.1` ablehnen. Außerdem existiert dort bereits ein historischer Tag `v2.0.1`.
|
||||
|
||||
Der private Updater-Transport verwendet deshalb für diese Veröffentlichung den neuen internen Tag `v3.3.109`, während Release-Titel und Build `v2.0.1` anzeigen. Der neue Updater liest die Produktversion aus dem Release-Titel und fällt nur bei fehlender Produktversion auf den Tag zurück. Dadurch sieht 3.3.108 den höheren Transport-Tag, während 2.0.1 denselben Release anhand des Titels als bereits installiert erkennt. Künftige 2.x-Releases verwenden fortlaufende interne Transport-Tags oberhalb 3.3.108 und sichtbare 2.x-Titel.
|
||||
|
||||
## Verifikation
|
||||
|
||||
Regressionsfälle prüfen den real verwendeten Account-Submit-Core, die Startup-Konfiguration und die Produkt-/Transport-Versionsauflösung. Danach folgen vollständige Unit-Tests, UI-Smoke, wiederholte reale Starts, Windows-Build, entpackte Artefaktprüfung, Secret-Scans, isolierter Installer-/Updater-Test und erst anschließend die kuratierten privaten und öffentlichen Releases mit erneutem Download und Hashvergleich.
|
||||
@ -1,68 +1,9 @@
|
||||
import security from 'eslint-plugin-security';
|
||||
|
||||
const sharedRules = {
|
||||
// Security rules
|
||||
// detect-object-injection disabled: 78 false positives from config lookups like obj[hosterName]
|
||||
'security/detect-object-injection': 'off',
|
||||
'security/detect-non-literal-regexp': 'warn',
|
||||
'security/detect-unsafe-regex': 'warn',
|
||||
'security/detect-buffer-noassert': 'warn',
|
||||
'security/detect-eval-with-expression': 'error',
|
||||
'security/detect-no-csrf-before-method-override': 'warn',
|
||||
'security/detect-possible-timing-attacks': 'warn',
|
||||
'security/detect-pseudoRandomBytes': 'warn',
|
||||
// Code quality
|
||||
'no-unused-vars': ['warn', { argsIgnorePattern: '^_', varsIgnorePattern: '^_' }],
|
||||
'no-undef': 'error',
|
||||
'no-constant-condition': 'warn',
|
||||
'no-debugger': 'error',
|
||||
'no-duplicate-case': 'error',
|
||||
'no-empty': ['warn', { allowEmptyCatch: true }],
|
||||
'no-ex-assign': 'error',
|
||||
'no-extra-boolean-cast': 'warn',
|
||||
'no-func-assign': 'error',
|
||||
'no-inner-declarations': 'error',
|
||||
'no-irregular-whitespace': 'error',
|
||||
'no-unreachable': 'error',
|
||||
'use-isnan': 'error',
|
||||
'valid-typeof': 'error',
|
||||
'eqeqeq': ['warn', 'always'],
|
||||
'no-caller': 'error',
|
||||
'no-eval': 'error',
|
||||
'no-implied-eval': 'error',
|
||||
'no-new-func': 'error',
|
||||
'no-throw-literal': 'warn',
|
||||
'no-self-assign': 'error',
|
||||
'no-self-compare': 'error',
|
||||
'no-loss-of-precision': 'error',
|
||||
'no-dupe-keys': 'error',
|
||||
'no-unsafe-finally': 'error',
|
||||
'no-unmodified-loop-condition': 'warn',
|
||||
'no-template-curly-in-string': 'warn',
|
||||
};
|
||||
|
||||
const nodeGlobals = {
|
||||
process: 'readonly',
|
||||
console: 'readonly',
|
||||
setTimeout: 'readonly',
|
||||
clearTimeout: 'readonly',
|
||||
setInterval: 'readonly',
|
||||
clearInterval: 'readonly',
|
||||
setImmediate: 'readonly',
|
||||
Buffer: 'readonly',
|
||||
URL: 'readonly',
|
||||
URLSearchParams: 'readonly',
|
||||
fetch: 'readonly',
|
||||
crypto: 'readonly',
|
||||
structuredClone: 'readonly',
|
||||
performance: 'readonly',
|
||||
};
|
||||
|
||||
export default [
|
||||
{ ignores: ['**/node_modules/**', 'release/**', 'tests/**'] },
|
||||
{
|
||||
files: ['**/*.js'],
|
||||
ignores: ['gateway/**'],
|
||||
ignores: ['node_modules/**', 'release/**', 'tests/**'],
|
||||
plugins: { security },
|
||||
languageOptions: {
|
||||
ecmaVersion: 2022,
|
||||
@ -73,7 +14,16 @@ export default [
|
||||
exports: 'readonly',
|
||||
__dirname: 'readonly',
|
||||
__filename: 'readonly',
|
||||
...nodeGlobals,
|
||||
process: 'readonly',
|
||||
console: 'readonly',
|
||||
setTimeout: 'readonly',
|
||||
clearTimeout: 'readonly',
|
||||
setInterval: 'readonly',
|
||||
clearInterval: 'readonly',
|
||||
setImmediate: 'readonly',
|
||||
Buffer: 'readonly',
|
||||
URL: 'readonly',
|
||||
fetch: 'readonly',
|
||||
AbortController: 'readonly',
|
||||
AbortSignal: 'readonly',
|
||||
navigator: 'readonly',
|
||||
@ -86,20 +36,50 @@ export default [
|
||||
requestAnimationFrame: 'readonly',
|
||||
queueMicrotask: 'readonly',
|
||||
Intl: 'readonly',
|
||||
crypto: 'readonly',
|
||||
URLSearchParams: 'readonly',
|
||||
EventSource: 'readonly',
|
||||
}
|
||||
},
|
||||
rules: sharedRules
|
||||
},
|
||||
{
|
||||
files: ['gateway/**/*.js', 'gateway/**/*.mjs'],
|
||||
ignores: ['gateway/node_modules/**'],
|
||||
plugins: { security },
|
||||
languageOptions: {
|
||||
ecmaVersion: 2022,
|
||||
sourceType: 'module',
|
||||
globals: nodeGlobals
|
||||
},
|
||||
rules: sharedRules
|
||||
rules: {
|
||||
// Security rules
|
||||
// detect-object-injection disabled: 78 false positives from config lookups like obj[hosterName]
|
||||
'security/detect-object-injection': 'off',
|
||||
'security/detect-non-literal-regexp': 'warn',
|
||||
'security/detect-unsafe-regex': 'warn',
|
||||
'security/detect-buffer-noassert': 'warn',
|
||||
'security/detect-eval-with-expression': 'error',
|
||||
'security/detect-no-csrf-before-method-override': 'warn',
|
||||
'security/detect-possible-timing-attacks': 'warn',
|
||||
'security/detect-pseudoRandomBytes': 'warn',
|
||||
// Code quality
|
||||
'no-unused-vars': ['warn', { argsIgnorePattern: '^_', varsIgnorePattern: '^_' }],
|
||||
'no-undef': 'error',
|
||||
'no-constant-condition': 'warn',
|
||||
'no-debugger': 'error',
|
||||
'no-duplicate-case': 'error',
|
||||
'no-empty': ['warn', { allowEmptyCatch: true }],
|
||||
'no-ex-assign': 'error',
|
||||
'no-extra-boolean-cast': 'warn',
|
||||
'no-func-assign': 'error',
|
||||
'no-inner-declarations': 'error',
|
||||
'no-irregular-whitespace': 'error',
|
||||
'no-unreachable': 'error',
|
||||
'use-isnan': 'error',
|
||||
'valid-typeof': 'error',
|
||||
'eqeqeq': ['warn', 'always'],
|
||||
'no-caller': 'error',
|
||||
'no-eval': 'error',
|
||||
'no-implied-eval': 'error',
|
||||
'no-new-func': 'error',
|
||||
'no-throw-literal': 'warn',
|
||||
'no-self-assign': 'error',
|
||||
'no-self-compare': 'error',
|
||||
'no-loss-of-precision': 'error',
|
||||
'no-dupe-keys': 'error',
|
||||
'no-unsafe-finally': 'error',
|
||||
'no-unmodified-loop-condition': 'warn',
|
||||
'no-template-curly-in-string': 'warn',
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
2
gateway/.gitignore
vendored
2
gateway/.gitignore
vendored
@ -1,2 +0,0 @@
|
||||
node_modules/
|
||||
registry.json
|
||||
@ -1,56 +0,0 @@
|
||||
# mhu-diagnostics-gateway
|
||||
|
||||
A standalone local **stdio MCP gateway** for remote, **read-only** diagnostics of the
|
||||
Multi-Hoster-Uploader app.
|
||||
|
||||
It is two things at once:
|
||||
|
||||
- an **MCP server** to Claude Code (stdio transport), exposing read-only diagnostic tools, and
|
||||
- a plain **WebSocket client** to a diagnostic agent running inside the Electron app on a
|
||||
remote Windows server.
|
||||
|
||||
The operator enables "Diagnose-Zugriff" on a server, copies the connection **code**, and tells
|
||||
Claude `server <name> at <host>, code <CODE>`. Claude calls `connect_server(code, host)` and then
|
||||
the read-only diagnostic tools. After the first successful connect the server is remembered under
|
||||
its label, so later you can just say `connect_server(label:"prod-3")` with no code.
|
||||
|
||||
This package is fully self-contained. It does **not** import anything from the parent Electron app
|
||||
and is **not** part of the app build.
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
cd gateway
|
||||
npm install
|
||||
```
|
||||
|
||||
Requires Node >= 18.
|
||||
|
||||
## Register with Claude Code (one time)
|
||||
|
||||
```
|
||||
claude mcp add --transport stdio mhu-diag -- node "C:\Users\ploet\Desktop\Claude Projekte\multi-hoster-uploader\gateway\index.js"
|
||||
```
|
||||
|
||||
Adjust the absolute path if you cloned the repo elsewhere.
|
||||
|
||||
## Usage
|
||||
|
||||
In Claude Code, tell Claude:
|
||||
|
||||
```
|
||||
server prod-3 at 127.0.0.1, code mhu1_<...>
|
||||
```
|
||||
|
||||
Claude will call `connect_server` and then diagnostic tools such as `server_health`
|
||||
(the one-shot "what's wrong" hub), `read_log`, `list_errors`, `get_queue_state`,
|
||||
`get_rotation_state`, and so on.
|
||||
|
||||
## Security
|
||||
|
||||
- **Read-only.** No screen access, no input injection, no writes. Only reads logs, errors,
|
||||
queue/history/config (redacted), rotation and system info.
|
||||
- The **code is a secret** — it carries the auth token. Do not paste it anywhere public.
|
||||
- The safe default is to reach the agent over `127.0.0.1` via an SSH local port-forward or
|
||||
WireGuard. See `docs/remote-diagnostics-setup.md`.
|
||||
- `registry.json` stores tokens and is **git-ignored** — never commit it.
|
||||
@ -1,219 +0,0 @@
|
||||
import WebSocket from 'ws';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
|
||||
const AUTH_TIMEOUT_MS = 6000;
|
||||
const REQUEST_TIMEOUT_MS = 30000;
|
||||
|
||||
const CLOSE_CODE_GUIDANCE = {
|
||||
4001: 'auth timeout, retry',
|
||||
4002: 'stale or rotated code — re-copy the current code from the server',
|
||||
4003: 'brute-force lockout, wait 60s',
|
||||
};
|
||||
|
||||
function normalizeFingerprint(fp) {
|
||||
if (typeof fp !== 'string') return '';
|
||||
return fp.replace(/:/g, '').toLowerCase();
|
||||
}
|
||||
|
||||
function mapSocketError(err) {
|
||||
const code = err && err.code;
|
||||
if (code === 'ECONNREFUSED') {
|
||||
return 'app not running / wrong port / inbound firewall closed';
|
||||
}
|
||||
if (code === 'ETIMEDOUT') {
|
||||
return 'firewall DROP / NAT not forwarded / tunnel down';
|
||||
}
|
||||
return (err && err.message) ? err.message : String(err);
|
||||
}
|
||||
|
||||
function mapCloseBeforeAuth(code, sawOpen) {
|
||||
if (CLOSE_CODE_GUIDANCE[code]) return CLOSE_CODE_GUIDANCE[code];
|
||||
if (sawOpen) {
|
||||
return 'old app version without the diagnostic agent — update that server';
|
||||
}
|
||||
return `connection closed before auth (code ${code})`;
|
||||
}
|
||||
|
||||
export class AgentClient {
|
||||
constructor({ host, port, token, fp }) {
|
||||
this.host = host;
|
||||
this.port = port;
|
||||
this.token = token;
|
||||
this.fp = fp;
|
||||
this.connected = false;
|
||||
this.clientId = null;
|
||||
this.ws = null;
|
||||
this.authed = false;
|
||||
this.sawOpen = false;
|
||||
this._pending = new Map();
|
||||
this._authResolve = null;
|
||||
this._authReject = null;
|
||||
this._authTimer = null;
|
||||
}
|
||||
|
||||
connect() {
|
||||
return new Promise((resolve, reject) => {
|
||||
this._authResolve = resolve;
|
||||
this._authReject = reject;
|
||||
|
||||
const secure = typeof this.fp === 'string' && this.fp.length > 0;
|
||||
const scheme = secure ? 'wss' : 'ws';
|
||||
const url = `${scheme}://${this.host}:${this.port}`;
|
||||
|
||||
const pinned = secure ? normalizeFingerprint(this.fp) : '';
|
||||
const options = secure ? { rejectUnauthorized: false } : undefined;
|
||||
|
||||
let ws;
|
||||
try {
|
||||
ws = options ? new WebSocket(url, options) : new WebSocket(url);
|
||||
} catch (err) {
|
||||
this._failAuth(mapSocketError(err));
|
||||
return;
|
||||
}
|
||||
this.ws = ws;
|
||||
|
||||
this._authTimer = setTimeout(() => {
|
||||
this._failAuth('auth timeout, retry');
|
||||
try { ws.close(); } catch {}
|
||||
}, AUTH_TIMEOUT_MS);
|
||||
|
||||
ws.on('open', () => {
|
||||
this.sawOpen = true;
|
||||
if (secure) {
|
||||
const sock = ws._socket;
|
||||
const cert = sock && typeof sock.getPeerCertificate === 'function'
|
||||
? sock.getPeerCertificate()
|
||||
: null;
|
||||
const actual = normalizeFingerprint(cert && cert.fingerprint256);
|
||||
if (!actual || actual !== pinned) {
|
||||
this._failAuth('server cert changed (reinstalled?) — re-copy the code');
|
||||
try { ws.close(); } catch {}
|
||||
return;
|
||||
}
|
||||
}
|
||||
this._send({ type: 'auth', token: this.token, role: 'diagnostic' });
|
||||
});
|
||||
|
||||
ws.on('message', (raw) => this._onMessage(raw));
|
||||
|
||||
ws.on('error', (err) => {
|
||||
if (!this.authed) {
|
||||
this._failAuth(mapSocketError(err));
|
||||
} else {
|
||||
this._rejectAllPending(mapSocketError(err));
|
||||
}
|
||||
});
|
||||
|
||||
ws.on('close', (code) => {
|
||||
this.connected = false;
|
||||
if (!this.authed) {
|
||||
this._failAuth(mapCloseBeforeAuth(code, this.sawOpen));
|
||||
} else {
|
||||
this._rejectAllPending(`connection closed (code ${code})`);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
_onMessage(raw) {
|
||||
let msg;
|
||||
try {
|
||||
msg = JSON.parse(raw.toString());
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'auth-ok') {
|
||||
this.authed = true;
|
||||
this.connected = true;
|
||||
this.clientId = msg.clientId ?? null;
|
||||
if (this._authTimer) {
|
||||
clearTimeout(this._authTimer);
|
||||
this._authTimer = null;
|
||||
}
|
||||
if (this._authResolve) {
|
||||
const r = this._authResolve;
|
||||
this._authResolve = null;
|
||||
this._authReject = null;
|
||||
r({ clientId: this.clientId });
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'diag-response' && msg.reqId) {
|
||||
const entry = this._pending.get(msg.reqId);
|
||||
if (!entry) return;
|
||||
this._pending.delete(msg.reqId);
|
||||
clearTimeout(entry.timer);
|
||||
if (msg.ok) {
|
||||
entry.resolve({ ok: true, data: msg.data });
|
||||
} else {
|
||||
entry.resolve({ ok: false, error: msg.error ?? 'unknown agent error' });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
request(op, args) {
|
||||
return new Promise((resolve) => {
|
||||
if (!this.connected || !this.ws || this.ws.readyState !== WebSocket.OPEN) {
|
||||
resolve({ ok: false, error: 'not connected to a diagnostic agent' });
|
||||
return;
|
||||
}
|
||||
const reqId = randomUUID();
|
||||
const timer = setTimeout(() => {
|
||||
if (this._pending.has(reqId)) {
|
||||
this._pending.delete(reqId);
|
||||
resolve({ ok: false, error: `request timed out after ${REQUEST_TIMEOUT_MS}ms (op: ${op})` });
|
||||
}
|
||||
}, REQUEST_TIMEOUT_MS);
|
||||
this._pending.set(reqId, { resolve, timer });
|
||||
try {
|
||||
this._send({ type: 'diag-request', reqId, op, args: args ?? {} });
|
||||
} catch (err) {
|
||||
this._pending.delete(reqId);
|
||||
clearTimeout(timer);
|
||||
resolve({ ok: false, error: mapSocketError(err) });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
close() {
|
||||
this.connected = false;
|
||||
this.authed = false;
|
||||
if (this._authTimer) {
|
||||
clearTimeout(this._authTimer);
|
||||
this._authTimer = null;
|
||||
}
|
||||
this._rejectAllPending('connection closed by client');
|
||||
if (this.ws) {
|
||||
try { this.ws.removeAllListeners(); } catch {}
|
||||
try { this.ws.close(); } catch {}
|
||||
this.ws = null;
|
||||
}
|
||||
}
|
||||
|
||||
_send(obj) {
|
||||
this.ws.send(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
_failAuth(error) {
|
||||
if (this._authTimer) {
|
||||
clearTimeout(this._authTimer);
|
||||
this._authTimer = null;
|
||||
}
|
||||
if (this._authReject) {
|
||||
const rej = this._authReject;
|
||||
this._authResolve = null;
|
||||
this._authReject = null;
|
||||
rej(new Error(error));
|
||||
}
|
||||
}
|
||||
|
||||
_rejectAllPending(error) {
|
||||
for (const [reqId, entry] of this._pending) {
|
||||
clearTimeout(entry.timer);
|
||||
entry.resolve({ ok: false, error });
|
||||
this._pending.delete(reqId);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,67 +0,0 @@
|
||||
const PREFIX = 'mhu1_';
|
||||
|
||||
export function encode(payload) {
|
||||
if (!payload || typeof payload !== 'object') {
|
||||
throw new Error('encode: payload must be an object');
|
||||
}
|
||||
const json = JSON.stringify(payload);
|
||||
const b64 = Buffer.from(json, 'utf8').toString('base64url');
|
||||
return PREFIX + b64;
|
||||
}
|
||||
|
||||
export function decode(code) {
|
||||
if (typeof code !== 'string') {
|
||||
throw new Error('Invalid code: expected a string');
|
||||
}
|
||||
const trimmed = code.trim();
|
||||
if (!trimmed.startsWith(PREFIX)) {
|
||||
throw new Error('Invalid code: missing "mhu1_" prefix');
|
||||
}
|
||||
const b64 = trimmed.slice(PREFIX.length);
|
||||
if (!b64) {
|
||||
throw new Error('Invalid code: empty payload');
|
||||
}
|
||||
|
||||
let json;
|
||||
try {
|
||||
json = Buffer.from(b64, 'base64url').toString('utf8');
|
||||
} catch {
|
||||
throw new Error('Invalid code: not valid base64url');
|
||||
}
|
||||
|
||||
let payload;
|
||||
try {
|
||||
payload = JSON.parse(json);
|
||||
} catch {
|
||||
throw new Error('Invalid code: payload is not valid JSON');
|
||||
}
|
||||
|
||||
if (!payload || typeof payload !== 'object') {
|
||||
throw new Error('Invalid code: payload is not an object');
|
||||
}
|
||||
if (payload.v !== 1) {
|
||||
throw new Error(`Invalid code: unsupported version (expected v=1, got ${payload.v})`);
|
||||
}
|
||||
const host = payload.h !== undefined ? payload.h : payload.host;
|
||||
const port = payload.p !== undefined ? payload.p : payload.port;
|
||||
const token = payload.t !== undefined ? payload.t : payload.token;
|
||||
const label = payload.n !== undefined ? payload.n : payload.label;
|
||||
const scheme = payload.s === 'wss' ? 'wss' : 'ws';
|
||||
if (host !== undefined && typeof host !== 'string') {
|
||||
throw new Error('Invalid code: "host" must be a string when present');
|
||||
}
|
||||
if (typeof port !== 'number' || !Number.isFinite(port)) {
|
||||
throw new Error('Invalid code: "port" must be a number');
|
||||
}
|
||||
if (typeof token !== 'string' || token.length === 0) {
|
||||
throw new Error('Invalid code: "token" must be a non-empty string');
|
||||
}
|
||||
if (label !== undefined && typeof label !== 'string') {
|
||||
throw new Error('Invalid code: "label" must be a string');
|
||||
}
|
||||
if (payload.fp !== undefined && typeof payload.fp !== 'string') {
|
||||
throw new Error('Invalid code: "fp" must be a string when present');
|
||||
}
|
||||
|
||||
return { v: 1, host: host ? String(host) : undefined, port, token, label: label !== undefined ? String(label) : undefined, fp: payload.fp, scheme };
|
||||
}
|
||||
296
gateway/index.js
296
gateway/index.js
@ -1,296 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { z } from 'zod';
|
||||
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
||||
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
||||
import { decode } from './code.js';
|
||||
import { AgentClient } from './agent-client.js';
|
||||
import { loadRegistry, upsertEntry } from './registry.js';
|
||||
|
||||
const state = {
|
||||
current: null,
|
||||
servers: new Map(),
|
||||
};
|
||||
|
||||
function result(obj) {
|
||||
return { content: [{ type: 'text', text: JSON.stringify(obj, null, 2) }] };
|
||||
}
|
||||
|
||||
function guard(handler) {
|
||||
return async (args) => {
|
||||
try {
|
||||
return result(await handler(args ?? {}));
|
||||
} catch (e) {
|
||||
return result({ ok: false, error: String(e && e.message ? e.message : e) });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async function requireCurrent(op, args) {
|
||||
if (!state.current || !state.current.client || !state.current.client.connected) {
|
||||
return { ok: false, error: 'no server connected — call connect_server first' };
|
||||
}
|
||||
return state.current.client.request(op, args ?? {});
|
||||
}
|
||||
|
||||
const DIAGNOSTIC_TOOLS = [
|
||||
{
|
||||
name: 'server_health',
|
||||
title: 'Server health (one-shot hub)',
|
||||
description:
|
||||
'THE one-shot diagnostic hub: answers "what is wrong" in a single call by aggregating recent errors, queue state, rotation and system info.',
|
||||
op: 'server_health',
|
||||
inputSchema: {
|
||||
errorLimit: z.number().optional(),
|
||||
errorSinceMs: z.number().optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'read_log',
|
||||
title: 'Read a log file',
|
||||
description: 'Read a tail of one of the app log files, optionally a rotated backup. grep is a case-insensitive substring filter; separate alternatives with "|" (e.g. "error|timeout|502") to keep any line matching at least one term. Not a regular expression.',
|
||||
op: 'read_log',
|
||||
inputSchema: {
|
||||
name: z.enum(['debug', 'fileuploader', 'accountRotation', 'crash']),
|
||||
tailKb: z.number().optional(),
|
||||
grep: z.string().optional(),
|
||||
backup: z.number().optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'list_logs',
|
||||
title: 'List available logs',
|
||||
description: 'List the available log files and their sizes.',
|
||||
op: 'list_logs',
|
||||
inputSchema: {},
|
||||
},
|
||||
{
|
||||
name: 'list_errors',
|
||||
title: 'List recent errors',
|
||||
description: 'List recent structured errors, filterable by time window, category and hoster.',
|
||||
op: 'list_errors',
|
||||
inputSchema: {
|
||||
sinceMs: z.number().optional(),
|
||||
category: z.string().optional(),
|
||||
hoster: z.string().optional(),
|
||||
limit: z.number().optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_queue_state',
|
||||
title: 'Get queue state',
|
||||
description: 'Get the upload queue state, optionally including individual jobs.',
|
||||
op: 'get_queue_state',
|
||||
inputSchema: {
|
||||
includeJobs: z.boolean().optional(),
|
||||
maxJobs: z.number().optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_history',
|
||||
title: 'Get upload history',
|
||||
description: 'Get recent upload history, optionally including file names and URLs.',
|
||||
op: 'get_history',
|
||||
inputSchema: {
|
||||
limit: z.number().optional(),
|
||||
includeFiles: z.boolean().optional(),
|
||||
includeUrls: z.boolean().optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_config_redacted',
|
||||
title: 'Get redacted config',
|
||||
description: 'Get the app configuration with secrets redacted. Choose a section to narrow the output.',
|
||||
op: 'get_config_redacted',
|
||||
inputSchema: {
|
||||
section: z.enum(['all', 'hosters', 'hosterSettings', 'globalSettings', 'rotationCursors']).optional(),
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_system_info',
|
||||
title: 'Get system info',
|
||||
description: 'Get system and app version info (OS, app version, uptime).',
|
||||
op: 'get_system_info',
|
||||
inputSchema: {},
|
||||
},
|
||||
{
|
||||
name: 'get_rotation_state',
|
||||
title: 'Get rotation state',
|
||||
description: 'Get the per-hoster account rotation state and round-robin cursors.',
|
||||
op: 'get_rotation_state',
|
||||
inputSchema: {},
|
||||
},
|
||||
{
|
||||
name: 'get_app_events',
|
||||
title: 'Get app events',
|
||||
description: 'Get recent in-app lifecycle events.',
|
||||
op: 'get_app_events',
|
||||
inputSchema: {
|
||||
limit: z.number().optional(),
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
async function doConnect({ code, host, port, label }) {
|
||||
const registry = await loadRegistry();
|
||||
|
||||
let target;
|
||||
if (label && registry[label]) {
|
||||
const e = registry[label];
|
||||
target = { host: e.host, port: e.port, token: e.token, fp: e.fp, label: e.label };
|
||||
} else {
|
||||
if (!code) {
|
||||
return { ok: false, error: 'provide a known label, or a code (the host is taken from the code; pass host only to override)' };
|
||||
}
|
||||
let payload;
|
||||
try {
|
||||
payload = decode(code);
|
||||
} catch (e) {
|
||||
return { ok: false, error: String(e.message ?? e) };
|
||||
}
|
||||
const effHost = host || payload.host;
|
||||
if (!effHost) {
|
||||
return { ok: false, error: 'no host in the code and none provided — pass host (e.g. the Tailscale IP/MagicDNS name)' };
|
||||
}
|
||||
target = {
|
||||
host: effHost,
|
||||
port: typeof port === 'number' ? port : payload.port,
|
||||
token: payload.token,
|
||||
fp: payload.fp,
|
||||
label: label || payload.label || effHost,
|
||||
};
|
||||
}
|
||||
|
||||
if (typeof port === 'number') target.port = port;
|
||||
|
||||
const client = new AgentClient(target);
|
||||
try {
|
||||
await client.connect();
|
||||
} catch (e) {
|
||||
return { ok: false, error: String(e.message ?? e) };
|
||||
}
|
||||
|
||||
let version;
|
||||
const info = await client.request('get_system_info', {});
|
||||
if (info && info.ok && info.data) {
|
||||
const d = info.data;
|
||||
version = d.version ?? d.appVersion ?? (d.app && d.app.version) ?? (d.agent && d.agent.version) ?? undefined;
|
||||
}
|
||||
|
||||
const id = `${target.label}@${target.host}:${target.port}`;
|
||||
if (state.current && state.current.client && state.current.client !== client) {
|
||||
state.current.client.close();
|
||||
}
|
||||
state.servers.set(id, { id, client, target });
|
||||
state.current = { id, client, target };
|
||||
|
||||
await upsertEntry({
|
||||
host: target.host,
|
||||
port: target.port,
|
||||
token: target.token,
|
||||
fp: target.fp,
|
||||
label: target.label,
|
||||
version,
|
||||
lastConnectedAt: new Date().toISOString(),
|
||||
});
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
server: { label: target.label, host: target.host, port: target.port, version },
|
||||
};
|
||||
}
|
||||
|
||||
export function buildServer() {
|
||||
const server = new McpServer({ name: 'mhu-diagnostics-gateway', version: '1.0.0' });
|
||||
|
||||
server.registerTool(
|
||||
'list_servers',
|
||||
{
|
||||
title: 'List known servers',
|
||||
description: 'List the registry of known diagnostic servers and which one is currently connected.',
|
||||
inputSchema: {},
|
||||
},
|
||||
guard(async () => {
|
||||
const registry = await loadRegistry();
|
||||
return {
|
||||
ok: true,
|
||||
servers: Object.values(registry),
|
||||
current: state.current ? state.current.id : null,
|
||||
};
|
||||
}),
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
'connect_server',
|
||||
{
|
||||
title: 'Connect to a diagnostic server',
|
||||
description:
|
||||
'Connect to a remote diagnostic agent. Use label for a known server, or a code for a new one — the host (e.g. a Tailscale IP/MagicDNS name) is taken from the code. Pass host only to override what the code carries.',
|
||||
inputSchema: {
|
||||
code: z.string().optional(),
|
||||
host: z.string().optional(),
|
||||
port: z.number().optional(),
|
||||
label: z.string().optional(),
|
||||
},
|
||||
},
|
||||
guard((args) => doConnect(args)),
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
'disconnect_server',
|
||||
{
|
||||
title: 'Disconnect the current server',
|
||||
description: 'Close the connection to the currently connected diagnostic server.',
|
||||
inputSchema: {},
|
||||
},
|
||||
guard(async () => {
|
||||
if (!state.current) return { ok: true, disconnected: false };
|
||||
const id = state.current.id;
|
||||
try { state.current.client.close(); } catch {}
|
||||
state.servers.delete(id);
|
||||
state.current = null;
|
||||
return { ok: true, disconnected: true, id };
|
||||
}),
|
||||
);
|
||||
|
||||
server.registerTool(
|
||||
'current_server',
|
||||
{
|
||||
title: 'Show the current server',
|
||||
description: 'Show the currently connected diagnostic target, or null if none.',
|
||||
inputSchema: {},
|
||||
},
|
||||
guard(async () => {
|
||||
if (!state.current) return { ok: true, current: null };
|
||||
const t = state.current.target;
|
||||
return {
|
||||
ok: true,
|
||||
current: { id: state.current.id, label: t.label, host: t.host, port: t.port },
|
||||
};
|
||||
}),
|
||||
);
|
||||
|
||||
for (const tool of DIAGNOSTIC_TOOLS) {
|
||||
server.registerTool(
|
||||
tool.name,
|
||||
{ title: tool.title, description: tool.description, inputSchema: tool.inputSchema },
|
||||
guard((args) => requireCurrent(tool.op, args)),
|
||||
);
|
||||
}
|
||||
|
||||
return server;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const server = buildServer();
|
||||
const transport = new StdioServerTransport();
|
||||
await server.connect(transport);
|
||||
console.error('[mhu-diag] gateway ready (stdio MCP)');
|
||||
}
|
||||
|
||||
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
|
||||
main().catch((e) => {
|
||||
console.error('[mhu-diag] fatal:', e);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
1197
gateway/package-lock.json
generated
1197
gateway/package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@ -1,22 +0,0 @@
|
||||
{
|
||||
"name": "mhu-diagnostics-gateway",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"description": "Local stdio MCP gateway for remote diagnostics of the Multi-Hoster-Uploader app.",
|
||||
"bin": {
|
||||
"mhu-diag": "index.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "node --test \"test/**/*.test.js\"",
|
||||
"verify": "node verify/e2e-verify.mjs && node verify/integration-mcp.mjs && node verify/adversarial-probe.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "~1.29.0",
|
||||
"ws": "^8",
|
||||
"zod": "^3.25.0"
|
||||
}
|
||||
}
|
||||
@ -1 +0,0 @@
|
||||
{}
|
||||
@ -1,63 +0,0 @@
|
||||
import { readFile, writeFile, chmod } from 'node:fs/promises';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { execFile } from 'node:child_process';
|
||||
import { userInfo } from 'node:os';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const REGISTRY_PATH = join(HERE, 'registry.json');
|
||||
|
||||
export async function loadRegistry() {
|
||||
try {
|
||||
const raw = await readFile(REGISTRY_PATH, 'utf8');
|
||||
const parsed = JSON.parse(raw);
|
||||
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
|
||||
return {};
|
||||
}
|
||||
return parsed;
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
function tightenWindowsAcl(path) {
|
||||
return new Promise((resolve) => {
|
||||
let user;
|
||||
try { user = userInfo().username; } catch { resolve(); return; }
|
||||
if (!user) { resolve(); return; }
|
||||
execFile('icacls', [path, '/inheritance:r', '/grant:r', `${user}:F`], { windowsHide: true }, () => resolve());
|
||||
});
|
||||
}
|
||||
|
||||
export async function saveRegistry(registry) {
|
||||
const data = registry && typeof registry === 'object' ? registry : {};
|
||||
await writeFile(REGISTRY_PATH, JSON.stringify(data, null, 2) + '\n', { encoding: 'utf8', mode: 0o600 });
|
||||
try { await chmod(REGISTRY_PATH, 0o600); } catch {}
|
||||
if (process.platform === 'win32') { try { await tightenWindowsAcl(REGISTRY_PATH); } catch {} }
|
||||
}
|
||||
|
||||
export async function upsertEntry(entry) {
|
||||
if (!entry || typeof entry.label !== 'string' || entry.label.length === 0) {
|
||||
throw new Error('upsertEntry: entry.label is required');
|
||||
}
|
||||
const registry = await loadRegistry();
|
||||
registry[entry.label] = {
|
||||
host: entry.host,
|
||||
port: entry.port,
|
||||
token: entry.token,
|
||||
fp: entry.fp,
|
||||
label: entry.label,
|
||||
version: entry.version,
|
||||
lastConnectedAt: entry.lastConnectedAt ?? new Date().toISOString(),
|
||||
};
|
||||
await saveRegistry(registry);
|
||||
return registry[entry.label];
|
||||
}
|
||||
|
||||
export async function getEntry(label) {
|
||||
if (typeof label !== 'string' || label.length === 0) return null;
|
||||
const registry = await loadRegistry();
|
||||
return registry[label] ?? null;
|
||||
}
|
||||
|
||||
export { REGISTRY_PATH };
|
||||
@ -1,64 +0,0 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { encode, decode } from '../code.js';
|
||||
|
||||
test('decode reads the host-bearing short-key format (h/p/t/n/s/fp)', () => {
|
||||
const code = encode({ v: 1, h: '100.64.0.5', p: 9110, t: 'deadbeefcafe1234', n: 'prod-3', s: 'wss', fp: 'AB:CD:EF:01' });
|
||||
assert.ok(code.startsWith('mhu1_'));
|
||||
const d = decode(code);
|
||||
assert.equal(d.host, '100.64.0.5');
|
||||
assert.equal(d.port, 9110);
|
||||
assert.equal(d.token, 'deadbeefcafe1234');
|
||||
assert.equal(d.label, 'prod-3');
|
||||
assert.equal(d.scheme, 'wss');
|
||||
assert.equal(d.fp, 'AB:CD:EF:01');
|
||||
});
|
||||
|
||||
test('decode is tolerant of the legacy long-key format (port/token/label, no host -> ws)', () => {
|
||||
const d = decode(encode({ v: 1, port: 9110, token: 'token-abc', label: 'localhost' }));
|
||||
assert.equal(d.host, undefined);
|
||||
assert.equal(d.port, 9110);
|
||||
assert.equal(d.token, 'token-abc');
|
||||
assert.equal(d.label, 'localhost');
|
||||
assert.equal(d.scheme, 'ws');
|
||||
});
|
||||
|
||||
test('decode rejects a string without the mhu1_ prefix', () => {
|
||||
assert.throws(() => decode('hello-world'), /missing "mhu1_" prefix/);
|
||||
});
|
||||
|
||||
test('decode rejects a wrong-version payload', () => {
|
||||
const bad = 'mhu1_' + Buffer.from(
|
||||
JSON.stringify({ v: 2, port: 9110, token: 'x', label: 'l' }),
|
||||
'utf8',
|
||||
).toString('base64url');
|
||||
assert.throws(() => decode(bad), /unsupported version/);
|
||||
});
|
||||
|
||||
test('decode rejects garbage after the prefix', () => {
|
||||
assert.throws(() => decode('mhu1_!!!not-base64-or-json!!!'), /Invalid code/);
|
||||
});
|
||||
|
||||
test('decode rejects an empty payload', () => {
|
||||
assert.throws(() => decode('mhu1_'), /empty payload/);
|
||||
});
|
||||
|
||||
test('decode rejects a non-string input', () => {
|
||||
assert.throws(() => decode(null), /expected a string/);
|
||||
});
|
||||
|
||||
test('decode rejects a missing token', () => {
|
||||
const bad = 'mhu1_' + Buffer.from(
|
||||
JSON.stringify({ v: 1, port: 9110, label: 'l' }),
|
||||
'utf8',
|
||||
).toString('base64url');
|
||||
assert.throws(() => decode(bad), /token/);
|
||||
});
|
||||
|
||||
test('decode rejects a non-number port', () => {
|
||||
const bad = 'mhu1_' + Buffer.from(
|
||||
JSON.stringify({ v: 1, port: 'nope', token: 'x', label: 'l' }),
|
||||
'utf8',
|
||||
).toString('base64url');
|
||||
assert.throws(() => decode(bad), /port/);
|
||||
});
|
||||
@ -1,137 +0,0 @@
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { writeFileSync, mkdtempSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import WebSocket from 'ws';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const appRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
const support = require(join(appRoot, 'lib', 'support-bundle.js'));
|
||||
const stats = require(join(appRoot, 'lib', 'stats.js'));
|
||||
const RemoteServer = require(join(appRoot, 'lib', 'remote-server.js'));
|
||||
const { createCollectors } = require(join(appRoot, 'lib', 'diagnostics-collectors.js'));
|
||||
const { createAgent } = require(join(appRoot, 'lib', 'diagnostics-agent.js'));
|
||||
|
||||
const findings = [];
|
||||
function leakCheck(label, text, needle, { realistic }) {
|
||||
if (text.includes(needle)) findings.push({ label, needle: needle.slice(0, 24), realistic });
|
||||
}
|
||||
|
||||
console.log('=== A. redactLogText pattern-scrub battery (no config secrets; pure shape detection) ===');
|
||||
const battery = [
|
||||
{ line: 'Authorization: Bearer abcDEF123456ghiJKL789', needle: 'abcDEF123456ghiJKL789', realistic: true },
|
||||
{ line: 'using Bearer eyJhbGciOiJIUzI1Nib-longtokenvalue-0099', needle: 'eyJhbGciOiJIUzI1Nib-longtokenvalue-0099', realistic: true },
|
||||
{ line: 'resp token=sess_9f8e7d6c5b4a3210ffee', needle: 'sess_9f8e7d6c5b4a3210ffee', realistic: true },
|
||||
{ line: 'access_token: ya29.A0ARrdaM-longgoogletoken-123', needle: 'ya29.A0ARrdaM-longgoogletoken-123', realistic: true },
|
||||
{ line: 'refresh_token = 1//0ggLongRefreshToken_abcdef', needle: '1//0ggLongRefreshToken_abcdef', realistic: true },
|
||||
{ line: 'x-api-key: SuperSecretApiKeyValue99', needle: 'SuperSecretApiKeyValue99', realistic: true },
|
||||
{ line: 'api_key=AKIAIOSFODNN7EXAMPLEKEY', needle: 'AKIAIOSFODNN7EXAMPLEKEY', realistic: true },
|
||||
{ line: 'GET /up?key=querykeysecret12345 HTTP/1.1', needle: 'querykeysecret12345', realistic: true },
|
||||
{ line: 'POST https://discord.com/api/webhooks/123456789012345678/WEBHOOKTOKENsecretvalue', needle: 'WEBHOOKTOKENsecretvalue', realistic: true },
|
||||
{ line: 'set-cookie: session=COOKIEsecretvalue9988; HttpOnly', needle: 'COOKIEsecretvalue9988', realistic: true },
|
||||
{ line: 'Cookie: sess_id=ABCcookievalue12345', needle: 'ABCcookievalue12345', realistic: true },
|
||||
{ line: 'sessionId: SESSIONsecret009988aa', needle: 'SESSIONsecret009988aa', realistic: true },
|
||||
{ line: 'two leaks: token=firsttok12345678 and api_key=secondkey87654321', needle: 'secondkey87654321', realistic: true },
|
||||
{ line: 'proxy https://admin:Sup3rProxyPass@proxy.internal:8080/path', needle: 'Sup3rProxyPass', realistic: true },
|
||||
{ line: 'Authorization: Basic dXNlcjpwYXNzd29yZF9zZWNyZXQ=', needle: 'dXNlcjpwYXNzd29yZF9zZWNyZXQ', realistic: true },
|
||||
{ line: 'jwt eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhYmMxMjMifQ.SflKxwRJSMeKKF2QT4fwpMabc', needle: 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhYmMxMjMifQ', realistic: true },
|
||||
{ line: 'session=BareSessionSecret998877', needle: 'BareSessionSecret998877', realistic: true },
|
||||
{ line: 'password: PlainTextPassword123 was used', needle: 'PlainTextPassword123', realistic: true },
|
||||
{ line: 'random high-entropy blob 9f8e7d6c5b4a3210ffeeddccbbaa with no key context', needle: '9f8e7d6c5b4a3210ffeeddccbbaa', realistic: false },
|
||||
];
|
||||
for (const t of battery) {
|
||||
const out = support.redactLogText(t.line, []);
|
||||
leakCheck('redactLogText: ' + t.line.slice(0, 40), out, t.needle, t);
|
||||
console.log(` ${out.includes(t.needle) ? 'LEAK ' : 'scrub'} ${t.line.slice(0, 52)}`);
|
||||
}
|
||||
|
||||
console.log('\n=== B. value-scrub: config secret in odd encodings (deepRedact via collectors) ===');
|
||||
const SECRET = 'CFGsecret_aabbccddeeff';
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'mhu-adv-'));
|
||||
writeFileSync(join(tmp, 'debug.log'), `plain ${SECRET}\nurlenc CFGsecret_aabbccddeeff also\n`);
|
||||
const cfg = {
|
||||
hosters: { doodstream: [{ accountId: 'a', apiKey: SECRET }] }, hosterSettings: {},
|
||||
globalSettings: { diagnostics: { enabled: true, token: 'd'.repeat(64) },
|
||||
pendingQueue: { savedAt: 1, selectedUploadHosters: [], selectedFiles: [], queueJobs: [
|
||||
{ file: 'x', fileName: 'x', hoster: 'doodstream', status: 'error', error: `failed ${SECRET}` }] } },
|
||||
history: [{ timestamp: 't', files: [{ name: 'x', results: [{ hoster: 'doodstream', status: 'error', error: `e ${SECRET}` }] }] }],
|
||||
rotationCursors: {},
|
||||
};
|
||||
const cols = createCollectors({
|
||||
loadConfig: () => JSON.parse(JSON.stringify(cfg)),
|
||||
getAllLogPaths: () => ({ fileuploader: join(tmp, 'f.log'), debug: join(tmp, 'debug.log'), accountRotation: join(tmp, 'r.log'), doodstreamDebug: join(tmp, 'doodstream-debug.log'), crashLog: join(tmp, 'c.log'), logDir: tmp }),
|
||||
support, stats, appInfo: () => ({ version: '3.3.84' }), systemInfo: () => ({}), agentInfo: () => ({}),
|
||||
});
|
||||
for (const [name, fn] of [
|
||||
['getConfigRedacted(all)', () => cols.getConfigRedacted({ section: 'all' })],
|
||||
['getQueueState(includeJobs)', () => cols.getQueueState({ includeJobs: true })],
|
||||
['getQueueState(default)', () => cols.getQueueState({})],
|
||||
['listErrors', () => cols.listErrors({})],
|
||||
['getHistory(files)', () => cols.getHistory({ includeFiles: true })],
|
||||
['serverHealth', () => cols.serverHealth({})],
|
||||
['readLog(debug)', () => cols.readLog({ name: 'debug' })],
|
||||
]) {
|
||||
const text = JSON.stringify(fn());
|
||||
leakCheck('collector:' + name, text, SECRET, { realistic: true });
|
||||
console.log(` ${text.includes(SECRET) ? 'LEAK ' : 'scrub'} ${name}`);
|
||||
}
|
||||
|
||||
console.log('\n=== C. abuse / DoS: ReDoS grep, oversized tailKb, malformed args (must not hang/crash) ===');
|
||||
const t0 = Date.now();
|
||||
writeFileSync(join(tmp, 'debug.log'), 'a'.repeat(80) + '! catastrophic-bait line\n' + `plain ${SECRET}\nurlenc CFGsecret_aabbccddeeff also\n`);
|
||||
const redos = cols.readLog({ name: 'debug', grep: '(a+)+$', tailKb: 1 });
|
||||
const redosMs = Date.now() - t0;
|
||||
if (redosMs > 1500) findings.push({ label: 'grep ReDoS hang (' + redosMs + 'ms) on 80-a line', needle: '(a+)+$', realistic: true });
|
||||
console.log(` grep "(a+)+$" vs 80-a line returned in ${redosMs}ms (must be <1500: ${redosMs < 1500})`);
|
||||
const longGrep = cols.readLog({ name: 'debug', grep: 'a'.repeat(5000) });
|
||||
console.log(` grep 5000-char pattern: ${longGrep && (longGrep.matchedLines !== undefined || longGrep.content !== undefined) ? 'handled' : 'handled'}`);
|
||||
const bigTail = cols.readLog({ name: 'debug', tailKb: 9999999 });
|
||||
console.log(` tailKb 9999999 clamped to: ${bigTail.tailKb} (<=1024:${bigTail.tailKb <= 1024})`);
|
||||
let crashed = false;
|
||||
for (const bad of [null, undefined, 42, [], { name: 123 }, { name: ['debug'] }, { name: 'debug', backup: 'evil' }, { name: 'debug', tailKb: -5 }, { limit: 'NaN' }]) {
|
||||
try { cols.readLog(bad); cols.listErrors(bad); cols.getQueueState(bad); cols.getHistory(bad); cols.getAppEvents(bad); }
|
||||
catch (e) { crashed = true; findings.push({ label: 'collector THREW on malformed args: ' + JSON.stringify(bad), needle: String(e.message), realistic: true }); }
|
||||
}
|
||||
console.log(` malformed-args battery: ${crashed ? 'THREW (bad)' : 'no throw (good)'}`);
|
||||
|
||||
console.log('\n=== D. agent whitelist: write/exec/unknown ops rejected, never throws ===');
|
||||
const agent = createAgent(cols);
|
||||
let agentThrew = false;
|
||||
for (const op of ['save_config', 'run_health_check', 'exec', 'eval', 'delete_log', '__proto__', 'constructor', 'getConfigRedacted', '', null, 'get_config_redacted; drop']) {
|
||||
try { const r = agent.handle(op, {}); if (r && r.ok === true && !['get_config_redacted'].includes(op)) findings.push({ label: 'agent ACCEPTED non-whitelisted op: ' + op, needle: op, realistic: true }); }
|
||||
catch (e) { agentThrew = true; findings.push({ label: 'agent THREW on op ' + op, needle: String(e.message), realistic: true }); }
|
||||
}
|
||||
console.log(` non-whitelisted ops: ${agentThrew ? 'THREW (bad)' : 'all returned {ok:false} (good)'}`);
|
||||
|
||||
console.log('\n=== E. transport abuse: brute-force lockout + concurrent clients (live RemoteServer) ===');
|
||||
const TOKEN = 'z'.repeat(64);
|
||||
const srv = new RemoteServer();
|
||||
await srv.start({ port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true, onDiagnosticRequest: (m, _c, reply) => reply(agent.handle(m.op, m.args)) });
|
||||
const port = srv.getPort();
|
||||
function wsOnce(sendToken) {
|
||||
return new Promise((resolve) => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${port}`);
|
||||
let authed = false;
|
||||
ws.on('open', () => ws.send(JSON.stringify({ type: 'auth', token: sendToken, role: 'diagnostic' })));
|
||||
ws.on('message', (raw) => { try { const m = JSON.parse(raw); if (m.type === 'auth-ok') { authed = true; ws.close(); resolve({ authed: true }); } } catch {} });
|
||||
ws.on('close', (code) => resolve({ authed, code }));
|
||||
ws.on('error', () => {});
|
||||
});
|
||||
}
|
||||
const okClients = await Promise.all([wsOnce(TOKEN), wsOnce(TOKEN), wsOnce(TOKEN)]);
|
||||
console.log(` 3 concurrent valid clients all authed: ${okClients.every(c => c.authed)}`);
|
||||
let lastCode = null;
|
||||
for (let i = 0; i < 6; i++) lastCode = (await wsOnce('wrongtoken')).code;
|
||||
console.log(` after 6 bad-token attempts, close code = ${lastCode} (4003 lockout expected: ${lastCode === 4003})`);
|
||||
const afterLock = await wsOnce(TOKEN);
|
||||
console.log(` valid token DURING lockout window: authed=${afterLock.authed} closeCode=${afterLock.code} (locked out even with right token: ${!afterLock.authed})`);
|
||||
srv.stop();
|
||||
|
||||
console.log('\n=== SUMMARY ===');
|
||||
const real = findings.filter(f => f.realistic);
|
||||
const theo = findings.filter(f => !f.realistic);
|
||||
if (theo.length) console.log(` ${theo.length} THEORETICAL (acknowledged denylist limit): ${theo.map(f => f.label).join(' | ')}`);
|
||||
if (real.length) { console.log(` ${real.length} REAL finding(s):`); for (const f of real) console.log(` - ${f.label} :: ${f.needle}`); process.exit(2); }
|
||||
console.log(' No REAL leaks/crashes found. (Theoretical = standalone secret with zero key/Bearer/URL context — inherent to denylist.)');
|
||||
process.exit(0);
|
||||
@ -1,133 +0,0 @@
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { writeFileSync, mkdtempSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import assert from 'node:assert';
|
||||
import { encode, decode } from '../code.js';
|
||||
import { AgentClient } from '../agent-client.js';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const root = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
const RemoteServer = require(join(root, 'lib', 'remote-server.js'));
|
||||
const support = require(join(root, 'lib', 'support-bundle.js'));
|
||||
const stats = require(join(root, 'lib', 'stats.js'));
|
||||
const { createCollectors } = require(join(root, 'lib', 'diagnostics-collectors.js'));
|
||||
const { createAgent } = require(join(root, 'lib', 'diagnostics-agent.js'));
|
||||
|
||||
const SECRET_API = 'SUPERSECRET_apikey_9f8e7d6c5b4a';
|
||||
const SECRET_PW = 'hunter2_password_zxcv';
|
||||
const SECRET_TOKEN = 'bearer_tok_qwerty12345';
|
||||
const WEBHOOK = 'https://discord.com/api/webhooks/123456789012345678/abcDEF_secretWebhookToken-xyz';
|
||||
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'mhu-e2e-'));
|
||||
const debugLog = join(tmp, 'debug.log');
|
||||
const dood = join(tmp, 'doodstream-debug.log');
|
||||
writeFileSync(debugLog, [
|
||||
`[2026-06-19T10:00:00Z] starting upload with apiKey=${SECRET_API}`,
|
||||
`[2026-06-19T10:00:01Z] Authorization: Bearer ${SECRET_TOKEN}`,
|
||||
`[2026-06-19T10:00:02Z] posting to ${WEBHOOK}`,
|
||||
`[2026-06-19T10:00:03Z] password ${SECRET_PW} used`,
|
||||
`[2026-06-19T10:00:04Z] normal benign line`,
|
||||
].join('\n'));
|
||||
writeFileSync(dood, `[doodstream] live apiKey=${SECRET_API}\n`);
|
||||
|
||||
const fakeConfig = {
|
||||
hosters: { doodstream: [{ accountId: 'acc1', apiKey: SECRET_API, password: SECRET_PW }] },
|
||||
hosterSettings: {},
|
||||
globalSettings: {
|
||||
webhookUrl: WEBHOOK,
|
||||
diagnostics: { enabled: true, port: 9110, token: 'x'.repeat(64) },
|
||||
pendingQueue: { savedAt: '2026-06-19T09:00:00Z', queueJobs: [{ file: 'a.mp4', fileName: 'a.mp4', hoster: 'doodstream', status: 'error', error: `failed with apiKey=${SECRET_API}` }, { file: 'b.mp4', fileName: 'b.mp4', hoster: 'streamtape', status: 'error', error: `upload rejected: token=${SECRET_TOKEN}` }], selectedUploadHosters: ['doodstream'], selectedFiles: ['a.mp4', 'b.mp4'] },
|
||||
},
|
||||
rotationCursors: { doodstream: 1 },
|
||||
history: [{ timestamp: '2026-06-19T08:00:00Z', files: [{ name: 'a.mp4', results: [{ hoster: 'doodstream', status: 'error', error: `boom token=${SECRET_TOKEN}` }] }] }],
|
||||
};
|
||||
|
||||
function getAllLogPaths() {
|
||||
return { fileuploader: join(tmp, 'fileuploader.log'), debug: debugLog, accountRotation: join(tmp, 'rot.log'), doodstreamDebug: dood, crashLog: join(tmp, 'crash.log'), logDir: tmp };
|
||||
}
|
||||
|
||||
const collectors = createCollectors({
|
||||
loadConfig: () => fakeConfig,
|
||||
getAllLogPaths,
|
||||
support,
|
||||
stats,
|
||||
appInfo: () => ({ version: '9.9.9' }),
|
||||
systemInfo: () => ({ platform: 'win32', hostname: 'TESTHOST' }),
|
||||
agentInfo: () => ({ version: '9.9.9', port: 9110 }),
|
||||
});
|
||||
const agent = createAgent(collectors);
|
||||
|
||||
const TOKEN = 'z'.repeat(64);
|
||||
const srv = new RemoteServer();
|
||||
|
||||
const SECRETS = [SECRET_API, SECRET_PW, SECRET_TOKEN, 'abcDEF_secretWebhookToken-xyz'];
|
||||
|
||||
function assertNoLeak(label, payload) {
|
||||
const text = JSON.stringify(payload);
|
||||
for (const s of SECRETS) {
|
||||
assert.ok(!text.includes(s), `LEAK in ${label}: secret "${s.slice(0, 12)}…" appeared in response`);
|
||||
}
|
||||
}
|
||||
|
||||
(async () => {
|
||||
await srv.start({
|
||||
port: 0,
|
||||
host: '127.0.0.1',
|
||||
token: TOKEN,
|
||||
diagnosticMode: true,
|
||||
onDiagnosticRequest: (msg, _client, reply) => {
|
||||
let r;
|
||||
try { r = agent.handle(msg.op, msg.args); }
|
||||
catch (e) { r = { ok: false, error: String(e && e.message || e) }; }
|
||||
reply(r);
|
||||
},
|
||||
});
|
||||
const port = srv.getPort();
|
||||
|
||||
const code = encode({ v: 1, port, token: TOKEN, label: 'e2e' });
|
||||
const payload = decode(code);
|
||||
assert.equal(payload.token, TOKEN);
|
||||
|
||||
const client = new AgentClient({ host: '127.0.0.1', port, token: payload.token });
|
||||
await client.connect();
|
||||
|
||||
const health = await client.request('server_health', { errorLimit: 10 });
|
||||
assert.equal(health.ok, true, 'server_health must succeed: ' + JSON.stringify(health));
|
||||
assertNoLeak('server_health', health);
|
||||
assert.ok(health.data.errors, 'server_health has errors section');
|
||||
assert.ok(health.data.queue, 'server_health has queue section');
|
||||
|
||||
const log = await client.request('read_log', { name: 'debug', tailKb: 64 });
|
||||
assert.equal(log.ok, true, 'read_log debug must succeed');
|
||||
assert.ok(log.data.content.includes('normal benign line'), 'benign content preserved');
|
||||
assertNoLeak('read_log:debug', log);
|
||||
|
||||
const dl = await client.request('read_log', { name: 'doodstream', tailKb: 64 });
|
||||
assert.equal(dl.ok, false, 'doodstream log MUST NOT be readable (live api keys)');
|
||||
|
||||
const trav = await client.request('read_log', { name: '../../../etc/passwd' });
|
||||
assert.equal(trav.ok, false, 'path traversal must be rejected');
|
||||
|
||||
const cfg = await client.request('get_config_redacted', { section: 'all' });
|
||||
assert.equal(cfg.ok, true, 'get_config_redacted must succeed');
|
||||
assertNoLeak('get_config_redacted', cfg);
|
||||
|
||||
const queue = await client.request('get_queue_state', { includeJobs: true });
|
||||
assert.equal(queue.ok, true, 'get_queue_state must succeed');
|
||||
assert.ok(Array.isArray(queue.data.jobs) && queue.data.jobs.length >= 2, 'jobs present');
|
||||
assertNoLeak('get_queue_state:includeJobs', queue);
|
||||
|
||||
const queueDefault = await client.request('get_queue_state', {});
|
||||
assert.equal(queueDefault.ok, true, 'get_queue_state (default args) must succeed');
|
||||
assertNoLeak('get_queue_state:default', queueDefault);
|
||||
|
||||
const writeAttempt = await client.request('save_config', { x: 1 });
|
||||
assert.equal(writeAttempt.ok, false, 'unknown/write op must be rejected by whitelist');
|
||||
|
||||
client.close();
|
||||
srv.stop();
|
||||
console.log('E2E PASS: connect → server_health/read_log/get_config_redacted succeeded; NO secrets leaked; doodstream + traversal + write rejected.');
|
||||
process.exit(0);
|
||||
})().catch((e) => { console.error('E2E FAIL:', e && e.stack || e); try { srv.stop(); } catch {} process.exit(1); });
|
||||
@ -1,218 +0,0 @@
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { writeFileSync, mkdtempSync, readFileSync, existsSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import assert from 'node:assert';
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import { encode } from '../code.js';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const gwRoot = join(here, '..');
|
||||
const appRoot = join(gwRoot, '..');
|
||||
const indexPath = join(gwRoot, 'index.js');
|
||||
const registryPath = join(gwRoot, 'registry.json');
|
||||
|
||||
const RemoteServer = require(join(appRoot, 'lib', 'remote-server.js'));
|
||||
const support = require(join(appRoot, 'lib', 'support-bundle.js'));
|
||||
const stats = require(join(appRoot, 'lib', 'stats.js'));
|
||||
const { createCollectors } = require(join(appRoot, 'lib', 'diagnostics-collectors.js'));
|
||||
const { createAgent } = require(join(appRoot, 'lib', 'diagnostics-agent.js'));
|
||||
|
||||
const SECRET_API = 'APIKEY_live_77ffee0011aabb';
|
||||
const SECRET_PW = 'pw_S3cr3t_zzqqww';
|
||||
const SECRET_DIAGTOK = 'd'.repeat(64);
|
||||
const OPAQUE_TOK = 'OPAQUE_session_tok_5a4b3c2d1e';
|
||||
const WEBHOOK = 'https://discord.com/api/webhooks/987654321098765432/IntegrationWebhookSecretXyz';
|
||||
const SECRETS = [SECRET_API, SECRET_PW, SECRET_DIAGTOK, OPAQUE_TOK, 'IntegrationWebhookSecretXyz'];
|
||||
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'mhu-int-'));
|
||||
const debugLog = join(tmp, 'debug.log');
|
||||
const rotLog = join(tmp, 'account-rotation.log');
|
||||
const dood = join(tmp, 'doodstream-debug.log');
|
||||
writeFileSync(debugLog, [
|
||||
`[2026-06-19T10:00:00Z] boot ok`,
|
||||
`[2026-06-19T10:00:01Z] upload with apiKey=${SECRET_API}`,
|
||||
`[2026-06-19T10:00:02Z] Authorization: Bearer ${OPAQUE_TOK}`,
|
||||
`[2026-06-19T10:00:03Z] grepneedle marker line`,
|
||||
].join('\n'));
|
||||
writeFileSync(rotLog, `[2026-06-19T10:00:00Z] rotating to acc2\n`);
|
||||
writeFileSync(dood, `[dood] api_key=${SECRET_API}\n`);
|
||||
|
||||
const config = {
|
||||
hosters: { doodstream: [{ accountId: 'acc1', apiKey: SECRET_API, password: SECRET_PW }] },
|
||||
hosterSettings: {},
|
||||
globalSettings: {
|
||||
webhookUrl: WEBHOOK,
|
||||
diagnostics: { enabled: true, port: 9110, token: SECRET_DIAGTOK },
|
||||
pendingQueue: { savedAt: '2026-06-19T09:00:00Z', selectedUploadHosters: ['doodstream'], selectedFiles: ['a.mp4', 'b.mp4'], queueJobs: [
|
||||
{ file: 'a.mp4', fileName: 'a.mp4', hoster: 'doodstream', status: 'error', error: `rejected token=${OPAQUE_TOK}` },
|
||||
{ file: 'b.mp4', fileName: 'b.mp4', hoster: 'doodstream', status: 'uploading', error: null },
|
||||
] },
|
||||
},
|
||||
rotationCursors: { doodstream: 1 },
|
||||
history: [{ timestamp: '2026-06-19T08:00:00Z', files: [{ name: 'a.mp4', results: [
|
||||
{ hoster: 'doodstream', status: 'error', error: `boom apiKey=${SECRET_API}` },
|
||||
{ hoster: 'voe.sx', status: 'done', url: 'https://voe.sx/x' },
|
||||
] }] }],
|
||||
};
|
||||
|
||||
function getAllLogPaths() {
|
||||
return { fileuploader: join(tmp, 'fileuploader.log'), debug: debugLog, accountRotation: rotLog, doodstreamDebug: dood, crashLog: join(tmp, 'crash.log'), logDir: tmp };
|
||||
}
|
||||
|
||||
const collectors = createCollectors({
|
||||
loadConfig: () => JSON.parse(JSON.stringify(config)),
|
||||
getAllLogPaths, support, stats,
|
||||
appInfo: () => ({ name: 'mhu', version: '3.3.84' }),
|
||||
systemInfo: () => ({ platform: 'win32', hostname: 'INT-HOST', cpuCount: 8 }),
|
||||
agentInfo: () => ({ version: '3.3.84', port: 9110, clientCount: 1, lastAccess: null }),
|
||||
});
|
||||
const agent = createAgent(collectors);
|
||||
|
||||
const TOKEN = 't'.repeat(64);
|
||||
const failures = [];
|
||||
function check(name, cond, detail) {
|
||||
if (cond) { console.log(` PASS ${name}`); }
|
||||
else { console.log(` FAIL ${name}${detail ? ' :: ' + detail : ''}`); failures.push(name); }
|
||||
}
|
||||
function noLeak(name, payloadText) {
|
||||
for (const s of SECRETS) {
|
||||
if (payloadText.includes(s)) { check(`${name} (no-leak:${s.slice(0, 10)})`, false, 'secret present'); return; }
|
||||
}
|
||||
check(`${name} (no-leak)`, true);
|
||||
}
|
||||
|
||||
const srv = new RemoteServer();
|
||||
let client, transport;
|
||||
const savedRegistry = existsSync(registryPath) ? readFileSync(registryPath, 'utf8') : null;
|
||||
|
||||
async function callJSON(name, args) {
|
||||
try {
|
||||
const r = await client.callTool({ name, arguments: args || {} });
|
||||
const text = r.content ? r.content.map(c => c.text || '').join('') : '';
|
||||
let data; try { data = JSON.parse(text); } catch { data = null; }
|
||||
return { text, data, isError: !!r.isError, threw: false };
|
||||
} catch (e) {
|
||||
return { text: String(e && e.message || e), data: null, isError: true, threw: true };
|
||||
}
|
||||
}
|
||||
function rejected(r) { return r.threw || r.isError || (r.data && r.data.ok === false); }
|
||||
|
||||
(async () => {
|
||||
await srv.start({
|
||||
port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true,
|
||||
onDiagnosticRequest: (msg, _c, reply) => {
|
||||
let res; try { res = agent.handle(msg.op, msg.args); } catch (e) { res = { ok: false, error: String(e && e.message || e) }; }
|
||||
reply(res);
|
||||
},
|
||||
});
|
||||
const port = srv.getPort();
|
||||
const code = encode({ v: 1, h: '127.0.0.1', p: port, t: TOKEN, n: 'integration' });
|
||||
|
||||
transport = new StdioClientTransport({ command: process.execPath, args: [indexPath] });
|
||||
client = new Client({ name: 'mhu-int-test', version: '1.0.0' });
|
||||
await client.connect(transport);
|
||||
|
||||
const toolList = await client.listTools();
|
||||
const names = toolList.tools.map(t => t.name).sort();
|
||||
const expected = ['connect_server', 'current_server', 'disconnect_server', 'get_app_events', 'get_config_redacted', 'get_history', 'get_queue_state', 'get_rotation_state', 'get_system_info', 'list_errors', 'list_logs', 'list_servers', 'read_log', 'server_health'].sort();
|
||||
check('listTools returns all 14 tools', JSON.stringify(names) === JSON.stringify(expected), names.join(','));
|
||||
|
||||
let r = await callJSON('connect_server', { code });
|
||||
check('connect_server ok (host taken from the code, no host arg)', r.data && r.data.ok === true, r.text.slice(0, 120));
|
||||
check('connect_server resolved host from code', r.data && r.data.server && r.data.server.host === '127.0.0.1');
|
||||
check('connect_server reports version 3.3.84', r.data && r.data.server && r.data.server.version === '3.3.84');
|
||||
|
||||
r = await callJSON('current_server');
|
||||
check('current_server shows host 127.0.0.1', r.data && r.data.current && r.data.current.host === '127.0.0.1');
|
||||
|
||||
r = await callJSON('server_health', { errorLimit: 10 });
|
||||
check('server_health ok + sections', r.data && r.data.ok && r.data.data.errors && r.data.data.queue && r.data.data.logs);
|
||||
noLeak('server_health', r.text);
|
||||
|
||||
r = await callJSON('read_log', { name: 'debug', tailKb: 64 });
|
||||
check('read_log debug ok + benign content kept', r.data && r.data.ok && r.data.data.content.includes('grepneedle marker line'));
|
||||
noLeak('read_log:debug', r.text);
|
||||
|
||||
r = await callJSON('read_log', { name: 'debug', grep: 'grepneedle' });
|
||||
check('read_log grep filters to matching line', r.data && r.data.ok && r.data.data.content.includes('grepneedle') && !r.data.data.content.includes('boot ok'));
|
||||
|
||||
r = await callJSON('read_log', { name: 'doodstream' });
|
||||
check('read_log doodstream REJECTED (live keys)', rejected(r), r.text.slice(0, 100));
|
||||
noLeak('read_log:doodstream-reject', r.text);
|
||||
|
||||
r = await callJSON('read_log', { name: '../../../etc/passwd' });
|
||||
check('read_log path traversal REJECTED', rejected(r), r.text.slice(0, 100));
|
||||
noLeak('read_log:traversal-reject', r.text);
|
||||
|
||||
r = await callJSON('list_logs');
|
||||
check('list_logs lists debug+fileuploader+accountRotation+crash', r.data && r.data.ok && r.data.data.files.length === 4);
|
||||
check('list_logs marks doodstream NOT readable (not in files)', r.data && !r.data.data.files.some(f => f.name === 'doodstream'));
|
||||
|
||||
r = await callJSON('list_errors', { limit: 50 });
|
||||
check('list_errors finds the 1 history error', r.data && r.data.ok && r.data.data.total === 1);
|
||||
noLeak('list_errors', r.text);
|
||||
|
||||
r = await callJSON('get_queue_state', { includeJobs: true });
|
||||
check('get_queue_state jobs present (2)', r.data && r.data.ok && Array.isArray(r.data.data.jobs) && r.data.data.jobs.length === 2);
|
||||
check('get_queue_state stale flag set', r.data && r.data.data.stale === true);
|
||||
noLeak('get_queue_state:includeJobs', r.text);
|
||||
|
||||
r = await callJSON('get_queue_state', {});
|
||||
noLeak('get_queue_state:default-args', r.text);
|
||||
|
||||
r = await callJSON('get_history', { limit: 10, includeFiles: true, includeUrls: true });
|
||||
check('get_history returns batches + perHoster', r.data && r.data.ok && Array.isArray(r.data.data.batches) && r.data.data.perHoster);
|
||||
noLeak('get_history', r.text);
|
||||
|
||||
r = await callJSON('get_config_redacted', { section: 'all' });
|
||||
check('get_config_redacted ok + history omitted', r.data && r.data.ok && r.data.data.config && r.data.data.config.history === undefined);
|
||||
noLeak('get_config_redacted:all', r.text);
|
||||
|
||||
r = await callJSON('get_config_redacted', { section: 'hosters' });
|
||||
noLeak('get_config_redacted:hosters', r.text);
|
||||
|
||||
r = await callJSON('get_rotation_state');
|
||||
check('get_rotation_state returns cursors', r.data && r.data.ok && r.data.data.rotationCursors);
|
||||
noLeak('get_rotation_state', r.text);
|
||||
|
||||
r = await callJSON('get_system_info');
|
||||
check('get_system_info returns app+system+agent', r.data && r.data.ok && r.data.data.app && r.data.data.system);
|
||||
noLeak('get_system_info', r.text);
|
||||
|
||||
r = await callJSON('get_app_events', { limit: 20 });
|
||||
check('get_app_events ok', r.data && r.data.ok);
|
||||
noLeak('get_app_events', r.text);
|
||||
|
||||
r = await callJSON('list_servers');
|
||||
check('list_servers includes the connected one', r.data && r.data.ok && r.data.current && r.data.current.includes('integration'));
|
||||
|
||||
r = await callJSON('disconnect_server');
|
||||
check('disconnect_server ok', r.data && r.data.ok && r.data.disconnected === true);
|
||||
|
||||
r = await callJSON('server_health');
|
||||
check('tool after disconnect returns guidance (no server connected)', r.data && r.data.ok === false && /connect_server/.test(r.data.error || ''));
|
||||
|
||||
r = await callJSON('connect_server', { code: 'mhu1_not_valid_base64!!', host: '127.0.0.1' });
|
||||
check('connect with garbage code REJECTED', r.data && r.data.ok === false);
|
||||
|
||||
r = await callJSON('connect_server', { code, host: '127.0.0.1', port: 1 });
|
||||
check('connect to dead port REJECTED with guidance', r.data && r.data.ok === false && /not running|refused|firewall|closed|timeout/i.test(r.data.error || ''));
|
||||
|
||||
await client.close();
|
||||
srv.stop();
|
||||
if (savedRegistry !== null) writeFileSync(registryPath, savedRegistry, 'utf8');
|
||||
|
||||
console.log('');
|
||||
if (failures.length) { console.log(`INTEGRATION FAIL: ${failures.length} check(s) failed: ${failures.join(' | ')}`); process.exit(1); }
|
||||
console.log('INTEGRATION PASS: full gateway-MCP <-> live agent stack verified, every tool, no leaks, error paths correct.');
|
||||
process.exit(0);
|
||||
})().catch((e) => {
|
||||
console.error('INTEGRATION ERROR:', e && e.stack || e);
|
||||
try { srv.stop(); } catch {}
|
||||
try { if (savedRegistry !== null) writeFileSync(registryPath, savedRegistry, 'utf8'); } catch {}
|
||||
process.exit(1);
|
||||
});
|
||||
@ -1,27 +0,0 @@
|
||||
function enabledAccountsFor(hosters, hoster, hasCreds) {
|
||||
const list = hosters && hosters[hoster];
|
||||
if (!Array.isArray(list)) return [];
|
||||
return list.filter(a => a && a.enabled !== false && hasCreds(hoster, a));
|
||||
}
|
||||
|
||||
function createAccountPicker({ hosters, hosterSettings, hasCreds, indices }) {
|
||||
const rotIdx = Object.assign(Object.create(null), indices || {});
|
||||
let dirty = false;
|
||||
function pick(hoster) {
|
||||
const enabled = enabledAccountsFor(hosters, hoster, hasCreds);
|
||||
if (enabled.length === 0) return null;
|
||||
const hs = (hosterSettings && hosterSettings[hoster]) || {};
|
||||
if (hs.rotateAccounts === true && enabled.length > 1) {
|
||||
const cursor = Number.isFinite(rotIdx[hoster]) ? rotIdx[hoster] : 0;
|
||||
rotIdx[hoster] = cursor + 1;
|
||||
dirty = true;
|
||||
return enabled[cursor % enabled.length];
|
||||
}
|
||||
return enabled[0];
|
||||
}
|
||||
pick.indices = () => ({ ...rotIdx });
|
||||
pick.dirty = () => dirty;
|
||||
return pick;
|
||||
}
|
||||
|
||||
module.exports = { createAccountPicker, enabledAccountsFor };
|
||||
@ -105,7 +105,7 @@ class ClouddropUploader {
|
||||
let bytesRead = 0;
|
||||
async function* generate() {
|
||||
yield preambleBuf;
|
||||
const fileStream = fs.createReadStream(filePath, { highWaterMark: 1024 * 1024 });
|
||||
const fileStream = fs.createReadStream(filePath, { highWaterMark: 256 * 1024 });
|
||||
for await (const chunk of fileStream) {
|
||||
if (signal && signal.aborted) throw new Error('Aborted');
|
||||
if (throttle) await throttle.consume(chunk.length, signal);
|
||||
@ -159,7 +159,7 @@ class ClouddropUploader {
|
||||
// Reuse a single buffer for all chunks (only the last chunk may be smaller,
|
||||
// in which case we slice a view). Avoids 64× 16 MB allocations on a 1 GB
|
||||
// file — real GC pressure during busy uploads.
|
||||
const fh = await fs.promises.open(filePath, 'r');
|
||||
const fd = fs.openSync(filePath, 'r');
|
||||
let bytesSent = 0;
|
||||
const reusableBuf = Buffer.allocUnsafe(chunkSize);
|
||||
try {
|
||||
@ -169,7 +169,7 @@ class ClouddropUploader {
|
||||
const offset = i * chunkSize;
|
||||
const remaining = fileSize - offset;
|
||||
const thisChunkSize = Math.min(chunkSize, remaining);
|
||||
await fh.read(reusableBuf, 0, thisChunkSize, offset);
|
||||
fs.readSync(fd, reusableBuf, 0, thisChunkSize, offset);
|
||||
const body = thisChunkSize === chunkSize
|
||||
? reusableBuf
|
||||
: reusableBuf.subarray(0, thisChunkSize);
|
||||
@ -194,7 +194,7 @@ class ClouddropUploader {
|
||||
if (progressCb) progressCb(bytesSent, fileSize);
|
||||
}
|
||||
} finally {
|
||||
try { await fh.close(); } catch {}
|
||||
try { fs.closeSync(fd); } catch {}
|
||||
}
|
||||
|
||||
// 3. Complete session — all bytes are already on the server at this point.
|
||||
|
||||
@ -10,9 +10,7 @@ const HOSTER_SETTINGS_DEFAULTS = {
|
||||
restartBelowKbs: 0, // 0 = off
|
||||
timeIntervalSec: 0, // delay between jobs
|
||||
maxSizeMb: 0, // 0 = unlimited
|
||||
logToFile: true, // write this hoster's successful links to fileuploader.log
|
||||
rotateAccounts: false,
|
||||
sizeMemoEnabled: true
|
||||
logToFile: true // write this hoster's successful links to fileuploader.log
|
||||
};
|
||||
|
||||
// Template for each hoster type (used as defaults for new accounts)
|
||||
@ -61,11 +59,6 @@ const DEFAULTS = {
|
||||
logFilePath: '',
|
||||
sessionLog: false, // legacy boolean (kept for back-compat reads); normalized into logMode on load
|
||||
logVerbose: false, // when true, [DEBUG] level entries are written to debug.log
|
||||
webhookUrl: '', // POST target on batch-done (Discord or generic JSON)
|
||||
webhookMention: '', // optional Discord ping target: user-id, role:id, @here, @everyone
|
||||
autoRetryRounds: 0, // 0 = off; 1-5 automatic retry rounds for transient failures after batch end
|
||||
autoRetryDelayMin: 5, // base delay in minutes between auto-retry rounds (linear backoff: round N waits N*delay)
|
||||
historyRetention: 'all', // 'all' | '7d' | '30d' | '90d' | '1000' | '100' — storage cap for upload history
|
||||
// NOTE: logMode is intentionally NOT in DEFAULTS. If it were, the deep-merge
|
||||
// would seed logMode='single' for every load, which would beat (and silently
|
||||
// erase) the legacy sessionLog:true → "daily" migration. normalizeLogMode in
|
||||
@ -100,171 +93,23 @@ const DEFAULTS = {
|
||||
port: 9100,
|
||||
token: '',
|
||||
allowInput: true
|
||||
},
|
||||
diagnostics: {
|
||||
enabled: false,
|
||||
port: 9110,
|
||||
token: '',
|
||||
label: '',
|
||||
codeIssuedAt: 0,
|
||||
bindMode: 'local',
|
||||
publicHost: '',
|
||||
allowlist: [],
|
||||
bindAddress: '127.0.0.1'
|
||||
}
|
||||
},
|
||||
history: [],
|
||||
rotationCursors: {}
|
||||
history: []
|
||||
};
|
||||
|
||||
const HISTORY_RETENTION_OPTIONS = [
|
||||
{ value: 'all', label: 'Alles behalten' },
|
||||
{ value: '7d', label: 'Letzte 7 Tage' },
|
||||
{ value: '30d', label: 'Letzte 30 Tage' },
|
||||
{ value: '90d', label: 'Letzte 90 Tage' },
|
||||
{ value: '1000', label: 'Letzte 1000 Uploads' },
|
||||
{ value: '100', label: 'Letzte 100 Uploads' }
|
||||
];
|
||||
|
||||
function batchTimestampMs(batch) {
|
||||
const raw = batch && batch.timestamp;
|
||||
if (raw === null || raw === undefined || raw === '') return null;
|
||||
const ms = typeof raw === 'number' ? raw : Date.parse(raw);
|
||||
return Number.isFinite(ms) ? ms : null;
|
||||
}
|
||||
|
||||
function batchRowCount(batch) {
|
||||
let n = 0;
|
||||
const files = (batch && batch.files) || [];
|
||||
for (const file of files) {
|
||||
for (const result of (file.results || [])) {
|
||||
if (result.status === 'aborted' || result.status === 'error') continue;
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function countHistoryRows(history) {
|
||||
let n = 0;
|
||||
for (const batch of (history || [])) n += batchRowCount(batch);
|
||||
return n;
|
||||
}
|
||||
|
||||
function applyHistoryRetention(history, retention, nowMs) {
|
||||
if (!Array.isArray(history) || history.length === 0) return history;
|
||||
const policy = String(retention || 'all');
|
||||
if (policy === 'all') return history;
|
||||
|
||||
if (/^\d+d$/.test(policy)) {
|
||||
const days = parseInt(policy, 10);
|
||||
if (!Number.isFinite(days) || days <= 0) return history;
|
||||
const cutoff = nowMs - days * 86400000;
|
||||
return history.filter(b => {
|
||||
const ts = batchTimestampMs(b);
|
||||
return ts === null || ts >= cutoff;
|
||||
});
|
||||
}
|
||||
|
||||
const maxRows = parseInt(policy, 10);
|
||||
if (!Number.isFinite(maxRows) || maxRows <= 0) return history;
|
||||
const keptReversed = [];
|
||||
let acc = 0;
|
||||
for (let i = history.length - 1; i >= 0; i--) {
|
||||
keptReversed.push(history[i]);
|
||||
acc += batchRowCount(history[i]);
|
||||
if (acc >= maxRows) break;
|
||||
}
|
||||
return keptReversed.reverse();
|
||||
}
|
||||
|
||||
class ConfigStore {
|
||||
constructor(app) {
|
||||
const dir = app && app.isPackaged
|
||||
? app.getPath('userData')
|
||||
: path.join(__dirname, '..');
|
||||
this.filePath = path.join(dir, 'electron-config.json');
|
||||
this.historyPath = path.join(dir, 'electron-history.json');
|
||||
this._writeQueue = Promise.resolve(); // Serializes all writes to prevent race conditions
|
||||
this._historyWriteQueue = Promise.resolve();
|
||||
this._historyMigrated = false;
|
||||
this._cache = null;
|
||||
this._cacheKey = '';
|
||||
this._perfLog = null;
|
||||
this._wqDepth = 0;
|
||||
|
||||
// Migrate config from old location if current doesn't exist
|
||||
if (!fs.existsSync(this.filePath) && app && app.isPackaged) {
|
||||
this._migrateFromOldPath(app);
|
||||
}
|
||||
if (app && app.isPackaged) {
|
||||
this._migrateHistory();
|
||||
}
|
||||
}
|
||||
|
||||
_readHistoryFile() {
|
||||
try {
|
||||
const raw = fs.readFileSync(this.historyPath, 'utf-8');
|
||||
if (!raw || raw.trim().length < 2) return [];
|
||||
const parsed = JSON.parse(raw);
|
||||
if (Array.isArray(parsed)) return parsed;
|
||||
if (parsed && Array.isArray(parsed.history)) return parsed.history;
|
||||
return [];
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
_writeHistoryFileDurable(arr) {
|
||||
const tmp = this.historyPath + '.tmp';
|
||||
const fd = fs.openSync(tmp, 'w');
|
||||
try {
|
||||
fs.writeSync(fd, JSON.stringify(arr));
|
||||
fs.fsyncSync(fd);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
fs.renameSync(tmp, this.historyPath);
|
||||
}
|
||||
|
||||
_writeHistoryFileAtomic(arr) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const tmp = this.historyPath + '.tmp';
|
||||
fs.writeFile(tmp, JSON.stringify(arr), 'utf-8', (err) => {
|
||||
if (err) return reject(err);
|
||||
try { fs.renameSync(tmp, this.historyPath); } catch (e) { return reject(e); }
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
_enqueueHistoryWrite(fn) {
|
||||
this._historyWriteQueue = this._historyWriteQueue.then(fn, fn);
|
||||
return this._historyWriteQueue;
|
||||
}
|
||||
|
||||
_migrateHistory() {
|
||||
try {
|
||||
if (fs.existsSync(this.historyPath)) {
|
||||
this._historyMigrated = Array.isArray(this._readHistoryFile());
|
||||
return;
|
||||
}
|
||||
let cfg = null;
|
||||
try { cfg = this._readAndParse(this.filePath); } catch {}
|
||||
const hist = (cfg && Array.isArray(cfg.history)) ? cfg.history : [];
|
||||
this._writeHistoryFileDurable(hist);
|
||||
const check = this._readHistoryFile();
|
||||
if (Array.isArray(check) && check.length === hist.length) {
|
||||
if (hist.length > 0) {
|
||||
try { fs.copyFileSync(this.filePath, this.filePath + '.pre-history-split.bak'); } catch {}
|
||||
}
|
||||
this._historyMigrated = true;
|
||||
} else {
|
||||
this._historyMigrated = false;
|
||||
}
|
||||
} catch {
|
||||
this._historyMigrated = false;
|
||||
}
|
||||
}
|
||||
|
||||
_migrateFromOldPath(app) {
|
||||
@ -296,71 +141,15 @@ class ConfigStore {
|
||||
return JSON.parse(raw);
|
||||
}
|
||||
|
||||
_clone(obj) {
|
||||
try { return structuredClone(obj); }
|
||||
catch { return JSON.parse(JSON.stringify(obj)); }
|
||||
}
|
||||
|
||||
setPerfLog(fn) { this._perfLog = typeof fn === 'function' ? fn : null; }
|
||||
|
||||
_pqLen(globalSettings) {
|
||||
const pq = globalSettings && globalSettings.pendingQueue;
|
||||
return pq && Array.isArray(pq.queueJobs) ? pq.queueJobs.length : 0;
|
||||
}
|
||||
|
||||
_callerTag() {
|
||||
const lines = (new Error().stack || '').split('\n');
|
||||
const out = [];
|
||||
for (let i = 2; i < lines.length && out.length < 3; i++) {
|
||||
const line = lines[i].trim();
|
||||
if (/config-store\.js/.test(line)) continue;
|
||||
const m = line.match(/at (?:async )?([^ (]+)/);
|
||||
if (m) out.push(m[1].split('.').pop());
|
||||
}
|
||||
return out.join('<') || '?';
|
||||
}
|
||||
|
||||
load() {
|
||||
if (!this._perfLog) return this._loadImpl();
|
||||
const hadCache = !!this._cache;
|
||||
const t0 = performance.now();
|
||||
const r = this._loadImpl();
|
||||
const dt = performance.now() - t0;
|
||||
if (dt >= 20) {
|
||||
const q = this._pqLen(r && r.globalSettings);
|
||||
const h = (r && r.history || []).length;
|
||||
this._perfLog(`config-load wall=${dt.toFixed(0)}ms cache=${hadCache ? 'hit' : 'miss'} hist=${h} queue=${q} via=${this._callerTag()}`);
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
_loadImpl() {
|
||||
try {
|
||||
// In-memory cache keyed on the file's mtime+size. The processed config
|
||||
// (merged + credential-decrypted) is reparsed/re-decrypted from disk ONLY
|
||||
// when the file actually changes. Our own writes refresh the cache (see
|
||||
// _commit), and an external edit changes mtime/size so the cache misses
|
||||
// and we reread. Without this, every one of the ~38 main.js load() call
|
||||
// sites (incl. the per-500ms log-flush path) re-read disk + JSON.parse the
|
||||
// whole growing history + DPAPI-decrypt every credential — the dominant
|
||||
// long-running main-thread drag. load() always returns a CLONE so callers
|
||||
// can mutate the result without corrupting the cache.
|
||||
let stat = null;
|
||||
try { stat = fs.statSync(this.filePath); } catch {}
|
||||
const statKey = stat ? `${stat.mtimeMs}:${stat.size}` : '';
|
||||
if (stat && this._cache && this._cacheKey === statKey) {
|
||||
return this._clone(this._cache);
|
||||
}
|
||||
|
||||
let data = null;
|
||||
// Try main config
|
||||
try { data = this._readAndParse(this.filePath); } catch {}
|
||||
// Fallback to backup if main is empty/corrupt
|
||||
if (!data) {
|
||||
try { data = this._readAndParse(this.filePath + '.bak'); } catch {}
|
||||
}
|
||||
if (!data) {
|
||||
try { data = this._readAndParse(this.filePath + '.pre-history-split.bak'); } catch {}
|
||||
const backupPath = this.filePath + '.bak';
|
||||
try { data = this._readAndParse(backupPath); } catch {}
|
||||
}
|
||||
if (!data) {
|
||||
const fresh = JSON.parse(JSON.stringify(DEFAULTS));
|
||||
@ -433,18 +222,11 @@ class ConfigStore {
|
||||
// Downstream readers consume logMode only and must NOT derive from
|
||||
// sessionLog at call sites.
|
||||
globalSettings.logMode = normalizeLogMode(globalSettings);
|
||||
const rotationCursors = (data.rotationCursors && typeof data.rotationCursors === 'object' && !Array.isArray(data.rotationCursors))
|
||||
? data.rotationCursors
|
||||
: {};
|
||||
const result = { hosters, hosterSettings, globalSettings, history: this._historyMigrated ? [] : (data.history || []), rotationCursors };
|
||||
const result = { hosters, hosterSettings, globalSettings, history: data.history || [] };
|
||||
// Decrypt credentials stored with safeStorage so the rest of the app
|
||||
// keeps working with plaintext in memory.
|
||||
secretStore.decryptCredentials(result);
|
||||
if (stat) {
|
||||
this._cache = result;
|
||||
this._cacheKey = statKey;
|
||||
}
|
||||
return this._clone(result);
|
||||
return result;
|
||||
} catch {
|
||||
const fresh = JSON.parse(JSON.stringify(DEFAULTS));
|
||||
fresh.globalSettings.logMode = normalizeLogMode(fresh.globalSettings);
|
||||
@ -452,80 +234,30 @@ class ConfigStore {
|
||||
}
|
||||
}
|
||||
|
||||
// Encrypt credential fields without mutating the caller's plaintext object.
|
||||
// Only `hosters` carries credentials, so we clone ONLY that subtree — the rest
|
||||
// (history, globalSettings, …) is referenced read-only into the stringified
|
||||
// object. Deep-cloning the whole config here (incl. an ever-growing history)
|
||||
// on every write was a primary long-running main-thread stall.
|
||||
// Deep-clone a config and encrypt its credential fields. Never mutate the
|
||||
// caller's object — the rest of the app holds plaintext references.
|
||||
_serializeForDisk(config) {
|
||||
const hosters = this._clone(config.hosters || {});
|
||||
secretStore.encryptCredentials({ hosters });
|
||||
return JSON.stringify({ ...config, hosters }, null, 2);
|
||||
}
|
||||
|
||||
_commit(config) {
|
||||
if (!this._perfLog) return this._atomicWrite(this._serializeForDisk(config));
|
||||
const t0 = performance.now();
|
||||
const data = this._serializeForDisk(config);
|
||||
const dt = performance.now() - t0;
|
||||
if (dt >= 20) {
|
||||
const q = this._pqLen(config.globalSettings);
|
||||
const h = (config.history || []).length;
|
||||
this._perfLog(`config-serialize wall=${dt.toFixed(0)}ms bytes=${data.length} hist=${h} queue=${q} wqDepth=${this._wqDepth} via=${this._callerTag()}`);
|
||||
}
|
||||
return this._atomicWrite(data);
|
||||
const clone = JSON.parse(JSON.stringify(config));
|
||||
secretStore.encryptCredentials(clone);
|
||||
return JSON.stringify(clone, null, 2);
|
||||
}
|
||||
|
||||
_enqueueWrite(fn) {
|
||||
this._wqDepth++;
|
||||
const done = () => { this._wqDepth--; };
|
||||
this._writeQueue = this._writeQueue.then(fn, fn).then(done, done);
|
||||
this._writeQueue = this._writeQueue.then(fn, fn);
|
||||
return this._writeQueue;
|
||||
}
|
||||
|
||||
_anyHosters(cfg) {
|
||||
const h = cfg && cfg.hosters;
|
||||
return !!h && typeof h === 'object' && Object.values(h).some(a => Array.isArray(a) && a.length > 0);
|
||||
}
|
||||
|
||||
_recoverHostersFromDisk() {
|
||||
for (const p of [this.filePath, this.filePath + '.bak', this.filePath + '.pre-history-split.bak']) {
|
||||
try {
|
||||
const raw = fs.readFileSync(p, 'utf-8');
|
||||
if (!raw || raw.trim().length < 2) continue;
|
||||
const data = JSON.parse(raw);
|
||||
if (this._anyHosters(data)) return data.hosters;
|
||||
} catch {}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
_guardHosters(current, hostersIntentional) {
|
||||
if (!hostersIntentional && !this._anyHosters(current)) {
|
||||
const recovered = this._recoverHostersFromDisk();
|
||||
if (recovered) {
|
||||
current.hosters = recovered;
|
||||
if (this._perfLog) this._perfLog('config-guard: prevented account wipe — restored hosters from on-disk backup after a corrupt/empty read');
|
||||
}
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
save(config) {
|
||||
return this._enqueueWrite(() => {
|
||||
const current = this.load();
|
||||
if (config.hosters) current.hosters = config.hosters;
|
||||
if (config.hosterSettings) current.hosterSettings = config.hosterSettings;
|
||||
if (config.globalSettings) current.globalSettings = config.globalSettings;
|
||||
this._guardHosters(current, !!config.hosters);
|
||||
return this._commit(current);
|
||||
return this._atomicWrite(this._serializeForDisk(current));
|
||||
});
|
||||
}
|
||||
|
||||
loadHistory() {
|
||||
if (this._historyMigrated) {
|
||||
return this._readHistoryFile() || [];
|
||||
}
|
||||
const config = this.load();
|
||||
return config.history || [];
|
||||
}
|
||||
@ -534,113 +266,46 @@ class ConfigStore {
|
||||
return new Promise((resolve, reject) => {
|
||||
const tmpPath = this.filePath + '.tmp';
|
||||
const backupPath = this.filePath + '.bak';
|
||||
let fd;
|
||||
try {
|
||||
fd = fs.openSync(tmpPath, 'w');
|
||||
fs.writeSync(fd, data);
|
||||
fs.fsyncSync(fd);
|
||||
} catch (e) {
|
||||
try { if (fd !== undefined) fs.closeSync(fd); } catch {}
|
||||
return reject(e);
|
||||
}
|
||||
try { fs.closeSync(fd); } catch {}
|
||||
Promise.resolve().then(() => {
|
||||
fs.writeFile(tmpPath, data, 'utf-8', (err) => {
|
||||
if (err) return reject(err);
|
||||
try {
|
||||
// Refresh .bak from the previous live file. Wrapped in try/catch
|
||||
// so an AV/indexer briefly locking the file doesn't fail the whole
|
||||
// save — the rename to the live path is the part that matters,
|
||||
// a stale .bak is preferable to losing the new write entirely.
|
||||
try {
|
||||
if (fs.existsSync(this.filePath)) {
|
||||
const cur = fs.readFileSync(this.filePath, 'utf-8');
|
||||
if (cur && cur.trim().length > 2) fs.writeFileSync(backupPath, cur, 'utf-8');
|
||||
const existing = fs.readFileSync(this.filePath, 'utf-8');
|
||||
if (existing && existing.trim().length > 2) {
|
||||
let isValid = false;
|
||||
try {
|
||||
const parsed = JSON.parse(existing);
|
||||
isValid = parsed && typeof parsed === 'object' && (parsed.hosters || parsed.hosterSettings || parsed.globalSettings);
|
||||
} catch {}
|
||||
if (isValid) fs.writeFileSync(backupPath, existing, 'utf-8');
|
||||
}
|
||||
}
|
||||
} catch {}
|
||||
fs.renameSync(tmpPath, this.filePath);
|
||||
} catch (e) { return reject(e); }
|
||||
// Invalidate the read cache: the next load() re-reads + re-merges the
|
||||
// freshly-written file (the on-disk format is sparse — load() fills
|
||||
// defaults — so we must NOT serve a pre-merge in-memory object).
|
||||
this._cache = null;
|
||||
this._cacheKey = '';
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
appendHistory(entry) {
|
||||
if (this._historyMigrated) {
|
||||
return this._enqueueHistoryWrite(() => {
|
||||
const cur = this._readHistoryFile();
|
||||
if (cur === null && fs.existsSync(this.historyPath)) return;
|
||||
const arr = cur || [];
|
||||
arr.push(entry);
|
||||
const gs = this.load().globalSettings;
|
||||
const retention = (gs && gs.historyRetention) || 'all';
|
||||
const pruned = applyHistoryRetention(arr, retention, Date.now());
|
||||
return this._writeHistoryFileAtomic(pruned);
|
||||
});
|
||||
}
|
||||
return this._enqueueWrite(() => {
|
||||
const config = this.load();
|
||||
config.history.push(entry);
|
||||
const retention = (config.globalSettings && config.globalSettings.historyRetention) || 'all';
|
||||
config.history = applyHistoryRetention(config.history, retention, Date.now());
|
||||
return this._commit(config);
|
||||
});
|
||||
}
|
||||
|
||||
pruneHistory(retention, opts = {}) {
|
||||
const dryRun = !!opts.dryRun;
|
||||
if (this._historyMigrated) {
|
||||
return this._enqueueHistoryWrite(() => {
|
||||
const current = this._readHistoryFile() || [];
|
||||
const beforeBatches = current.length;
|
||||
const beforeRows = countHistoryRows(current);
|
||||
const pruned = applyHistoryRetention(current, retention, Date.now());
|
||||
const result = {
|
||||
removedBatches: beforeBatches - pruned.length,
|
||||
removedRows: beforeRows - countHistoryRows(pruned),
|
||||
keptBatches: pruned.length,
|
||||
keptRows: countHistoryRows(pruned)
|
||||
};
|
||||
if (dryRun) return result;
|
||||
return this._writeHistoryFileAtomic(pruned)
|
||||
.then(() => this.save({ globalSettings: { ...this.load().globalSettings, historyRetention: String(retention || 'all') } }))
|
||||
.then(() => result);
|
||||
});
|
||||
}
|
||||
return this._enqueueWrite(() => {
|
||||
const config = this.load();
|
||||
const beforeBatches = config.history.length;
|
||||
const beforeRows = countHistoryRows(config.history);
|
||||
const pruned = applyHistoryRetention(config.history, retention, Date.now());
|
||||
const result = {
|
||||
removedBatches: beforeBatches - pruned.length,
|
||||
removedRows: beforeRows - countHistoryRows(pruned),
|
||||
keptBatches: pruned.length,
|
||||
keptRows: countHistoryRows(pruned)
|
||||
};
|
||||
if (dryRun) return result;
|
||||
config.history = pruned;
|
||||
if (config.globalSettings) config.globalSettings.historyRetention = String(retention || 'all');
|
||||
return this._commit(config).then(() => result);
|
||||
return this._atomicWrite(this._serializeForDisk(config));
|
||||
});
|
||||
}
|
||||
|
||||
clearHistory() {
|
||||
if (this._historyMigrated) {
|
||||
return this._enqueueHistoryWrite(() => this._writeHistoryFileAtomic([]));
|
||||
}
|
||||
return this._enqueueWrite(() => {
|
||||
const config = this.load();
|
||||
config.history = [];
|
||||
return this._commit(config);
|
||||
});
|
||||
}
|
||||
|
||||
saveRotationCursors(cursors) {
|
||||
return this._enqueueWrite(() => {
|
||||
const config = this.load();
|
||||
config.rotationCursors = (cursors && typeof cursors === 'object' && !Array.isArray(cursors)) ? cursors : {};
|
||||
this._guardHosters(config, false);
|
||||
return this._commit(config);
|
||||
return this._atomicWrite(this._serializeForDisk(config));
|
||||
});
|
||||
}
|
||||
}
|
||||
@ -649,6 +314,3 @@ module.exports = ConfigStore;
|
||||
module.exports.HOSTER_ACCOUNT_TEMPLATES = HOSTER_ACCOUNT_TEMPLATES;
|
||||
module.exports.HOSTER_NAMES = HOSTER_NAMES;
|
||||
module.exports.HOSTER_ADD_OPTIONS = HOSTER_ADD_OPTIONS;
|
||||
module.exports.HISTORY_RETENTION_OPTIONS = HISTORY_RETENTION_OPTIONS;
|
||||
module.exports.applyHistoryRetention = applyHistoryRetention;
|
||||
module.exports.countHistoryRows = countHistoryRows;
|
||||
|
||||
@ -1,32 +0,0 @@
|
||||
function createAgent(collectors) {
|
||||
const OPS = {
|
||||
get_system_info: (a) => collectors.getSystemInfo(a),
|
||||
server_health: (a) => collectors.serverHealth(a),
|
||||
get_config_redacted: (a) => collectors.getConfigRedacted(a),
|
||||
list_logs: () => collectors.listLogs(),
|
||||
read_log: (a) => collectors.readLog(a),
|
||||
tail_log: (a) => collectors.readLog(a),
|
||||
get_app_events: (a) => collectors.getAppEvents(a),
|
||||
list_errors: (a) => collectors.listErrors(a),
|
||||
get_queue_state: (a) => collectors.getQueueState(a),
|
||||
get_history: (a) => collectors.getHistory(a),
|
||||
get_rotation_state: () => collectors.getRotationState(),
|
||||
get_health: () => collectors.getHealth()
|
||||
};
|
||||
|
||||
function handle(op, args) {
|
||||
const fn = (typeof op === 'string' && Object.prototype.hasOwnProperty.call(OPS, op)) ? OPS[op] : null;
|
||||
if (typeof fn !== 'function') return { ok: false, error: `unknown or non-readonly op: ${op}` };
|
||||
try {
|
||||
const data = fn(args || {});
|
||||
if (data && data.ok === false) return data;
|
||||
return { ok: true, data };
|
||||
} catch (e) {
|
||||
return { ok: false, error: String((e && e.message) || e) };
|
||||
}
|
||||
}
|
||||
|
||||
return { handle, ops: Object.keys(OPS) };
|
||||
}
|
||||
|
||||
module.exports = { createAgent };
|
||||
@ -1,277 +0,0 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const READABLE_LOGS = {
|
||||
debug: 'debug',
|
||||
fileuploader: 'fileuploader',
|
||||
accountRotation: 'accountRotation',
|
||||
crash: 'crashLog'
|
||||
};
|
||||
|
||||
const QUEUE_STATUSES = ['preview', 'queued', 'getting-server', 'uploading', 'retrying', 'done', 'error', 'aborted', 'skipped'];
|
||||
|
||||
function createCollectors(deps) {
|
||||
const { loadConfig, loadHistory, getAllLogPaths, support, stats, appInfo, systemInfo, agentInfo } = deps;
|
||||
|
||||
function _secrets() {
|
||||
try { return support.collectSecretValues(loadConfig()); } catch { return []; }
|
||||
}
|
||||
|
||||
function _deepRedact(value, secrets) {
|
||||
const s = secrets || _secrets();
|
||||
const walk = (v) => {
|
||||
if (typeof v === 'string') return support.redactLogText(v, s);
|
||||
if (Array.isArray(v)) return v.map(walk);
|
||||
if (v && typeof v === 'object') {
|
||||
const o = {};
|
||||
for (const k of Object.keys(v)) o[k] = walk(v[k]);
|
||||
return o;
|
||||
}
|
||||
return v;
|
||||
};
|
||||
try { return walk(value); } catch { return value; }
|
||||
}
|
||||
|
||||
function _resolveLogPath(name, backup) {
|
||||
const key = READABLE_LOGS[name];
|
||||
if (!key) return null;
|
||||
const paths = getAllLogPaths();
|
||||
let p = paths[key];
|
||||
if (!p) return null;
|
||||
if (backup === 1 || backup === 2) p = `${p}.${backup}`;
|
||||
return p;
|
||||
}
|
||||
|
||||
function getSystemInfo() {
|
||||
return { app: appInfo(), system: systemInfo(), agent: agentInfo() };
|
||||
}
|
||||
|
||||
function getConfigRedacted(args) {
|
||||
const section = (args && args.section) || 'all';
|
||||
const cfg = loadConfig();
|
||||
const secrets = support.collectSecretValues(cfg);
|
||||
const sanitized = support.sanitizeConfig(cfg);
|
||||
let pick;
|
||||
let note;
|
||||
if (section === 'all') {
|
||||
pick = { ...sanitized };
|
||||
delete pick.history;
|
||||
note = 'history omitted from config — use get_history';
|
||||
} else {
|
||||
pick = sanitized[section] !== undefined ? sanitized[section] : null;
|
||||
}
|
||||
return { section, note, config: _deepRedact(pick, secrets) };
|
||||
}
|
||||
|
||||
function listLogs() {
|
||||
const paths = getAllLogPaths();
|
||||
const dir = paths.logDir;
|
||||
const files = [];
|
||||
for (const [name, key] of Object.entries(READABLE_LOGS)) {
|
||||
const base = paths[key];
|
||||
if (!base) continue;
|
||||
const variants = [];
|
||||
for (const suffix of ['', '.1', '.2']) {
|
||||
const fp = base + suffix;
|
||||
try {
|
||||
const st = fs.statSync(fp);
|
||||
variants.push({ backup: suffix === '' ? 0 : Number(suffix.slice(1)), sizeBytes: st.size, mtime: st.mtime.toISOString() });
|
||||
} catch {}
|
||||
}
|
||||
files.push({ name, path: base, readable: true, present: variants.length > 0, variants });
|
||||
}
|
||||
let siblings = [];
|
||||
try {
|
||||
siblings = fs.readdirSync(dir)
|
||||
.filter(f => /\.log(\.\d+)?$/i.test(f))
|
||||
.filter(f => !files.some(x => path.basename(x.path) === f || f.startsWith(path.basename(x.path))));
|
||||
siblings = siblings.map(f => {
|
||||
let size = 0, mtime = null;
|
||||
try { const st = fs.statSync(path.join(dir, f)); size = st.size; mtime = st.mtime.toISOString(); } catch {}
|
||||
return { name: f, readable: false, sizeBytes: size, mtime };
|
||||
});
|
||||
} catch {}
|
||||
return { dir, files, otherLogs: siblings };
|
||||
}
|
||||
|
||||
function readLog(args) {
|
||||
const a = args || {};
|
||||
const name = a.name;
|
||||
const p = _resolveLogPath(name, a.backup);
|
||||
if (!p) return { ok: false, error: `unknown or non-readable log: ${name}` };
|
||||
const tailKb = Math.min(Math.max(Number(a.tailKb) || 256, 1), 1024);
|
||||
const raw = support.collectFile(p, name, tailKb * 1024);
|
||||
let content = support.redactLogText(raw, _secrets());
|
||||
let matchedLines;
|
||||
if (a.grep && typeof a.grep === 'string' && a.grep.length <= 200) {
|
||||
const terms = a.grep.split('|').map(s => s.trim().toLowerCase()).filter(Boolean);
|
||||
if (terms.length) {
|
||||
const lines = content.split('\n').filter(l => {
|
||||
const low = l.toLowerCase();
|
||||
return terms.some(t => low.includes(t));
|
||||
});
|
||||
matchedLines = lines.length;
|
||||
content = lines.join('\n');
|
||||
}
|
||||
}
|
||||
let sizeBytes = null;
|
||||
try { sizeBytes = fs.statSync(p).size; } catch {}
|
||||
return { name, path: p, sizeBytes, returnedBytes: Buffer.byteLength(content), tailKb, matchedLines, content };
|
||||
}
|
||||
|
||||
function getAppEvents(args) {
|
||||
const limit = Math.min(Math.max(Number(args && args.limit) || 50, 1), 500);
|
||||
const out = [];
|
||||
const secrets = _secrets();
|
||||
for (const name of ['crash', 'debug']) {
|
||||
const p = _resolveLogPath(name);
|
||||
if (!p) continue;
|
||||
const raw = support.redactLogText(support.collectFile(p, name, 256 * 1024), secrets);
|
||||
const lines = raw.split('\n').filter(l => l.trim() && !l.startsWith('==='));
|
||||
for (const line of lines.slice(-limit)) out.push({ source: name, text: line });
|
||||
}
|
||||
return { events: out.slice(-limit), truncated: out.length > limit };
|
||||
}
|
||||
|
||||
function _historyErrors(history, opts) {
|
||||
const o = opts || {};
|
||||
const sinceMs = Number.isFinite(o.sinceMs) ? o.sinceMs : null;
|
||||
const secrets = _secrets();
|
||||
const errors = [];
|
||||
const byCategory = {};
|
||||
for (const batch of (Array.isArray(history) ? history : [])) {
|
||||
if (!batch || !Array.isArray(batch.files)) continue;
|
||||
const ts = batch.timestamp ? Date.parse(batch.timestamp) : null;
|
||||
if (sinceMs !== null && ts !== null && ts < sinceMs) continue;
|
||||
for (const file of batch.files) {
|
||||
if (!file || !Array.isArray(file.results)) continue;
|
||||
for (const r of file.results) {
|
||||
if (!r || r.status === 'done') continue;
|
||||
const category = stats.classifyErrorCategory(r.error);
|
||||
if (o.category && o.category !== category) continue;
|
||||
if (o.hoster && o.hoster !== r.hoster) continue;
|
||||
byCategory[category] = (byCategory[category] || 0) + 1;
|
||||
errors.push({
|
||||
ts: batch.timestamp || null,
|
||||
fileName: file.name || file.fileName || '',
|
||||
hoster: r.hoster || '',
|
||||
accountId: r.accountId || undefined,
|
||||
category,
|
||||
error: support.redactLogText(String(r.error || ''), secrets)
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return { errors, byCategory };
|
||||
}
|
||||
|
||||
function listErrors(args) {
|
||||
const a = args || {};
|
||||
const cfg = loadConfig();
|
||||
const { errors, byCategory } = _historyErrors(cfg.history, a);
|
||||
const limit = Math.min(Math.max(Number(a.limit) || 100, 1), 1000);
|
||||
const window = Number.isFinite(a.sinceMs) ? `since ${new Date(a.sinceMs).toISOString()}` : 'all history';
|
||||
return { window, total: errors.length, byCategory, errors: errors.slice(-limit) };
|
||||
}
|
||||
|
||||
function getQueueState(args) {
|
||||
const a = args || {};
|
||||
const cfg = loadConfig();
|
||||
const pending = cfg.globalSettings && cfg.globalSettings.pendingQueue;
|
||||
if (!pending || typeof pending !== 'object') {
|
||||
return { source: 'empty', stale: false, counts: {}, selectedHosters: [] };
|
||||
}
|
||||
const counts = {};
|
||||
for (const s of QUEUE_STATUSES) counts[s] = 0;
|
||||
const jobs = Array.isArray(pending.queueJobs) ? pending.queueJobs : [];
|
||||
for (const j of jobs) { if (counts[j.status] !== undefined) counts[j.status]++; }
|
||||
const result = {
|
||||
source: 'persisted',
|
||||
stale: true,
|
||||
savedAt: pending.savedAt || null,
|
||||
selectedHosters: Array.isArray(pending.selectedUploadHosters) ? pending.selectedUploadHosters : [],
|
||||
fileCount: Array.isArray(pending.selectedFiles) ? pending.selectedFiles.length : 0,
|
||||
counts
|
||||
};
|
||||
if (a.includeJobs !== false) {
|
||||
const maxJobs = Math.min(Math.max(Number(a.maxJobs) || 200, 1), 2000);
|
||||
result.jobs = _deepRedact(jobs.slice(0, maxJobs).map(j => ({
|
||||
file: j.file, fileName: j.fileName, hoster: j.hoster, status: j.status, error: j.error || null
|
||||
})));
|
||||
result.jobsTruncated = jobs.length > maxJobs;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function getHistory(args) {
|
||||
const a = args || {};
|
||||
const history = typeof loadHistory === 'function'
|
||||
? (loadHistory() || [])
|
||||
: (Array.isArray(loadConfig().history) ? loadConfig().history : []);
|
||||
const limit = Math.min(Math.max(Number(a.limit) || 20, 1), 200);
|
||||
const perHoster = stats.summarizePerHoster(history);
|
||||
const recent = [...history].slice(-limit).reverse();
|
||||
const secrets = _secrets();
|
||||
const batches = recent.map(b => {
|
||||
const out = { timestamp: b.timestamp || null, fileCount: Array.isArray(b.files) ? b.files.length : 0 };
|
||||
if (a.includeFiles) {
|
||||
out.files = (b.files || []).map(f => ({
|
||||
name: f.name || f.fileName || '',
|
||||
results: (f.results || []).map(r => {
|
||||
const rr = { hoster: r.hoster, status: r.status };
|
||||
if (r.error) rr.error = support.redactLogText(String(r.error), secrets);
|
||||
if (a.includeUrls && r.url) rr.url = r.url;
|
||||
return rr;
|
||||
})
|
||||
}));
|
||||
}
|
||||
return out;
|
||||
});
|
||||
return { totalBatches: history.length, returned: batches.length, perHoster, batches };
|
||||
}
|
||||
|
||||
function getRotationState() {
|
||||
const cfg = loadConfig();
|
||||
return { rotationCursors: _deepRedact(cfg.rotationCursors || {}) };
|
||||
}
|
||||
|
||||
function getHealth() {
|
||||
const cfg = loadConfig();
|
||||
const hosters = cfg.hosters && typeof cfg.hosters === 'object' ? Object.keys(cfg.hosters).filter(h => Array.isArray(cfg.hosters[h]) && cfg.hosters[h].length > 0) : [];
|
||||
return {
|
||||
reachabilityKnown: false,
|
||||
hint: 'Live hoster probing (run_health_check) is disabled in this build. Configured hosters with at least one account are listed.',
|
||||
configuredHosters: hosters
|
||||
};
|
||||
}
|
||||
|
||||
function serverHealth(args) {
|
||||
const a = args || {};
|
||||
const errorLimit = Math.min(Math.max(Number(a.errorLimit) || 20, 1), 200);
|
||||
const errArgs = Number.isFinite(a.errorSinceMs) ? { sinceMs: a.errorSinceMs, limit: errorLimit } : { limit: errorLimit };
|
||||
const errors = listErrors(errArgs);
|
||||
const queue = getQueueState({ includeJobs: false });
|
||||
const history = getHistory({ limit: 5 });
|
||||
const warnings = [];
|
||||
if (queue.source === 'persisted' && queue.stale) warnings.push('queue state is from the persisted snapshot (may lag live state; UploadManager not introspected in this build).');
|
||||
if (errors.total > 0) warnings.push(`${errors.total} non-success result(s) in the error window.`);
|
||||
return {
|
||||
server: getSystemInfo(),
|
||||
queue,
|
||||
recentBatches: history.batches,
|
||||
perHoster: history.perHoster,
|
||||
errors,
|
||||
hosters: getHealth(),
|
||||
logs: listLogs(),
|
||||
warnings
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
getSystemInfo, getConfigRedacted, listLogs, readLog, getAppEvents,
|
||||
listErrors, getQueueState, getHistory, getRotationState, getHealth, serverHealth,
|
||||
READABLE_LOGS
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = { createCollectors, READABLE_LOGS };
|
||||
@ -27,11 +27,7 @@ function _doodstreamLogPath() {
|
||||
return path.join(__dirname, '..', 'doodstream-debug.log');
|
||||
}
|
||||
|
||||
let _debugVerbose = false;
|
||||
function setDebugVerbose(v) { _debugVerbose = !!v; }
|
||||
|
||||
function _debugLog(msg) {
|
||||
if (!_debugVerbose) return;
|
||||
try {
|
||||
const logPath = _doodstreamLogPath();
|
||||
maybeRotateLogFile(logPath, _DOODSTREAM_LOG_MAX_BYTES, _DOODSTREAM_LOG_MAX_BACKUPS);
|
||||
@ -339,7 +335,7 @@ class DoodstreamUploader {
|
||||
const epilogueBuf = Buffer.from(epilogue, 'utf-8');
|
||||
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
|
||||
|
||||
const CHUNK_SIZE = 1024 * 1024;
|
||||
const CHUNK_SIZE = 256 * 1024;
|
||||
let bytesRead = 0;
|
||||
|
||||
async function* generate() {
|
||||
@ -702,4 +698,3 @@ class DoodstreamUploader {
|
||||
}
|
||||
|
||||
module.exports = DoodstreamUploader;
|
||||
module.exports.setDebugVerbose = setDebugVerbose;
|
||||
|
||||
@ -8,11 +8,7 @@ const SIGNATURES = [
|
||||
{ kind: 'flv', test: (b) => b.length >= 3 && b.slice(0, 3).toString('ascii') === 'FLV' },
|
||||
{ kind: 'asf-wmv', test: (b) => b.length >= 4 && b[0] === 0x30 && b[1] === 0x26 && b[2] === 0xB2 && b[3] === 0x75 },
|
||||
{ kind: 'mpeg-ps', test: (b) => b.length >= 4 && b[0] === 0x00 && b[1] === 0x00 && b[2] === 0x01 && (b[3] === 0xBA || b[3] === 0xB3) },
|
||||
{ kind: 'gif', test: (b) => b.length >= 6 && (b.slice(0, 6).toString('ascii') === 'GIF87a' || b.slice(0, 6).toString('ascii') === 'GIF89a') },
|
||||
// TS demands the 0x47 sync byte every 188 bytes — a single leading 0x47
|
||||
// matches every GIF and every text file starting with "G", so require
|
||||
// three consecutive packet boundaries before classifying as video.
|
||||
{ kind: 'mpeg-ts', test: (b) => b.length >= 377 && b[0] === 0x47 && b[188] === 0x47 && b[376] === 0x47 },
|
||||
{ kind: 'mpeg-ts', test: (b) => b.length >= 1 && b[0] === 0x47 },
|
||||
{ kind: 'mp3', test: (b) => b.length >= 3 && (b.slice(0, 3).toString('ascii') === 'ID3' || (b[0] === 0xFF && (b[1] & 0xE0) === 0xE0)) },
|
||||
{ kind: 'ogg', test: (b) => b.length >= 4 && b.slice(0, 4).toString('ascii') === 'OggS' },
|
||||
{ kind: 'jpeg', test: (b) => b.length >= 3 && b[0] === 0xFF && b[1] === 0xD8 && b[2] === 0xFF },
|
||||
|
||||
@ -235,18 +235,8 @@ function parseByseResult(payload) {
|
||||
// format, too small/large) ARE per-file and rotation is pointless.
|
||||
const accountLevel = /(not enough (disk )?(space|storage)|insufficient (disk )?space|disk (space )?full|storage (exhausted|full|voll|limit)|quota (exceeded|voll|überschritten)|account (full|voll|suspended|banned))/i.test(perFileError);
|
||||
const err = new Error(`Byse lehnte Datei ab: ${perFileError}`);
|
||||
if (accountLevel) {
|
||||
err.accountError = true;
|
||||
} else {
|
||||
err.fileRejected = true;
|
||||
// "Not video file format" is byse's known-misleading status: observed
|
||||
// live (2026-06-09) ONLY on valid MKVs >2.7 GB while the same account
|
||||
// accepted 1100+ smaller MKVs. Per-account size tiers produce it, and
|
||||
// async registration can land the file anyway. Flag it suspect so the
|
||||
// recovery poll still runs and the upload manager may try the file on
|
||||
// the remaining accounts instead of failing it everywhere.
|
||||
if (/not video file format/i.test(perFileError)) err.suspectReject = true;
|
||||
}
|
||||
if (accountLevel) err.accountError = true;
|
||||
else err.fileRejected = true;
|
||||
throw err;
|
||||
}
|
||||
|
||||
@ -288,7 +278,7 @@ function createUploadBody(filePath, formFields, onProgress, throttle, signal) {
|
||||
const { boundary, preambleBuf, epilogueBuf, totalSize, fileSize } = buildMultipart(filePath, formFields);
|
||||
|
||||
let bytesRead = 0;
|
||||
const CHUNK_SIZE = 1024 * 1024;
|
||||
const CHUNK_SIZE = 256 * 1024;
|
||||
|
||||
async function* generate() {
|
||||
yield preambleBuf;
|
||||
@ -325,15 +315,11 @@ async function apiGet(url, signal) {
|
||||
try {
|
||||
data = JSON.parse(text);
|
||||
} catch {
|
||||
const err = new Error(`API-Antwort war kein JSON (HTTP ${res.status}): ${(text || '').slice(0, 200)}`);
|
||||
if (res.status >= 500) err.transientNetwork = true;
|
||||
throw err;
|
||||
throw new Error(`API-Antwort war kein JSON (HTTP ${res.status}): ${(text || '').slice(0, 200)}`);
|
||||
}
|
||||
|
||||
if (data.status && [401, 403, 429, 500].includes(data.status)) {
|
||||
const err = new Error(data.msg || data.message || JSON.stringify(data));
|
||||
if (data.status === 500) err.transientNetwork = true;
|
||||
throw err;
|
||||
throw new Error(data.msg || data.message || JSON.stringify(data));
|
||||
}
|
||||
return data;
|
||||
} finally {
|
||||
@ -346,7 +332,6 @@ async function apiGet(url, signal) {
|
||||
|
||||
async function getUploadServer(hosterName, hosterConfig, apiKey, signal) {
|
||||
let lastMessage = '';
|
||||
let lastTransient = false;
|
||||
|
||||
for (let attempt = 1; attempt <= SERVER_RETRY_ATTEMPTS; attempt++) {
|
||||
for (const endpoint of hosterConfig.serverEndpoints) {
|
||||
@ -366,7 +351,6 @@ async function getUploadServer(hosterName, hosterConfig, apiKey, signal) {
|
||||
} catch (err) {
|
||||
if (err.name === 'AbortError') throw err;
|
||||
if (err.message) lastMessage = err.message;
|
||||
if (err.transientNetwork === true) lastTransient = true;
|
||||
}
|
||||
}
|
||||
|
||||
@ -400,7 +384,6 @@ async function getUploadServer(hosterName, hosterConfig, apiKey, signal) {
|
||||
// Genuine auth failures (invalid key / unauthorized / forbidden) make
|
||||
// shouldRetryServerLookup return false and stay classified as account errors.
|
||||
if (shouldRetryServerLookup(lastMessage)) e.hosterTransient = true;
|
||||
if (lastTransient) e.transientNetwork = true;
|
||||
throw e;
|
||||
}
|
||||
throw new Error('Kein Upload-Server erhalten. API-Key pruefen.');
|
||||
@ -411,7 +394,7 @@ async function _fetchByseFileList(apiKey, signal) {
|
||||
// to match the upload we just did against what the server has. The API
|
||||
// shape is typical XFS: { status, msg, result: { files: [...] } } or
|
||||
// { status, msg, files: [...] }.
|
||||
const url = `https://api.byse.sx/file/list?key=${encodeURIComponent(apiKey)}&per_page=100&sort=date&order=desc`;
|
||||
const url = `https://api.byse.sx/api/file/list?key=${encodeURIComponent(apiKey)}&per_page=100&sort=date&order=desc`;
|
||||
try {
|
||||
const { body, statusCode } = await request(url, {
|
||||
method: 'GET', signal,
|
||||
@ -458,13 +441,7 @@ async function _resolveByseUploadByName(apiKey, fileName, baselineCodes, signal)
|
||||
file_code: match.file_code
|
||||
};
|
||||
}
|
||||
if (i < POLL_ATTEMPTS - 1) {
|
||||
try {
|
||||
await sleep(POLL_DELAY_MS, signal);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (i < POLL_ATTEMPTS - 1) await sleep(POLL_DELAY_MS, signal);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@ -517,13 +494,7 @@ async function _resolveDoodstreamUploadByName(apiKey, fileName, baselineCodes, s
|
||||
file_code: match.file_code
|
||||
};
|
||||
}
|
||||
if (i < POLL_ATTEMPTS - 1) {
|
||||
try {
|
||||
await sleep(POLL_DELAY_MS, signal);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (i < POLL_ATTEMPTS - 1) await sleep(POLL_DELAY_MS, signal);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@ -580,11 +551,9 @@ async function uploadFile(hosterName, filePath, apiKey, onProgress, signal, thro
|
||||
payload = rawBody ? JSON.parse(rawBody) : {};
|
||||
} catch {
|
||||
const snippet = rawBody ? rawBody.slice(0, 240).replace(/\s+/g, ' ').trim() : '';
|
||||
const err = new Error(
|
||||
throw new Error(
|
||||
`Upload-Antwort von ${hosterName} war kein JSON (HTTP ${statusCode}${snippet ? `): ${snippet}` : ')'}`
|
||||
);
|
||||
if (statusCode >= 500) err.transientNetwork = true;
|
||||
throw err;
|
||||
}
|
||||
// Normalize valid-but-not-object JSON (JSON.parse('null') → null;
|
||||
// JSON.parse('"foo"') → string; JSON.parse('[1]') → array). Without this
|
||||
@ -598,19 +567,15 @@ async function uploadFile(hosterName, filePath, apiKey, onProgress, signal, thro
|
||||
}
|
||||
|
||||
if (statusCode < 200 || statusCode >= 300) {
|
||||
const err = new Error(
|
||||
throw new Error(
|
||||
payload.msg
|
||||
|| payload.message
|
||||
|| `Upload fehlgeschlagen (HTTP ${statusCode}${headers?.['content-type'] ? `, ${headers['content-type']}` : ''})`
|
||||
);
|
||||
if (statusCode >= 500) err.transientNetwork = true;
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (payload.status && [401, 403, 429, 500].includes(payload.status)) {
|
||||
const err = new Error(payload.msg || payload.message || JSON.stringify(payload));
|
||||
if (payload.status === 500) err.transientNetwork = true;
|
||||
throw err;
|
||||
throw new Error(payload.msg || payload.message || JSON.stringify(payload));
|
||||
}
|
||||
|
||||
let result = null;
|
||||
@ -635,27 +600,12 @@ async function uploadFile(hosterName, filePath, apiKey, onProgress, signal, thro
|
||||
return result;
|
||||
}
|
||||
|
||||
// Explicit rejections skip the recovery poll — EXCEPT suspect ones
|
||||
// (byse "Not video file format", see parseByseResult): for those the file
|
||||
// may have registered asynchronously despite the rejection-looking status,
|
||||
// so the poll must still run. Without this exception the rescue below is
|
||||
// dead for the very case it documents (regression shipped in 3.3.5x).
|
||||
// When the caller's file probe positively says the upload is NOT a video
|
||||
// (opts.probeIsVideoLike === false), the rejection is genuine — skip the
|
||||
// 30s poll for it like any other explicit rejection.
|
||||
const suspectBypass = parseErr
|
||||
&& parseErr.suspectReject === true
|
||||
&& !(opts && opts.probeIsVideoLike === false);
|
||||
const explicitlyRejected = parseErr
|
||||
&& (parseErr.fileRejected === true || parseErr.accountError === true)
|
||||
&& !suspectBypass;
|
||||
|
||||
// Byse-specific async handling: server accepts the file but responds with
|
||||
// filecode="" + misleading status ("Not video file format"). The file shows
|
||||
// up in the account shortly after — poll the list to claim it. User observed
|
||||
// this with 2+ GB MKV uploads that appeared as "OK" on the byse dashboard
|
||||
// even after our uploader gave up.
|
||||
if (hosterName === 'byse.sx' && byseBaseline && !explicitlyRejected) {
|
||||
if (hosterName === 'byse.sx' && byseBaseline) {
|
||||
const fileName = path.basename(filePath);
|
||||
const polled = await _resolveByseUploadByName(apiKey, fileName, byseBaseline, signal);
|
||||
if (polled) return polled;
|
||||
@ -664,7 +614,7 @@ async function uploadFile(hosterName, filePath, apiKey, onProgress, signal, thro
|
||||
// Doodstream: the doodapi upload POST returned no filecode (the same backend
|
||||
// hiccup that empties the web form). Poll the account file list by name — if
|
||||
// the file did register, claim its code instead of failing the upload.
|
||||
if (hosterName === 'doodstream.com' && doodBaseline && !explicitlyRejected) {
|
||||
if (hosterName === 'doodstream.com' && doodBaseline) {
|
||||
const fileName = path.basename(filePath);
|
||||
const polled = await _resolveDoodstreamUploadByName(apiKey, fileName, doodBaseline, signal);
|
||||
if (polled) return polled;
|
||||
|
||||
@ -1,50 +0,0 @@
|
||||
function normalizeIp(ip) {
|
||||
return String(ip || '').trim().replace(/^::ffff:/i, '').toLowerCase();
|
||||
}
|
||||
|
||||
function isLoopbackIp(ip) {
|
||||
const c = normalizeIp(ip);
|
||||
return c === '' || c === '::1' || c === 'localhost' || /^127\./.test(c);
|
||||
}
|
||||
|
||||
function ipv4ToInt(ip) {
|
||||
const parts = String(ip).split('.');
|
||||
if (parts.length !== 4) return null;
|
||||
let n = 0;
|
||||
for (const p of parts) {
|
||||
if (!/^\d{1,3}$/.test(p)) return null;
|
||||
const v = Number(p);
|
||||
if (v < 0 || v > 255) return null;
|
||||
n = (n << 8) + v;
|
||||
}
|
||||
return n >>> 0;
|
||||
}
|
||||
|
||||
function matchIpRule(clientIp, rule) {
|
||||
const client = normalizeIp(clientIp);
|
||||
const r = String(rule || '').trim().toLowerCase();
|
||||
if (!r) return false;
|
||||
if (r === '*' || r === '0.0.0.0/0') return true;
|
||||
if (r === client) return true;
|
||||
const slash = r.indexOf('/');
|
||||
if (slash > 0) {
|
||||
const baseInt = ipv4ToInt(r.slice(0, slash));
|
||||
const clientInt = ipv4ToInt(client);
|
||||
const bits = Number(r.slice(slash + 1));
|
||||
if (baseInt === null || clientInt === null || !Number.isInteger(bits) || bits < 0 || bits > 32) return false;
|
||||
if (bits === 0) return true;
|
||||
const mask = bits === 32 ? 0xffffffff : (~((1 << (32 - bits)) - 1)) >>> 0;
|
||||
return (clientInt & mask) === (baseInt & mask);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function evaluateClientAllowed(clientIp, rules) {
|
||||
const client = normalizeIp(clientIp);
|
||||
if (isLoopbackIp(client)) return true;
|
||||
const list = Array.isArray(rules) ? rules : [];
|
||||
if (list.length === 0) return false;
|
||||
return list.some((rule) => matchIpRule(client, rule));
|
||||
}
|
||||
|
||||
module.exports = { normalizeIp, isLoopbackIp, ipv4ToInt, matchIpRule, evaluateClientAllowed };
|
||||
@ -1,7 +1,7 @@
|
||||
// Log-file mode resolution for fileuploader.log:
|
||||
// - "single" → one file: fileuploader.log
|
||||
// - "daily" → per-day: fileuploader-YYYY-MM-DD.log
|
||||
// - "session" → per-launch: DD-MM-YYYY-mdu-session-HH-MM-NNNNNN.log
|
||||
// - "session" → per-launch: fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log
|
||||
//
|
||||
// Pure functions only — no fs, no Date.now() at call time — so they unit-test
|
||||
// cleanly and the main.js call sites pass in `new Date()` + the session stamp.
|
||||
@ -38,11 +38,13 @@
|
||||
return `${date.getFullYear()}-${_two(date.getMonth() + 1)}-${_two(date.getDate())}`;
|
||||
}
|
||||
|
||||
function formatSessionStamp(date, rand) {
|
||||
const d = `${_two(date.getDate())}-${_two(date.getMonth() + 1)}-${date.getFullYear()}`;
|
||||
const t = `${_two(date.getHours())}-${_two(date.getMinutes())}`;
|
||||
const r = (rand !== undefined && rand !== null && String(rand).trim()) ? `-${String(rand).trim()}` : '';
|
||||
return `${d}-mdu-session-${t}${r}`;
|
||||
function formatSessionStamp(date, pid) {
|
||||
const d = `${date.getFullYear()}-${_two(date.getMonth() + 1)}-${_two(date.getDate())}`;
|
||||
const t = `${_two(date.getHours())}-${_two(date.getMinutes())}-${_two(date.getSeconds())}`;
|
||||
// PID disambiguates a same-second close→reopen — a human can't but two
|
||||
// automated runs might. Cheap belt to a suspenders-not-required problem.
|
||||
const pidStr = pid !== undefined && pid !== null ? `-${pid}` : '';
|
||||
return `${d}_${t}${pidStr}`;
|
||||
}
|
||||
|
||||
/**
|
||||
@ -65,10 +67,9 @@
|
||||
const date = a.date instanceof Date ? a.date : new Date();
|
||||
return `${base}-${formatDateStamp(date)}${ext}`;
|
||||
}
|
||||
// session — the stamp is the full app-defined stem (DD-MM-YYYY-mdu-session-HH-MM),
|
||||
// independent of baseName.
|
||||
// session
|
||||
const sid = a.sessionId && String(a.sessionId).trim();
|
||||
if (sid) return `${sid}${ext}`;
|
||||
if (sid) return `${base}-session-${sid}${ext}`;
|
||||
// Defensive: if a session-id wasn't passed, fall back to single rather
|
||||
// than emit a malformed name. main.js always supplies one.
|
||||
return `${base}${ext}`;
|
||||
@ -84,9 +85,6 @@
|
||||
*/
|
||||
function stripModeStampFromFileName(fileName) {
|
||||
if (!fileName || typeof fileName !== 'string') return fileName;
|
||||
const newSessionRe = /^\d{2}-\d{2}-\d{4}-mdu-session-\d{2}-\d{2}(?:-\d+)?(\.[^.]+)?$/;
|
||||
const mNew = fileName.match(newSessionRe);
|
||||
if (mNew) return `fileuploader${mNew[1] || ''}`;
|
||||
// Order matters: session first (longer, more specific) before daily.
|
||||
// Both regexes are anchored to $ with no nested/ambiguous quantifiers, so
|
||||
// matching is linear — the eslint security warning is precautionary.
|
||||
|
||||
@ -1,29 +0,0 @@
|
||||
(function (root) {
|
||||
'use strict';
|
||||
|
||||
function selectOrphanTmps(fileNames, opts) {
|
||||
const o = opts || {};
|
||||
const baseName = String(o.baseName || '');
|
||||
const currentPid = o.currentPid;
|
||||
const isAlive = typeof o.isAlive === 'function' ? o.isAlive : () => false;
|
||||
const out = [];
|
||||
if (!baseName || !Array.isArray(fileNames)) return out;
|
||||
const prefix = baseName + '.';
|
||||
const suffix = '.tmp';
|
||||
for (const file of fileNames) {
|
||||
if (typeof file !== 'string') continue;
|
||||
if (!file.startsWith(prefix) || !file.endsWith(suffix)) continue;
|
||||
const mid = file.slice(prefix.length, file.length - suffix.length);
|
||||
if (!/^\d+$/.test(mid)) continue;
|
||||
const pid = Number(mid);
|
||||
if (pid === currentPid) continue;
|
||||
if (isAlive(pid)) continue;
|
||||
out.push(file);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const api = { selectOrphanTmps };
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = api;
|
||||
else if (root) root.OrphanTmp = api;
|
||||
})(typeof window !== 'undefined' ? window : this);
|
||||
@ -7,21 +7,18 @@
|
||||
// runtime and tests — no drift.
|
||||
//
|
||||
// Behaviour: on launch the restored queue is compared against the lifetime
|
||||
// upload log. Two rules drop a job:
|
||||
// 1) a 'done' job whose fileName|hoster appears in the log (declutter of
|
||||
// already-finished work), and
|
||||
// 2) ANY job (incl. preview) whose newest matching log entry is timestamped
|
||||
// at/after the snapshot's savedAt — it provably completed AFTER the queue
|
||||
// was last persisted, so a restored 'preview' row for it is a stale ghost.
|
||||
// upload log. ONLY genuinely-completed ('done') jobs that also appear in the
|
||||
// log are dropped — that's pure decluttering of work that already finished.
|
||||
//
|
||||
// Rule 2 only fires when a savedAt is passed AND the log carries timestamps;
|
||||
// without them this falls back to rule 1 alone. That fallback is the invariant
|
||||
// the canary tests pin: a pending job matching an OLDER log line (ts < savedAt,
|
||||
// or no ts at all) is KEPT — it's an intentional re-upload of a file uploaded
|
||||
// before, not a ghost. The old code filtered on log-presence alone, regardless
|
||||
// of status, so the ENTIRE restored queue vanished on the next restart/update
|
||||
// whenever the files had been uploaded previously. Manual log import
|
||||
// (importUploadLog) stays separate and explicit for bulk dedup.
|
||||
// Pending jobs (preview / queued) and failed ones (error / aborted) are NEVER
|
||||
// dropped here, even if a same-name+hoster line exists in the log. Those are
|
||||
// work the user intentionally has queued (often a deliberate re-upload of a
|
||||
// file that was uploaded before). The old code filtered on log-presence alone,
|
||||
// regardless of status, so the ENTIRE restored queue vanished on the next
|
||||
// restart/update whenever the files had been uploaded previously — surfacing as
|
||||
// an empty "Dateien hierhin ziehen oder klicken" queue. Manual log import
|
||||
// (importUploadLog) stays separate and explicit for users who do want bulk
|
||||
// dedup of pending jobs.
|
||||
|
||||
(function (root) {
|
||||
'use strict';
|
||||
@ -37,46 +34,19 @@
|
||||
* @param {Array<{fileName:string,hoster:string}>} logEntries
|
||||
* @returns {{ kept: Array, removed: Array }}
|
||||
*/
|
||||
function partitionRestoredJobsByLog(jobs, logEntries, savedAt) {
|
||||
function partitionRestoredJobsByLog(jobs, logEntries) {
|
||||
const kept = [];
|
||||
const removed = [];
|
||||
if (!Array.isArray(jobs) || jobs.length === 0) return { kept, removed };
|
||||
|
||||
const logKeys = new Set();
|
||||
const logMaxTs = new Map();
|
||||
for (const e of (Array.isArray(logEntries) ? logEntries : [])) {
|
||||
if (e && e.fileName && e.hoster) {
|
||||
const k = _key(e.fileName, e.hoster);
|
||||
logKeys.add(k);
|
||||
if (typeof e.ts === 'number' && isFinite(e.ts)) {
|
||||
const prev = logMaxTs.get(k);
|
||||
if (prev === undefined || e.ts > prev) logMaxTs.set(k, e.ts);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const savedAtFloor = (typeof savedAt === 'number' && isFinite(savedAt))
|
||||
? Math.floor(savedAt / 1000) * 1000
|
||||
: null;
|
||||
|
||||
const filesPerKey = new Map();
|
||||
for (const job of jobs) {
|
||||
if (job && job.fileName && job.hoster) {
|
||||
const jk = _key(job.fileName, job.hoster);
|
||||
let set = filesPerKey.get(jk);
|
||||
if (!set) { set = new Set(); filesPerKey.set(jk, set); }
|
||||
set.add(job.file || '');
|
||||
}
|
||||
if (e && e.fileName && e.hoster) logKeys.add(_key(e.fileName, e.hoster));
|
||||
}
|
||||
|
||||
for (const job of jobs) {
|
||||
const hasIds = job && job.fileName && job.hoster;
|
||||
const k = hasIds ? _key(job.fileName, job.hoster) : null;
|
||||
const doneInLog = job && job.status === 'done' && hasIds && logKeys.has(k);
|
||||
const keyUnambiguous = k !== null && filesPerKey.get(k).size <= 1;
|
||||
const uploadedAfterSnapshot = savedAtFloor !== null && k !== null && keyUnambiguous
|
||||
&& logMaxTs.has(k) && logMaxTs.get(k) >= savedAtFloor;
|
||||
if (doneInLog || uploadedAfterSnapshot) {
|
||||
const isDone = job && job.status === 'done' && job.fileName && job.hoster;
|
||||
if (isDone && logKeys.has(_key(job.fileName, job.hoster))) {
|
||||
removed.push(job);
|
||||
} else {
|
||||
kept.push(job);
|
||||
@ -85,25 +55,7 @@
|
||||
return { kept, removed };
|
||||
}
|
||||
|
||||
function completedSelectionKeys(selectedFiles, hosters, logEntries, savedAt) {
|
||||
const out = [];
|
||||
if (!Array.isArray(selectedFiles) || !Array.isArray(hosters)) return out;
|
||||
if (!(typeof savedAt === 'number' && isFinite(savedAt))) return out;
|
||||
const synthetic = [];
|
||||
for (const f of selectedFiles) {
|
||||
if (!f || !f.path) continue;
|
||||
const name = f.name || String(f.path).split(/[\\/]/).pop();
|
||||
for (const h of hosters) {
|
||||
if (h) synthetic.push({ fileName: name, hoster: h, file: f.path, status: 'preview' });
|
||||
}
|
||||
}
|
||||
if (synthetic.length === 0) return out;
|
||||
const { removed } = partitionRestoredJobsByLog(synthetic, logEntries, savedAt);
|
||||
for (const job of removed) out.push(`${job.file}|${job.hoster}`);
|
||||
return out;
|
||||
}
|
||||
|
||||
const api = { partitionRestoredJobsByLog, completedSelectionKeys };
|
||||
const api = { partitionRestoredJobsByLog };
|
||||
|
||||
if (typeof module !== 'undefined' && module.exports) {
|
||||
module.exports = api;
|
||||
|
||||
@ -1,12 +1,5 @@
|
||||
const { WebSocketServer } = require('ws');
|
||||
const crypto = require('crypto');
|
||||
const { evaluateClientAllowed } = require('./ip-allowlist');
|
||||
|
||||
function timingSafeEqualStr(a, b) {
|
||||
const x = Buffer.from(String(a == null ? '' : a));
|
||||
const y = Buffer.from(String(b == null ? '' : b));
|
||||
return x.length === y.length && crypto.timingSafeEqual(x, y);
|
||||
}
|
||||
|
||||
class RemoteServer {
|
||||
constructor() {
|
||||
@ -14,16 +7,13 @@ class RemoteServer {
|
||||
this._clients = new Map(); // ws -> { id, role, authenticated }
|
||||
this._config = null;
|
||||
this._failedAttempts = new Map(); // ip -> { count, blockedUntil }
|
||||
this._lastAccess = null;
|
||||
}
|
||||
|
||||
start(opts) {
|
||||
return new Promise((resolve, reject) => {
|
||||
this._config = opts;
|
||||
|
||||
const wssOpts = { port: opts.port, maxPayload: 256 * 1024 };
|
||||
if (opts.host) wssOpts.host = opts.host;
|
||||
this._wss = new WebSocketServer(wssOpts, () => {
|
||||
this._wss = new WebSocketServer({ port: opts.port }, () => {
|
||||
resolve();
|
||||
});
|
||||
|
||||
@ -71,11 +61,6 @@ class RemoteServer {
|
||||
return;
|
||||
}
|
||||
|
||||
if (Array.isArray(this._config.allowlist) && !evaluateClientAllowed(ip, this._config.allowlist)) {
|
||||
ws.close(4005, 'Client IP not allowed');
|
||||
return;
|
||||
}
|
||||
|
||||
const clientId = crypto.randomUUID();
|
||||
this._clients.set(ws, { id: clientId, role: null, authenticated: false });
|
||||
|
||||
@ -98,13 +83,12 @@ class RemoteServer {
|
||||
authReceived = true;
|
||||
clearTimeout(authTimeout);
|
||||
|
||||
if (msg.type === 'auth' && timingSafeEqualStr(msg.token, this._config.token)) {
|
||||
if (msg.type === 'auth' && msg.token === this._config.token) {
|
||||
client.authenticated = true;
|
||||
client.role = this._config.diagnosticMode ? 'diagnostic' : (msg.role || 'viewer');
|
||||
this._lastAccess = Date.now();
|
||||
client.role = msg.role || 'viewer';
|
||||
ws.send(JSON.stringify({ type: 'auth-ok', clientId }));
|
||||
|
||||
if (!this._config.diagnosticMode && this.getClientCount() === 1) {
|
||||
if (this.getClientCount() === 1) {
|
||||
this._config.onCreateCaptureWindow();
|
||||
}
|
||||
} else {
|
||||
@ -115,16 +99,6 @@ class RemoteServer {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._config.diagnosticMode) {
|
||||
if (msg.type === 'diag-request' && typeof this._config.onDiagnosticRequest === 'function') {
|
||||
this._lastAccess = Date.now();
|
||||
this._config.onDiagnosticRequest(msg, client, (payload) => {
|
||||
this.sendToClient(client.id, { type: 'diag-response', reqId: msg.reqId, ...payload });
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (msg.type === 'offer' || msg.type === 'ice-candidate') {
|
||||
msg.clientId = client.id;
|
||||
msg.role = client.role;
|
||||
@ -138,7 +112,7 @@ class RemoteServer {
|
||||
const wasAuthenticated = client && client.authenticated;
|
||||
this._clients.delete(ws);
|
||||
|
||||
if (wasAuthenticated && !this._config.diagnosticMode) {
|
||||
if (wasAuthenticated) {
|
||||
this._config.onSignalingToCapture({
|
||||
type: 'client-disconnected',
|
||||
clientId: client.id
|
||||
@ -156,7 +130,7 @@ class RemoteServer {
|
||||
const wasAuthenticated = client && client.authenticated;
|
||||
this._clients.delete(ws);
|
||||
|
||||
if (wasAuthenticated && !this._config.diagnosticMode) {
|
||||
if (wasAuthenticated) {
|
||||
this._config.onSignalingToCapture({
|
||||
type: 'client-disconnected',
|
||||
clientId: client.id
|
||||
@ -168,16 +142,10 @@ class RemoteServer {
|
||||
});
|
||||
}
|
||||
|
||||
getLastAccess() {
|
||||
return this._lastAccess;
|
||||
}
|
||||
|
||||
sendToClient(clientId, data) {
|
||||
for (const [ws, client] of this._clients) {
|
||||
if (client.id === clientId && client.authenticated) {
|
||||
if (ws.readyState === 1) {
|
||||
try { ws.send(JSON.stringify(data)); } catch {}
|
||||
}
|
||||
ws.send(JSON.stringify(data));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,19 +0,0 @@
|
||||
function configureStartupRenderer(app) {
|
||||
app.disableHardwareAcceleration();
|
||||
}
|
||||
|
||||
function createStartupWindow(BrowserWindow, options) {
|
||||
const window = new BrowserWindow({ ...options, show: false });
|
||||
window.once('ready-to-show', () => {
|
||||
window.show();
|
||||
});
|
||||
|
||||
return {
|
||||
window,
|
||||
load(target, onLoadError) {
|
||||
return window.loadFile(target).catch(onLoadError);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = { configureStartupRenderer, createStartupWindow };
|
||||
@ -1,6 +1,6 @@
|
||||
const fs = require('fs');
|
||||
|
||||
const CRED_KEYS = new Set(['password', 'apiKey', 'token', 'cookie', 'sessionId', 'webhookUrl', 'diagToken']);
|
||||
const CRED_KEYS = new Set(['password', 'apiKey', 'token', 'cookie', 'sessionId']);
|
||||
const REDACTED = '<redacted>';
|
||||
|
||||
function sanitizeConfig(config) {
|
||||
@ -18,54 +18,6 @@ function sanitizeConfig(config) {
|
||||
return clone;
|
||||
}
|
||||
|
||||
function collectSecretValues(config) {
|
||||
const out = new Set();
|
||||
(function walk(o) {
|
||||
if (!o) return;
|
||||
if (Array.isArray(o)) { for (const e of o) walk(e); return; }
|
||||
if (typeof o !== 'object') return;
|
||||
for (const k of Object.keys(o)) {
|
||||
const v = o[k];
|
||||
if (CRED_KEYS.has(k) && typeof v === 'string' && v.length >= 6) out.add(v);
|
||||
else walk(v);
|
||||
}
|
||||
})(config);
|
||||
return Array.from(out);
|
||||
}
|
||||
|
||||
function redactLogText(text, secrets) {
|
||||
if (typeof text !== 'string' || !text) return text;
|
||||
let out = text;
|
||||
if (Array.isArray(secrets)) {
|
||||
for (const s of secrets) {
|
||||
if (typeof s === 'string' && s.length >= 6) out = out.split(s).join(REDACTED);
|
||||
}
|
||||
}
|
||||
out = out
|
||||
.replace(/https?:\/\/(?:ptb\.|canary\.)?discord(?:app)?\.com\/api\/webhooks\/\d+\/[\w-]+/gi, 'https://discord.com/api/webhooks/' + REDACTED)
|
||||
.replace(/(\/\/[^\s/:@]+:)[^\s/@]+(@)/g, '$1' + REDACTED + '$2')
|
||||
.replace(/(authorization:\s*(?:bearer|basic)\s+)\S+/gi, '$1' + REDACTED)
|
||||
.replace(/\bbearer\s+[A-Za-z0-9._\-/+]{16,}/gi, 'bearer ' + REDACTED)
|
||||
.replace(/\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{6,}/g, REDACTED)
|
||||
.replace(/([?&](?:api_?key|key|token|access_token|password|pass)=)[^\s&"'`]+/gi, '$1' + REDACTED)
|
||||
.replace(/("?\b(?:api[_-]?key|apikey|password|passwd|secret|(?:access|refresh|auth|session)[_-]?token|token|sessionid|session)"?\s*[:=]\s*"?)[A-Za-z0-9._\-/+]{8,}/gi, '$1' + REDACTED)
|
||||
.replace(/(\bset-cookie:|\bcookie:)\s*\S[^\n]*/gi, '$1 ' + REDACTED)
|
||||
.replace(/(\bsess(?:_?id)?\b["'=:\s]+)[A-Za-z0-9._\-]{8,}/gi, '$1' + REDACTED);
|
||||
return out;
|
||||
}
|
||||
|
||||
function valueScrub(value, secrets) {
|
||||
if (value == null) return value;
|
||||
const json = JSON.stringify(value);
|
||||
let scrubbed = json;
|
||||
if (Array.isArray(secrets)) {
|
||||
for (const s of secrets) {
|
||||
if (typeof s === 'string' && s.length >= 6) scrubbed = scrubbed.split(s).join(REDACTED);
|
||||
}
|
||||
}
|
||||
return JSON.parse(scrubbed);
|
||||
}
|
||||
|
||||
function collectFile(filePath, label, maxBytes) {
|
||||
if (!filePath) return `=== ${label} ===\n<no path configured>\n\n`;
|
||||
let stat;
|
||||
@ -109,4 +61,4 @@ function buildSupportBundleText({ header, sanitizedConfig, files }) {
|
||||
return parts.join('');
|
||||
}
|
||||
|
||||
module.exports = { sanitizeConfig, collectSecretValues, redactLogText, valueScrub, collectFile, buildSupportBundleText, CRED_KEYS, REDACTED };
|
||||
module.exports = { sanitizeConfig, collectFile, buildSupportBundleText, CRED_KEYS, REDACTED };
|
||||
|
||||
@ -1,59 +0,0 @@
|
||||
(function (root) {
|
||||
'use strict';
|
||||
|
||||
function makeThrottleTimer(opts) {
|
||||
const o = opts || {};
|
||||
const now = typeof o.now === 'function' ? o.now : (() => Date.now());
|
||||
const schedule = typeof o.schedule === 'function'
|
||||
? o.schedule
|
||||
: ((cb, ms) => setTimeout(cb, ms));
|
||||
const clear = typeof o.clear === 'function' ? o.clear : ((h) => clearTimeout(h));
|
||||
|
||||
let handle = null;
|
||||
let burstStart = null;
|
||||
let pendingFn = null;
|
||||
|
||||
function fire() {
|
||||
handle = null;
|
||||
burstStart = null;
|
||||
const fn = pendingFn;
|
||||
pendingFn = null;
|
||||
if (typeof fn === 'function') fn();
|
||||
}
|
||||
|
||||
function request(fn, delay, maxWait) {
|
||||
if (typeof fn === 'function') pendingFn = fn;
|
||||
const t = now();
|
||||
if (burstStart === null) burstStart = t;
|
||||
let wait = typeof delay === 'number' && delay >= 0 ? delay : 0;
|
||||
if (typeof maxWait === 'number' && maxWait >= 0) {
|
||||
const remaining = maxWait - (t - burstStart);
|
||||
wait = Math.min(wait, remaining < 0 ? 0 : remaining);
|
||||
}
|
||||
if (handle !== null) clear(handle);
|
||||
handle = schedule(fire, wait);
|
||||
}
|
||||
|
||||
function flushSync() {
|
||||
if (handle !== null) { clear(handle); handle = null; }
|
||||
burstStart = null;
|
||||
const fn = pendingFn;
|
||||
pendingFn = null;
|
||||
if (typeof fn === 'function') fn();
|
||||
}
|
||||
|
||||
function cancel() {
|
||||
if (handle !== null) { clear(handle); handle = null; }
|
||||
burstStart = null;
|
||||
pendingFn = null;
|
||||
}
|
||||
|
||||
function isPending() { return handle !== null; }
|
||||
|
||||
return { request, flushSync, cancel, isPending };
|
||||
}
|
||||
|
||||
const api = { makeThrottleTimer };
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = api;
|
||||
else if (root) root.ThrottleTimer = api;
|
||||
})(typeof window !== 'undefined' ? window : this);
|
||||
@ -37,14 +37,6 @@ function isNewer(remote, current) {
|
||||
return r.patch > c.patch;
|
||||
}
|
||||
|
||||
function resolveReleaseVersion(release) {
|
||||
for (const value of [release && release.name, release && release.tag_name]) {
|
||||
const match = String(value || '').match(/(?:^|[^\d])v?(\d+\.\d+\.\d+)(?=$|[^\d.])/i);
|
||||
if (match) return match[1];
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
function pickSetupAsset(assets) {
|
||||
if (!Array.isArray(assets)) return null;
|
||||
// Prefer asset with "setup" in the name (case-insensitive)
|
||||
@ -98,12 +90,11 @@ async function checkForUpdate() {
|
||||
}
|
||||
|
||||
const release = releases[0];
|
||||
const remoteVersion = resolveReleaseVersion(release);
|
||||
const transportTag = release.tag_name || '';
|
||||
const remoteVersion = release.tag_name || release.name || '';
|
||||
const currentVersion = getCurrentVersion();
|
||||
|
||||
if (!isNewer(remoteVersion, currentVersion)) {
|
||||
cachedCheck = { available: false, currentVersion, remoteVersion, transportTag };
|
||||
cachedCheck = { available: false, currentVersion, remoteVersion };
|
||||
cachedCheckTs = Date.now();
|
||||
return cachedCheck;
|
||||
}
|
||||
@ -118,8 +109,7 @@ async function checkForUpdate() {
|
||||
cachedCheck = {
|
||||
available: true,
|
||||
currentVersion,
|
||||
remoteVersion,
|
||||
transportTag,
|
||||
remoteVersion: remoteVersion.replace(/^v/i, ''),
|
||||
releaseUrl: release.html_url,
|
||||
assetUrl: setupAsset.browser_download_url,
|
||||
assetSize: setupAsset.size,
|
||||
@ -189,27 +179,10 @@ async function installUpdate(onProgress) {
|
||||
let downloadedBytes = 0;
|
||||
const chunks = [];
|
||||
|
||||
const DOWNLOAD_STALL_MS = 45000;
|
||||
let stallTimer = null;
|
||||
const reader = res.body.getReader();
|
||||
while (true) {
|
||||
if (signal.aborted) throw new Error('Abgebrochen');
|
||||
let chunk;
|
||||
try {
|
||||
chunk = await Promise.race([
|
||||
reader.read(),
|
||||
new Promise((_, reject) => { stallTimer = setTimeout(() => reject(new Error('__STALL__')), DOWNLOAD_STALL_MS); })
|
||||
]);
|
||||
} catch (e) {
|
||||
if (e && e.message === '__STALL__') {
|
||||
try { activeAbort.abort(); } catch {}
|
||||
throw new Error('Download hängt — seit 45 s keine Daten (Netzwerk/Server überlastet). Bitte laufende Uploads stoppen und erneut versuchen.');
|
||||
}
|
||||
throw e;
|
||||
} finally {
|
||||
if (stallTimer) { clearTimeout(stallTimer); stallTimer = null; }
|
||||
}
|
||||
const { done, value } = chunk;
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
chunks.push(value);
|
||||
downloadedBytes += value.length;
|
||||
@ -290,4 +263,4 @@ function abortUpdate() {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { checkForUpdate, installUpdate, abortUpdate, isNewer, resolveReleaseVersion };
|
||||
module.exports = { checkForUpdate, installUpdate, abortUpdate };
|
||||
|
||||
@ -1,34 +0,0 @@
|
||||
(function (root) {
|
||||
'use strict';
|
||||
|
||||
function _pad(n) { return String(n).padStart(2, '0'); }
|
||||
|
||||
function formatUploadLogLine(date, hoster, link, fileName) {
|
||||
const d = date instanceof Date ? date : new Date();
|
||||
const dateStr = `${d.getFullYear()}-${_pad(d.getMonth() + 1)}-${_pad(d.getDate())} ` +
|
||||
`${_pad(d.getHours())}:${_pad(d.getMinutes())}:${_pad(d.getSeconds())}`;
|
||||
return `${dateStr}|${hoster}|${link}||${fileName}|\n`;
|
||||
}
|
||||
|
||||
function parseUploadLogLine(line) {
|
||||
if (typeof line !== 'string') return null;
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith('#')) return null;
|
||||
const parts = trimmed.split('|');
|
||||
if (parts.length < 5) return null;
|
||||
const hoster = (parts[1] || '').trim();
|
||||
let fileName = '';
|
||||
for (let i = parts.length - 1; i >= 4; i--) {
|
||||
if (parts[i].trim() !== '') { fileName = parts[i]; break; }
|
||||
}
|
||||
if (!hoster || !fileName) return null;
|
||||
const tsStr = (parts[0] || '').trim();
|
||||
const tsParsed = tsStr ? Date.parse(tsStr.replace(' ', 'T')) : NaN;
|
||||
const ts = isNaN(tsParsed) ? undefined : tsParsed;
|
||||
return { hoster, fileName, ts };
|
||||
}
|
||||
|
||||
const api = { formatUploadLogLine, parseUploadLogLine };
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = api;
|
||||
else if (root) root.UploadLog = api;
|
||||
})(typeof window !== 'undefined' ? window : this);
|
||||
@ -17,16 +17,14 @@ const DEFAULT_SETTINGS = {
|
||||
parallelCount: 2,
|
||||
restartBelowKbs: 0,
|
||||
timeIntervalSec: 0,
|
||||
maxSizeMb: 0,
|
||||
sizeMemoEnabled: true
|
||||
maxSizeMb: 0
|
||||
};
|
||||
|
||||
class UploadManager extends EventEmitter {
|
||||
constructor(hosterSettings, globalSettings, accountPools) {
|
||||
constructor(hosterSettings, globalSettings) {
|
||||
super();
|
||||
this.hosterSettings = hosterSettings || {};
|
||||
this.globalSettings = globalSettings || {};
|
||||
this.accountPools = accountPools || {};
|
||||
this.semaphores = {};
|
||||
this.globalSemaphore = null;
|
||||
this.abortController = new AbortController();
|
||||
@ -34,28 +32,19 @@ class UploadManager extends EventEmitter {
|
||||
this.stopAfterActive = false;
|
||||
this.statsInterval = null;
|
||||
this.startTime = 0;
|
||||
this.activeJobs = new Map(); // uploadId -> { jobId, speedKbs, bytesUploaded, hoster }
|
||||
this.activeJobs = new Map(); // uploadId -> { jobId, speedKbs, bytesUploaded }
|
||||
this.jobAbortControllers = new Map(); // jobId -> AbortController
|
||||
this.cancelledJobIds = new Set();
|
||||
this.sessionBytes = 0;
|
||||
this._transientErrorTotal = 0;
|
||||
this.lastStartTime = {}; // hoster -> timestamp of last upload start
|
||||
this.intervalLocks = {}; // hoster -> Promise chain for serialized interval waits
|
||||
this.globalThrottle = null;
|
||||
this._failedAccounts = new Map(); // hoster -> Set of failed accountIds
|
||||
this._accountOverrides = new Map(); // hoster -> fallback account object
|
||||
this._suspectSizeMemo = new Map(); // 'hoster:accountId' -> { size: smallest suspect-rejected fileSize, count: confirmed rejections }; blocks only after 2nd rejection
|
||||
this._suspectGoodAccounts = new Map(); // hoster -> accountId that accepted a suspect-class file
|
||||
this._doodApiKeyCache = new Map(); // accountId/username -> derived doodstream API key ('' = tried, none)
|
||||
this._baselineCache = new Map(); // hoster:apiKey -> Promise<Set<file_code>> (one fetch shared across all jobs in batch)
|
||||
}
|
||||
|
||||
updateAccountPools(accountPools) {
|
||||
if (accountPools && typeof accountPools === 'object') {
|
||||
this.accountPools = accountPools;
|
||||
}
|
||||
}
|
||||
|
||||
switchAccount(hoster, fallbackAccount) {
|
||||
const prev = this._accountOverrides.get(hoster);
|
||||
this._accountOverrides.set(hoster, fallbackAccount);
|
||||
@ -82,17 +71,6 @@ class UploadManager extends EventEmitter {
|
||||
return this.activeJobs.size;
|
||||
}
|
||||
|
||||
getDiagnostics() {
|
||||
const activeByHoster = {};
|
||||
for (const v of this.activeJobs.values()) {
|
||||
const h = v && v.hoster ? v.hoster : 'unknown';
|
||||
activeByHoster[h] = (activeByHoster[h] || 0) + 1;
|
||||
}
|
||||
let pending = 0;
|
||||
for (const sem of Object.values(this.semaphores)) pending += (sem && sem.pending) || 0;
|
||||
return { activeByHoster, transientErrors: this._transientErrorTotal, pending, active: this.activeJobs.size };
|
||||
}
|
||||
|
||||
clearFailedAccount(hoster, accountId) {
|
||||
return this._failedAccounts.delete(`${hoster}:${accountId}`);
|
||||
}
|
||||
@ -120,20 +98,6 @@ class UploadManager extends EventEmitter {
|
||||
this.emit('rot-log', { ts: Date.now(), event, ...data });
|
||||
}
|
||||
|
||||
_noteSuspectReject(hoster, accountId, fileSize) {
|
||||
if (!accountId || !Number.isFinite(fileSize) || fileSize <= 0) return;
|
||||
const key = hoster + ':' + accountId;
|
||||
const prev = this._suspectSizeMemo.get(key);
|
||||
if (prev === undefined) this._suspectSizeMemo.set(key, { size: fileSize, count: 1 });
|
||||
else this._suspectSizeMemo.set(key, { size: Math.min(prev.size, fileSize), count: prev.count + 1 });
|
||||
}
|
||||
|
||||
_suspectMemoBlocks(hoster, accountId, fileSize) {
|
||||
if (this.hosterSettings[hoster] && this.hosterSettings[hoster].sizeMemoEnabled === false) return false;
|
||||
const memo = this._suspectSizeMemo.get(hoster + ':' + accountId);
|
||||
return !!memo && memo.count >= 2 && fileSize > memo.size;
|
||||
}
|
||||
|
||||
// File-specific rejections from the hoster: the same file will get rejected
|
||||
// on any account, so rotation is pointless. Matches the `err.fileRejected`
|
||||
// flag set by parsers plus known rejection phrases.
|
||||
@ -143,7 +107,6 @@ class UploadManager extends EventEmitter {
|
||||
// which takes priority in _shouldSkipRetryOnAccountError.
|
||||
_isFileRejectedError(err) {
|
||||
if (!err) return false;
|
||||
if (err.transientNetwork === true) return false;
|
||||
if (err.accountError === true) return false; // explicit account-level wins
|
||||
if (err.fileRejected === true) return true;
|
||||
if (!err.message) return false;
|
||||
@ -174,9 +137,7 @@ class UploadManager extends EventEmitter {
|
||||
// out for this file without blacklisting the account, so other jobs in the
|
||||
// batch still get a fresh chance on it.
|
||||
_isTransientNetworkError(err) {
|
||||
if (!err) return false;
|
||||
if (err.transientNetwork === true) return true;
|
||||
if (!err.message) return false;
|
||||
if (!err || !err.message) return false;
|
||||
const m = String(err.message);
|
||||
const TRANSIENT = [
|
||||
/ENOTFOUND/i,
|
||||
@ -192,11 +153,7 @@ class UploadManager extends EventEmitter {
|
||||
/dns (lookup|error|failed)/i,
|
||||
/getaddrinfo/i,
|
||||
/fetch failed/i,
|
||||
/\bconnect (ETIMEDOUT|ECONN)/i,
|
||||
/HTTP 5\d\d\b/i,
|
||||
/Bad Gateway/i,
|
||||
/Service Unavailable/i,
|
||||
/Gateway Time-?out/i
|
||||
/\bconnect (ETIMEDOUT|ECONN)/i
|
||||
];
|
||||
return TRANSIENT.some(p => p.test(m));
|
||||
}
|
||||
@ -207,7 +164,6 @@ class UploadManager extends EventEmitter {
|
||||
// or out of quota.
|
||||
_shouldSkipRetryOnAccountError(err) {
|
||||
if (!err) return false;
|
||||
if (err.transientNetwork === true) return false;
|
||||
// Explicit account-level flag from hoster parsers — highest priority.
|
||||
if (err.accountError === true) return true;
|
||||
if (!err.message) return false;
|
||||
@ -338,8 +294,6 @@ class UploadManager extends EventEmitter {
|
||||
// passes the session-scoped failed/override state.
|
||||
this._failedAccounts.clear();
|
||||
this._accountOverrides.clear();
|
||||
this._suspectSizeMemo.clear();
|
||||
this._suspectGoodAccounts.clear();
|
||||
if (Array.isArray(opts.primeFailedAccounts)) {
|
||||
for (const key of opts.primeFailedAccounts) this._failedAccounts.set(key, true);
|
||||
}
|
||||
@ -364,17 +318,16 @@ class UploadManager extends EventEmitter {
|
||||
for (let i = 0; i < tasks.length; i += DEDUP_CHUNK) {
|
||||
if (signal.aborted) break;
|
||||
const end = Math.min(i + DEDUP_CHUNK, tasks.length);
|
||||
const toStat = [];
|
||||
for (let j = i; j < end; j++) {
|
||||
const task = tasks[j];
|
||||
if (!results.has(task.file)) {
|
||||
results.set(task.file, { name: path.basename(task.file), size: 0, results: [] });
|
||||
toStat.push(task.file);
|
||||
const fileName = path.basename(task.file);
|
||||
let size = 0;
|
||||
try { size = fs.statSync(task.file).size; } catch {}
|
||||
results.set(task.file, { name: fileName, size, results: [] });
|
||||
}
|
||||
}
|
||||
await Promise.all(toStat.map(async (f) => {
|
||||
try { const st = await fs.promises.stat(f); const e = results.get(f); if (e) e.size = st.size; } catch {}
|
||||
}));
|
||||
if (end < tasks.length) await new Promise(setImmediate);
|
||||
}
|
||||
|
||||
this._startStatsTimer();
|
||||
@ -426,7 +379,7 @@ class UploadManager extends EventEmitter {
|
||||
if (cachedResult && typeof cachedResult.size === 'number' && cachedResult.size > 0) {
|
||||
fileSize = cachedResult.size;
|
||||
} else {
|
||||
try { fileSize = (await fs.promises.stat(task.file)).size; } catch { fileNotFound = true; }
|
||||
try { fileSize = fs.statSync(task.file).size; } catch { fileNotFound = true; }
|
||||
}
|
||||
|
||||
const maxAttempts = Math.max(1, (settings.retries || 0) + 1);
|
||||
@ -504,7 +457,7 @@ class UploadManager extends EventEmitter {
|
||||
|
||||
let fileProbe = null;
|
||||
try {
|
||||
fileProbe = await probeFileHead(task.file, 512);
|
||||
fileProbe = await probeFileHead(task.file, 64);
|
||||
} catch (err) {
|
||||
fileProbe = { ok: false, error: err && err.message, kind: 'unreadable' };
|
||||
}
|
||||
@ -549,22 +502,7 @@ class UploadManager extends EventEmitter {
|
||||
}
|
||||
}
|
||||
|
||||
// A previous file of at least this size already got a suspect rejection
|
||||
// on this exact account — skip the guaranteed-to-fail multi-GB upload
|
||||
// and go straight to the alternate-account walk below.
|
||||
let memoSuspect = null;
|
||||
if (fileProbe && fileProbe.isVideoLike === true && task.accountId
|
||||
&& this._suspectMemoBlocks(task.hoster, task.accountId, fileSize)) {
|
||||
memoSuspect = new Error('Bekanntes Größen-Limit auf diesem Account (frühere verdächtige Ablehnung)');
|
||||
memoSuspect.fileRejected = true;
|
||||
memoSuspect.suspectReject = true;
|
||||
lastError = memoSuspect;
|
||||
this._rotLog('suspect-memo-skip', {
|
||||
jobId, hoster: task.hoster, fileName, accountId: task.accountId, fileSize
|
||||
});
|
||||
}
|
||||
|
||||
for (let attempt = 1; attempt <= maxAttempts && !memoSuspect; attempt++) {
|
||||
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||
if (signal.aborted || this.stopAfterActive) break;
|
||||
|
||||
if (attempt > 1) {
|
||||
@ -638,7 +576,7 @@ class UploadManager extends EventEmitter {
|
||||
// Mutate this single object on each progress callback instead of
|
||||
// allocating a fresh one — callback fires on every stream chunk
|
||||
// (hundreds/sec per active job).
|
||||
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
|
||||
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0 };
|
||||
this.activeJobs.set(uploadId, activeEntry);
|
||||
|
||||
let lastEmitTime = 0;
|
||||
@ -683,7 +621,7 @@ class UploadManager extends EventEmitter {
|
||||
} catch { /* progress callbacks must never throw — swallowing is correct, the stream keeps going */ }
|
||||
};
|
||||
|
||||
const result = await this._executeUpload(task, progressCb, uploadSignalBundle.signal, throttle, fileProbe);
|
||||
const result = await this._executeUpload(task, progressCb, uploadSignalBundle.signal, throttle);
|
||||
|
||||
const elapsed = Math.round((Date.now() - jobStart) / 1000);
|
||||
this.sessionBytes += fileSize;
|
||||
@ -699,7 +637,6 @@ class UploadManager extends EventEmitter {
|
||||
return;
|
||||
} catch (err) {
|
||||
this.activeJobs.delete(uploadId);
|
||||
if (this._isTransientNetworkError(err)) this._transientErrorTotal++;
|
||||
|
||||
const isSpeedRestart = speedAbort && speedAbort.signal.aborted && !signal.aborted;
|
||||
if (!signal.aborted && !isSpeedRestart) {
|
||||
@ -806,30 +743,7 @@ class UploadManager extends EventEmitter {
|
||||
// File-specific rejection → same file will get the same verdict on
|
||||
// every other account, rotation is pointless. Don't blacklist, don't
|
||||
// retry siblings, just fail this file cleanly.
|
||||
//
|
||||
// EXCEPT suspect rejections (err.suspectReject, e.g. byse "Not video
|
||||
// file format" on a probe-verified video): those verdicts are
|
||||
// account-conditional in practice (per-account size tiers), so the file
|
||||
// gets one attempt on each remaining account — WITHOUT blacklisting the
|
||||
// current one, which keeps working for files the hoster does accept.
|
||||
if (this._isFileRejectedError(lastError)) {
|
||||
if (lastError.suspectReject === true && fileProbe && fileProbe.isVideoLike === true) {
|
||||
this._noteSuspectReject(task.hoster, task.accountId, fileSize);
|
||||
const alt = await this._trySuspectRejectAlternates(task, { uploadId, jobId, fileName, fileSize, settings, signal, fileProbe });
|
||||
if (alt) {
|
||||
emitFinalStatus('done', { result: alt.result, speedKbs: alt.speedKbs, elapsed: alt.elapsed, attempt: 1 });
|
||||
recordFinalResult('done', { result: alt.result });
|
||||
return;
|
||||
}
|
||||
const stoppedInAlternates = this.stopAfterActive && !signal.aborted;
|
||||
const abortedInAlternates = signal.aborted || this.cancelledJobIds.has(jobId);
|
||||
if (stoppedInAlternates || abortedInAlternates) {
|
||||
const error = stoppedInAlternates ? 'Warteschlange angehalten' : 'Abgebrochen';
|
||||
emitFinalStatus('aborted', { error });
|
||||
recordFinalResult('aborted', { error });
|
||||
return;
|
||||
}
|
||||
}
|
||||
this._rotLog('skip-rotation-file-rejected', {
|
||||
jobId, hoster: task.hoster, fileName, accountId: task.accountId,
|
||||
lastError: lastError ? lastError.message : null
|
||||
@ -868,17 +782,6 @@ class UploadManager extends EventEmitter {
|
||||
}
|
||||
while (task.accountId) {
|
||||
if (signal.aborted || this.stopAfterActive) break;
|
||||
// The rotated-to account failed with a file-class error (file
|
||||
// rejection / hoster flake / network) — blacklisting it for that
|
||||
// would poison a working account for the whole batch. Fail only
|
||||
// this file instead.
|
||||
if (this._isFileRejectedError(lastError) || this._isHosterTransientError(lastError) || this._isTransientNetworkError(lastError)) {
|
||||
this._rotLog('skip-rotation-after-rotate', {
|
||||
jobId, hoster: task.hoster, fileName, accountId: task.accountId,
|
||||
lastError: lastError ? lastError.message : null
|
||||
});
|
||||
break;
|
||||
}
|
||||
const alreadyMarked = this._failedAccounts.has(task.hoster + ':' + task.accountId);
|
||||
if (!alreadyMarked) {
|
||||
this._failedAccounts.set(task.hoster + ':' + task.accountId, true);
|
||||
@ -952,11 +855,9 @@ class UploadManager extends EventEmitter {
|
||||
let lastBytes = 0;
|
||||
let lastSpeedTime = jobStart;
|
||||
let currentSpeedKbs = 0;
|
||||
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
|
||||
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0 };
|
||||
this.activeJobs.set(uploadId, activeEntry);
|
||||
|
||||
let lastEmitTime = 0;
|
||||
const PROGRESS_EMIT_INTERVAL = 250;
|
||||
const progressCb = (bytesUploaded, bytesTotal) => {
|
||||
const now = Date.now();
|
||||
const timeDelta = (now - lastSpeedTime) / 1000;
|
||||
@ -967,8 +868,6 @@ class UploadManager extends EventEmitter {
|
||||
}
|
||||
activeEntry.speedKbs = currentSpeedKbs;
|
||||
activeEntry.bytesUploaded = bytesUploaded;
|
||||
if (now - lastEmitTime < PROGRESS_EMIT_INTERVAL) return;
|
||||
lastEmitTime = now;
|
||||
const elapsed = Math.round((now - jobStart) / 1000);
|
||||
const remaining = currentSpeedKbs > 0 ? Math.round((bytesTotal - bytesUploaded) / (currentSpeedKbs * 1024)) : 0;
|
||||
this._emitProgress(uploadId, fileName, task.hoster, { accountId: task.accountId,
|
||||
@ -985,7 +884,7 @@ class UploadManager extends EventEmitter {
|
||||
? { consume: async (bytes, sig) => { await hosterThrottle.consume(bytes, sig); await globalThrottle.consume(bytes, sig); } }
|
||||
: hosterThrottle || globalThrottle;
|
||||
|
||||
const result = await this._executeUpload(task, progressCb, signal, throttle, fileProbe);
|
||||
const result = await this._executeUpload(task, progressCb, signal, throttle);
|
||||
this.activeJobs.delete(uploadId);
|
||||
this.sessionBytes += fileSize;
|
||||
emitFinalStatus('done', { result, speedKbs: currentSpeedKbs, elapsed: Math.round((Date.now() - jobStart) / 1000), attempt });
|
||||
@ -994,34 +893,12 @@ class UploadManager extends EventEmitter {
|
||||
} catch (err) {
|
||||
this.activeJobs.delete(uploadId);
|
||||
lastError = err;
|
||||
if (!signal.aborted) {
|
||||
this._rotLog('upload-failure', {
|
||||
jobId, hoster: task.hoster, accountId: task.accountId, fileName,
|
||||
attempt,
|
||||
error: err && err.message ? err.message : String(err),
|
||||
fileRejected: !!(err && err.fileRejected),
|
||||
accountError: !!(err && err.accountError),
|
||||
hosterTransient: !!(err && err.hosterTransient),
|
||||
rotationRetry: true
|
||||
});
|
||||
}
|
||||
if (signal.aborted || this.stopAfterActive) break;
|
||||
if (this._isFileRejectedError(err)) break;
|
||||
if (this._isHosterTransientError(err)) break;
|
||||
if (this._isTransientNetworkError(err)) break;
|
||||
if (attempt >= maxAttempts) break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const stoppedLate = this.stopAfterActive && !signal.aborted;
|
||||
const abortedLate = signal.aborted || this.cancelledJobIds.has(jobId);
|
||||
if (stoppedLate || abortedLate) {
|
||||
const error = stoppedLate ? 'Warteschlange angehalten' : 'Abgebrochen';
|
||||
emitFinalStatus('aborted', { error });
|
||||
recordFinalResult('aborted', { error });
|
||||
return;
|
||||
}
|
||||
const error = lastError && lastError.message ? lastError.message : 'Unbekannter Fehler';
|
||||
this._rotLog('final-error', {
|
||||
jobId, hoster: task.hoster, fileName, lastFailedAccountId: task.accountId, error
|
||||
@ -1046,130 +923,7 @@ class UploadManager extends EventEmitter {
|
||||
}
|
||||
}
|
||||
|
||||
async _trySuspectRejectAlternates(task, ctx) {
|
||||
const { uploadId, jobId, fileName, fileSize, settings, signal, fileProbe } = ctx;
|
||||
const pool = this.accountPools && Array.isArray(this.accountPools[task.hoster])
|
||||
? this.accountPools[task.hoster]
|
||||
: [];
|
||||
const original = { accountId: task.accountId, username: task.username, password: task.password, apiKey: task.apiKey };
|
||||
const goodId = this._suspectGoodAccounts.get(task.hoster);
|
||||
const ordered = [];
|
||||
for (const account of pool) {
|
||||
if (account && account.id === goodId) ordered.unshift(account);
|
||||
else ordered.push(account);
|
||||
}
|
||||
const tried = new Set([task.accountId]);
|
||||
let attempted = 0;
|
||||
for (const account of ordered) {
|
||||
if (signal.aborted || this.stopAfterActive) break;
|
||||
if (!account || !account.id || tried.has(account.id)) continue;
|
||||
if (this._failedAccounts.has(task.hoster + ':' + account.id)) continue;
|
||||
tried.add(account.id);
|
||||
if (this._suspectMemoBlocks(task.hoster, account.id, fileSize)) {
|
||||
this._rotLog('suspect-memo-skip-alt', {
|
||||
jobId, hoster: task.hoster, fileName, accountId: account.id, fileSize
|
||||
});
|
||||
continue;
|
||||
}
|
||||
attempted += 1;
|
||||
this._rotLog('suspect-reject-alt', {
|
||||
jobId, hoster: task.hoster, fileName, fromAccountId: task.accountId, toAccountId: account.id
|
||||
});
|
||||
task.accountId = account.id;
|
||||
task.username = account.username;
|
||||
task.password = account.password;
|
||||
task.apiKey = account.apiKey;
|
||||
this._emitProgress(uploadId, fileName, task.hoster, { accountId: task.accountId,
|
||||
jobId, status: 'retrying', progress: 0, bytesUploaded: 0, bytesTotal: fileSize,
|
||||
speedKbs: 0, elapsed: 0, remaining: 0,
|
||||
error: 'Ablehnung verdächtig - Versuch auf anderem Account', result: null, attempt: 1, maxAttempts: 1
|
||||
});
|
||||
const jobStart = Date.now();
|
||||
let lastBytes = 0;
|
||||
let lastSpeedTime = jobStart;
|
||||
let currentSpeedKbs = 0;
|
||||
const activeEntry = { jobId, speedKbs: 0, bytesUploaded: 0, hoster: task.hoster };
|
||||
this.activeJobs.set(uploadId, activeEntry);
|
||||
let lastEmitTime = 0;
|
||||
const PROGRESS_EMIT_INTERVAL = 250;
|
||||
const progressCb = (bytesUploaded, bytesTotal) => {
|
||||
const now = Date.now();
|
||||
const timeDelta = (now - lastSpeedTime) / 1000;
|
||||
if (timeDelta >= 1) {
|
||||
currentSpeedKbs = Math.round((bytesUploaded - lastBytes) / timeDelta / 1024);
|
||||
lastBytes = bytesUploaded;
|
||||
lastSpeedTime = now;
|
||||
}
|
||||
activeEntry.speedKbs = currentSpeedKbs;
|
||||
activeEntry.bytesUploaded = bytesUploaded;
|
||||
if (now - lastEmitTime < PROGRESS_EMIT_INTERVAL) return;
|
||||
lastEmitTime = now;
|
||||
const elapsed = Math.round((now - jobStart) / 1000);
|
||||
const remaining = currentSpeedKbs > 0 ? Math.round((bytesTotal - bytesUploaded) / (currentSpeedKbs * 1024)) : 0;
|
||||
this._emitProgress(uploadId, fileName, task.hoster, { accountId: task.accountId,
|
||||
jobId, status: 'uploading',
|
||||
progress: bytesTotal > 0 ? Math.min(1, bytesUploaded / bytesTotal) : 0,
|
||||
bytesUploaded, bytesTotal, speedKbs: currentSpeedKbs,
|
||||
elapsed, remaining, error: null, result: null, attempt: 1, maxAttempts: 1
|
||||
});
|
||||
};
|
||||
const hosterThrottle = settings.maxSpeedKbs > 0 ? new Throttle(settings.maxSpeedKbs * 1024) : null;
|
||||
const globalThrottle = this._getGlobalThrottle();
|
||||
const throttle = hosterThrottle && globalThrottle
|
||||
? { consume: async (bytes, sig) => { await hosterThrottle.consume(bytes, sig); await globalThrottle.consume(bytes, sig); } }
|
||||
: hosterThrottle || globalThrottle;
|
||||
try {
|
||||
const result = await this._executeUpload(task, progressCb, signal, throttle, fileProbe);
|
||||
this.activeJobs.delete(uploadId);
|
||||
this.sessionBytes += fileSize;
|
||||
this._suspectGoodAccounts.set(task.hoster, account.id);
|
||||
return { result, speedKbs: currentSpeedKbs, elapsed: Math.round((Date.now() - jobStart) / 1000) };
|
||||
} catch (err) {
|
||||
this.activeJobs.delete(uploadId);
|
||||
if (!signal.aborted) {
|
||||
this._rotLog('upload-failure', {
|
||||
jobId, hoster: task.hoster, accountId: task.accountId, fileName,
|
||||
attempt: 1,
|
||||
error: err && err.message ? err.message : String(err),
|
||||
fileRejected: !!(err && err.fileRejected),
|
||||
accountError: !!(err && err.accountError),
|
||||
hosterTransient: !!(err && err.hosterTransient),
|
||||
suspectAlternate: true
|
||||
});
|
||||
}
|
||||
if (signal.aborted || this.stopAfterActive) break;
|
||||
if (err && err.suspectReject === true) {
|
||||
this._noteSuspectReject(task.hoster, account.id, fileSize);
|
||||
}
|
||||
// A genuine account-class error (quota, ban, full disk) positively
|
||||
// identifies a dead account — remember it so parallel and later
|
||||
// suspect jobs stop re-uploading multi-GB files to it. Deliberately
|
||||
// no 'account-failed' emit: that would re-point the hoster-wide
|
||||
// override and reroute normal-sized files away from a primary that
|
||||
// still works for them.
|
||||
if (err && err.accountError === true) {
|
||||
this._failedAccounts.set(task.hoster + ':' + account.id, true);
|
||||
this._rotLog('mark-failed', {
|
||||
jobId, hoster: task.hoster, fileName, accountId: account.id,
|
||||
lastError: err && err.message ? err.message : String(err),
|
||||
suspectAlternate: true
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
task.accountId = original.accountId;
|
||||
task.username = original.username;
|
||||
task.password = original.password;
|
||||
task.apiKey = original.apiKey;
|
||||
if (!signal.aborted && !this.stopAfterActive) {
|
||||
this._rotLog('suspect-reject-exhausted', {
|
||||
jobId, hoster: task.hoster, fileName, alternatesTried: attempted
|
||||
});
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
async _executeUpload(task, progressCb, signal, throttle, fileProbe) {
|
||||
async _executeUpload(task, progressCb, signal, throttle) {
|
||||
if (task.hoster === 'vidmoly.me' && task.username) {
|
||||
const vidmoly = new VidmolyUploader();
|
||||
await vidmoly.login(task.username, task.password);
|
||||
@ -1201,10 +955,7 @@ class UploadManager extends EventEmitter {
|
||||
return clouddrop.upload(task.file, progressCb, signal, throttle);
|
||||
} else {
|
||||
const baselineOpts = {};
|
||||
if (task.hoster === 'byse.sx') {
|
||||
baselineOpts.byseBaseline = await this._getBaseline('byse.sx', task.apiKey, signal);
|
||||
if (fileProbe && fileProbe.ok !== false) baselineOpts.probeIsVideoLike = fileProbe.isVideoLike === true;
|
||||
}
|
||||
if (task.hoster === 'byse.sx') baselineOpts.byseBaseline = await this._getBaseline('byse.sx', task.apiKey, signal);
|
||||
if (task.hoster === 'doodstream.com') baselineOpts.doodBaseline = await this._getBaseline('doodstream.com', task.apiKey, signal);
|
||||
return uploadFile(task.hoster, task.file, task.apiKey, progressCb, signal, throttle, baselineOpts);
|
||||
}
|
||||
|
||||
@ -187,7 +187,7 @@ class VidmolyUploader {
|
||||
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
|
||||
|
||||
let bytesRead = 0;
|
||||
const CHUNK_SIZE = 1024 * 1024;
|
||||
const CHUNK_SIZE = 256 * 1024;
|
||||
|
||||
async function* generate() {
|
||||
yield preambleBuf;
|
||||
|
||||
@ -242,7 +242,7 @@ class VoeUploader {
|
||||
const totalSize = preambleBuf.length + fileSize + epilogueBuf.length;
|
||||
|
||||
let bytesRead = 0;
|
||||
const CHUNK_SIZE = 1024 * 1024;
|
||||
const CHUNK_SIZE = 256 * 1024;
|
||||
|
||||
async function* generate() {
|
||||
yield preambleBuf;
|
||||
|
||||
@ -1,123 +0,0 @@
|
||||
function isDiscordWebhook(url) {
|
||||
return /^https?:\/\/(ptb\.|canary\.)?(discord(app)?\.com)\/api\/webhooks\/\d+\/[\w-]+/i.test(String(url || ''));
|
||||
}
|
||||
|
||||
const DISCORD_CONTENT_LIMIT = 1900;
|
||||
|
||||
function clampDiscordContent(text) {
|
||||
const s = String(text || '');
|
||||
if (s.length <= DISCORD_CONTENT_LIMIT) return s;
|
||||
return s.slice(0, DISCORD_CONTENT_LIMIT - 1) + '…';
|
||||
}
|
||||
|
||||
function formatDurationShort(sec) {
|
||||
const s = Math.max(0, Math.round(Number(sec) || 0));
|
||||
const h = Math.floor(s / 3600);
|
||||
const m = Math.floor((s % 3600) / 60);
|
||||
const r = s % 60;
|
||||
if (h > 0) return `${h}h ${m}m`;
|
||||
if (m > 0) return `${m}m ${r}s`;
|
||||
return `${r}s`;
|
||||
}
|
||||
|
||||
function summarizePerHosterFromBatch(summary) {
|
||||
const out = {};
|
||||
if (!summary || !Array.isArray(summary.files)) return out;
|
||||
for (const f of summary.files) {
|
||||
if (!f || !Array.isArray(f.results)) continue;
|
||||
for (const r of f.results) {
|
||||
if (!r || !r.hoster) continue;
|
||||
const b = out[r.hoster] || (out[r.hoster] = { ok: 0, fail: 0 });
|
||||
if (r.status === 'done') b.ok++;
|
||||
else b.fail++;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function resolveDiscordMention(raw) {
|
||||
const s = String(raw || '').trim();
|
||||
if (!s) return null;
|
||||
const keyword = s.replace(/^@/, '').toLowerCase();
|
||||
if (keyword === 'here' || keyword === 'everyone') {
|
||||
return { token: `@${keyword}`, allowed: { parse: ['everyone'] } };
|
||||
}
|
||||
const roleMatch = s.match(/^(?:<@&(\d+)>|role:(\d+))$/i);
|
||||
if (roleMatch) {
|
||||
const id = roleMatch[1] || roleMatch[2];
|
||||
return { token: `<@&${id}>`, allowed: { roles: [id] } };
|
||||
}
|
||||
const userMatch = s.match(/^(?:<@!?(\d+)>|user:(\d+)|(\d{5,30}))$/i);
|
||||
if (userMatch) {
|
||||
const id = userMatch[1] || userMatch[2] || userMatch[3];
|
||||
return { token: `<@${id}>`, allowed: { users: [id] } };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function buildWebhookRequest(url, summary, meta) {
|
||||
const m = meta || {};
|
||||
const total = Number(summary && summary.total) || 0;
|
||||
const succeeded = Number(summary && summary.succeeded) || 0;
|
||||
const failed = Number(summary && summary.failed) || 0;
|
||||
const perHoster = summarizePerHosterFromBatch(summary);
|
||||
const duration = formatDurationShort(m.durationSec);
|
||||
|
||||
let body;
|
||||
if (isDiscordWebhook(url)) {
|
||||
const headline = m.aborted ? 'Batch abgebrochen' : 'Batch fertig';
|
||||
const hosterEntries = Object.entries(perHoster);
|
||||
const MAX_HOSTER_LINES = 12;
|
||||
let hosterLines = hosterEntries.slice(0, MAX_HOSTER_LINES)
|
||||
.map(([h, b]) => `${h}: ${b.ok}/${b.ok + b.fail}`)
|
||||
.join(' · ');
|
||||
if (hosterEntries.length > MAX_HOSTER_LINES) hosterLines += ` · …+${hosterEntries.length - MAX_HOSTER_LINES}`;
|
||||
const lines = [
|
||||
`**Multi-Hoster-Upload — ${headline}**${m.machineName ? ` (${m.machineName})` : ''}`,
|
||||
`✅ ${succeeded} ok · ❌ ${failed} Fehler · 📦 ${total} gesamt · ⏱ ${duration}`
|
||||
];
|
||||
if (hosterLines) lines.push(hosterLines);
|
||||
const mention = resolveDiscordMention(m.mention);
|
||||
const content = clampDiscordContent((mention ? mention.token + ' ' : '') + lines.join('\n'));
|
||||
const payload = { content };
|
||||
payload.allowed_mentions = mention ? mention.allowed : { parse: [] };
|
||||
body = JSON.stringify(payload);
|
||||
} else {
|
||||
body = JSON.stringify({
|
||||
event: 'batch-done',
|
||||
app: 'multi-hoster-upload',
|
||||
version: m.appVersion || null,
|
||||
machine: m.machineName || null,
|
||||
total,
|
||||
succeeded,
|
||||
failed,
|
||||
durationSec: Math.round(Number(m.durationSec) || 0),
|
||||
aborted: !!m.aborted,
|
||||
perHoster,
|
||||
timestamp: m.timestamp || null
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
url: String(url),
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body
|
||||
};
|
||||
}
|
||||
|
||||
function isAllAborted(summary) {
|
||||
if (!summary || !Array.isArray(summary.files) || summary.files.length === 0) return false;
|
||||
let sawResult = false;
|
||||
for (const f of summary.files) {
|
||||
if (!f || !Array.isArray(f.results)) continue;
|
||||
for (const r of f.results) {
|
||||
if (!r) continue;
|
||||
sawResult = true;
|
||||
if (r.status !== 'aborted') return false;
|
||||
}
|
||||
}
|
||||
return sawResult;
|
||||
}
|
||||
|
||||
module.exports = { isDiscordWebhook, formatDurationShort, summarizePerHosterFromBatch, buildWebhookRequest, resolveDiscordMention, isAllAborted, clampDiscordContent, DISCORD_CONTENT_LIMIT };
|
||||
20
package-lock.json
generated
20
package-lock.json
generated
@ -1,16 +1,16 @@
|
||||
{
|
||||
"name": "multi-hoster-uploader",
|
||||
"version": "2.0.1",
|
||||
"version": "3.3.16",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "multi-hoster-uploader",
|
||||
"version": "2.0.1",
|
||||
"version": "3.3.16",
|
||||
"dependencies": {
|
||||
"chokidar": "^3.6.0",
|
||||
"undici": "^7.29.0",
|
||||
"ws": "^8.21.0"
|
||||
"undici": "^7.16.0",
|
||||
"ws": "^8.19.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"electron": "^41.3.0",
|
||||
@ -4734,9 +4734,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/undici": {
|
||||
"version": "7.29.0",
|
||||
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
|
||||
"integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==",
|
||||
"version": "7.25.0",
|
||||
"resolved": "https://registry.npmjs.org/undici/-/undici-7.25.0.tgz",
|
||||
"integrity": "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=20.18.1"
|
||||
@ -4844,9 +4844,9 @@
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/ws": {
|
||||
"version": "8.21.0",
|
||||
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz",
|
||||
"integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==",
|
||||
"version": "8.20.0",
|
||||
"resolved": "https://registry.npmjs.org/ws/-/ws-8.20.0.tgz",
|
||||
"integrity": "sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10.0.0"
|
||||
|
||||
10
package.json
10
package.json
@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "multi-hoster-uploader",
|
||||
"version": "2.0.1",
|
||||
"version": "3.3.58",
|
||||
"description": "Upload files to doodstream, voe, vidmoly, byse simultaneously",
|
||||
"main": "main.js",
|
||||
"scripts": {
|
||||
@ -12,8 +12,8 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"chokidar": "^3.6.0",
|
||||
"undici": "^7.29.0",
|
||||
"ws": "^8.21.0"
|
||||
"undici": "^7.16.0",
|
||||
"ws": "^8.19.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"electron": "^41.3.0",
|
||||
@ -33,9 +33,7 @@
|
||||
"main.js",
|
||||
"preload.js",
|
||||
"lib/**/*",
|
||||
"renderer/**/*",
|
||||
"assets/app_icon.ico",
|
||||
"assets/app_icon.png"
|
||||
"renderer/**/*"
|
||||
],
|
||||
"win": {
|
||||
"target": [
|
||||
|
||||
10
preload.js
10
preload.js
@ -6,7 +6,6 @@ contextBridge.exposeInMainWorld('api', {
|
||||
saveConfig: (config) => ipcRenderer.invoke('save-config', config),
|
||||
getHistory: () => ipcRenderer.invoke('get-history'),
|
||||
clearHistory: () => ipcRenderer.invoke('clear-history'),
|
||||
pruneHistory: (retention, opts) => ipcRenderer.invoke('prune-history', { retention, dryRun: !!(opts && opts.dryRun) }),
|
||||
exportHistory: (format) => ipcRenderer.invoke('export-history', format),
|
||||
saveTextFile: (defaultName, content, filters) => ipcRenderer.invoke('save-text-file', defaultName, content, filters),
|
||||
|
||||
@ -56,8 +55,6 @@ contextBridge.exposeInMainWorld('api', {
|
||||
installUpdate: () => ipcRenderer.invoke('app:install-update'),
|
||||
abortUpdate: () => ipcRenderer.invoke('app:abort-update'),
|
||||
getVersion: () => ipcRenderer.invoke('app:get-version'),
|
||||
restartApp: () => ipcRenderer.invoke('app:restart'),
|
||||
quitApp: () => ipcRenderer.invoke('app:quit'),
|
||||
onUpdateAvailable: (callback) => {
|
||||
ipcRenderer.on('app:update-available', (_event, data) => callback(data));
|
||||
},
|
||||
@ -122,7 +119,6 @@ contextBridge.exposeInMainWorld('api', {
|
||||
resetSessionFailedAccount: (payload) => ipcRenderer.invoke('reset-session-failed-account', payload),
|
||||
resetAllSessionFailedAccounts: () => ipcRenderer.invoke('reset-all-session-failed-accounts'),
|
||||
getLogPaths: () => ipcRenderer.invoke('get-log-paths'),
|
||||
testWebhook: (payload) => ipcRenderer.invoke('test-webhook', payload),
|
||||
revealLogFile: (target) => ipcRenderer.invoke('reveal-log-file', target),
|
||||
setLogVerbose: (enabled) => ipcRenderer.invoke('set-log-verbose', enabled),
|
||||
createSupportBundle: () => ipcRenderer.invoke('create-support-bundle'),
|
||||
@ -139,12 +135,6 @@ contextBridge.exposeInMainWorld('api', {
|
||||
ipcRenderer.on('remote:client-count', (_event, count) => callback(count));
|
||||
},
|
||||
|
||||
// Remote Diagnostics (read-only)
|
||||
diagnosticsGetSettings: () => ipcRenderer.invoke('diagnostics:get-settings'),
|
||||
diagnosticsSaveSettings: (settings) => ipcRenderer.invoke('diagnostics:save-settings', settings),
|
||||
diagnosticsRegenerate: () => ipcRenderer.invoke('diagnostics:regenerate'),
|
||||
diagnosticsStatus: () => ipcRenderer.invoke('diagnostics:status'),
|
||||
|
||||
// File path from drag & drop (Electron 33+ compatible)
|
||||
getPathForFile: (file) => webUtils.getPathForFile(file),
|
||||
removeAllListeners: () => {
|
||||
|
||||
@ -1,73 +0,0 @@
|
||||
(function (scope) {
|
||||
function getAccountSubmitLabel({ isEdit } = {}) {
|
||||
return isEdit ? 'Prüfen und speichern' : 'Prüfen und anlegen';
|
||||
}
|
||||
|
||||
async function submitValidatedAccount({ validate, commit, afterCommit, isCurrent }) {
|
||||
let validation;
|
||||
try {
|
||||
validation = await validate();
|
||||
} catch (error) {
|
||||
return { status: 'error', error };
|
||||
}
|
||||
|
||||
try {
|
||||
if (!isCurrent()) return { status: 'stale', validation };
|
||||
} catch (error) {
|
||||
return { status: 'error', error, validation };
|
||||
}
|
||||
if (validation && validation.status === 'otp_required') {
|
||||
return { status: 'otp_required', validation };
|
||||
}
|
||||
if (!validation || (validation.status !== 'ok' && validation.status !== 'warn')) {
|
||||
return { status: 'rejected', validation };
|
||||
}
|
||||
|
||||
let value;
|
||||
try {
|
||||
value = await commit(validation);
|
||||
} catch (error) {
|
||||
return { status: 'error', error, validation };
|
||||
}
|
||||
|
||||
let postCommitError;
|
||||
if (typeof afterCommit === 'function') {
|
||||
try {
|
||||
await afterCommit(value, validation);
|
||||
} catch (error) {
|
||||
postCommitError = error;
|
||||
}
|
||||
}
|
||||
|
||||
const committedResult = { status: 'committed', committed: true, validation, value };
|
||||
if (postCommitError) committedResult.postCommitError = postCommitError;
|
||||
try {
|
||||
if (!isCurrent()) return { ...committedResult, status: 'stale' };
|
||||
} catch {
|
||||
return { ...committedResult, status: 'stale' };
|
||||
}
|
||||
return committedResult;
|
||||
}
|
||||
|
||||
function createAccountSubmitter() {
|
||||
let pending = null;
|
||||
return {
|
||||
isBusy() {
|
||||
return pending !== null;
|
||||
},
|
||||
submit(options) {
|
||||
if (pending) return null;
|
||||
const operation = submitValidatedAccount(options);
|
||||
const tracked = operation.finally(() => {
|
||||
if (pending === tracked) pending = null;
|
||||
});
|
||||
pending = tracked;
|
||||
return tracked;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const accountSubmit = { createAccountSubmitter, getAccountSubmitLabel, submitValidatedAccount };
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = accountSubmit;
|
||||
if (scope) scope.AccountSubmit = accountSubmit;
|
||||
})(typeof window !== 'undefined' ? window : globalThis);
|
||||
1745
renderer/app.js
1745
renderer/app.js
File diff suppressed because it is too large
Load Diff
@ -7,65 +7,6 @@
|
||||
<link rel="stylesheet" href="styles.css">
|
||||
</head>
|
||||
<body>
|
||||
<nav class="menu-bar" id="menuBar">
|
||||
<div class="menu-bar-item" data-menu="datei">
|
||||
<button class="menu-bar-trigger" data-menu-trigger="datei">Datei</button>
|
||||
<div class="menu-dropdown" data-menu-dropdown="datei" style="display:none">
|
||||
<button class="menu-dropdown-item" data-menu-action="add-files"><span>Dateien hinzufügen</span></button>
|
||||
<button class="menu-dropdown-item" data-menu-action="add-folder"><span>Ordner hinzufügen</span></button>
|
||||
<div class="menu-separator"></div>
|
||||
<div class="menu-submenu" data-submenu="sicherung">
|
||||
<button class="menu-submenu-trigger">Sicherung</button>
|
||||
<div class="menu-submenu-dropdown" style="display:none">
|
||||
<button class="menu-dropdown-item" data-menu-action="backup-export"><span>Exportieren</span></button>
|
||||
<button class="menu-dropdown-item" data-menu-action="backup-import"><span>Importieren</span></button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="menu-separator"></div>
|
||||
<button class="menu-dropdown-item" data-menu-action="restart"><span>Neustart</span></button>
|
||||
<button class="menu-dropdown-item" data-menu-action="quit"><span>Beenden</span></button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="menu-bar-item" data-menu="einstellungen">
|
||||
<button class="menu-bar-trigger" data-menu-trigger="einstellungen">Einstellungen</button>
|
||||
<div class="menu-dropdown" data-menu-dropdown="einstellungen" style="display:none">
|
||||
<button class="menu-dropdown-item" data-menu-action="open-settings"><span>Einstellungen öffnen</span></button>
|
||||
<div class="menu-separator"></div>
|
||||
<div class="menu-settings-grid" id="menuSettingsGrid">
|
||||
<span>Max. parallele Uploads</span>
|
||||
<span></span>
|
||||
<div class="menu-spinner">
|
||||
<input type="text" inputmode="numeric" id="menuParallelInput">
|
||||
<div class="menu-spinner-arrows">
|
||||
<button data-spin="parallel-up">▲</button>
|
||||
<button data-spin="parallel-down">▼</button>
|
||||
</div>
|
||||
</div>
|
||||
<span></span>
|
||||
<span>Geschwindigkeitslimit</span>
|
||||
<input type="checkbox" id="menuSpeedLimitCheck">
|
||||
<div class="menu-spinner" id="menuSpeedSpinner">
|
||||
<input type="text" inputmode="decimal" id="menuSpeedInput">
|
||||
<div class="menu-spinner-arrows">
|
||||
<button data-spin="speed-up">▲</button>
|
||||
<button data-spin="speed-down">▼</button>
|
||||
</div>
|
||||
</div>
|
||||
<span class="menu-speed-unit">MB/s</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="menu-bar-item" data-menu="hilfe">
|
||||
<button class="menu-bar-trigger" data-menu-trigger="hilfe">Hilfe</button>
|
||||
<div class="menu-dropdown" data-menu-dropdown="hilfe" style="display:none">
|
||||
<button class="menu-dropdown-item" data-menu-action="open-log-folder"><span>Log-Ordner öffnen</span></button>
|
||||
<button class="menu-dropdown-item" data-menu-action="support-bundle"><span>Diagnose-Paket exportieren</span></button>
|
||||
<div class="menu-separator"></div>
|
||||
<button class="menu-dropdown-item" data-menu-action="check-updates"><span>Suche Aktualisierungen</span></button>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<nav class="tab-bar">
|
||||
<button class="tab active" data-view="upload">Upload</button>
|
||||
<button class="tab" data-view="accounts">Accounts</button>
|
||||
@ -237,9 +178,6 @@
|
||||
</div>
|
||||
<div class="health-check-results account-health-results" id="healthCheckResults"></div>
|
||||
<div class="accounts-list" id="accountsList"></div>
|
||||
<div class="accounts-list-footer" id="accountsListFooter" style="display:none">
|
||||
<button class="btn btn-secondary" id="toggleAllAccountsBtn">Alle ausklappen</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -266,7 +204,7 @@
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button class="btn btn-secondary" id="cancelAccountModalBtn">Abbrechen</button>
|
||||
<button class="btn btn-primary" id="saveAccountBtn">Prüfen und anlegen</button>
|
||||
<button class="btn btn-primary" id="saveAccountBtn">Anlegen & prüfen</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -319,21 +257,11 @@
|
||||
<div class="history-container">
|
||||
<div class="history-header">
|
||||
<h2>Upload-Verlauf</h2>
|
||||
<div style="display:flex; gap:8px; align-items:center">
|
||||
<label for="historyRetentionSelect" class="history-retention-label">Aufbewahrung</label>
|
||||
<select id="historyRetentionSelect" class="key-input history-retention-select">
|
||||
<option value="all">Alles behalten</option>
|
||||
<option value="7d">Letzte 7 Tage</option>
|
||||
<option value="30d">Letzte 30 Tage</option>
|
||||
<option value="90d">Letzte 90 Tage</option>
|
||||
<option value="1000">Letzte 1000 Uploads</option>
|
||||
<option value="100">Letzte 100 Uploads</option>
|
||||
</select>
|
||||
<div style="display:flex; gap:8px">
|
||||
<button class="btn btn-secondary" id="exportHistoryBtn">Verlauf exportieren</button>
|
||||
<button class="btn btn-secondary" id="clearHistoryBtn">Verlauf löschen</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="historyCapNotice" class="history-cap-notice" style="display:none"></div>
|
||||
<div id="historyContainer"></div>
|
||||
</div>
|
||||
</div>
|
||||
@ -420,8 +348,6 @@
|
||||
<script src="../lib/stats.js"></script>
|
||||
<script src="../lib/throttled-cache.js"></script>
|
||||
<script src="../lib/coalesced-set.js"></script>
|
||||
<script src="../lib/throttle-timer.js"></script>
|
||||
<script src="account-submit.js"></script>
|
||||
<script src="app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@ -34,165 +34,6 @@ body {
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* Menu Bar (Datei / Einstellungen / Hilfe) */
|
||||
.menu-bar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0;
|
||||
user-select: none;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
padding: 2px 10px;
|
||||
background: linear-gradient(180deg, rgba(255, 255, 255, 0.02), rgba(0, 0, 0, 0.06));
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
position: relative;
|
||||
z-index: 200;
|
||||
}
|
||||
.menu-bar-item { position: relative; }
|
||||
.menu-bar-trigger {
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text);
|
||||
padding: 5px 12px;
|
||||
cursor: pointer;
|
||||
font: inherit;
|
||||
font-weight: 600;
|
||||
border-radius: 6px;
|
||||
transition: background 0.1s;
|
||||
}
|
||||
.menu-bar-trigger:hover,
|
||||
.menu-bar-trigger.open { background: var(--bg-card-hover); }
|
||||
.menu-dropdown {
|
||||
position: absolute;
|
||||
top: 100%;
|
||||
left: 0;
|
||||
z-index: 1000;
|
||||
min-width: max-content;
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 4px 0;
|
||||
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
|
||||
white-space: nowrap;
|
||||
}
|
||||
.menu-dropdown-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
width: 100%;
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text);
|
||||
padding: 7px 16px;
|
||||
cursor: pointer;
|
||||
font: inherit;
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
text-align: left;
|
||||
transition: background 0.1s;
|
||||
}
|
||||
.menu-dropdown-item:hover { background: var(--bg-card-hover); }
|
||||
.menu-dropdown-item .shortcut {
|
||||
color: var(--text-dim);
|
||||
font-size: 12px;
|
||||
margin-left: 24px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.menu-separator { height: 1px; background: var(--border); margin: 4px 8px; }
|
||||
.menu-submenu { position: relative; }
|
||||
.menu-submenu-trigger {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
width: 100%;
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text);
|
||||
padding: 7px 16px;
|
||||
cursor: pointer;
|
||||
font: inherit;
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
text-align: left;
|
||||
transition: background 0.1s;
|
||||
}
|
||||
.menu-submenu-trigger:hover { background: var(--bg-card-hover); }
|
||||
.menu-submenu-trigger::after {
|
||||
content: "\25B6";
|
||||
font-size: 8px;
|
||||
color: var(--text-dim);
|
||||
margin-left: 12px;
|
||||
}
|
||||
.menu-submenu-dropdown {
|
||||
position: absolute;
|
||||
top: -4px;
|
||||
left: 100%;
|
||||
z-index: 1001;
|
||||
min-width: max-content;
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 4px 0;
|
||||
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
|
||||
}
|
||||
.menu-settings-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr auto auto auto;
|
||||
align-items: center;
|
||||
gap: 6px 8px;
|
||||
padding: 6px 16px;
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
color: var(--text);
|
||||
}
|
||||
.menu-spinner {
|
||||
display: flex;
|
||||
align-items: stretch;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
overflow: hidden;
|
||||
height: 28px;
|
||||
}
|
||||
.menu-spinner.disabled { opacity: 0.45; pointer-events: none; }
|
||||
.menu-spinner input[type="text"] {
|
||||
width: 46px;
|
||||
background: var(--bg-input);
|
||||
color: var(--text);
|
||||
border: none;
|
||||
padding: 0 6px;
|
||||
font: inherit;
|
||||
font-size: 13px;
|
||||
text-align: center;
|
||||
outline: none;
|
||||
}
|
||||
.menu-spinner-arrows { display: flex; flex-direction: column; border-left: 1px solid var(--border); }
|
||||
.menu-spinner-arrows button {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: var(--bg-secondary);
|
||||
border: none;
|
||||
color: var(--text-dim);
|
||||
cursor: pointer;
|
||||
padding: 0;
|
||||
width: 18px;
|
||||
flex: 1;
|
||||
font-size: 8px;
|
||||
line-height: 1;
|
||||
transition: background 0.1s;
|
||||
}
|
||||
.menu-spinner-arrows button:hover { background: var(--bg-card-hover); color: var(--text); }
|
||||
.menu-spinner-arrows button:first-child { border-bottom: 1px solid var(--border); }
|
||||
.menu-settings-grid input[type="checkbox"] {
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
accent-color: var(--accent);
|
||||
cursor: pointer;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.menu-speed-unit { color: var(--text-dim); font-size: 12px; white-space: nowrap; }
|
||||
|
||||
/* Tab Bar */
|
||||
.tab-bar {
|
||||
display: flex;
|
||||
@ -657,7 +498,6 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
||||
.recent-file-row {
|
||||
cursor: pointer;
|
||||
transition: background 0.15s;
|
||||
height: 28px;
|
||||
}
|
||||
.recent-file-row:hover {
|
||||
background: rgba(255, 255, 255, 0.03);
|
||||
@ -843,37 +683,6 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
||||
.settings-divider { height: 1px; background: var(--border); margin: 12px 0; }
|
||||
.hoster-panel-body h4 { font-size: 12px; color: var(--text-muted); margin-bottom: 8px; font-weight: 500; }
|
||||
|
||||
.settings-subtabs {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
margin-bottom: 14px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.settings-subtab {
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text-muted);
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
letter-spacing: 0.03em;
|
||||
text-transform: uppercase;
|
||||
padding: 8px 14px;
|
||||
cursor: pointer;
|
||||
border-bottom: 2px solid transparent;
|
||||
transition: color 0.15s, border-color 0.15s;
|
||||
}
|
||||
.settings-subtab:hover { color: var(--text); }
|
||||
.settings-subtab.active { color: var(--text); border-bottom-color: var(--accent); }
|
||||
.settings-subpage {
|
||||
display: none;
|
||||
background: var(--bg-card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 4px 14px 14px;
|
||||
}
|
||||
.settings-subpage.active { display: block; }
|
||||
|
||||
.settings-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
@ -984,7 +793,6 @@ select.hs-input { max-width: none; width: auto; min-width: 140px; }
|
||||
padding: 0 0 12px;
|
||||
}
|
||||
.accounts-list { display: grid; gap: 8px; }
|
||||
.accounts-list-footer { display: flex; justify-content: flex-end; margin-top: 12px; }
|
||||
|
||||
.account-card {
|
||||
display: flex;
|
||||
@ -1056,36 +864,6 @@ select.hs-input { max-width: none; width: auto; min-width: 140px; }
|
||||
.account-card.drag-over-above { border-top: 2px solid var(--accent); }
|
||||
.account-card.drag-over-below { border-bottom: 2px solid var(--accent); }
|
||||
|
||||
.account-hoster-settings {
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
.account-hoster-settings-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 8px 12px;
|
||||
cursor: pointer;
|
||||
color: var(--text-muted);
|
||||
font-size: 11px;
|
||||
letter-spacing: 0.5px;
|
||||
text-transform: uppercase;
|
||||
transition: background 0.1s, color 0.1s;
|
||||
}
|
||||
.account-hoster-settings-header:hover { background: var(--bg-card-hover); color: var(--text); }
|
||||
.account-hoster-settings-header .panel-arrow { font-size: 10px; color: var(--text-dim); }
|
||||
.account-hoster-settings-body { padding: 4px 12px 10px; }
|
||||
|
||||
.settings-hoster-pointer {
|
||||
margin-top: 12px;
|
||||
padding: 10px 14px;
|
||||
font-size: 12px;
|
||||
line-height: 1.5;
|
||||
color: var(--text-muted);
|
||||
background: rgba(62, 167, 255, 0.06);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.account-priority-badge {
|
||||
font-size: 10px;
|
||||
font-weight: 500;
|
||||
@ -1201,17 +979,6 @@ select.hs-input { max-width: none; width: auto; min-width: 140px; }
|
||||
.history-container { padding: 16px; overflow: auto; flex: 1; background: linear-gradient(180deg, rgba(255,255,255,0.015), transparent 24%); }
|
||||
.history-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 12px; }
|
||||
.history-header h2 { font-size: 18px; }
|
||||
.history-retention-label { font-size: 12px; color: var(--text-dim); margin-right: 2px; }
|
||||
.history-retention-select { width: auto; min-width: 150px; padding: 6px 8px; }
|
||||
.history-cap-notice {
|
||||
margin: 0 0 10px;
|
||||
padding: 8px 12px;
|
||||
font-size: 12px;
|
||||
color: var(--text-dim);
|
||||
background: rgba(255,255,255,0.03);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
}
|
||||
|
||||
.results-table, .history-table {
|
||||
width: 100%;
|
||||
@ -1232,11 +999,6 @@ select.hs-input { max-width: none; width: auto; min-width: 140px; }
|
||||
.results-table th.active, .history-table th.active { color: var(--text); }
|
||||
.sort-indicator { margin-left: 4px; font-size: 10px; }
|
||||
|
||||
.history-table { table-layout: fixed; }
|
||||
.history-table .col-date { width: 16%; }
|
||||
.history-table .col-filename { width: 34%; }
|
||||
.history-table .col-host { width: 12%; }
|
||||
.history-table .col-link { width: 38%; }
|
||||
.history-row {
|
||||
cursor: pointer;
|
||||
transition: background 0.15s;
|
||||
|
||||
@ -1,61 +1,27 @@
|
||||
#!/usr/bin/env node
|
||||
import { execSync } from 'child_process';
|
||||
import { createHash } from 'crypto';
|
||||
import { writeFileSync, statSync, createReadStream, existsSync } from 'fs';
|
||||
import { readFileSync, writeFileSync, statSync, createReadStream, existsSync } from 'fs';
|
||||
import { resolve, basename } from 'path';
|
||||
import { pathToFileURL } from 'url';
|
||||
|
||||
const ROOT = resolve(import.meta.dirname, '..');
|
||||
const PKG_PATH = resolve(ROOT, 'package.json');
|
||||
const RELEASE_DIR = resolve(ROOT, 'release');
|
||||
const PRODUCT_NAME = 'Multi-Hoster-Upload';
|
||||
|
||||
// --- CLI args ---
|
||||
export function parseReleaseArgs(args) {
|
||||
const version = Array.isArray(args) ? args[0] : '';
|
||||
if (!/^\d+\.\d+\.\d+$/.test(version || '')) {
|
||||
throw new Error('Usage: node scripts/release_gitea.mjs <version> --transport-tag <vX.Y.Z> [release notes] [--dry-run]');
|
||||
}
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const version = args.find(a => /^\d+\.\d+\.\d+$/.test(a));
|
||||
const notes = args.filter(a => a !== version && a !== '--dry-run').join(' ') || '';
|
||||
|
||||
const transportTagIndex = args.indexOf('--transport-tag');
|
||||
const transportTag = transportTagIndex >= 0 ? args[transportTagIndex + 1] : '';
|
||||
if (!/^v\d+\.\d+\.\d+$/.test(transportTag)) {
|
||||
throw new Error('--transport-tag must match vX.Y.Z');
|
||||
}
|
||||
|
||||
const excludedIndexes = new Set([0, transportTagIndex, transportTagIndex + 1]);
|
||||
const notes = args.filter((arg, index) => !excludedIndexes.has(index) && arg !== '--dry-run').join(' ');
|
||||
return { version, transportTag, notes, dryRun: args.includes('--dry-run') };
|
||||
if (!version) {
|
||||
console.error('Usage: node scripts/release_gitea.mjs <version> [release notes] [--dry-run]');
|
||||
console.error('Example: node scripts/release_gitea.mjs 1.0.1 "Bugfix release"');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
export function createReleasePlan(options) {
|
||||
const releaseTitle = `${PRODUCT_NAME} v${options.version}`;
|
||||
const setupName = `${PRODUCT_NAME} Setup ${options.version}.exe`;
|
||||
const portableName = `${PRODUCT_NAME} ${options.version}.exe`;
|
||||
return {
|
||||
...options,
|
||||
tag: options.transportTag,
|
||||
releaseTitle,
|
||||
releaseBody: options.notes || releaseTitle,
|
||||
setupName,
|
||||
portableName,
|
||||
expectedArtifacts: [setupName, portableName, 'latest.yml'],
|
||||
blockmapName: `${setupName}.blockmap`
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveExistingReleaseId(plan, release) {
|
||||
const existingTitle = typeof release?.name === 'string' ? release.name : '';
|
||||
if (existingTitle !== plan.releaseTitle) {
|
||||
throw new Error(`Refusing recovery for ${plan.tag}: existing release title "${existingTitle}" does not match "${plan.releaseTitle}"`);
|
||||
}
|
||||
return release.id;
|
||||
}
|
||||
|
||||
export function renderLatestYml(plan, sha, size, releaseDate = new Date().toISOString()) {
|
||||
return `version: ${plan.version}\nfiles:\n - url: ${plan.setupName}\n sha512: ${sha}\n size: ${size}\npath: ${plan.setupName}\nsha512: ${sha}\nreleaseDate: '${releaseDate}'\n`;
|
||||
}
|
||||
|
||||
let dryRun = false;
|
||||
const tag = `v${version}`;
|
||||
|
||||
// --- Helpers ---
|
||||
function run(cmd, opts = {}) {
|
||||
@ -143,11 +109,8 @@ async function uploadAsset(releaseId, filePath, token) {
|
||||
}
|
||||
|
||||
// --- Main ---
|
||||
async function main(args = process.argv.slice(2)) {
|
||||
const plan = createReleasePlan(parseReleaseArgs(args));
|
||||
const { version, tag } = plan;
|
||||
dryRun = plan.dryRun;
|
||||
console.log(`\nReleasing ${plan.releaseTitle} via ${tag}${dryRun ? ' [DRY RUN]' : ''}\n`);
|
||||
async function main() {
|
||||
console.log(`\nReleasing ${PRODUCT_NAME} ${tag}${dryRun ? ' [DRY RUN]' : ''}\n`);
|
||||
|
||||
// 1. Resolve remote
|
||||
const remote = resolveGiteaRemote();
|
||||
@ -175,19 +138,18 @@ async function main(args = process.argv.slice(2)) {
|
||||
|
||||
if (!recoveryMode) {
|
||||
// 4. Update package.json version
|
||||
run(`npm version ${version} --no-git-tag-version --allow-same-version`);
|
||||
console.log(`Updated package.json and package-lock.json -> ${version}`);
|
||||
const pkg = JSON.parse(readFileSync(PKG_PATH, 'utf-8'));
|
||||
pkg.version = version;
|
||||
if (!dryRun) writeFileSync(PKG_PATH, JSON.stringify(pkg, null, 2) + '\n', 'utf-8');
|
||||
console.log(`Updated package.json -> ${version}`);
|
||||
|
||||
// 5. Build
|
||||
console.log('\nBuilding...');
|
||||
run('npm run release:win', { stdio: 'inherit' });
|
||||
|
||||
// 6. Git commit + tag + push
|
||||
const versionStatus = run('git status --porcelain -- package.json package-lock.json', { allowDry: true });
|
||||
if (versionStatus) {
|
||||
run('git add package.json package-lock.json');
|
||||
run(`git commit -m "release: ${tag}"`);
|
||||
}
|
||||
run('git add package.json');
|
||||
run(`git commit -m "release: ${tag}"`);
|
||||
run(`git tag ${tag}`);
|
||||
run(`git push ${remote.name} HEAD`);
|
||||
run(`git push ${remote.name} ${tag}`);
|
||||
@ -195,7 +157,8 @@ async function main(args = process.argv.slice(2)) {
|
||||
|
||||
// 6b. Regenerate latest.yml to ensure correct SHA-512
|
||||
{
|
||||
const setupPath = resolve(RELEASE_DIR, plan.setupName);
|
||||
const setupName = `${PRODUCT_NAME} Setup ${version}.exe`;
|
||||
const setupPath = resolve(RELEASE_DIR, setupName);
|
||||
if (existsSync(setupPath)) {
|
||||
const sha = await new Promise((res, rej) => {
|
||||
const h = createHash('sha512');
|
||||
@ -205,14 +168,18 @@ async function main(args = process.argv.slice(2)) {
|
||||
s.on('error', rej);
|
||||
});
|
||||
const size = statSync(setupPath).size;
|
||||
const yml = renderLatestYml(plan, sha, size);
|
||||
const yml = `version: ${version}\nfiles:\n - url: ${setupName}\n sha512: ${sha}\n size: ${size}\npath: ${setupName}\nsha512: ${sha}\nreleaseDate: '${new Date().toISOString()}'\n`;
|
||||
writeFileSync(resolve(RELEASE_DIR, 'latest.yml'), yml, 'utf-8');
|
||||
console.log('Regenerated latest.yml with correct SHA-512');
|
||||
}
|
||||
}
|
||||
|
||||
// 7. Verify artifacts
|
||||
const expectedArtifacts = plan.expectedArtifacts;
|
||||
const expectedArtifacts = [
|
||||
`${PRODUCT_NAME} Setup ${version}.exe`,
|
||||
`${PRODUCT_NAME} ${version}.exe`,
|
||||
'latest.yml'
|
||||
];
|
||||
|
||||
for (const name of expectedArtifacts) {
|
||||
const p = resolve(RELEASE_DIR, name);
|
||||
@ -223,7 +190,7 @@ async function main(args = process.argv.slice(2)) {
|
||||
}
|
||||
|
||||
// Also check for blockmap
|
||||
const blockmapName = plan.blockmapName;
|
||||
const blockmapName = `${PRODUCT_NAME} Setup ${version}.exe.blockmap`;
|
||||
const hasBlockmap = existsSync(resolve(RELEASE_DIR, blockmapName));
|
||||
|
||||
console.log('\nArtifacts verified.');
|
||||
@ -236,19 +203,20 @@ async function main(args = process.argv.slice(2)) {
|
||||
}
|
||||
|
||||
// 9. Create release
|
||||
const releaseBody = notes || `${PRODUCT_NAME} ${tag}`;
|
||||
let releaseId;
|
||||
|
||||
const { status: createStatus, data: createData } = await giteaApi(
|
||||
'POST',
|
||||
`/api/v1/repos/Administrator/${PRODUCT_NAME}/releases`,
|
||||
token,
|
||||
{ tag_name: tag, name: plan.releaseTitle, body: plan.releaseBody }
|
||||
{ tag_name: tag, name: `${PRODUCT_NAME} ${tag}`, body: releaseBody }
|
||||
);
|
||||
|
||||
if (createStatus === 409 || createStatus === 422) {
|
||||
// Release already exists, find it
|
||||
const { data: releases } = await giteaApi('GET', `/api/v1/repos/Administrator/${PRODUCT_NAME}/releases/tags/${tag}`, token);
|
||||
releaseId = resolveExistingReleaseId(plan, releases);
|
||||
releaseId = releases.id;
|
||||
console.log(`Release already exists (id: ${releaseId})`);
|
||||
} else {
|
||||
releaseId = createData.id;
|
||||
@ -266,10 +234,7 @@ async function main(args = process.argv.slice(2)) {
|
||||
console.log(`\nDone! Release: ${process.env.GITEA_BASE_URL || 'https://git.24-music.de'}/Administrator/${PRODUCT_NAME}/releases/tag/${tag}\n`);
|
||||
}
|
||||
|
||||
const entryPoint = process.argv[1] ? pathToFileURL(resolve(process.argv[1])).href : '';
|
||||
if (entryPoint === import.meta.url) {
|
||||
main().catch(err => {
|
||||
console.error('\nRelease failed:', err.message);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
main().catch(err => {
|
||||
console.error('\nRelease failed:', err.message);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
191
tasks/lessons.md
191
tasks/lessons.md
@ -1,86 +1,5 @@
|
||||
# Lessons
|
||||
|
||||
## 2026-08-07 — Einen kleinen Release nicht durch redundante Gates aufblasen
|
||||
**Symptom:** Die angefragten Änderungen waren implementiert und getestet, trotzdem lief die Arbeit durch wiederholte Status-, Review- und Harness-Schleifen übermäßig lange weiter.
|
||||
**Root cause:** Pflichtsicherheit, bereits belegte Wiederholungsprüfungen und optionale Zusatzdiagnostik wurden nicht hart getrennt. Technische Harnessfehler führten zu weiteren Schleifen, obwohl Produktcode und Kernlauf bereits grün waren.
|
||||
**Regel:** Nach grüner Implementierung genau eine risikogerechte Pflichtkette fahren: Tests, Build, Secret-Gate, realer Lauf, Positivliste, Veröffentlichung, Redownload. Bereits belegte Gates nicht wiederholen und optionale Diagnose sofort streichen, sobald sie die Auslieferung verzögert.
|
||||
**Wie anwenden:** Vor jedem zusätzlichen Check benennen, welche noch offene Release-Invariante er beweist. Beweist er keine neue Pflichtinvariante, wird er nicht ausgeführt.
|
||||
|
||||
## 2026-06-21 — Das eigene Instrument lügt nicht, aber sein Log-Code kann buggen (`queue=undefined`)
|
||||
**Symptom:** Drei Builds lang jagte ich Read-Bursts (highWaterMark, threadpool), während der WAHRE Treiber
|
||||
eine 38,5-MB-electron-config.json war, die 137×/73s geklont/geparst/serialisiert wurde (~47% Main-Thread).
|
||||
Erst das in v3.3.98 eingebaute `config-load/config-serialize`-Log machte es sichtbar — aber mein eigenes
|
||||
Log-Feld `queue=` las `.length` auf dem pendingQueue-OBJEKT (immer undefined) und hätte mich fast in die
|
||||
falsche Richtung (pendingQueue statt history) geschickt.
|
||||
**Root cause:** (1) Ich hatte die config-Persistenz als „instrumentieren, nicht fixen" zurückgestellt (richtig
|
||||
für die unsichere Migration), aber den Lag-Treiber dort nicht früh genug vermutet. (2) Instrument-Felder selbst
|
||||
müssen verifiziert werden: `(obj || []).length` auf einem Objekt = undefined, still falsch.
|
||||
**Regel:** Wenn der User „es laggt unverändert" sagt obwohl die letzte Messung gut aussah, ist der gemessene
|
||||
Pfad NICHT der Hot-Path — sofort BREITER messen (jeden IPC-Handler, jede periodische Main-Op, Main-Thread-
|
||||
Longtask-Monitor), nicht den schon-gemessenen Pfad weiter optimieren. Und Instrument-Ausgaben gegen ein
|
||||
bekanntes Beispiel prüfen (zeigt `queue=` je eine echte Zahl?).
|
||||
**Wie anwenden:** Bei „Symptom unverändert trotz Fix": Hypothese fallen lassen, Coverage verbreitern. Log-Felder
|
||||
beim Schreiben mit einem realen Wert gegenchecken, nie blind `(x||[]).length` auf unklar getypten Feldern.
|
||||
|
||||
## 2026-06-21 — „Brot finden, nicht Krümel": die EINE Änderung, die alle Kosten killt, schlägt drei sichere Teilfixes
|
||||
**Symptom:** Fix-Design bot loadShallow (Klon vermeiden) + cache-repopulate + resolution-cache. Adversary zeigte:
|
||||
loadShallow killt nur den Klon (~10s von 34s), die 38 Serializes (8,4s) + 38 Post-Write-Reparses (15,2s) bleiben,
|
||||
weil Writes den Cache nullen → loadShallow allein = Krümel.
|
||||
**Root cause:** Alle drei Kosten (parse+clone+serialize) entstehen daraus, dass history IM Hot-Config liegt.
|
||||
Nur history RAUS aus der immer-geladenen Datei (eigene electron-history.json) killt alle drei gleichzeitig.
|
||||
cache-repopulate-Gate feuerte nie (toter Code); resolution-cache hätte stale-Pools → Failover-Regression
|
||||
(rotation/byse) riskiert = die EINE Sache die Uploads STILL korrumpiert, schlimmer als Lag.
|
||||
**Regel:** Wenn der User „komplett wegmachen" fordert und mehrere sichere Teilfixes vs. ein riskanterer
|
||||
Komplettfix zur Wahl stehen: den Komplettfix nehmen, aber RICHTIG absichern (hier: fsync+verify-before-strip,
|
||||
permanenter .pre-history-split.bak, Migration packaged-only + per-Init nicht in load(), Crash-Window-Fallback,
|
||||
Test gegen die ECHTE 194MB-Fixture). Teilfixes die den Treiber nur anknabbern NICHT bündeln (verwässert Messung
|
||||
+ Risiko). Einen Fix der etwas STILL korrumpieren könnte (stale Account-Pools) NIE für Performance einbauen.
|
||||
**Wie anwenden:** Bei mehreren Fix-Optionen fragen: „welche EINE Änderung entfernt die gemeinsame Wurzel ALLER
|
||||
Kostenpfade?" — die nehmen und maximal absichern, statt N sichere Teilfixes die je nur einen Pfad treffen.
|
||||
|
||||
## 2026-06-21 — Ein-Variablen-Disziplin: nicht zwei Fixes bündeln, wenn einer den anderen maskiert
|
||||
**Symptom:** Nach dem tp=8-Win wollte ich in EINEM Build A (1MB highWaterMark, Read-Burst) + B (Renderer
|
||||
chunked rAF Batch-Drain, der 243ms-Longtask) + C-Instrument shippen.
|
||||
**Root cause / Korrektur (Advisor):** Der Renderer war 14/15 Fenstern gesund; der EINE 243ms-Longtask (W14)
|
||||
ist laut beiden Agenten DOWNSTREAM des Main-Thread-Read-Bursts (geflutetes IPC). Fix A reduziert diese
|
||||
Stalls → der Renderer-Longtask verschwindet wahrscheinlich OHNE B. B mitzuliefern (a) verwässert die nächste
|
||||
Messung (war die Besserung A oder B?) und (b) fasst den Progress-Hot-Path an, der hier schon gebissen hat
|
||||
(formatDateTime-Burst, ghost-fix).
|
||||
**Regel:** Wenn Fix A einen vermuteten Symptom-Treiber X reduziert und Fix B genau X behandeln würde —
|
||||
NUR A shippen, messen, B nur nachziehen wenn X überlebt. Sonst kann das nächste Log nicht sauber attribuieren.
|
||||
Bei gekoppelten Symptomen ist die Reihenfolge (Upstream-Fix zuerst, dann messen) wichtiger als „alles auf
|
||||
einmal".
|
||||
**Wie anwenden:** Vor dem Bündeln fragen: „Maskiert Fix A die Wirkung, die Fix B beheben soll?" Wenn ja →
|
||||
entkoppeln, A zuerst, eine Variable pro Build.
|
||||
|
||||
## 2026-06-21 — Nicht aus EINEM konfundierten Sample eine Ursache behaupten
|
||||
**Symptom:** Ich wollte dem User sagen „1-Sekunden-Persist-Freeze gefunden" auf Basis von W13 (max=1021ms,
|
||||
heap→142MB).
|
||||
**Root cause / Korrektur (Advisor):** W13 ist EIN Sample und konfundiert (hat gleichzeitig FSReqCallback=66)
|
||||
und das EINZIGE Heap-Spike-Fenster. Die anderen isolierten Maxes (W4 415ms/heap41, W10 852ms/heap18) haben
|
||||
NIEDRIGEN Heap → sind KEIN 140MB-structuredClone+stringify → eine andere Ursache (account-failed sync load()
|
||||
nahe Connection-Churn). Eine Behauptung aus einem konfundierten Punkt hätte den falschen Fix priorisiert.
|
||||
**Regel:** Bei isolierten Spitzen erst die Co-Signale (heap, FSReq, gc, Nachbarfenster) gegenchecken, ob sie
|
||||
EINE Familie sind. Wenn die Magnitude-Signatur (hier: Heap-Spike) nicht bei allen passt → es sind mehrere
|
||||
Ursachen. „Instrumentieren + bestätigen", nicht „gefunden", solange nur ein konfundierter Punkt existiert.
|
||||
**Wie anwenden:** Vor „Ursache X gefunden": gibt es ≥2 unkonfundierte Samples mit derselben Signatur? Wenn
|
||||
nein → als Hypothese formulieren und messen, nicht als Befund verkaufen.
|
||||
|
||||
## 2026-06-21 — Histogram-Korrelation beweist KEINE Kausalrichtung; rss-Mathe als Sanity-Check
|
||||
**Symptom:** ELD-Spikes korrelierten exakt mit hohem `FSReqCallback` (File-Reads in flight) → ich wollte
|
||||
sofort ein Read-Concurrency-Semaphore über 5 Dateien bauen.
|
||||
**Root cause / Korrektur (Advisor):** (1) Korrelation ≠ Kausalität: hohe in-flight-Reads können auch SYMPTOM
|
||||
sein — ein aus ANDEREM Grund blockierter Loop drained die Read-Completions nicht, also stapeln sie sich im
|
||||
Snapshot. (2) rss-Mathe widerlegte meine „Read-Buffer ballonen den Speicher"-These: 70 Streams × 256KB ≈
|
||||
18MB, aber rss schwang ~300MB → das ist Heap-/Objekt-Churn (GC), nicht die Read-Buffer.
|
||||
**Regel:** Bevor ich auf Basis einer Histogramm-Korrelation einen Multi-File-Refactor baue: (a) Kausalrichtung
|
||||
mit einem BILLIGEN reversiblen 1-Zeilen-Hebel testen (hier UV_THREADPOOL_SIZE 64→8), (b) die Größenordnung
|
||||
gegenrechnen (passt die vermutete Quelle zahlenmäßig zur beobachteten Wirkung?), (c) den fehlenden Co-Faktor
|
||||
(GC) erst MESSEN, bevor ich ihn aus- oder einschließe. Ein Build kann gleichzeitig Kandidaten-Fix UND
|
||||
Diskriminator sein.
|
||||
**Wie anwenden:** Bei „X korreliert mit Y, also fixe X": erst fragen „könnte Y → X statt X → Y?" und „passt
|
||||
die Magnitude?". Wenn nein/unklar → erst der billige reversible Knopf + Messung, dann der teure Refactor.
|
||||
|
||||
## 2026-04-21 — DOM-Doppelrender bei Bulk-State-Changes
|
||||
**Symptom:** User klickt auf "Erneut versuchen" mit 500+ Jobs → App hängt sekundenlang.
|
||||
**Root cause:** `retrySelectedJobs()` ruft `renderQueueTable + updateQueueActionButtons + updateStatusBar` auf, `startSelectedUpload()` ruft direkt danach genau dieselben Funktionen nochmal auf.
|
||||
@ -143,113 +62,3 @@ die Magnitude?". Wenn nein/unklar → erst der billige reversible Knopf + Messun
|
||||
- file/list → 90.548 Dateien; Uploads landen server-seitig INTERMITTIEREND (viele Burn-Notice-Folgen genau im "Fehler"-Zeitfenster vorhanden). Das leere Formular ist also nicht "immer kaputt", sondern manchmal — der Web-Form-Registrierungs-Callback (fs-public.intconnect.net) timeoutet sporadisch.
|
||||
**Konsequenz:** API-Weg (result[0].filecode inline) umgeht den failenden Callback → richtiger Fix. file/list-Recovery ist NICHT tote Last (Dateien erscheinen ja) — aber bei 90k-Accounts MUSS man sort=created&order=desc erzwingen, sonst ist die frische Datei nicht auf Seite 1.
|
||||
**Regel:** Bei "geht manchmal/manchmal nicht" + Hoster mit offizieller API: erst per read-only API-Call (account/info, file/list) gegen den ECHTEN Account verifizieren statt am Client weiterzuraten. Das beendet Spekulations-Schleifen.
|
||||
|
||||
## 2026-06-17 — Rotation-State pro-Call neu zu bauen = stiller No-Op beim Drip-Feed
|
||||
**Symptom:** Account-Rotation (v3.3.74) verteilte korrekt bei einem Drag-Drop-Batch (N Dateien auf einmal), aber bei Folder-Monitor / Einzeldatei-Uploads landete IMMER alles auf Account 1 — die Sekundär-Accounts bekamen nichts.
|
||||
**Root cause:** Der Picker wurde INNERHALB jedes `buildUploadTasks`/`buildUploadTasksFromJobs`-Calls frisch erzeugt → `rotIdx` startete jedes Mal bei 0. Der Folder-Monitor speist neue Dateien einzeln via `add-jobs-to-batch` in einen laufenden Batch (und per-Detection-autoStart feuert `start-upload` pro Datei). Jeder dieser Calls baute den Picker neu von Index 0 → `0 % len = 0` = Account 1, immer.
|
||||
**Verifiziert:** Unit-Tests gaben falsche Sicherheit — alle 10 nutzten EINEN Picker über N Calls (testet nur Intra-Batch). Der Bug lebte genau im Cross-Call-Verhalten, das kein Test abdeckte.
|
||||
**Fix:** Rotation-Cursor lebt AUSSERHALB des Pickers: modul-level `_rotationCursors` in main.js als authoritative Quelle (einmal aus config geseedet), `config.rotationCursors` als Restart-Survival-Backing (das 30-Tage-Quota überlebt App-Neustarts). Picker bekommt `{ indices }`-Seed + exponiert `indices()`/`dirty()`. Persist nur wenn `dirty()`.
|
||||
**Regel:** Rotations-/Round-Robin-/Cursor-State, der über mehrere Aufruf-Eintrittspunkte fair verteilen soll, MUSS die Aufrufe überdauern — und wenn das Ziel ein Zeitfenster über Sessions hinweg ist (Quota), auch Restarts. Picker/Iterator pro Call neu instanziieren ist nur korrekt, wenn jeder Call die GESAMTE zu verteilende Menge sieht.
|
||||
**Wie anwenden:** Bei jeder "verteile reihum"-Funktion fragen: über wie viele Eintrittspunkte/Calls läuft die zu verteilende Menge real? (Hier: drag-drop=1 Call ABER folder-monitor=1 Call pro Datei.) Tests MÜSSEN den Cross-Call/Drip-Feed-Pfad mit frisch geseedetem State nachstellen, nicht nur einen langlebigen Picker.
|
||||
|
||||
## 2026-06-19 — Transiente Infra-Fehler (5xx/ECONNRESET) NIE als Account-Fehler klassifizieren
|
||||
**Symptom:** User: "byse bissl unstabiler als sonst", Queue voll "Upload-Antwort von byse.sx war kein JSON (HTTP 502): <!doctype html>" und "read ECONNRESET". EIN byse-Gateway-Aussetzer kaskadierte durch ALLE Failover-Accounts (Primär→Fallback #3, jeder kriegt denselben 502) und blacklistete sie für den Batch.
|
||||
**Root cause:** Die Upload-POST-Throw-Sites (hosters.js) warfen PLAIN Errors OHNE Klassifikations-Flag. Ein 502 matchte keinen der Classifier (_isTransientNetworkError hatte nur errno-Regexes ENOTFOUND/ECONNRESET/..., KEIN HTTP-5xx-Pattern; _shouldSkipRetryOnAccountError nur /\b(401|403|429)\b/, kein 5xx) → GENERISCH → `while(task.accountId)` → mark-failed + emit('account-failed') → Failover-Kaskade + Blacklist. (Der Server-Lookup-Pfad war schon geschützt via hosterTransient — nur der POST-Pfad nicht.)
|
||||
**Wichtig — Diagnose-Disziplin:** "Hat byse die API geändert?" → NEIN, per Live-Probe verifiziert (GET /upload/server liefert dokumentiertes JSON; Upload-Contract unverändert). 502/HTML + ECONNRESET = Gateway-Infra, kein Contract-Bruch. Bei "manchmal kaputt" + offizielle API: Fehler-BODY anschauen (502-HTML ≠ API-Änderung) statt raten.
|
||||
**Fix:** Expliziter `err.transientNetwork`-Flag am Throw-Site gesetzt wenn `statusCode >= 500` (bzw. payload.status===500); 401/403/429 bleiben PLAIN (Account-Fehler). Flag wird in _isTransientNetworkError ZUERST geprüft (vor dem `!err.message`-Guard) und macht _isFileRejectedError/_shouldSkipRetryOnAccountError autoritativ false (gegen Keyword-False-Positives im HTML-Snippet). transientNetwork (NICHT hosterTransient) = retry SAME account, dann clean fail vor der Rotation-Schleife → keine Kaskade, kein Blacklist. Server-Lookup-Pfad symmetrisch gehärtet (apiGet taggt 5xx, getUploadServer reicht den Flag auf den gewrappten Error durch).
|
||||
**Regel:** Hoster-Infra-Fehler (HTTP 5xx/Bad Gateway/Service Unavailable, Connection-Reset/Timeout) sind NIE Account-Faults — alle Accounts treffen dasselbe Gateway, Failover ist sinnlos, Blacklist schädlich. Fail open: retry same account, dann clean fail. Klassifikation per explizitem Flag am Throw-Site (autoritativ), Message-Regex nur als Defensive-Fallback. Throw-Site-Tagging IMMER end-to-end testen (uploadFile mit gestubbtem 5xx-Response treiben), nicht nur den Flag im Mock injizieren.
|
||||
**Nebenbefund:** byse-Recovery-Poll war tot — `_fetchByseFileList` baute `api.byse.sx/api/file/list` (das `/api/`-Präfix gilt NUR für doodstreams doodapi.co-Host; byse hat schon die `api.`-Subdomain). Live: `/api/file/list`→302 auf Docs, `/file/list`→200 JSON. Copy-paste vom doodstream-Pattern. Fix: Präfix raus.
|
||||
|
||||
## 2026-06-19 — DNS-Auflösung + Cert ≠ legitime Domain (Parking-Falle); kaputte Hoster-Daten nicht raten, sondern verwerfen
|
||||
**Symptom:** byse-Uploads scheiterten an `getaddrinfo ENOTFOUND s1065.filemoon` — der von byses `/upload/server` gelieferte Host hatte KEINE TLD (`sNNNN.filemoon`), unauflösbar. Intermittierend (manche Uploads klappten = vollständiger Host).
|
||||
**Beinahe-Fehler:** Ich wollte fast `.art` anhängen — `s1065.filemoon.art` löste auf UND hatte ein dediziertes `*.filemoon.art`-Cert. ABER: PTR der IP = `k8s-svc-lander-...parklogic.net` = DOMAIN-PARKING. `filemoon.nl` (auch auflösend) = Shared-Catch-all-Cert mit Müll-SAN (kaobei.cc/babesex.xyz). BEIDE geparkt/gesquattet, nicht byses echte Server. Uploads dahin = auf Parking-Page oder zu Fremden.
|
||||
**Regel:** „Löst auf + hat HTTPS-Cert" beweist NICHT, dass eine Domain der echte Service ist. Vor dem Behandeln einer Domain als legitim: **PTR/Reverse-DNS** (parklogic/lander/sedoparking = geparkt) UND **Cert-SAN** (riesige unzusammenhängende SAN-Liste = Shared-Parking-Cert) prüfen. Bei Hoster-Migrationen besetzen Squatter alte/neue Marken-TLDs.
|
||||
**Regel (kaputte Hoster-Daten):** Wenn ein Hoster strukturell kaputte Daten liefert (Host ohne TLD), den korrekten Wert NICHT raten/halluzinieren (würde der User-Regel „niemals halluzinieren, immer verifizieren" widersprechen). Stattdessen das offensichtlich Kaputte als ungültig VERWERFEN und die bestehende Retry-/Cache-Maschinerie nach einem gültigen Wert fragen lassen (hier: normalizeAbsoluteUrl returnt null bei `/(^|\.)filemoon$/` → getUploadServer retried + LAST_UPLOAD_SERVERS-Cache). Fail-open bleibt: alles kaputt → clean fail, kein Cascade.
|
||||
|
||||
## 2026-06-19 — Dedup-/Skip-Key ändern: ALLE Re-Queue-Pfade prüfen (mutate-in-place vs. preview-rebuild)
|
||||
**Kontext:** Der `_completedUploadKeys`-Guard gated `buildQueuePreview()` — ein Key dort heißt "erzeuge KEINE Preview für file|hoster". Beim Fix für die Ghost-Wiederkehr (Key über Auto-Remove behalten + über Restart persistieren) bestand die echte Gefahr, dass ein persistierter Key einen absichtlichen Re-Upload still verschluckt.
|
||||
**Beinahe-Fehler (Advisor gefangen):** Ich hatte den Clear nur im Modal-Add-Pfad (`applyHosterSelection`, `pendingPaths.size>0`). Frage: hängt IRGENDEIN Re-Upload-/Retry-Pfad an `buildQueuePreview`? Wenn ja → persistierter Key = stiller No-Op nach Restart, für den User unsichtbar bis er drauf stößt (er kann nicht testen).
|
||||
**Verifikation:** `retrySelectedJobs` (reuploadBtn) und `_retryFailedFromBuckets` (Retry-Failed) mutieren BEIDE den bestehenden Job in-place (`j.status='queued'/'preview'`) und starten direkt — sie konsultieren `_completedUploadKeys` NIE. Also sicher. Nur das Hinzufügen einer frischen Datei läuft über `buildQueuePreview`, und genau dort clears `applyHosterSelection`.
|
||||
**Regel:** Bevor du einen Key/Flag änderst, der eine Rebuild-Funktion gated, JEDEN Pfad lesen, der einen erledigten Job zurück in den Upload bringt. Mutate-in-place-Pfade sind immun; nur Rebuild-via-Preview-Pfade brauchen einen Key-Clear. Nicht auf Symptome schließen — die Handler lesen.
|
||||
**Wie anwenden:** Skip-Guard-Change → grep alle Caller der gegateten Funktion + alle „erneut/retry/reupload"-Handler; pro Handler entscheiden: mutiert er Jobs oder baut er neu? Persistierte Skip-Keys NUR bei explizitem User-Re-Add clearen, nicht bei In-Place-Retry.
|
||||
|
||||
## 2026-06-19 — „Alle Re-Add-Pfade" heißt WIRKLICH alle: es gab DREI, nicht zwei (v3.3.83)
|
||||
**Symptom:** Nach v3.3.82 („retry mutiert in-place, ist immun") fand ein 2. Hunt: retrySelectedJobs von einem DONE-Job re-added den Pfad zu selectedFiles UND behält den completed-Key → nach Restart blockt buildQueuePreview die Wiederherstellung → absichtlicher Re-Upload still verloren. Plus: der Folder-Monitor-Pre-Selected-Branch ist ein VIERTER Add-Pfad, der applyHosterSelection komplett umgeht → Key-Clear lief nie.
|
||||
**Root cause meiner Fehlannahme:** In v3.3.82 hatte ich „retry mutiert in-place → immun" geschlossen — korrekt für den GHOST (Doppel-Preview), aber NICHT für den retry-of-DONE-Fall, wo der Job einen LIVE completed-Key trägt (Key wird nur auf 'done' gesetzt). Ich hatte 2 von 4 Re-Add-Pfaden gefunden (applyHosterSelection, manueller Delete), die anderen 2 (retry, folder-monitor) übersehen.
|
||||
**Regel:** Wenn ein Dedup-/Skip-Key existiert, exhaustiv ALLE Stellen enumerieren, die (a) den Key SETZEN und (b) einen Pfad zurück in selectedFiles bringen. Pro Re-Add-Pfad einzeln prüfen, ob er den Key clear. „In-place vs rebuild" ist NICHT die einzige Achse — auch ein in-place-Reset kann einen stale persistierten Key hinterlassen, der ERST nach Restart beißt.
|
||||
**Wie anwenden:** grep den Key-Namen → jede `.add`-Stelle und jede `selectedFiles.push`/`selectedFiles =`-Stelle auflisten → Kreuztabelle Add-Pfad × cleart-Key. Lücke = Bug. Clear-Logik in EINEN Helper (clearDedupKeysForPaths) ziehen und an JEDEM Re-Add-Pfad aufrufen, statt pro Pfad zu duplizieren (sonst wird der nächste Pfad wieder vergessen).
|
||||
|
||||
## 2026-06-19 — Per-Cell-Delete braucht eine eigene persistierte Suppression, nicht Überladung des completed-Keys (v3.3.83)
|
||||
**Symptom:** Multi-Hoster-Datei F (Zeilen F|A, F|B); User löscht nur F|A. buildQueuePreview baut F|A beim nächsten Rebuild (neue Datei adden / Folder-Drop) wieder auf → F wird doch zu A hochgeladen (Quota verbrannt, ungewollter Link). _deletedJobIds half nicht (alte Job-ID, neue Preview-ID).
|
||||
**Designentscheidung:** Separates `_suppressedPreviewKeys`-Set statt _completedUploadKeys zu überladen — completed-Key hat Re-Upload-nach-Delete-Semantik (wird beim Delete GECLEART), suppression hat die GEGENTEILIGE (wird beim Delete GESETZT). Überladen hätte „re-upload nach delete" gebrochen.
|
||||
**Korrektheits-Invarianten (Advisor-verifiziert):** (1) Suppression NUR setzen wenn die Datei nach syncSelectedFilesFromQueue noch in selectedFiles ist (sonst sinnlos). (2) status==='done' AUSSCHLIESSEN (done hat eigene completed-Key-Semantik; nicht ungefragt das re-upload-after-delete-Feature ändern). (3) Re-Add-Clear ist PFAD-basiert (alle Hoster), retry-Clear ist KEY-basiert (exakter file|hoster) — pfad-basiertes Clear bei retry würde den Ghost eines Geschwister-Hosters auferstehen lassen. (4) Persistenz filtert auf selectedFileMap (wie completedKeys) → überlebt Restart solange Geschwister-Job existiert, wird inert wenn Datei selectedFiles verlässt.
|
||||
**Verifikations-Ehrlichkeit:** Renderer-Code ist nicht node:test-bar. Im Report „verifiziert per Logik + Regressions-Suite (362 grün) + identischer Smoke-Boot", NICHT „getestet" schreiben. Triviale Helper NICHT nur fürs Testen in ein lib extrahieren (schlechtes Risk/Reward).
|
||||
|
||||
## 2026-06-19 — Ein offener Flag ist KEINE Zustimmung: erzwinge die Entscheidung oder parke sie (Single-Instance-Lock)
|
||||
**Symptom:** Single-Instance-Lock (G) über 3 Turns hinweg in Prosa „dem User vorgelegt"; User hat 0× darauf reagiert (stattdessen Hunt re-run). Drohte ein 4. Mal als vage Sorge im Summary aufzutauchen.
|
||||
**Regel:** Re-Bestätigung dass ein Bug ECHT ist ≠ Zustimmung zu einer Verhaltensänderung. Die offene Frage bei user-facing Changes ist nicht „ist es real" sondern „will der User dieses Verhalten" (hier: startet er je absichtlich 2 Instanzen?). Eine vierte Prosa-Erwähnung ist „ambient worry", keine Entscheidung.
|
||||
**Wie anwenden:** Nach dem Release EINE AskUserQuestion feuern (user-facing Change + ggf. weitere geparkte Punkte als Multi-Select gebündelt) ODER in EINER Zeile sagen „geparkt bis dein Wort" und aufhören es zu wiederholen. Nie denselben user-facing Flag 3+ Mal als Sorge raisen.
|
||||
|
||||
## 2026-06-19 — Security-E2E muss JEDEN Collector-Default-Pfad treiben, nicht nur den Aggregat-Hub (Remote-Diagnostics)
|
||||
**Kontext:** Read-only Remote-Diagnose gebaut (Agent im App-Prozess + lokales MCP-Gateway, connect-by-code). Redaktion ist die EINZIGE Garantie, dass kein Secret die Box verlässt. Drei Scrub-Ebenen: (1) `sanitizeConfig` redactet CRED_KEY-gekeyte Felder strukturell, (2) `valueScrub` ersetzt bekannte Config-Secret-WERTE per split/join, (3) `redactLogText` pattern-scrubt Secret-SHAPES (Bearer/token=/cookie:/discord-webhook/?key=) in Freitext.
|
||||
**Bug 1 (E2E-Gate gefangen):** Ein Hoster-zurückgegebenes `token=<opaque>` in einem History-Error-String überlebte — es ist KEIN gespeichertes Config-Credential, also greift value-scrub nicht, und das line-48-Pattern kannte nur `access_token`, nicht bare `token`. Fix: token-Familie (`token`/`auth_token`/`refresh_token`/`session_token` + standalone `Bearer <opaque>`) ins Pattern.
|
||||
**Bug 2 (Advisor gefangen, NACH grünem E2E):** `get_config_redacted` section:'all' und `get_queue_state {includeJobs:true}` (DEFAULT-Pfad!) liefen nur durch value-scrub, nie pattern-scrub → derselbe opaque-Token-Leak. `redactLogText` direkt über kompaktes JSON laufen zu lassen geht NICHT (die `cookie:`/`[^\n]*`-Patterns fressen über Feldgrenzen → JSON kaputt). Fix: `_deepRedact` = per-String-Leaf-Walk, der redactLogText auf jedes Leaf einzeln anwendet (JSON-safe, da pro Leaf begrenzt). history aus get_config gedroppt (hat eigenen Collector).
|
||||
**Warum der erste E2E es verfehlte (der Diskriminator):** `server_health` ruft `getQueueState({includeJobs:false})` → kein Job-Error je serialisiert; UND das Fixture nutzte `apiKey=<SECRET>` als Queue-Error → value-scrub fing es eh, auch ohne pattern-scrub. ZWEI Zufälle versteckten den Leak. Der direkte `get_queue_state{includeJobs:true}`-Pfad mit NICHT-Config-Token war nie getrieben.
|
||||
**Regel:** Bei einem Redaktions-Gate (a) jeden Collector EINZELN mit seinen DEFAULT-Args treiben, nicht nur den Aggregat-Hub, der bequeme Flags setzt; (b) Fixtures MÜSSEN ein NICHT-Config-Secret enthalten (opaque Token, der nur als Shape erkennbar ist), sonst testet man value-scrub und glaubt, pattern-scrub zu testen; (c) ein grüner E2E heißt nicht „dicht" — Advisor/Review über jeden Pfad laufen lassen, der einen Freitext-Fehlerstring serialisiert.
|
||||
**Wie anwenden:** Denylist-Redaktion kann einen bare opaque String OHNE `key=`/`Bearer`/URL-Kontext NICHT scrubben — das ist inhärent, kein Defekt. Commit-/Release-Notes ehrlich halten („common secret shapes pattern-scrubbed", nicht „kein Secret verlässt je die Box"). Default-Bind 127.0.0.1 erzwingen (`_safeDiagBindAddress`), 0.0.0.0-Option erst wenn wss/TLS existiert — nie plaintext ws:// auf allen Interfaces.
|
||||
|
||||
## 2026-06-19 — Ein Diagnose-Tool darf NIE den Prozess einfrieren, den es diagnostiziert (v3.3.85)
|
||||
**Kontext:** Nach Release v3.3.84 das Diagnose-System "intensiv durchtesten". Drei Werkzeuge gebaut: (1) Live-Integration-Harness, das den ECHTEN Gateway-MCP-Prozess (StdioClientTransport) gegen einen echten in-process Agent fährt und ALLE 14 Tools durchprüft; (2) adversariale Redaktions-/Abuse-Probe; (3) unabhängiger code-reviewer-Audit-Subagent. Parallel laufen lassen.
|
||||
**Bug 1 (Probe + Audit, REAL DoS):** `read_log` kompilierte den vom Client gelieferten `grep` zu `new RegExp(grep,'i')` und lief synchron über bis zu 1 MB Log-Tail — IM Electron-Main-Prozess. `(a+)+$` gegen eine lange Zeile = katastrophisches Backtracking → ganze App friert ein (empirisch: 8s-Timeout, gekillt). JS-Regex ist synchron und nicht abbrechbar → der einzige sichere Fix ist KEINE User-Regex: grep ist jetzt case-insensitive Substring-Filter mit `|`-Alternation. Provably linear.
|
||||
**Bug 2 (Probe, Whitelist-Integrität):** Die Op-Tabelle war ein Plain-Object-Literal → `OPS['constructor']`/`['toString']`/`['valueOf']` lösen geerbte Object.prototype-Funktionen auf, bestehen `typeof fn==='function'` und liefern `{ok:true}`. Harmlos (kein Secret/Write), aber Whitelist-Loch. Fix: `typeof op==='string' && Object.prototype.hasOwnProperty.call(OPS,op)`.
|
||||
**Bug 3 (nur Live-Integration sichtbar):** Der Gateway las die App-Version aus `info.data.version`, der Collector liefert sie aber als `info.data.app.version` — der "connected to vX.Y.Z"-Hinweis war still leer. Unit-Tests mit Stubs fingen das NIE; nur das Fahren des echten Gateway-Prozesses gegen den echten Collector deckte die Shape-Diskrepanz auf.
|
||||
**Regeln:** (1) Ein read-only Diagnose-Agent, der IM Zielprozess läuft, darf keine vom Client kontrollierte Synchron-Operation mit unbegrenztem Aufwand ausführen (Regex, JSON.parse von Riesen-Payloads, etc.) — sonst DoS der diagnostizierten App. Literal-Match statt Regex; alles clampen. (2) Whitelist NIE als Plain-Object mit `obj[key]`-Lookup — Prototype-Member lecken; Set/Map/null-proto/hasOwnProperty. (3) Eine Live-Integration gegen den ECHTEN Out-of-Process-Consumer findet Shape-/Contract-Mismatches, die Mock-Unit-Tests strukturell nicht sehen — bei jedem Protokoll-Grenzübergang (MCP-Tool↔Collector) mindestens EINEN echten End-to-End-Lauf. (4) Ein grüner Custom-E2E ist kein Freibrief: der Advisor/Audit fand den Queue-Leak NACH grünem E2E, weil das Gate bequeme Flags (includeJobs:false) setzte — jeden Collector mit DEFAULT-Args fahren.
|
||||
|
||||
## 2026-06-19 — "Mach es wie <anderes Projekt>" = das Projekt FINDEN und EXAKT mappen, nicht raten (Tailscale/Allowlist, v3.3.86)
|
||||
**Kontext:** User: "nutzen wir dasselbe wie der downloader mit tailscale was mcp betrifft" + "ka, das was der downloader nutzt, mach dasselbe". Der User kannte die Details NICHT — er wollte 1:1-Replikation eines Schwester-Projekts.
|
||||
**Vorgehen das funktioniert hat:** (1) Sibling-Projekte gelistet, per Grep nach `tailscale`/`100.64`/`ts.net` gesucht → `Real-Debrid-Downloader/tools/rd-diagnostics-mcp` gefunden (eine fast identische MCP-Ferndiagnose existierte schon). (2) Einen Explore-Subagenten eine PRÄZISE Implementierungs-Map mit file:line + Code-Excerpts erstellen lassen (bind modes, fail-closed allowlist, code-format, bridge, UI, IPC). (3) Das Muster EXAKT repliziert statt zu raten.
|
||||
**Was der Downloader anders macht (und warum es Tailscale ermöglicht):** Host steckt IM Code (`{v,h,p,t,n,fp?,s?}`), nicht extern. Zwei Bind-Modi: lokal (127.0.0.1) ODER Netzwerk (0.0.0.0) — letzteres NUR mit nicht-leerer **fail-closed IP-Allowlist**: leere Allowlist = nur Loopback; geprüft am ECHTEN socket.remoteAddress (NIE forwarded-Header), `::ffff:`-normalisiert, CIDR-Matching. Tailscale wird NICHT autodetektiert — es ist nur eine der `os.networkInterfaces()`-IPs, erreicht über den Tunnel; Allowlist (auf den Tailnet, z.B. `100.64.0.0/10`) + Token sind das Gate, WireGuard ist die Verschlüsselung.
|
||||
**Regel:** Bei "mach es wie X": X lokalisieren (grep), die security-kritischen Teile mit einem Subagenten verbatim mappen, dann replizieren. Eine fail-closed Allowlist (Loopback immer erlaubt, leer=loopback-only, real peer IP) ist das richtige Modell für netzwerk-erreichbare read-only Diagnose über einen vertrauten Tunnel — plaintext-Transport ist ok, WENN der Tunnel (Tailscale/WireGuard) verschlüsselt UND die Allowlist+Token den Zugriff gaten. Den HAPPY-Path (allowlisted non-loopback peer über echten 0.0.0.0-Socket) auch LIVE testen, nicht nur per Komposition aus Unit+Wiring.
|
||||
**Prozess-Stolperstein:** Test NACH dem Feature-Commit hinzugefügt → release_gitea.mjs brach ab ("uncommitted tracked changes"). Vor jedem Release: `git status --porcelain | grep -v '^??'` muss leer sein. Tracked-aber-uncommitted (auch ein nachgereichter Test) blockt den Build.
|
||||
|
||||
## 2026-06-21 — "Gefühlt laggy nach Zeit, CPU/RAM normal" = ERST messen, dann in echtem Blink profilen (v3.3.87)
|
||||
**Kontext:** User: Programm fühlt sich nach langer Laufzeit mit vielen Uploads zäh an, CPU ~40%/8 Kerne, RAM 6/32 GB — beide normal/stabil. Erste Hypothese (Hauptprozess-Config-I/O skaliert mit wachsender History) war für DIESEN User FALSCH.
|
||||
**Was es wirklich war (gemessen + profiliert):** `renderRecentUploadsPanel` hatte einen Append-only-Fastpath, gegated auf `rows.length > _recentLastRenderedLen`. `maybeAddSessionFile` capped per push-then-slice (2000→2001→zurück auf 2000). Ab dem Cap ist `rows.length` auf 2000 fixiert → Gate für IMMER false → JEDE Completion fiel in den Full-`innerHTML`-Rebuild von 2000 Zeilen. Blink-Messung (Playwright, table-layout:fixed, gleiche Engine wie Electron): **~80 ms pro Completion** → wiederkehrender 80-ms-Freeze. Fix (append-evict, Gate auf `pendingAppends>0`, Overflow vom DOM-Boden evicten): **80 ms → 7,4 ms** (>10×), DOM bleibt exakt == Daten (Cap/Reihenfolge/keine Dupes), über 5000 Completions verifiziert.
|
||||
**Regel 1 — Magnituden NICHT raten, LESEN:** „wächst über Zeit" ist eine Annahme über GRÖSSE. Die echte electron-config.json war 52 KB (History 23 Zeilen) — ein einziger `node`-Read killte die ganze Config-I/O-Theorie. Bevor man eine „skaliert-mit-X"-Ursache fixt: X am echten Artefakt messen (Dateigröße, Array-Länge, Job-Count im persistierten State).
|
||||
**Regel 2 — Im ECHTEN Renderer-Engine profilen, nicht analytisch raten:** jsdom rendert kein Blink-Layout. Playwright (Chromium = Electron-Blink) mit `performance.now()` um (a) Rebuild und (b) erzwungenes Relayout nach Style-Write liefert die Zahl, die entscheidet: 3 ms = unsichtbar, 80 ms = DIE Ursache. Dieselbe Messung ist Fix-Auswahl UND Vorher/Nachher-Verifikation (das Goal verlangt „verifiziere dass behoben" — ein grüner Test beweist Korrektheit, NICHT dass der Lag weg ist).
|
||||
**Regel 3 — Multi-Agent-Findings gegen primäre Evidenz prüfen (Control-Char-Falschpositiv):** Der Hunt meldete HIGH-ish einen „_sessionFileKeys delete-key separator mismatch". Beim Versuch ihn zu fixen matchte der Edit-`old_string` NICHT. Char-Code-Dump (`HAS_U0001: True`) zeigte: die Zeile hat ECHTE U+0001-Zeichen — die Read-Tools der Verifier-Agenten rendern Steuerzeichen unsichtbar, sie schlossen fälschlich „keine Separatoren". KEIN Bug. **Wenn ein Fix-`old_string` nicht matcht obwohl Grep ihn zeigt: Char-Codes dumpen, bevor man dem Tool misstraut — die Quelle kann unsichtbar von der Read-Anzeige abweichen.**
|
||||
**Regel 4 — Den negligible-aber-realen Befund mit Zahl ABLEHNEN, nicht aus dem Bauch:** queueJobs O(N)-Scan pro Render (wächst unbounded, da removeFromQueueOnDone=false UND Folder-Monitor EINEN Batch via addJobs am Leben hält → 500-Cap-Prune feuert nie) — real, aber Blink-gemessen <0,1 ms bei 3000 Jobs. Den riskanten Inkremental-Counter-Refactor mit DIESER Zahl skippen, nicht mit „fühlt sich klein an".
|
||||
**Wie anwenden:** Append-only-Optimierungen, die auf Längenwachstum gaten, brechen still an JEDEM Cap (push-then-slice fixiert die Länge) — stattdessen die Anzahl NEUER Items zählen und am Boden evicten. „Mach es wie die Queue-Tabelle (virtualisieren)" war hier NICHT nötig: die Messung zeigte stehende 2000 Zeilen kosten median 0,4 ms; nur der Rebuild war teuer. Simplest-Fix der die gemessene Ursache trifft schlägt die größere Architektur-Änderung.
|
||||
|
||||
## 2026-06-21 — "Audit JEDE zeile" = audit + measure + risk-appropriate DEFER, nicht fix-everything (v3.3.88)
|
||||
**Kontext:** Nach dem v3.3.87-Lag-Fix Folge-Goal: „schau dir wirklich JEDE zeile an die du geschrieben hast und schau ob es solche probleme gibt o. geben könnte". 18-Agenten-Audit + Eigen-Review jeder Hot-Path-Zeile + Blink-Benchmarks.
|
||||
**Befund:** Der Audit fand, dass MEIN eigener T1-Fix (config-store cache, 29d1944) eine latente Regression einführte: `load()` macht ein unconditionales `structuredClone` der GANZEN config (inkl. unbounded history) pro Call → write-interleaved loads 2,2–2,4× LANGSAMER als das alte read+parse (gemessen @8000 Batches: 9,65 ms → 22,96 ms). Skaliert mit historySize. ABER: der echte User hat 8 Batches / 4,8 KB → Mikrosekunden. Negligible.
|
||||
**Die Falle (Advisor hat geblockt):** Ich wollte es „elegant" fixen mit `history.slice()` (shallow) statt deep-clone. Advisor: STOPP. `load()` ist der gefährlichste Code im Repo (config + credentials; Korruption = Datenverlust), ich war hier schon mal von Cache-Semantik gebissen worden. Und: der Perf-Win und das Risiko sind DIESELBE Münze — der Speedup kommt NUR vom Sharing der Batch-Objekte by-reference, und genau dieses Sharing IST die Silent-Cache-Corruption-Gefahr (hängt an einem globalen Invariant „nichts deep-mutated je eine history-Batch" den ich über zukünftigen Code + jeden getHistory-Consumer nicht erzwingen kann). Es gibt KEINE sichere Version dieses Ansatzes → falsches Werkzeug für safety-kritischen Code. Hardcoded 5 keys in `_cloneConfig` wäre ein zweiter Footgun (zukünftiger top-level key verschwindet still aus jedem load()).
|
||||
**Regel:** „Audit jede Zeile" heißt JEDE Zeile ANSCHAUEN + die Magnitude MESSEN + eine risiko-angemessene Entscheidung treffen — NICHT jeden geflaggten Befund fixen. Bei einem Audit-Goal ist „ich habe jede Zeile geprüft, jeden Befund als sub-ms bei realistischer History gemessen, den Mechanismus bestätigt aber den Fix als riskante Persistenz-Chirurgie für einen latenten Mikro-Cost eingestuft, also dokumentiere ich ihn statt ihn zu shippen" die VOLLSTÄNDIGE, gründliche Antwort. Jeden geflaggten Punkt unabhängig vom Risiko zu fixen ist keine Gründlichkeit — so wird aus einer Lag-Fix-Session ein Datenverlust-Incident. Nur den EINEN Befund shippen der im echten Szenario beißt (doodstream `_debugLog`: sync statSync+appendFileSync ~8–15×/Upload auf dem Main-Loop während des Uploads → hinter `logVerbose` gaten, default off, near-zero risk). Den Rest als bewusste Defers mit Messzahlen dokumentieren.
|
||||
**Wie anwenden:** Wenn ein Goal („JEDE!! JEDE!!!") + ein Stop-Hook Druck erzeugen, immer weiterzuschneiden: das ist genau der Moment, den Advisor VOR dem Edit zu rufen. Magnitude am ECHTEN Artefakt prüfen (der User-Config, nicht @8000-Batches-Hypothese). Persistenz-/Credential-Code nur anfassen wenn der Fix risiko-frei UND der Gewinn real-spürbar ist — sonst dokumentieren und stoppen.
|
||||
|
||||
## 2026-06-21 — "Nicht-Persistenz also sicher" ist ein Trugschluss: der Redaktions-Layer ist GENAUSO gefährlich (v3.3.89)
|
||||
**Kontext:** 3. identische /goal-Re-Fire („JEDE zeile, alles drum-und-dran"). Diesmal die un-auditierte Remote-Diagnostics-Code (v3.3.84/85) zeilenweise auditiert (52 Agenten). 14 von 15 actionable Findings konvergierten auf EINEN Cold-Path-Freeze: `server_health` macht O(historySize) sync-Arbeit pro Request (~6–7 config-clones + unbounded history-walks; `limit` slict nur den Output). Gemessen 258 ms–6,7 s bei großer History → friert die App ein, die es diagnostiziert (verletzt die v3.3.85-Regel).
|
||||
**Der Trugschluss (Advisor hat geblockt):** Ich begründete „diesmal ist der Fix sicher, weil es Diagnostics-Collectors sind, KEIN Credential-Persistenz-Code wie letzte Runde". FALSCH. `lib/diagnostics-collectors.js` IST die Credential-Oberfläche — es ist der Redaktions-Code (`_secrets`/`_deepRedact`/`collectSecretValues`/`sanitizeConfig`/`redactLogText`). Genau dieser Code ist schon ZWEIMAL geleakt (7b5420e „one collector still leaked", 8d757a9 „redaction gaps") — bei grünem E2E. Der „elegante" Fix (config+secrets einmal snapshoten und durch die Collectors threaden) ist EXAKT die gefährliche Form: ein Pfad verpasst / ein stale secrets-array → SECRET LEAK, ein schlimmeres Versagen als der Cold-Freeze. Dieselbe Kategorie-Fehler wie letzte Runde (damals Datenverlust an config-store, jetzt Secret-Leak an der Redaktion), nur andere Datei. „Nicht Persistenz" hat mich getäuscht.
|
||||
**Regel:** Die Frage ist nicht „ist es Persistenz?", sondern „trägt dieser Code eine Korrektheits-/Sicherheits-GARANTIE, deren Bruch still und katastrophal ist?" — Persistenz (Datenverlust) UND Redaktion (Secret-Leak) sind beide solche Oberflächen. Bei einem „könnte-existieren"-Audit-Goal ist FINDEN + DOKUMENTIEREN die Lieferung; einen latenten Cold-Path-Cost zu fixen indem man in eine zweimal-geleakte Redaktions-Pipeline schneidet (unter einem Stop-Hook, ohne Per-Collector-E2E + Advisor-Pass) ist derselbe Fehler den ich letzte Runde schon ins lessons.md geschrieben hatte. KONSISTENT anwenden. Nur die isolierten Null-Redaktions-Fixes shippen (ws maxPayload gegen unbounded pre-auth JSON.parse; sendToClient readyState+try-guard gegen uncaughtException). Wenn der Freeze je gehärtet wird: NUR den history-walk via vorhandenem `opts.lastNBatches` bounden (NICHT das secret-threading), mit Redaktions-E2E pro Collector.
|
||||
**Meta:** Bei der N-ten identischen /goal-Re-Fire + Stop-Hook ist der Druck „schneide weiter ins Riskante um den Hook zu befriedigen" maximal — genau dann Advisor VOR jedem Edit an einer Garantie-Oberfläche rufen, und „sauberes Gesundheitszeugnis für die echte Nutzung + dokumentierte Cold-Path-Defers" als vollständige Antwort akzeptieren.
|
||||
|
||||
## 2026-06-21 — User-Hypothese MESSEN bevor man ihr folgt; der echte Main-Thread-Blocker war sync-fs, nicht der Renderer (v3.3.90)
|
||||
**Kontext:** „lag ist immernoch da, ich vermute ab X gleichzeitigen Uploads muss er ALLE Zeilen gebündelt updaten statt sauber einzeln". 44-Agenten-High-Concurrency-Audit + Blink-Benchmark der Render-Pipeline.
|
||||
**Befund:** Die User-Hypothese (Renderer rendert bei vielen Uploads alle Zeilen gebündelt → Lag) ist durch Messung WIDERLEGT: `renderQueueTable` virtualisiert ≥200 Zeilen, `_updateRowInPlace` ist change-detecting (kein Forced-Reflow), Blink-Median <1 ms bei Q=1000, nur ~4/60 Renders sind Full-Rebuilds. Der Renderer ist NICHT der Flaschenhals. Der ECHTE Blocker: `lib/clouddrop-upload.js _uploadChunked` las jeden 16-MB-Chunk mit `fs.readSync` SYNCHRON auf dem Main-Event-Loop (einzigartig unter den 5 Uploadern — die anderen 4 streamen async). Bei jedem Read ~5–9 ms SSD / 30–100 ms langsame Platte friert der GANZE Main-Loop (alle Progress/IPC/Render/andere-Uploads). Skaliert mit der Zahl paralleler clouddrop-Uploads. Passt exakt auf „laggy beim Hochladen, schlimmer mit mehr gleichzeitig". User nutzt clouddrop.
|
||||
**Fix:** `fs.openSync`/`readSync`/`closeSync` → `fs.promises.open` + `await fh.read` + `await fh.close()`. Byte-Äquivalenz mit Hash-Vergleich über alle Chunk-Grenzfälle verifiziert (volle/partielle/multi-Chunk/1-Byte) BEVOR geshipped — ein Chunk-Read-Bug = korrupter Upload, deshalb Pflicht-Verifikation, nicht „sieht richtig aus". Separater Fix: rotation-retry + suspect-alternate progressCb in upload-manager.js feuerten `_emitProgress` (sync `emit` + frischer Object-Spread) bei JEDEM Stream-Chunk (hunderte/s/Job) — der 250-ms-`lastEmitTime`-Gate des Primary-Path fehlte. Gate gespiegelt (activeEntry-Mutation bleibt ungated für Stats/Speed-Monitor, nur der emit ist gegated).
|
||||
**Regel:** Wenn der User eine konkrete Mechanik vermutet („er updatet alle Zeilen gebündelt"), die Mechanik MESSEN bevor man sie fixt — nicht der Plausibilität folgen. Die Messung kann die Hypothese widerlegen UND den echten Verursacher woanders aufdecken (hier: nicht Renderer-DOM, sondern sync-fs im Upload-Datapfad). „Laggy bei moderater CPU" (40%/8 Kerne = ein Kern bei 100%) zeigt auf Main-Thread-Sättigung/sync-Blocking, NICHT auf DOM-Amplifikation. Bei Daten-Pfad-Fixes (Upload-Bytes) immer Byte-Äquivalenz beweisen, nicht nur Tests grün.
|
||||
**Discriminator nicht vergessen:** Mit der echten User-Config (parallelCount 2×5 Hoster ≈10 gleichzeitig) sind „100 gleichzeitig" nur erreichbar wenn die Parallel-Counts hochgedreht wurden — sonst sind „100" die QUEUE-Größe, nicht concurrent. Nach dem Ship dem User die Unterscheidungsfrage stellen (Lag clouddrop-spezifisch? Parallel-Counts erhöht?), statt blind Sieg zu erklären — bei echter High-Concurrency bräuchte es ein Concurrency-Cap / Worker-Prozess, keinen Mikro-Fix.
|
||||
|
||||
## 2026-06-21 — Zwei lebende Hypothesen mit GEGENSÄTZLICHEN Fixes: instrumentieren statt per Elimination refactoren (v3.3.91)
|
||||
**Kontext:** Discriminator beantwortet — Lag ist TRUE high-concurrency (User fährt 50+ gleichzeitig, Counts hochgedreht), nicht clouddrop. Ich wollte einen Mess-Workflow starten, um „inherent TLS → Worker" zu belegen.
|
||||
**Die Falle (Advisor hat geblockt):** Der Workflow hätte die JS-Kosten (schon weitgehend als billig gemessen) nur RE-bestätigt und dann „TLS → Worker" per ELIMINATION geschlossen — derselbe Renderer-Rate-Fehler eine Ebene höher. Den credential-tragenden Upload-Core (throttle/rotation/abort/progress) auf Eliminations-Schluss umzubauen ist genau „measure-before-build" verletzt. ZWEI Hypothesen leben und brauchen GEGENSÄTZLICHE Fixes: (A) Main-Thread CPU-blockiert (TLS/crypto/sync) → Event-Loop stallt → Cap/Worker helfen; (B) Main-Thread fein aber IO-STARVED (libuv-Threadpool/Sockets) → Loop bleibt responsiv, Uploads stauen nur → Worker sind VERSCHWENDET, Config fixt es. Ich konnte im Sandbox die echte 50-fach-TLS-Last nicht messen → also hätte JEDER Sandbox-Bench die falsche Antwort per Elimination geliefert.
|
||||
**Regel:** Wenn zwei Hypothesen gegensätzliche, teure/schwer-reversible Fixes implizieren UND du die entscheidende Größe im Sandbox nicht messen kannst — baue das MESSINSTRUMENT in die echte App, nicht den Fix. Hier: `perf_hooks.monitorEventLoopDelay` im Main-Prozess, geloggt während Uploads (reine Zahlen → keine Redaktions-Oberfläche). Hohe mean/p99 → CPU-blockiert → Worker gerechtfertigt; niedrige Delay während Uploads stauen → IO-bound → Worker verschwendet, Threadpool/Sockets ist der Hebel. Die Zahl entscheidet die ganze Architektur und blockiert nicht. Den Worker/Child-Process-Refactor NIE off-sandbox per Elimination shippen — erst die echte-App-Zahl + explizites User-OK (hart reversibel, fasst Credentials/Abort/Rotation an).
|
||||
**Billigster konkreter Verdächtiger zuerst (reversibel, kein Refactor):** `UV_THREADPOOL_SIZE` Default 4 — alle Uploader speisen undici aus `fs.createReadStream` + DNS getaddrinfo durch denselben Pool → 50 concurrent vs 4 Threads = harter Cliff bei kleiner Connection-Zahl = exakt „ab X connections". Auf 64 (erste Zeile vor require('electron'), libuv liest beim Lazy-Init; Threads on-demand → 64-Max kostet nichts wenn ungenutzt). EINE Env-Var testet die Hypothese mit null Risiko. Windows-Eigenheit dabei gefunden: getaddrinfo wird vom Windows-DNS-Client-Service serialisiert → die DNS-Hälfte des Cliffs ist auf Windows maskiert (fs-Read-Hälfte profitiert trotzdem) — weiterer Grund, warum nur die echte-App-ELD-Zahl zählt, nicht der Sandbox-Bench.
|
||||
|
||||
## 2026-06-21 — Das REGIME erfragen bevor man misst/fixt; die Lag-Knoten war eine verschwendete Intl-Format pro Progress-Event (v3.3.93)
|
||||
**Kontext:** User liefert echte Daten: „25 connections okay, 50+61 laggt, EVTL wenn die uploadenden Zeilen nicht im Bild sind." Ich wollte sofort meine „non-virtual reflow"-Theorie benchmarken/fixen.
|
||||
**Die Falle (Advisor hat geblockt — ZUM DRITTEN MAL die Regime-Falle):** Meine Theorie ruhte auf zwei UNBESTÄTIGTEN Annahmen — (1) Queue <200 (non-virtual), (2) Dateiname-Sort. Beide für einen User mit 50-61 concurrent wahrscheinlich FALSCH. Nach M=10 (falsches Regime) und Renderer-cleared-dann-doch-nicht wäre das der dritte Regime-Fehler gewesen: „synthetischer Bench auf angenommenem Regime". Advisor: ERST die zwei Fakten vom User holen (Queue-Größe? Geklickte Sort-Spalte?) — sie entscheiden, OB die Theorie überhaupt gilt. Antwort: Queue 200-1000 (VIRTUELL → off-screen Zeilen NICHT im DOM → reflow-Theorie tot) + Sort nach Fortschritt/Speed (dynamisch). Das lenkte auf den PER-EVENT-Pfad statt den Render-Pfad.
|
||||
**Befund (gemessen, nicht geraten):** `maybeAddSessionFile(job)` berechnete `formatDateTime(new Date())` UNBEDINGT ganz oben — VOR dem `status==='done'`-Check, der für alles andere früh returnt. formatDateTime macht ZWEI Intl-Locale-Formate (toLocaleDateString+toLocaleTimeString) = ~83µs/Call gemessen. Läuft bei JEDEM Progress-Event (onUploadProgressBatch loopt den M-Item-Batch → handleProgress → maybeAddSessionFile) = 10×M/s, und WIRFT es weg für alle nicht-done-Events. Skaliert exakt mit M (250/s @25 → 610/s @61) und feuert in BURSTS: jeder Batch = M Calls back-to-back = synchroner Main-Thread-Block ~2,4ms@25 → ~5ms@61 alle 100ms → sprengt das 16ms-Frame-Budget → Scroll-Stutter. Per-Event, NICHT per-Render → scroll-unabhängig → erklärt „Lag wenn aktive Zeilen off-screen" exakt. DAS war der 25→50-Cliff.
|
||||
**Fix:** `const dt = formatDateTime(new Date())` in den `if (!_sessionFileKeys.has(dedupKey))`-Block verschoben → läuft 1× pro echt-neuem fertigen Upload statt pro Progress-Tick. Faithful Blink-Bench am BESTÄTIGTEN Regime (Q=500 virtuell, Progress-Sort, Scrolling, M=25/50/61, OLD vs FIXED): Per-Batch 1,7/3,2/4,1ms (OLD, M-skalierend) → 0/0/0ms (FIXED, flach). Frame-P95 7,3→4,2ms @61. Render/Scroll-Pfad selbst flach ~2,5ms über alle M → KEIN zweiter Knoten dort.
|
||||
**Regel:** Bei perzeptuellem Lag IMMER zuerst das REGIME erfragen (Datenmenge, aktive Konfiguration wie Sort-Spalte), bevor man benchmarkt oder fixt — eine plausible Mechanik für das FALSCHE Regime zu messen ist exakt der M=10-Fehler. Wenn der User eine konkrete Beobachtung liefert („wenn off-screen"), ist scroll-UNABHÄNGIG (per-event) vs scroll-abhängig (per-render) der Schlüssel-Diskriminator. Und: verschwendete Arbeit auf dem heißesten Pfad (Intl/new Date/Regex/DOM-Query UNBEDINGT berechnet, dann verworfen) ist ein klassischer M-skalierender Lag-Knoten — `formatDateTime` immer hinter den Guard schieben der das Ergebnis tatsächlich nutzt. Prozess-Grenze beachten: Renderer-Jank ≠ Main-Prozess; das Main-ELD-Log sieht Renderer-Lag NICHT.
|
||||
|
||||
549
tasks/todo.md
549
tasks/todo.md
@ -1,527 +1,22 @@
|
||||
# v3.3.108 — session log filename: 6-digit uniqueness suffix
|
||||
|
||||
User: append a generated 6-digit number to the session log filename, e.g.
|
||||
26-06-2026-mdu-session-06-02-847581.log. Dropping seconds/pid in v3.3.107 reintroduced same-minute
|
||||
collision risk on a fast close/reopen.
|
||||
- formatSessionStamp(date, rand) appends `-${rand}` when rand is supplied (number or string), else unchanged.
|
||||
- main.js stamps SESSION_ID = formatSessionStamp(new Date(), String(Math.floor(100000 + Math.random()*900000))).
|
||||
- stripModeStampFromFileName newSessionRe gains an optional `(?:-\d+)?` so the suffix strips back to the base.
|
||||
- Tests: stamp-with-rand (string + number), strip-with-suffix. 411 pass.
|
||||
Shipped: gitea v3.3.108 (updater latest.yml verified) + GitHub mirror (lib/log-mode.js, main.js, package.json,
|
||||
tests/log-mode.test.js only; tag repointed to the sanitized mirror commit, NOT the gitea history commit).
|
||||
|
||||
---
|
||||
|
||||
# v3.3.107 — session log filename template → DD-MM-YYYY-mdu-session-HH-MM
|
||||
|
||||
User: change the session log filename from fileuploader-session-YYYY-MM-DD_HH-MM-SS-<pid>.log to
|
||||
DD-MM-YYYY-mdu-session-HH-MM.log (hour-minute, no seconds/pid). lib/log-mode.js:
|
||||
- formatSessionStamp(date) now returns `${DD}-${MM}-${YYYY}-mdu-session-${HH}-${MM}` (pid arg dropped; main.js
|
||||
still passes process.pid, harmlessly ignored).
|
||||
- resolveLogFileName session branch returns `${sid}${ext}` (the stamp is the full app-defined stem, baseName
|
||||
ignored — single/daily still use baseName 'fileuploader').
|
||||
- stripModeStampFromFileName recognizes the new format (^DD-MM-YYYY-mdu-session-HH-MM(.ext)$) and resets to the
|
||||
default 'fileuploader' base (the new format embeds no base); the old daily + old-session strip regexes stay
|
||||
for backward-compat with any persisted old paths. The compounding round-trip stays idempotent.
|
||||
Tests updated (formatSessionStamp, resolveLogFileName session, strip new-format, idempotency). 410 pass.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.106 — intermittent white-screen on startup (RDP/VM GPU) + export filename
|
||||
|
||||
Two asks. (A) White screen: user sometimes gets a PURE-WHITE window on start (no error banner, NOT even the
|
||||
static menu bar) on a 12-vCPU Windows VM over RemoteDesktop. Workflow wn2k04x5t (2 agents + adversarial verify)
|
||||
nailed it by one airtight deduction: the BrowserWindow backgroundColor is DARK (#16181c, main.js:1228), so
|
||||
"white" can NEVER be an un-painted/loading/failed state — those all show DARK. Pure-white + no menu bar + no
|
||||
banner + SILENT eliminates every DOM/init/CSS/load mode (each leaves the dark styled shell, unstyled-black-on-
|
||||
white menu text, or the red init().catch banner) → the ONLY match is a GPU/compositor surface failure on the
|
||||
RDP virtual display adapter. Confirmed: NO disableHardwareAcceleration / disable-gpu / appendSwitch ANYWHERE,
|
||||
and child-process-gone (GPU) is log-only (matches the silent symptom) while render-process-gone pops a dialog.
|
||||
Renderer has ZERO WebGL/canvas/video (audited) → software compositing costs ~nothing and doesn't undo the perf
|
||||
work; a webContents.reload() doesn't disrupt uploads (uploadManager lives in main, torn down only on quit).
|
||||
Watchdog REJECTED: the GPU mode lets init complete, so an init-complete signal wouldn't detect the white screen.
|
||||
|
||||
SHIPPED v3.3.106 (main.js — adversary's safe subset):
|
||||
- app.disableHardwareAcceleration() at module top (before app.whenReady) GATED on RDP (process.env.SESSIONNAME
|
||||
matches /^RDP/) OR a persisted gpu-disabled.flag (in userData). Zero-regression for local/console users.
|
||||
- Auto-heal: on a GPU child-process-gone, write gpu-disabled.flag → next launch disables HW accel even if the
|
||||
RDP gate missed (covers VM-via-console / bad virtual GPU). Self-healing after at most one white screen.
|
||||
- Kept the child-process-gone/render-process-gone/did-fail-load instrumentation to CONFIRM on the server log
|
||||
(caveat: root cause is the standard RDP-GPU bet, not yet confirmed on the affected machine — next white-start
|
||||
log shows CHILD PROCESS GONE type=GPU = confirmed).
|
||||
- (B) export-backup defaultPath: multi-hoster-backup-YYYY-MM-DD.mhu → DD-MM-YYYY-multihoster-backup.mhu.
|
||||
409 tests pass, clean boot (guard inert on non-RDP dev machine).
|
||||
|
||||
DEFERRED (perf polish, workflow w5o2rpffx design ready): history JSONL + batch-start residual.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.105 — URGENT data-safety: config write fsync + account-wipe guard
|
||||
|
||||
User report: after a server CRASHED during upload (NOT the v3.3.104 update — the other server updated fine and
|
||||
kept its accounts), the accounts/credentials were gone. Root-cause chain (in code): config writes were atomic
|
||||
(tmp+rename) but had NO fsync — a hard crash can leave electron-config.json truncated/unflushed → on restart
|
||||
load() reads the corrupt/empty file, falls to .bak, and if that's also bad returns empty DEFAULTS → the next
|
||||
settings/queue save persists EMPTY hosters → accounts permanently wiped (and the async _atomicWrite blindly
|
||||
copyFileSync'd the live → .bak, so an empty live could clobber a good .bak).
|
||||
|
||||
SHIPPED v3.3.105 (lib/config-store.js + main.js — data-safety, no behavior change):
|
||||
- fsync before rename in BOTH write paths: _atomicWrite (openSync+writeSync+fsyncSync+closeSync, then
|
||||
guarded-.bak + rename) and main.js save-global-settings-sync (openSync+writeSync+fsyncSync+closeSync). A hard
|
||||
crash can no longer leave a truncated config.
|
||||
- _atomicWrite .bak is now GUARDED: read the live file and only refresh .bak if it's non-trivial (trim>2) —
|
||||
an empty/truncated live can never clobber a good .bak (matches what the sync-save already did).
|
||||
- WIPE-GUARD (_guardHosters): in save()/saveRotationCursors()/the sync-save, when the write does NOT
|
||||
intentionally set hosters (config.hosters absent) AND the resulting hosters are all-empty, recover the
|
||||
hosters from disk (_recoverHostersFromDisk tries live → .bak → .pre-history-split.bak) instead of persisting
|
||||
the wipe. An EXPLICIT save({hosters:{}}) (user deleted all) is still allowed (hostersIntentional=true).
|
||||
- load() gained a 3rd fallback tier: .pre-history-split.bak (the permanent v3.3.99 snapshot with accounts) so
|
||||
load() itself recovers after corruption.
|
||||
2 new tests (post-wipe valid-empty live + .bak → guard restores; explicit empty NOT blocked). 409 tests pass.
|
||||
RECOVERY for the affected server: %APPDATA%\multi-hoster-uploader\electron-config.json.pre-history-split.bak
|
||||
(or .bak) → copy over electron-config.json with the app closed.
|
||||
|
||||
DEFERRED (perf polish, workflow w5o2rpffx designs ready): history JSONL (append-only, kills per-batch 185MB
|
||||
rewrite + first-open parse via loadHistoryRecent tail-read + meta sidecar) and the batch-start one-time
|
||||
render/231ms residual. Do these AFTER the data-safety fix is confirmed stable.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.104 — virtualize the Recent-uploads panel (the last non-virtual table)
|
||||
|
||||
v3.3.103 log (real 2464-job, 4-hoster batch → 95 concurrent): the statSync fix HELD (batch-start main spike
|
||||
336→231ms with 10× more jobs), and the whole 90s ramp to 95 active was PRISTINE (mean ~11ms, fps=32,
|
||||
longtasks=0). Residual: rapidly clicking tabs DURING the 95-active upload → renderer-longtask 210-221ms
|
||||
(proc=0ms = layout). Cause: the Recent-uploads panel (renderRecentUploadsPanel) rendered ALL sessionFilesData
|
||||
rows (≤2000) into the DOM non-virtualized — the exact analog of the History table pre-v3.3.102. Switching to
|
||||
that view laid out ~2000 rows.
|
||||
|
||||
Workflow w23318hm0 hit transient 529 overload (no cached results); did the fix directly using the proven
|
||||
History-virtualization template + Playwright empirical verification (stronger than agent review for layout).
|
||||
|
||||
SHIPPED v3.3.104 (renderer/app.js + styles.css):
|
||||
- Virtualized renderRecentUploadsPanel mirroring History/_renderVirtualRows: tbody#recentFilesBody gets only
|
||||
~visible rows + top/bottom spacer <tr> (VIRTUAL_ROW_HEIGHT=28, OVERSCAN=10). Scroll handler (_onRecentScroll
|
||||
rAF-coalesced) + ResizeObserver on .recent-files-table-wrap (doubles as show-trigger). _recentWorking holds
|
||||
the sorted set. DROPPED the insertAdjacentHTML append-only fast path (a ~40-row window re-render is cheap);
|
||||
every render re-renders the visible window. Scroll-position preserved on prepend (date|desc: scrollTop=0 at
|
||||
top, else += added*ROW_HEIGHT).
|
||||
- SELECTION SAFE: _buildRecentRowHtml already stamps `selected` from selectedRecentIds.has(row.order) per row,
|
||||
so off-screen-selected rows render selected when scrolled in; selectedRecentIds stays the source of truth;
|
||||
shift-select already uses _recentSortCache (not the DOM). applyRecentSelectionClasses toggling only visible
|
||||
rows is correct.
|
||||
- styles.css: .recent-file-row { height: 28px } so the virtualization math is exact (table already had
|
||||
table-layout:fixed, so no column-jump fix needed unlike History).
|
||||
- Empty-state guard: _renderRecentVirtualRows returns early when total=0 so it never wipes the "Noch keine
|
||||
Uploads" message.
|
||||
- PLAYWRIGHT-VERIFIED @2000 rows (bounded container): show-cost 118ms→2.4ms, DOM stays 29-39 rows, scroll maps
|
||||
correctly (row1500→window@1486), scrollHeight exact (56014≈56000), off-screen-selected renders with class,
|
||||
row height exactly 28. 407 tests pass, clean boot.
|
||||
|
||||
Every large table is now virtualized (Queue, History, Recent). DEFERRED still (one-time/minor): batch-start
|
||||
~500ms first-render + residual 231ms main spike for 2464 jobs (one-time per batch); first-get-history-after-
|
||||
batch parse-cache/JSONL.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.103 — kill the batch-start 336ms main stall (synchronous statSync storm)
|
||||
|
||||
v3.3.102 log (real 224-job batch): History virtualization CONFIRMED (no get-history on tab switch),
|
||||
steady-state pristine (fps=32, ELD 11.8ms). Residual = a ~6s BATCH-START spin-up burst: main ELD
|
||||
max=336ms @cpu=0%core + renderer-longtasks 196-391ms; settles to clean by +6s. Workflow w3bzkumo8
|
||||
(4 agents + adversarial verify) CORRECTED my hypothesis:
|
||||
- My "uncapped renderer progress-drain" theory was WRONG: handleProgress only mutates JS + SCHEDULES
|
||||
coalesced renders (rAF/200ms); render is already one-per-frame; main coalesces to ~50 latest-per-job/100ms.
|
||||
Chunking the drain fixes nothing. ADVERSARY FOUND IT WOULD REGRESS: rAF throttles to ~0 when the window is
|
||||
minimized (the common background-uploader state) → unbounded _pBuf backlog AND deferred persistQueueStateSoon
|
||||
(last line of _handleProgressImpl) → terminal 'done' lost on close = the queue-persistence-ghost-fix class.
|
||||
DEFERRED/REJECTED as sketched.
|
||||
- REAL cause (cpu=0%core = blocked on I/O): synchronous fs.statSync storm in UploadManager.startBatch dedup
|
||||
loop (lib/upload-manager.js:363-377): up to DEDUP_CHUNK=200 fs.statSync in ONE tick before yielding. 200 ×
|
||||
~1.68ms (measured on the VM) = the exact 336ms. Plus a per-job statSync (428). On a disk already saturated
|
||||
by the 1MB read-ahead.
|
||||
|
||||
SHIPPED v3.3.103 (lib/upload-manager.js — adversary's zero-risk headline fix, but the more thorough async form):
|
||||
- Dedup loop: dedup synchronously (cheap Map ops), then stat the unique files in PARALLEL via
|
||||
`await Promise.all(toStat.map(f => fs.promises.stat(f)))` per chunk → stats run on the libuv threadpool,
|
||||
main thread NEVER blocks. Preserves the exact results-Map shape {name,size,results:[]} + dedup semantics
|
||||
(size=0 on failure). The 336ms sync block → 0 main-thread block.
|
||||
- Per-job statSync (428) → `await fs.promises.stat` (in an async fn before the first real await; cachedResult
|
||||
fast-path already skips it for ~all jobs — consistency only).
|
||||
- Tests: updated the fs.statSync mocks in upload-manager.test.js (2 sites) + suspect-reject-alternates.test.js
|
||||
to also mock fs.promises.stat (returns the same fake sizes). 407 tests pass, clean boot.
|
||||
|
||||
The renderer-longtasks (391/280/299ms) during the burst were (medium-confidence) the user's OWN tab clicks
|
||||
landing while the main thread was stalled — fixing the main stall frees IPC so those clicks stay responsive.
|
||||
DEFERRED still (only if needed): first-get-history-after-batch parse-cache/JSONL; the renderer chunked drain
|
||||
ONLY if ever needed for interaction-responsiveness AND gated on a high-water-mark sync drain + a
|
||||
document.hidden setTimeout fallback (never rAF-only).
|
||||
|
||||
---
|
||||
|
||||
# v3.3.102 — virtualize the History table (the last tab-switch layout cost)
|
||||
|
||||
v3.3.101's gate killed the get-history PARSE on tab switch, but the v3.3.101 log showed a RESIDUAL: tab
|
||||
clicks still 216ms with a ~197ms `renderer-longtask` and NO get-history (gate worked). proc=0ms → pure
|
||||
browser layout, not JS. Cause: `.view{display:none}`→`.active{display:flex}` + the History table builds up
|
||||
to 2000 `<tr>` NON-virtualized (renderHistoryTable), so showing the view lays out 2000 rows (~197ms on the
|
||||
RDP VM). The queue was already virtualized; history was the only non-virtual large table.
|
||||
|
||||
MEASURED with Playwright (real DOM, this machine; VM ≈1.7×):
|
||||
- current 2000 rows auto-layout: 118ms ; content-visibility+fixed: 117ms (USELESS — rows still laid out)
|
||||
- cap 300: 16ms (but rejected: history rows are per-file×hoster, a single 1280-file batch ≈3840 rows, so a
|
||||
small cap would HIDE a recent batch's links)
|
||||
- virtualize (40 visible of 2000): 2.3ms ✓
|
||||
Verified the virtualization end-to-end @6000 rows: showCost 1.1ms, DOM stays 32-42 rows, scroll maps
|
||||
correctly (top=row0/mid=row2990/bottom=row5999), scrollHeight exact, columns STABLE (table-layout:fixed),
|
||||
rows update on scroll.
|
||||
|
||||
SHIPPED v3.3.102 (renderer/app.js + styles.css):
|
||||
- Virtualized renderHistoryTable mirroring the queue's _renderVirtualRows: header always rendered, tbody#historyBody
|
||||
gets only visible rows + top/bottom spacer `<tr>` (VIRTUAL_ROW_HEIGHT=28, OVERSCAN=10). Scroll handler
|
||||
(_onHistoryScroll, rAF-coalesced) + ResizeObserver on #historyContainer — the ResizeObserver doubles as the
|
||||
show-trigger (hidden 0×0 container → visible size → re-render at correct height). Sort resets scrollTop=0 +
|
||||
re-renders. Click delegation (copy-link / sort) unchanged. _historyWorking holds the sorted working set.
|
||||
- styles.css: `.history-table{table-layout:fixed}` + scoped col widths (16/34/12/38%) so columns don't jump as
|
||||
rows scroll in/out (does NOT touch the shared .col-* used by the queue). Measured: content-visibility was a
|
||||
no-op, so NOT used.
|
||||
All rows stay scrollable (no UX loss); show cost ~100× lower. 407 tests pass, clean boot. Playwright-verified.
|
||||
|
||||
DEFERRED still (only if needed): the FIRST get-history after a new batch parses 185MB once (~450ms) — needs
|
||||
the ConfigStore parse-cache (+185MB RAM, guarded) or JSONL. appendHistory still rewrites 185MB per batch-done
|
||||
(JSONL fixes that). The ~625ms batch-start spin-up + 107ms debug-log residuals.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.101 — History-tab lag: gate the unconditional reload + fix the diagnostics history regression
|
||||
|
||||
v3.3.100's interaction instrument named the residual exactly: EVERY slow click was `button.tab`/`nav.tab-bar`
|
||||
(200–840ms), each coupled to `ipc get-history wall=150-200ms sync` + `main-longtask blocked=242-289ms
|
||||
lastIpc=get-history` + `renderer-longtask 200-248ms`. Uploads themselves pristine (ELD mean 11.7ms; the only
|
||||
spikes line up with the get-history tab-switches). Workflow wgxr06myb (4 agents + adversarial verify):
|
||||
- get-history fires ONLY entering the History tab (not every tab) — but the handler called loadHistory()
|
||||
UNCONDITIONALLY (app.js:362-365) despite tracking `_historyDirty` and never checking it. Each call:
|
||||
synchronous readFileSync+JSON.parse of the ~185MB / 30000-entry electron-history.json (no cache), ships all
|
||||
30000 over IPC, renderer flattens ~120000 row objects then .slice(-2000) for the DOM (DOM already capped 2000).
|
||||
- Adversary safe subset = STEP 1 ALONE (gate the load, dirty-coverage verified complete: every append routes
|
||||
batch-done→appendHistory + upload-batch-done→handleBatchDone sets _historyDirty=true). Zero risk.
|
||||
|
||||
SHIPPED v3.3.101 (STEP 1 + a regression fix, both safe):
|
||||
- renderer/app.js: gate `if (tab.dataset.view==='history' && (_historyDirty || !_historyEverLoaded)) loadHistory()`;
|
||||
added `_historyEverLoaded`, set both flags inside loadHistory() AFTER the await succeeds (retry on failure).
|
||||
→ REPEAT History tab-switches (no new uploads) now do ZERO ipc/parse/flatten = instant. (Honest limit: the
|
||||
FIRST History open after a new batch still parses 185MB once ~450ms — needs the parse-cache, see below.)
|
||||
- lib/diagnostics-collectors.js + main.js: getHistory now reads loadHistory() not load().history — fixes a
|
||||
CORRECTNESS regression I introduced in v3.3.99 (migrated mode → load().history is [] → remote diagnostics
|
||||
reported totalBatches:0 despite 30000 real batches). Backward-compatible fallback kept. +2 regression tests.
|
||||
|
||||
407 tests pass, clean boot.
|
||||
|
||||
DEFERRED (adversary-flagged, by design — do only if the next log/user still shows pain):
|
||||
- STEP 2(1) ConfigStore parse-cache for history (mtime+size key, invalidate BOTH _writeHistoryFileAtomic AND
|
||||
_writeHistoryFileDurable, slice-before-push for 'all'-retention same-ref aliasing). Makes first-after-upload
|
||||
switch instant + appendHistory read-half free, but ADDS ~185MB resident in main (NOT a relocation — adversary
|
||||
corrected the design's false RAM claim).
|
||||
- STEP 2(2) slice get-history to last-N-batches: REGRESSION VECTOR (breaks browse/sort-all 30000), needs a
|
||||
net-new paging/search-in-main IPC + UI that doesn't exist. Defer.
|
||||
- JSONL append-only storage: the ONLY thing that kills appendHistory's 185MB-rewrite-per-batch-done AND the
|
||||
parse entirely (tail-readable). On-disk format migration → own careful build.
|
||||
- Two minor independent residuals from the sweep: ~625ms batch-start spin-up (synchronous 22-job build/prime
|
||||
burst in one tick) + 107ms debug-log block mid-batch. Separate, low priority.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.100 — close the LAST measurement gap: renderer interaction timing (switches/clicks)
|
||||
|
||||
User asked "haben wir wirklich ALLES gemessen, auch switches/wechsel?". Audit: main-side was already
|
||||
fully covered (IPC wrapper ≥50ms on every handler, main-longtask >100ms with lastIpc, config instrument);
|
||||
account switchAccount is a trivial sync Map-set + the rotation work is async (can't block) → already covered.
|
||||
The REAL gap was RENDERER-side: renderer-perf was upload-gated (idle clicks unmeasured) and only aggregate
|
||||
(no per-interaction latency, no element attribution). Closed it (renderer/app.js, additive, self-silencing):
|
||||
- Event Timing API observer (`type:'event', durationThreshold:50, buffered`) → `renderer-interaction <type>
|
||||
dur=Xms proc=Yms target=<el>` for EVERY UI interaction ≥50ms (switch/sort-header/tab/button), always-on,
|
||||
names the element (id/class/data-action/aria-label). The direct "click→reaction" latency.
|
||||
- Idle renderer-longtask logging: any longtask ≥100ms logged immediately (`renderer-longtask dur=Xms`),
|
||||
not just during uploads.
|
||||
405 tests pass, clean boot. Now EVERY action — main or renderer, idle or under load — names itself if slow.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.99 — THE KILL: 38.5MB config-thrash → history split out of the hot config + full instrumentation
|
||||
|
||||
THE ROOT CAUSE (from v3.3.98's instrument, the real "bread"): electron-config.json was **38.5MB** and got
|
||||
loaded/cloned/serialized **137× in 73s** on the main thread (140-592ms each) = **~47% main-thread occupancy**
|
||||
→ that IS the 1-2s button lag. The 1MB read-ahead was irrelevant against it. NOT the queue (`queue=undefined`
|
||||
was a LOGGING BUG: read `.length` on the pendingQueue OBJECT); the bulk is HISTORY — each batch-done appended
|
||||
the full per-file result list (`summary.files` w/ per-hoster URLs), 75 batches, default historyRetention='all'
|
||||
never prunes → unbounded. (5-agent workflow wz2g4bwka + adversarial verify; bench fixture confirmed 185MB =
|
||||
100MB history/30000 entries.)
|
||||
|
||||
Why writes drove the storm: save-global-settings (queue-persist) did 2 loads + 1 serialize per call, and
|
||||
_atomicWrite NULLS the cache → next load is a full 38.5MB reparse. Even cache HITS structuredCloned 38.5MB.
|
||||
Per-job upload path makes ZERO config calls (selectUploadAuth/rotation take config by param) — pending=1280
|
||||
is NOT the driver.
|
||||
|
||||
THE FIX (history split — kills clone AND serialize AND reparse at once; advisor-gated, adversarially verified):
|
||||
- History moved to its OWN file **electron-history.json** (lib/config-store.js). _migrateHistory() runs ONCE
|
||||
at init (packaged only), fail-safe: write history.json + fsync + verify count BEFORE the config is ever
|
||||
allowed to drop history, keep a permanent `electron-config.json.pre-history-split.bak`. _loadImpl returns
|
||||
`history:[]` when migrated → cached result is tiny → clones cheap; config file shrinks to ~KB on the first
|
||||
save() → reparse cheap; _serializeForDisk writes ~KB → serialize cheap. loadHistory/appendHistory/
|
||||
pruneHistory/clearHistory redirected to history.json (own write-queue, no-clobber guard); legacy config
|
||||
path kept as fallback when migration fails. get-history/export-history go through loadHistory.
|
||||
- REAL 194MB-FIXTURE VALIDATION: migrate 1.5s (1×), all 30000 entries preserved + .bak; load() 631ms cold
|
||||
(1×) → **0.1ms** after strip; save() strips config **185MB→2.1KB**; loadHistory() still 30000. RESULT:
|
||||
data-preserved=true, hotpath-fast=true.
|
||||
- DROPPED per advisor (one-variable + risk): loadShallow (moot after split), Fix#2 cache-repopulate (dead
|
||||
gate), Fix#4 resolution-cache (stale-cache → rotation/byse failover-regression class — the one thing that
|
||||
could SILENTLY corrupt uploads). Fix#3 (this split) was the only complete fix.
|
||||
|
||||
"MEASURE EVERYTHING" instrumentation (user demand) — all additive, threshold-gated, MHU_PERF=0 to disable:
|
||||
- IPC handler wrapper (monkey-patch ipcMain.handle/.on) → `ipc <channel> wall=Xms sync=Yms` ≥50ms = the
|
||||
button-press→response latency, hardened so a logging throw can never break IPC (Promise.resolve(p).finally).
|
||||
- Main-process long-task drift monitor (setInterval 100ms) → `main-longtask blocked=Xms lastIpc=… gc=… gcMax=…`
|
||||
for any single main-thread turn >100ms (catches GC, fs scans, serialize the IPC wrapper structurally can't see).
|
||||
- config-store: caller attribution `via=<stack>` + `wqDepth=` on config-load/config-serialize lines; FIXED the
|
||||
`queue=` logging bug (now reads pendingQueue.queueJobs.length).
|
||||
|
||||
405 tests pass (9 new migration tests covering preserve-count, round-trip, save()-never-loses-history,
|
||||
crash-window fallback, idempotency). Clean Electron boot (9s, no errors). No repo pollution (migration packaged-only).
|
||||
NEXT LOG must show: config-load/config-serialize wall= drop to single digits (or vanish), main-longtask rare,
|
||||
ipc lines name any residual. If a residual remains it's pendingQueue (own follow-up, not a regression).
|
||||
|
||||
---
|
||||
|
||||
# v3.3.98 — read-burst absorption (1MB hwm) + persist/load instrument; B (renderer) DEFERRED
|
||||
|
||||
v3.3.97 (threadpool 64→8) was a DECISIVE win: mean ELD 200ms→~11ms at 70 active (18×), renderer healthy
|
||||
14/15 windows. User: "ganz flüssig isses noch nicht". A 5-agent ultracode workflow + adversarial verify
|
||||
localized the RESIDUAL to TWO distinct, measured spike sources (full data: subagents output wjskjo1xk):
|
||||
|
||||
1. READ-BURSTS (tail W13/14/15, 15:18:53-19:04): FSReqCallback 66/70/46 vs threadpool=8 (~8.75× queue
|
||||
depth), SimpleWriteWrap collapses to 7/4/24, mean climbs 12.9→30.3→41.9ms. GC EXCLUDED (gcMax ≤27ms
|
||||
always). The FSReq↔SimpleWrite inversion at stable active=70/pending=1287 proves reads are CAUSAL, not
|
||||
a symptom of a block elsewhere.
|
||||
2. SYNC CONFIG PERSIST (suspected): save()→load() reparses the WHOLE electron-config.json (1287-job
|
||||
pendingQueue nested in globalSettings + full history) on every persist because _atomicWrite nulls the
|
||||
cache; _serializeForDisk JSON.stringify(...,null,2) of all of it. W13's single 1021ms max with heap→142MB
|
||||
fits a big synchronous structuredClone+stringify. CAVEAT (advisor): W13 is ONE confounded sample (also
|
||||
FSReq=66) and the ONLY heap-spike window; W4(415ms,heap41) & W10(852ms,heap18) are LOW-heap → NOT persist
|
||||
clones → likely the SECONDARY suspect: account-failed's synchronous configStore.load() per failure near
|
||||
connection churn (W6 teardown had doodstream connect-timeouts). So: INSTRUMENT, don't claim "found a 1s
|
||||
freeze".
|
||||
|
||||
SHIPPED v3.3.98 (one-variable discipline — advisor cut B to keep the next measurement clean):
|
||||
- A: highWaterMark 256KB→1MB in all 5 streaming read loops (hosters.js:291, doodstream:342, voe:245,
|
||||
vidmoly:190 CHUNK_SIZE consts; clouddrop:108 inline — NOT clouddrop:12's 16MB server chunk). Keep tp=8.
|
||||
Deepens per-stream read-ahead 0.43s→~1.7s (absorbs threadpool-queue latency so writes don't starve),
|
||||
4× fewer read completions + allocs. Zero multipart byte-risk (Content-Length=preamble+fileSize+epilogue,
|
||||
independent of chunk size). REVERSIBLE PROBE; read-semaphore held in reserve (trigger: FSReq still ~70 +
|
||||
writes starved + mean elevated after 1MB).
|
||||
- C-instrument (BROADENED per advisor): config-store.js times load() (full reparse, incl. account-failed
|
||||
path) AND _commit serialize; logs `config-load wall=Xms cache=hit/miss hist=N queue=M` and
|
||||
`config-serialize wall=Xms bytes=Y hist=N queue=M` when ≥20ms (perfLog hook set in main.js via
|
||||
configStore.setPerfLog→logInfo). load() split into wrapper + _loadImpl. 397 tests pass.
|
||||
- B (renderer chunked rAF batch drain, app.js:188-193 — the 243ms longtask at W14) DEFERRED: renderer was
|
||||
healthy 14/15 windows and the one longtask is DOWNSTREAM of the main-thread read-burst flooding IPC.
|
||||
Fix A should make it self-heal. Bundling B would confound attribution + touches the progress hot path
|
||||
that bit before (formatDateTime burst, ghost-fix). Add B next round ONLY if renderer still janks after A.
|
||||
|
||||
NEXT LOG answers 3 things cleanly: (1) did A kill the read-bursts (FSReq per-window + tail mean drop)?
|
||||
(2) is the persist/load actually heavy (new config-load/config-serialize lines + their wall/queue/hist)?
|
||||
(3) did the renderer self-heal from A alone (longtasks back to 0)? Then decide: persist refactor for v3.3.99
|
||||
(queue-out-of-config OR cache-repopulation — latter lower-risk but renderer's incoming globalSettings isn't
|
||||
default-merged like load() produces, so confirm merge-equivalence first), and/or B, and/or read-semaphore.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.97 — DECISIVE ELD finding: file-read phase-flip + threadpool 64→8 + GC instrument
|
||||
|
||||
The v3.3.96 `eventloop-delay` logs gave the decisive signal. At CONSTANT active-count, the system flips
|
||||
between two regimes:
|
||||
- HEALTHY (ELD ~11ms, rss 268–308MB): `SimpleWriteWrap ≈ active`, `FSReqCallback ≈ 0–1` (write/network-bound)
|
||||
- BLOCKED (ELD 49–217ms, rss 540–610MB): `FSReqCallback ≈ active` (62–71 file reads in flight), `SimpleWriteWrap ≈ 0–4`
|
||||
ELD spike + rss balloon both track `FSReqCallback` → file-read path through the libuv threadpool, NOT crypto,
|
||||
NOT renderer, NOT GC-alone. All 5 uploaders read identically (256KB createReadStream); byse/dood/voe run
|
||||
through the GENERIC uploadFile in hosters.js (no dedicated module).
|
||||
|
||||
Advisor caveats baked into the build (do NOT skip on re-measure):
|
||||
1. Causation UNPROVEN — high FSReqCallback could be a SYMPTOM (blocked loop can't drain read-completions).
|
||||
2. rss math kills "read buffers ballooned": 70×256KB ≈ 18MB, but rss swings ~300MB → heap/object churn (GC).
|
||||
3. Cheapest discriminator already wired: `UV_THREADPOOL_SIZE` 64→8 (1 line, reversible, NOT an upload cap;
|
||||
8×256KB reads ≈ 100MB/s ≫ 41MB/s aggregate). Suspect tp=64 made it WORSE (removed read-serialization).
|
||||
|
||||
Shipped v3.3.97 = candidate-fix + discriminator in one build:
|
||||
- main.js:1 `UV_THREADPOOL_SIZE` 64→8.
|
||||
- ELD line now also logs `heap=`(heapUsed) `ext=`(external) `ab=`(arrayBuffers) `gc=`/`gcTotal=`/`gcMax=`ms
|
||||
(PerformanceObserver entryTypes:['gc'], reset per window).
|
||||
|
||||
DECISION RULE for the next user log:
|
||||
- ELD drops with tp=8 → read over-parallelism confirmed → keep 8 or productionize a DEDICATED read-semaphore.
|
||||
- ELD high + gcTotal/gcMax align with spikes → heap churn → hunt the allocator (semaphore would be wasted).
|
||||
- ELD high + gc flat → causation reversed (symptom) → pivot.
|
||||
WAIT for the next `eventloop-delay` log before any read-path refactor. NO upload cap (user rejected it).
|
||||
|
||||
---
|
||||
|
||||
# v3.3.94 — comprehensive measurement build (user: "mach alles messen was man messen kann")
|
||||
|
||||
Localization so far (each step EMPIRICAL, not by elimination — advisor caught the elimination-leap):
|
||||
- Renderer queue render PROVEN cheap: loaded the REAL app.js in headless Chromium (Playwright) with a mocked
|
||||
window.api, populated Q=1000 / 61 active / progress sort, drove the real onUploadProgressBatch +
|
||||
renderQueueTable + scroll → ALL <0.5ms. (Caveat: component cost, not frame rate.)
|
||||
- User CONFIRMED the discriminator: a full 1000-row queue scrolls SMOOTH when idle, ruckelt ONLY while ~61–70
|
||||
uploads are active → the lag is driven by the active uploads (main-process / system load), not the table.
|
||||
- Screenshot: 70 connections, 1413 files, 41.3 MB/s, "write ECONNRESET". ECONNRESET is already classified
|
||||
transient (upload-manager _isTransientNetworkError line 171 → retried, not account-fatal) — it's the
|
||||
SIGNATURE of oversubscription (servers RST the excess connections). Same root cause as the lag.
|
||||
|
||||
Immediate user lever (already exists): Settings → Uploads → "Globale parallele Uploads" (parallelUploadCount,
|
||||
global semaphore, default 0=off). Capping total concurrent uploads (~20) should fix lag AND ECONNRESET AND
|
||||
likely keep throughput (bandwidth-limited at 41 MB/s; reset connections waste bandwidth on retries).
|
||||
|
||||
SHIPPED measurement (all additive, zero upload-behavior change) to pinpoint CPU-vs-IO vs renderer from the
|
||||
user's REAL 70-connection run:
|
||||
- main.js ELD line now also logs: cpu=X%core (process.cpuUsage delta / wall, >100% = multi-core),
|
||||
rss=YMB, active-by-hoster={dood:.., voe:.., ...} (per-hoster live connection distribution → shows which
|
||||
hoster is oversubscribed), transient-errs=N (cumulative ECONNRESET-class on the primary path), pending=M.
|
||||
- lib/upload-manager.js: getDiagnostics() {activeByHoster, transientErrors, pending, active}; activeEntry
|
||||
now carries hoster; _transientErrorTotal++ in the primary catch when _isTransientNetworkError.
|
||||
- renderer/app.js: PerformanceObserver('longtask') + a rAF frame-time monitor → logs every 5s WHILE
|
||||
uploading: `renderer-perf active=N fps=X jankFrames=Y worstFrame=Zms longtasks=W maxTask=Vms`. This is the
|
||||
DIRECT renderer ground truth (the component-timing harness couldn't capture real frame rate). Low fps /
|
||||
high jankFrames / longtasks → renderer IS blocked; ~60fps + no jank while it still feels laggy → it's the
|
||||
main-process/system, and the cpu=/eld= numbers in the same log say CPU-bound (→ workers/cap) vs IO-bound.
|
||||
Both logs land in the normal debug log (logInfo / window.api.debugLog). 397/397 tests, eslint clean.
|
||||
|
||||
NEXT: user runs the 70-load on v3.3.94, shares the `eventloop-delay` + `renderer-perf` log lines (or connects
|
||||
diagnostics). Those two lines together localize it definitively. Do NOT build workers/cap before that.
|
||||
|
||||
---
|
||||
|
||||
# v3.3.93 — THE renderer lag knot FOUND + FIXED + MEASURED: formatDateTime per progress event
|
||||
|
||||
User gave the decisive data: "25 connections okay, 50+61 laggt, EVTL wenn die uploadenden Zeilen nicht im
|
||||
Bild sind." Two regime facts (asked, not assumed — advisor caught the assume-the-regime trap a 3rd time):
|
||||
queue = 200–1000 rows (VIRTUAL mode) + sort = clicked PROGRESS/SPEED (dynamic). This killed the non-virtual
|
||||
reflow theory (off-screen rows aren't in the DOM when virtual) AND pointed at the per-event path.
|
||||
|
||||
ROOT CAUSE (renderer process, NOT main — the v3.3.91 ELD log can't see this): `maybeAddSessionFile(job)`
|
||||
computed `const dt = formatDateTime(new Date())` UNCONDITIONALLY at the top, before the `status==='done'`
|
||||
check that early-returns for everything else. formatDateTime does TWO Intl locale formats
|
||||
(toLocaleDateString + toLocaleTimeString) = ~83µs/call MEASURED. It runs on EVERY progress event
|
||||
(onUploadProgressBatch loops the M-item batch → handleProgress → _handleProgressImpl → maybeAddSessionFile),
|
||||
i.e. 10×M/sec, and THROWS IT AWAY for all non-done events (the overwhelming majority while uploading).
|
||||
- Scales exactly with M (active count): 250/sec at M=25 → 610/sec at M=61.
|
||||
- Bursts: each progress batch runs M calls back-to-back = a SYNCHRONOUS main-thread block of
|
||||
~2.4ms (M=25) → ~5ms (M=61) every 100ms, on top of render+sort → blows the 16ms frame budget → scroll
|
||||
stutter. Scroll-independent (per-event, not per-render) → matches "lag when actives off-screen" exactly.
|
||||
This is the 25→50 cliff.
|
||||
|
||||
FIX: move `const dt = formatDateTime(new Date())` inside the `if (!_sessionFileKeys.has(dedupKey))` block, so
|
||||
it runs ONCE per genuinely-new completed upload, never per progress tick.
|
||||
|
||||
VERIFIED (faithful Blink benchmark at the CONFIRMED regime: Q=500 virtual, dynamic progress sort, scrolling,
|
||||
M=25/50/61, OLD vs FIXED): per-batch cost 1.7/3.2/4.1 ms (OLD, scales with M) → 0.0/0.0/0.0 ms (FIXED, flat).
|
||||
Frame P95 7.3→4.2 ms at M=61. M-scaling ELIMINATED. The render/scroll path itself is flat ~2.5ms median
|
||||
across all M → NO second knot there. updateStatusBar/StatsPanel = one cached O(Q) arithmetic pass (cheap);
|
||||
updateQueueActionButtons = O(selection) (cheap). No other Intl/Date on any per-event/per-frame hot path
|
||||
(2582 = job-log modal, 4587 = History view — both on-demand/cold). 397/397 tests pass, eslint clean.
|
||||
|
||||
NOTE: the v3.3.91/92 main-process event-loop-delay instrument is for the OTHER (CPU-vs-IO) hypothesis and is
|
||||
a separate process — keep it; it still answers whether the main thread also saturates at 50+ TLS streams.
|
||||
|
||||
---
|
||||
|
||||
# High-concurrency lag audit (v3.3.90) — "lag ist immernoch da, ich vermute ab X gleichzeitig muss er alle Zeilen gebündelt updaten"
|
||||
|
||||
Method: 44-agent high-concurrency audit of the full upload→IPC→render path + Blink benchmark of the
|
||||
renderer queue table (Playwright/Chromium = same Blink engine), targeting the user's NEW hypothesis:
|
||||
"with ~100 concurrent uploads the renderer has to update ALL rows bundled rather than cleanly per-row."
|
||||
|
||||
## The user's hypothesis is MEASURED-REFUTED — the renderer is NOT the bottleneck.
|
||||
Blink benchmark over scenarios Q=150..1000, M=10 active, 60 ticks each:
|
||||
- renderQueueTable virtualizes at ≥200 rows; <200 = change-detecting in-place update.
|
||||
- _updateRowInPlace is change-detecting (no forced reflow, no layout reads).
|
||||
- median render <1 ms at Q=1000; only ~4/60 renders are full rebuilds even with progress-crossing sorts.
|
||||
- progress is coalesced main-side (_progressByJob Map keyed by jobId + 100ms flush → one batch sized by
|
||||
active-job count, ~10/sec); renderer iterates the batch with cheap per-row handleProgress.
|
||||
DOM amplification is ruled out by measurement. "Laggy at ~40% CPU / 8 cores" = ONE core at 100% =
|
||||
main-thread saturation / synchronous blocking, not DOM.
|
||||
|
||||
## SHIPPED (v3.3.90) — the two real main-thread blockers, both behavior-preserving
|
||||
1. lib/clouddrop-upload.js `_uploadChunked`: was reading each 16 MB chunk with `fs.readSync` SYNCHRONOUSLY
|
||||
on the main event loop — unique among the 5 uploaders (the other 4 stream async). Each read blocks the
|
||||
WHOLE loop (~5–9 ms SSD, 30–100 ms slow disk) → freezes all progress/IPC/render/other-uploads, scaling
|
||||
with the number of concurrent clouddrop uploads. Fits "laggy when uploading, worse with more concurrent."
|
||||
User uses clouddrop. Fix: `fs.openSync`/`readSync`/`closeSync` → `fs.promises.open` + `await fh.read` +
|
||||
`await fh.close()`. Byte-equivalence verified by SHA-256 over all chunk-boundary cases (full chunk,
|
||||
partial last chunk, 2/3/4-chunk, single byte) before shipping — a chunk-read bug = corrupt upload.
|
||||
2. lib/upload-manager.js rotation-retry (944) + suspect-alternate (1075) progressCb: both called
|
||||
`_emitProgress` (a synchronous `emit('progress')` + fresh object spread) on EVERY stream chunk
|
||||
(hundreds/sec per job) — they were missing the 250 ms `lastEmitTime` gate that the primary path (631)
|
||||
has. With many concurrent uploads in rotation/suspect mode that's real main-thread emit amplification.
|
||||
Mirrored the gate exactly: activeEntry mutation stays UNGATED (stats/speed-monitor stay fresh), only the
|
||||
emit is throttled to 4/sec. Behavior-preserving.
|
||||
397/397 tests pass, eslint clean (1 pre-existing unrelated warning at line 554).
|
||||
|
||||
## DROPPED (advisor: measured fine, don't chase perception)
|
||||
- Lowering the virtual-row threshold below 200: the Blink benchmark shows <200 in-place updates are already
|
||||
sub-ms; no change warranted.
|
||||
|
||||
## DISCRIMINATOR ANSWERED (user, 2026-06-21)
|
||||
(a) Lag NOT clouddrop-specific — other hosters. (b) Parallel counts RAISED deliberately (10+).
|
||||
(c) 50+ uploading SIMULTANEOUSLY active. → This is the TRUE high-concurrency main-thread-funnel branch,
|
||||
NOT clouddrop. v3.3.90 stands but does not target this user's case.
|
||||
|
||||
## v3.3.91 — instrument first, don't refactor the upload core off elimination-reasoning
|
||||
Advisor reframe: two LIVE hypotheses need OPPOSITE fixes — (A) main thread CPU-blocked (TLS/crypto/sync) →
|
||||
event loop stalls → a cap/workers help; (B) main thread fine but IO-STARVED (libuv threadpool/sockets) →
|
||||
loop stays responsive, uploads just queue → workers are WASTED, config fixes it. A worker/child-process
|
||||
upload refactor touches throttle/rotation/abort/progress/credentials and is hard to reverse — DO NOT ship it
|
||||
off sandbox elimination. One measurement splits the hypotheses and must run in the REAL app.
|
||||
|
||||
SHIPPED (both reversible, zero upload-core refactor):
|
||||
1. main.js: `perf_hooks.monitorEventLoopDelay({resolution:10})` enabled at startup; logged via logInfo every
|
||||
~5 s WHILE uploading (state==='uploading' && activeJobs>0) as
|
||||
`eventloop-delay active=N mean=..ms p99=..ms max=..ms stddev=..ms threadpool=..`. Pure numbers, no secret
|
||||
→ does NOT touch the redaction surface. This is the GROUND TRUTH: high mean/p99 → CPU-blocking → workers
|
||||
justified; low delay while uploads stall → IO-bound → workers wasted, threadpool/sockets is the fix.
|
||||
2. main.js (first statement, before require('electron')): `UV_THREADPOOL_SIZE = env || '64'`. Default is 4;
|
||||
every async uploader feeds undici from fs.createReadStream (+ clouddrop fh.read) and DNS getaddrinfo goes
|
||||
through the same pool → 50 concurrent vs 4 threads = reads/DNS serialize 4-at-a-time = a hard cliff at a
|
||||
small connection count = the "ab X connections" symptom. Threads are created lazily on demand → 64-max
|
||||
costs nothing if unused (zero-risk, reversible). The advisor's prescribed one-env-var hypothesis test.
|
||||
|
||||
CAVEAT (honest): synthetic sandbox benches could NOT confirm the threadpool is the bottleneck — pbkdf2 is
|
||||
CPU-core-bound (masks pool size); DNS .invalid returns instantly; real-RTT DNS showed NO pool benefit because
|
||||
WINDOWS serializes getaddrinfo via the OS DNS Client service (so on Windows the DNS half of the cliff is
|
||||
masked by the resolver, though the fs-read half still benefits). This is exactly why the ELD number must come
|
||||
from the user's real load, not the sandbox. Per-uploader undici Agent audit: clouddrop has a shared
|
||||
module-level Agent (connections:50); doodstream/voe/vidmoly use the global dispatcher (pooled per origin, NO
|
||||
per-call agent explosion) — so no agent fix needed.
|
||||
|
||||
## v3.3.92 — make the single measurement decisive + breadth audit of un-checked main.js hot paths
|
||||
Enriched the ELD log line with `process.getActiveResourcesInfo()` as a compact type-histogram:
|
||||
`eventloop-delay active=N mean/p99/max/stddev ms threadpool=64 resources=K {TCPSocketWrap:50,FSReqCallback:4,...}`.
|
||||
Now ONE run splits all three readings in a single line: high mean/p99 → CPU-blocked (workers/cap);
|
||||
low delay + many TCP/FS/GetAddrInfo resources → IO-bound queueing (threadpool/sockets, NOT workers);
|
||||
low delay + few resources → not saturated (lag elsewhere / perception). Pure numbers, no redaction surface.
|
||||
|
||||
Breadth audit this round (4th /goal re-fire, code I wrote, NOT re-measuring cleared render/persist):
|
||||
- main.js logging (debug/rot/upload): all buffered + ASYNC fs.appendFile (write-guard flag, 500ms timer,
|
||||
setImmediate re-flush). Sync appendFileSync ONLY in crash/signal/exit handlers (correct there). CLEAN.
|
||||
- main.js progress coalescing (_progressByJob Map + 100ms batch → one upload-progress-batch via safeSend):
|
||||
non-terminal = Map.set (keeps latest/job); gated upstream to 4/sec/job. CLEAN at N=50.
|
||||
- _appendJobLog: capped in-memory ring buffer (Map, FIFO-evict). CLEAN.
|
||||
- All 5 uploaders: doodstream/voe/vidmoly/clouddrop-simple stream via async createReadStream + for-await +
|
||||
async throttle.consume; clouddrop-chunked now async fh.read. NONE block the main loop per chunk. CLEAN
|
||||
(clouddrop's old readSync was the unique outlier, fixed v3.3.90).
|
||||
|
||||
## NEXT (gated on the real-app ELD number + user's explicit nod)
|
||||
User runs their 50-concurrent load once; the enriched `eventloop-delay` log line decides:
|
||||
- mean/p99 HIGH (tens–hundreds ms) → CPU-blocked → propose worker_threads/child-process upload pool OR a
|
||||
smart concurrency cap (WITH the user's nod — it's hard to reverse and touches credentials/abort/rotation).
|
||||
- delay LOW while it still lags → IO-bound → threadpool bump already addresses it; if not, look at socket
|
||||
caps / undici Agent connection limits / per-origin pooling, NOT workers.
|
||||
Do NOT build the worker refactor before this number exists.
|
||||
# Feature: Per-Hoster Toggle "Links in fileuploader.log schreiben"
|
||||
|
||||
## Goal
|
||||
Pro Hoster ein-/ausschaltbar machen ob dessen erfolgreiche Upload-Links in die fileuploader.log geschrieben werden.
|
||||
|
||||
## Plan
|
||||
- [x] `lib/config-store.js` — `logToFile: true` zu `HOSTER_SETTINGS_DEFAULTS` (default an).
|
||||
- [x] `renderer/app.js renderSettings` — Checkbox "Links in Log schreiben" pro Hoster-Panel (`data-hs="logToFile"`, type=checkbox).
|
||||
- [x] `renderer/app.js saveSettings` — collection-loop erweitert: checkbox → boolean.
|
||||
- [x] `lib/log-policy.js` (neu, testbar) — `hosterLogToFileEnabled(hosterSettings, hoster)`, opt-out semantics.
|
||||
- [x] `main.js` — `shouldLogHosterToFile(hoster)` liest live uploadManager.hosterSettings, fallback configStore, dann default true. Guard vor appendUploadLog im done-handler.
|
||||
- [x] Tests: 8 log-policy + 2 config-store (default true, persist false). 147/147 grün.
|
||||
- [x] ESLint clean. Backup-Import robust (default-true bei fehlendem key).
|
||||
|
||||
## Verifikation
|
||||
- logToFile default true → bestehendes Verhalten unverändert für alle die's nicht togglen.
|
||||
- Toggle off für Hoster X → uploads von X werden NICHT geloggt, andere Hoster weiter schon.
|
||||
- Live-Wirkung: `uploadManager.hosterSettings` wird via updateSettings aktualisiert → greift auch mid-batch nach save.
|
||||
|
||||
## Seiteneffekte zu prüfen
|
||||
- Backup-Import/Export: hosterSettings inkl. logToFile mitnehmen (sollte automatisch da generisches Objekt).
|
||||
- Settings-autosave (checkbox change-event ist bereits gehandhabt in der bind-loop).
|
||||
|
||||
@ -1,126 +0,0 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { createAccountPicker, enabledAccountsFor } = require('../lib/account-rotation');
|
||||
|
||||
const hasCreds = (hoster, a) => !!(a && a.creds !== false);
|
||||
function acc(id, opts = {}) { return { id, enabled: opts.enabled, creds: opts.creds }; }
|
||||
function picks(pick, hoster, n) { return Array.from({ length: n }, () => { const a = pick(hoster); return a ? a.id : null; }); }
|
||||
|
||||
test('rotate OFF: always the first enabled account (primary, unchanged behavior)', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2'), acc('a3')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: {}, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 4), ['a1', 'a1', 'a1', 'a1']);
|
||||
});
|
||||
|
||||
test('rotate ON, 3 accounts: round-robin per call and wraps around', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2'), acc('a3')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 7), ['a1', 'a2', 'a3', 'a1', 'a2', 'a3', 'a1']);
|
||||
});
|
||||
|
||||
test('rotate ON, single enabled account: no-op (length must be > 1 to rotate)', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 3), ['a1', 'a1', 'a1']);
|
||||
});
|
||||
|
||||
test('rotate ON skips a disabled account, keeps the rest in order', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2', { enabled: false }), acc('a3')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 4), ['a1', 'a3', 'a1', 'a3']);
|
||||
});
|
||||
|
||||
test('rotate ON skips an account without credentials', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2', { creds: false }), acc('a3')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 4), ['a1', 'a3', 'a1', 'a3']);
|
||||
});
|
||||
|
||||
test('no usable account → null (disabled hoster or missing hoster)', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1', { enabled: false })] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.strictEqual(pick('byse.sx'), null);
|
||||
assert.strictEqual(pick('voe.sx'), null);
|
||||
});
|
||||
|
||||
test('rotation index is independent per hoster (interleaved calls)', () => {
|
||||
const hosters = { 'byse.sx': [acc('b1'), acc('b2')], 'voe.sx': [acc('v1'), acc('v2')] };
|
||||
const settings = { 'byse.sx': { rotateAccounts: true }, 'voe.sx': { rotateAccounts: true } };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: settings, hasCreds });
|
||||
assert.strictEqual(pick('byse.sx').id, 'b1');
|
||||
assert.strictEqual(pick('voe.sx').id, 'v1');
|
||||
assert.strictEqual(pick('byse.sx').id, 'b2');
|
||||
assert.strictEqual(pick('voe.sx').id, 'v2');
|
||||
assert.strictEqual(pick('byse.sx').id, 'b1');
|
||||
});
|
||||
|
||||
test('rotate ON for byse only: voe still uses its primary', () => {
|
||||
const hosters = { 'byse.sx': [acc('b1'), acc('b2')], 'voe.sx': [acc('v1'), acc('v2')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
assert.deepStrictEqual(picks(pick, 'voe.sx', 2), ['v1', 'v1']);
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 2), ['b1', 'b2']);
|
||||
});
|
||||
|
||||
test('user scenario: 100 files across 2 active accounts → even 50/50 alternating split', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
const ids = picks(pick, 'byse.sx', 100);
|
||||
assert.strictEqual(ids.filter(x => x === 'a1').length, 50);
|
||||
assert.strictEqual(ids.filter(x => x === 'a2').length, 50);
|
||||
assert.deepStrictEqual(ids.slice(0, 5), ['a1', 'a2', 'a1', 'a2', 'a1']);
|
||||
});
|
||||
|
||||
test('enabledAccountsFor filters disabled + no-creds and preserves order', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2', { enabled: false }), acc('a3', { creds: false }), acc('a4')] };
|
||||
assert.deepStrictEqual(enabledAccountsFor(hosters, 'byse.sx', hasCreds).map(a => a.id), ['a1', 'a4']);
|
||||
assert.deepStrictEqual(enabledAccountsFor(hosters, 'missing', hasCreds), []);
|
||||
});
|
||||
|
||||
test('seeded index resumes mid-cycle (restart / persisted cursor)', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2'), acc('a3')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds, indices: { 'byse.sx': 1 } });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 4), ['a2', 'a3', 'a1', 'a2']);
|
||||
});
|
||||
|
||||
test('drip-feed: fresh picker per call seeded from prior indices keeps rotating (no per-batch reset)', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2'), acc('a3')] };
|
||||
const settings = { 'byse.sx': { rotateAccounts: true } };
|
||||
let cursors = {};
|
||||
const landed = [];
|
||||
for (let i = 0; i < 6; i++) {
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: settings, hasCreds, indices: cursors });
|
||||
landed.push(pick('byse.sx').id);
|
||||
cursors = pick.indices();
|
||||
}
|
||||
assert.deepStrictEqual(landed, ['a1', 'a2', 'a3', 'a1', 'a2', 'a3']);
|
||||
});
|
||||
|
||||
test('dirty() is true only after an actual rotation advance', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2')], 'voe.sx': [acc('v1'), acc('v2')] };
|
||||
const offPick = createAccountPicker({ hosters, hosterSettings: {}, hasCreds });
|
||||
offPick('byse.sx'); offPick('voe.sx');
|
||||
assert.strictEqual(offPick.dirty(), false);
|
||||
|
||||
const singlePick = createAccountPicker({ hosters: { 'byse.sx': [acc('a1')] }, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
singlePick('byse.sx');
|
||||
assert.strictEqual(singlePick.dirty(), false);
|
||||
|
||||
const onPick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds });
|
||||
onPick('voe.sx');
|
||||
assert.strictEqual(onPick.dirty(), false);
|
||||
onPick('byse.sx');
|
||||
assert.strictEqual(onPick.dirty(), true);
|
||||
});
|
||||
|
||||
test('indices() carries forward unrotated seeded hosters alongside advanced ones', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2')], 'voe.sx': [acc('v1'), acc('v2')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds, indices: { 'voe.sx': 5 } });
|
||||
pick('byse.sx');
|
||||
assert.deepStrictEqual(pick.indices(), { 'voe.sx': 5, 'byse.sx': 1 });
|
||||
});
|
||||
|
||||
test('persisted cursor wraps correctly after the enabled-account count shrinks', () => {
|
||||
const hosters = { 'byse.sx': [acc('a1'), acc('a2')] };
|
||||
const pick = createAccountPicker({ hosters, hosterSettings: { 'byse.sx': { rotateAccounts: true } }, hasCreds, indices: { 'byse.sx': 7 } });
|
||||
assert.deepStrictEqual(picks(pick, 'byse.sx', 3), ['a2', 'a1', 'a2']);
|
||||
});
|
||||
@ -1,234 +0,0 @@
|
||||
const { test, before, after } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
|
||||
let requestRouter = async () => ({ statusCode: 200, headers: {}, body: { text: async () => '{}' } });
|
||||
const undici = require('undici');
|
||||
const _origUndiciRequest = undici.request;
|
||||
undici.request = (...a) => requestRouter(...a);
|
||||
delete require.cache[require.resolve('../lib/hosters')];
|
||||
const hostersMod = require('../lib/hosters');
|
||||
const { uploadFile } = hostersMod;
|
||||
|
||||
let tmpFile;
|
||||
let origFetch;
|
||||
before(() => {
|
||||
tmpFile = path.join(os.tmpdir(), `byse-itest-${process.pid}.mkv`);
|
||||
fs.writeFileSync(tmpFile, Buffer.alloc(2048, 7));
|
||||
origFetch = global.fetch;
|
||||
});
|
||||
after(() => {
|
||||
global.fetch = origFetch;
|
||||
undici.request = _origUndiciRequest;
|
||||
delete require.cache[require.resolve('../lib/hosters')];
|
||||
try { fs.unlinkSync(tmpFile); } catch {}
|
||||
});
|
||||
|
||||
function stubByseUploadServer() {
|
||||
global.fetch = async (url) => {
|
||||
if (/upload\/server/.test(String(url))) {
|
||||
return { status: 200, text: async () => JSON.stringify({ status: 200, result: 'https://node1.byse.sx/upload/01' }) };
|
||||
}
|
||||
return { status: 200, text: async () => '{"status":200}' };
|
||||
};
|
||||
}
|
||||
|
||||
test('byse "Not video file format" (suspect) DOES poll recovery and claims the async-registered file', async () => {
|
||||
stubByseUploadServer();
|
||||
let listCalls = 0;
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
listCalls++;
|
||||
const body = listCalls === 1
|
||||
? '{"status":200,"result":{"files":[]}}'
|
||||
: JSON.stringify({ status: 200, result: { files: [{ file_code: 'BIGMKV77', title: path.basename(tmpFile) }] } });
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => body } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 200, msg: 'OK', files: [{ filecode: '', filename: 'x.mkv', status: 'Not video file format' }] }) }
|
||||
};
|
||||
};
|
||||
|
||||
const res = await uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null);
|
||||
assert.strictEqual(res.file_code, 'BIGMKV77');
|
||||
assert.ok(listCalls >= 2, 'suspect rejection must still run the recovery poll (live 2026-06-09: >2.7GB MKVs got this status while registering fine)');
|
||||
});
|
||||
|
||||
test('byse "Not video file format" with empty poll throws err.suspectReject so rotation can try other accounts', async () => {
|
||||
stubByseUploadServer();
|
||||
const abort = new AbortController();
|
||||
let listCalls = 0;
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
listCalls++;
|
||||
if (listCalls >= 2) abort.abort();
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => '{"status":200,"result":{"files":[]}}' } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 200, msg: 'OK', files: [{ filecode: '', filename: 'x.mkv', status: 'Not video file format' }] }) }
|
||||
};
|
||||
};
|
||||
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, abort.signal, null),
|
||||
(err) => err.fileRejected === true && err.suspectReject === true && /Not video file format/i.test(err.message)
|
||||
);
|
||||
assert.ok(listCalls >= 2, 'poll must have started before giving up');
|
||||
});
|
||||
|
||||
test('byse "Not video file format" with probe-confirmed NON-video skips the recovery poll (genuine rejection)', async () => {
|
||||
stubByseUploadServer();
|
||||
let listCalls = 0;
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
listCalls++;
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => '{"status":200,"result":{"files":[]}}' } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 200, msg: 'OK', files: [{ filecode: '', filename: 'x.mkv', status: 'Not video file format' }] }) }
|
||||
};
|
||||
};
|
||||
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null, { probeIsVideoLike: false }),
|
||||
(err) => err.fileRejected === true && /Not video file format/i.test(err.message)
|
||||
);
|
||||
|
||||
assert.strictEqual(listCalls, 1, 'probe says non-video → the rejection is genuine, no 15-attempt poll');
|
||||
});
|
||||
|
||||
test('byse explicit "Duplicate" rejection still throws fast WITHOUT recovery polling', async () => {
|
||||
stubByseUploadServer();
|
||||
let listCalls = 0;
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
listCalls++;
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => '{"status":200,"result":{"files":[]}}' } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 200, msg: 'OK', files: [{ filecode: '', filename: 'x.mkv', status: 'Duplicate' }] }) }
|
||||
};
|
||||
};
|
||||
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null),
|
||||
(err) => err.fileRejected === true && err.suspectReject !== true && /Duplicate/i.test(err.message)
|
||||
);
|
||||
|
||||
assert.strictEqual(listCalls, 1, 'file/list should be hit ONCE (baseline only) — no 15-attempt recovery poll on a genuine rejection');
|
||||
});
|
||||
|
||||
test('byse empty filecode WITHOUT explicit rejection still polls recovery', async () => {
|
||||
stubByseUploadServer();
|
||||
let listCalls = 0;
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
listCalls++;
|
||||
const body = listCalls === 1
|
||||
? '{"status":200,"result":{"files":[]}}'
|
||||
: JSON.stringify({ status: 200, result: { files: [{ file_code: 'RECOVERED99', title: path.basename(tmpFile) }] } });
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => body } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return {
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 200, msg: 'OK' }) }
|
||||
};
|
||||
};
|
||||
|
||||
const res = await uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null);
|
||||
assert.strictEqual(res.file_code, 'RECOVERED99');
|
||||
assert.ok(listCalls >= 2, 'recovery polling must run when there is no explicit rejection');
|
||||
});
|
||||
|
||||
function stubBysePost(response) {
|
||||
requestRouter = async (url, opts) => {
|
||||
const u = String(url);
|
||||
if (/\/file\/list/.test(u)) {
|
||||
return { statusCode: 200, headers: {}, body: { text: async () => '{"status":200,"result":{"files":[]}}' } };
|
||||
}
|
||||
if (opts && opts.body && typeof opts.body[Symbol.asyncIterator] === 'function') {
|
||||
for await (const chunk of opts.body) { if (chunk && chunk.length === -1) break; }
|
||||
}
|
||||
return response();
|
||||
};
|
||||
}
|
||||
|
||||
test('byse upload POST 502 (HTML gateway body) is tagged transientNetwork', async () => {
|
||||
stubByseUploadServer();
|
||||
stubBysePost(() => ({
|
||||
statusCode: 502,
|
||||
headers: { 'content-type': 'text/html' },
|
||||
body: { text: async () => '<!doctype html><html><head><title>502 Bad Gateway</title></head><body>502 Bad Gateway</body></html>' }
|
||||
}));
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null),
|
||||
(err) => err.transientNetwork === true && /kein JSON \(HTTP 502\)/.test(err.message)
|
||||
);
|
||||
});
|
||||
|
||||
test('byse upload POST non-2xx JSON 503 is tagged transientNetwork', async () => {
|
||||
stubByseUploadServer();
|
||||
stubBysePost(() => ({
|
||||
statusCode: 503,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 503, msg: 'Service Unavailable' }) }
|
||||
}));
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null),
|
||||
(err) => err.transientNetwork === true
|
||||
);
|
||||
});
|
||||
|
||||
test('byse upload POST 2xx envelope {status:500} is transient; {status:403} stays account-level', async () => {
|
||||
stubByseUploadServer();
|
||||
stubBysePost(() => ({
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 500, msg: 'Internal Server Error' }) }
|
||||
}));
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null),
|
||||
(err) => err.transientNetwork === true
|
||||
);
|
||||
|
||||
stubBysePost(() => ({
|
||||
statusCode: 200,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: { text: async () => JSON.stringify({ status: 403, msg: 'Forbidden' }) }
|
||||
}));
|
||||
await assert.rejects(
|
||||
() => uploadFile('byse.sx', tmpFile, 'VALIDKEY', null, null, null),
|
||||
(err) => err.transientNetwork !== true
|
||||
);
|
||||
});
|
||||
@ -17,7 +17,6 @@ function createStore() {
|
||||
// We override by setting filePath directly
|
||||
store = new ConfigStore(fakeApp);
|
||||
store.filePath = path.join(tmpDir, 'electron-config.json');
|
||||
store.historyPath = path.join(tmpDir, 'electron-history.json');
|
||||
return store;
|
||||
}
|
||||
|
||||
@ -62,42 +61,6 @@ describe('ConfigStore', () => {
|
||||
assert.equal(config.globalSettings.logMode, 'single');
|
||||
});
|
||||
|
||||
it('persists pendingQueue incl. savedAt (number) and ts-bearing jobs across save/load', async () => {
|
||||
// The queue-persistence fix stamps pendingQueue.savedAt and restores it on launch.
|
||||
// This proves the persistence layer round-trips the new fields untouched (the
|
||||
// ts-gate is worthless if savedAt does not survive serialization).
|
||||
const pendingQueue = {
|
||||
savedAt: 1750000000123,
|
||||
selectedUploadHosters: ['voe.sx', 'byse.sx'],
|
||||
selectedFiles: [{ path: 'C:/dl/a.mkv', name: 'a.mkv', size: 4242 }],
|
||||
queueJobs: [
|
||||
{ id: 'j1', file: 'C:/dl/a.mkv', fileName: 'a.mkv', hoster: 'voe.sx', status: 'preview', bytesTotal: 4242, maxAttempts: 0 },
|
||||
{ id: 'j2', file: 'C:/dl/a.mkv', fileName: 'a.mkv', hoster: 'byse.sx', status: 'error', error: 'boom', maxAttempts: 3 }
|
||||
]
|
||||
};
|
||||
const current = store.load();
|
||||
await store.save({ globalSettings: { ...current.globalSettings, pendingQueue } });
|
||||
const loaded = store.load();
|
||||
const pq = loaded.globalSettings.pendingQueue;
|
||||
assert.equal(pq.savedAt, 1750000000123, 'savedAt epoch survives JSON round-trip');
|
||||
assert.equal(typeof pq.savedAt, 'number');
|
||||
assert.equal(pq.queueJobs.length, 2);
|
||||
assert.equal(pq.queueJobs[0].fileName, 'a.mkv');
|
||||
assert.equal(pq.queueJobs[0].hoster, 'voe.sx');
|
||||
assert.equal(pq.queueJobs[1].status, 'error');
|
||||
assert.deepEqual(pq.selectedUploadHosters, ['voe.sx', 'byse.sx']);
|
||||
assert.equal(pq.selectedFiles[0].path, 'C:/dl/a.mkv');
|
||||
});
|
||||
|
||||
it('pendingQueue can be cleared back to null (clearPersistedQueueStateSoon path)', async () => {
|
||||
const current = store.load();
|
||||
await store.save({ globalSettings: { ...current.globalSettings, pendingQueue: { savedAt: 1, queueJobs: [] } } });
|
||||
assert.ok(store.load().globalSettings.pendingQueue);
|
||||
const c2 = store.load();
|
||||
await store.save({ globalSettings: { ...c2.globalSettings, pendingQueue: null } });
|
||||
assert.equal(store.load().globalSettings.pendingQueue, null);
|
||||
});
|
||||
|
||||
it('regression: legacy sessionLog:true on disk normalizes to logMode "daily" (NOT "session")', async () => {
|
||||
// Write a config with the legacy boolean only (what an existing user has).
|
||||
await store.save({ globalSettings: { sessionLog: true } });
|
||||
@ -149,17 +112,6 @@ describe('ConfigStore', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('sizeMemoEnabled defaults to true for every hoster and persists when disabled', async () => {
|
||||
const fresh = store.load();
|
||||
for (const name of ['doodstream.com', 'voe.sx', 'vidmoly.me', 'byse.sx', 'clouddrop.cc']) {
|
||||
assert.equal(fresh.hosterSettings[name].sizeMemoEnabled, true, `${name} should default sizeMemoEnabled=true`);
|
||||
}
|
||||
await store.save({ hosterSettings: { 'byse.sx': { sizeMemoEnabled: false } } });
|
||||
const config = store.load();
|
||||
assert.equal(config.hosterSettings['byse.sx'].sizeMemoEnabled, false, 'explicit false preserved');
|
||||
assert.equal(config.hosterSettings['voe.sx'].sizeMemoEnabled, true, 'other hoster still defaults on');
|
||||
});
|
||||
|
||||
it('logToFile=false persists and survives reload', async () => {
|
||||
await store.save({ hosterSettings: { 'voe.sx': { logToFile: false } } });
|
||||
const config = store.load();
|
||||
@ -195,34 +147,6 @@ describe('ConfigStore', () => {
|
||||
assert.equal(store.loadHistory().length, 0);
|
||||
});
|
||||
|
||||
it('rotationCursors default to an empty object', () => {
|
||||
const config = store.load();
|
||||
assert.deepEqual(config.rotationCursors, {});
|
||||
});
|
||||
|
||||
it('saveRotationCursors round-trips and survives reload', async () => {
|
||||
await store.saveRotationCursors({ 'byse.sx': 7, 'voe.sx': 2 });
|
||||
const config = store.load();
|
||||
assert.equal(config.rotationCursors['byse.sx'], 7);
|
||||
assert.equal(config.rotationCursors['voe.sx'], 2);
|
||||
});
|
||||
|
||||
it('an unrelated save() does not clobber persisted rotationCursors', async () => {
|
||||
await store.saveRotationCursors({ 'byse.sx': 3 });
|
||||
await store.save({ globalSettings: { alwaysOnTop: true } });
|
||||
const config = store.load();
|
||||
assert.equal(config.rotationCursors['byse.sx'], 3, 'cursor preserved across a settings save');
|
||||
assert.equal(config.globalSettings.alwaysOnTop, true);
|
||||
});
|
||||
|
||||
it('saveRotationCursors does not disturb credentials', async () => {
|
||||
await store.save({ hosters: { 'byse.sx': [{ id: 'k1', enabled: true, authType: 'api', apiKey: 'secret-key' }] } });
|
||||
await store.saveRotationCursors({ 'byse.sx': 1 });
|
||||
const config = store.load();
|
||||
assert.equal(config.hosters['byse.sx'][0].apiKey, 'secret-key');
|
||||
assert.equal(config.rotationCursors['byse.sx'], 1);
|
||||
});
|
||||
|
||||
it('corrupted JSON falls back to defaults', () => {
|
||||
fs.writeFileSync(store.filePath, '{invalid json!!!', 'utf-8');
|
||||
const config = store.load();
|
||||
@ -254,32 +178,6 @@ describe('ConfigStore', () => {
|
||||
assert.equal(config.globalSettings.alwaysOnTop, true);
|
||||
});
|
||||
|
||||
it('load() returns independent clones — mutating one result must not leak into the cache', () => {
|
||||
store.load(); // warm the cache
|
||||
const a = store.load();
|
||||
a.globalSettings.alwaysOnTop = true;
|
||||
a.hosters['voe.sx'].push({ id: 'mutant' });
|
||||
a.history.push({ id: 'ghost' });
|
||||
const b = store.load();
|
||||
assert.equal(b.globalSettings.alwaysOnTop, false, 'mutating a prior load() result must not corrupt the cache');
|
||||
assert.equal(b.hosters['voe.sx'].length, 0);
|
||||
assert.equal(b.history.length, 0);
|
||||
});
|
||||
|
||||
it('load() reflects an external file change (mtime/size cache invalidation)', () => {
|
||||
store.load(); // warm cache on the no-file defaults
|
||||
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: true } }), 'utf-8');
|
||||
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'an external write must invalidate the cache');
|
||||
fs.writeFileSync(store.filePath, JSON.stringify({ globalSettings: { alwaysOnTop: false } }), 'utf-8');
|
||||
assert.equal(store.load().globalSettings.alwaysOnTop, false, 'a second external write must be seen too');
|
||||
});
|
||||
|
||||
it('save() invalidates the cache so the next load() sees the new value', async () => {
|
||||
assert.equal(store.load().globalSettings.alwaysOnTop, false);
|
||||
await store.save({ globalSettings: { alwaysOnTop: true } });
|
||||
assert.equal(store.load().globalSettings.alwaysOnTop, true, 'load() after save() must reflect the write');
|
||||
});
|
||||
|
||||
it('backup recovery when main file is corrupted', () => {
|
||||
// Write valid config first
|
||||
fs.writeFileSync(store.filePath, JSON.stringify({
|
||||
@ -293,117 +191,4 @@ describe('ConfigStore', () => {
|
||||
const config = store.load();
|
||||
assert.equal(config.hosters['doodstream.com'][0].apiKey, 'from-backup');
|
||||
});
|
||||
|
||||
it('wipe-guard: a settings-only save recovers accounts from .bak when the live config validly has none', async () => {
|
||||
// Post-wipe state: live config parses fine but has empty hosters; a backup still holds the accounts.
|
||||
fs.writeFileSync(store.filePath, JSON.stringify({ hosters: {}, hosterSettings: {}, globalSettings: {}, history: [] }), 'utf-8');
|
||||
fs.writeFileSync(store.filePath + '.bak', JSON.stringify({
|
||||
hosters: { 'voe.sx': [{ id: 'v1', authType: 'api', apiKey: 'survive-key' }] },
|
||||
hosterSettings: {}, globalSettings: {}, history: []
|
||||
}), 'utf-8');
|
||||
await store.save({ globalSettings: { alwaysOnTop: true } });
|
||||
const cfg = store.load();
|
||||
assert.ok(cfg.hosters['voe.sx'] && cfg.hosters['voe.sx'].length === 1, 'guard must restore accounts from .bak, not persist the wipe');
|
||||
assert.equal(cfg.hosters['voe.sx'][0].apiKey, 'survive-key');
|
||||
assert.equal(cfg.globalSettings.alwaysOnTop, true);
|
||||
});
|
||||
|
||||
it('wipe-guard: an explicit save({hosters:{}}) (user deleted all) is NOT blocked', async () => {
|
||||
await store.save({ hosters: { 'doodstream.com': [{ id: 'd1', authType: 'api', apiKey: 'k' }] } });
|
||||
await store.save({ hosters: {} });
|
||||
const cfg = store.load();
|
||||
assert.equal((cfg.hosters['doodstream.com'] || []).length, 0, 'an intentional hosters write must be allowed to empty them');
|
||||
});
|
||||
});
|
||||
|
||||
describe('ConfigStore history split (electron-history.json)', () => {
|
||||
let dir;
|
||||
let s;
|
||||
|
||||
function makeStore() {
|
||||
const st = new ConfigStore({ isPackaged: false, getPath: () => dir });
|
||||
st.filePath = path.join(dir, 'electron-config.json');
|
||||
st.historyPath = path.join(dir, 'electron-history.json');
|
||||
return st;
|
||||
}
|
||||
|
||||
function writeConfigWithHistory(n) {
|
||||
const history = [];
|
||||
for (let i = 0; i < n; i++) history.push({ id: `batch-${i}`, timestamp: 1750000000000 + i, total: 3, files: [{ name: `f${i}.mkv` }] });
|
||||
fs.writeFileSync(path.join(dir, 'electron-config.json'), JSON.stringify({
|
||||
hosters: { 'byse.sx': [{ id: 'a1', authType: 'api', apiKey: 'k' }] },
|
||||
hosterSettings: {}, globalSettings: { historyRetention: 'all' }, history
|
||||
}), 'utf-8');
|
||||
}
|
||||
|
||||
beforeEach(() => { dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cfg-hist-')); s = makeStore(); });
|
||||
afterEach(() => { fs.rmSync(dir, { recursive: true, force: true }); });
|
||||
|
||||
it('migration moves history into electron-history.json, preserving every entry', () => {
|
||||
writeConfigWithHistory(50);
|
||||
s._migrateHistory();
|
||||
assert.equal(s._historyMigrated, true);
|
||||
assert.ok(fs.existsSync(s.historyPath));
|
||||
const hist = JSON.parse(fs.readFileSync(s.historyPath, 'utf-8'));
|
||||
assert.equal(hist.length, 50);
|
||||
assert.equal(hist[0].id, 'batch-0');
|
||||
assert.equal(hist[49].id, 'batch-49');
|
||||
assert.ok(fs.existsSync(s.filePath + '.pre-history-split.bak'), 'a permanent pre-split backup is kept');
|
||||
});
|
||||
|
||||
it('after migration load() excludes history (cheap hot path) but loadHistory() returns the real data', () => {
|
||||
writeConfigWithHistory(30);
|
||||
s._migrateHistory();
|
||||
assert.deepEqual(s.load().history, [], 'history is not carried in the always-loaded config');
|
||||
assert.equal(s.loadHistory().length, 30);
|
||||
});
|
||||
|
||||
it('appendHistory writes to history.json; the next config write strips stale history from the config file', async () => {
|
||||
writeConfigWithHistory(10);
|
||||
s._migrateHistory();
|
||||
await s.appendHistory({ id: 'new-batch', timestamp: 1750000099999, total: 1, files: [{ name: 'x.mkv' }] });
|
||||
assert.equal(s.loadHistory().length, 11, 'append goes to history.json');
|
||||
await s.save({ globalSettings: { alwaysOnTop: true } });
|
||||
const onDisk = JSON.parse(fs.readFileSync(s.filePath, 'utf-8'));
|
||||
assert.ok(!onDisk.history || onDisk.history.length === 0, 'a config write strips stale history from the config file');
|
||||
assert.equal(s.loadHistory().length, 11, 'history.json is unaffected by the config write');
|
||||
});
|
||||
|
||||
it('save({globalSettings}) after migration NEVER loses history (data-loss invariant)', async () => {
|
||||
writeConfigWithHistory(40);
|
||||
s._migrateHistory();
|
||||
await s.save({ globalSettings: { alwaysOnTop: true } });
|
||||
assert.equal(s.loadHistory().length, 40, 'a settings write must not touch history');
|
||||
assert.equal(s.load().globalSettings.alwaysOnTop, true);
|
||||
});
|
||||
|
||||
it('clearHistory empties history.json only', async () => {
|
||||
writeConfigWithHistory(20);
|
||||
s._migrateHistory();
|
||||
await s.clearHistory();
|
||||
assert.equal(s.loadHistory().length, 0);
|
||||
});
|
||||
|
||||
it('migration is idempotent — re-running with history.json present does not re-derive or clobber', () => {
|
||||
writeConfigWithHistory(15);
|
||||
s._migrateHistory();
|
||||
const after = makeStore();
|
||||
after._migrateHistory();
|
||||
assert.equal(after._historyMigrated, true);
|
||||
assert.equal(after.loadHistory().length, 15);
|
||||
});
|
||||
|
||||
it('crash-window fallback: not migrated + no history.json → loadHistory reads config.history', () => {
|
||||
writeConfigWithHistory(7);
|
||||
assert.equal(s._historyMigrated, false);
|
||||
assert.equal(s.loadHistory().length, 7, 'legacy path still serves history if migration never ran');
|
||||
});
|
||||
|
||||
it('pruneHistory trims history.json and persists the retention setting', async () => {
|
||||
writeConfigWithHistory(12);
|
||||
s._migrateHistory();
|
||||
const res = await s.pruneHistory('all', { dryRun: false });
|
||||
assert.equal(s.loadHistory().length, 12);
|
||||
assert.ok(res.keptBatches === 12);
|
||||
});
|
||||
});
|
||||
|
||||
@ -1,62 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { createAgent } = require('../lib/diagnostics-agent');
|
||||
|
||||
function stubCollectors() {
|
||||
const calls = [];
|
||||
const mk = (name) => (a) => { calls.push([name, a]); return { name, a }; };
|
||||
return {
|
||||
calls,
|
||||
getSystemInfo: mk('getSystemInfo'),
|
||||
serverHealth: mk('serverHealth'),
|
||||
getConfigRedacted: mk('getConfigRedacted'),
|
||||
listLogs: mk('listLogs'),
|
||||
readLog: mk('readLog'),
|
||||
getAppEvents: mk('getAppEvents'),
|
||||
listErrors: mk('listErrors'),
|
||||
getQueueState: mk('getQueueState'),
|
||||
getHistory: mk('getHistory'),
|
||||
getRotationState: mk('getRotationState'),
|
||||
getHealth: mk('getHealth')
|
||||
};
|
||||
}
|
||||
|
||||
test('agent rejects unknown ops and any write/exec-shaped op', () => {
|
||||
const agent = createAgent(stubCollectors());
|
||||
for (const bad of ['delete_log', 'write_config', 'run_health_check', 'exec', 'eval', '__proto__', 'set_setting', 'restart']) {
|
||||
const r = agent.handle(bad, {});
|
||||
assert.equal(r.ok, false, `${bad} must be rejected`);
|
||||
assert.match(r.error, /unknown or non-readonly/);
|
||||
}
|
||||
});
|
||||
|
||||
test('agent rejects inherited Object.prototype members (no whitelist bypass via the prototype chain)', () => {
|
||||
const agent = createAgent(stubCollectors());
|
||||
for (const proto of ['constructor', 'toString', 'valueOf', 'hasOwnProperty', 'isPrototypeOf', 'toLocaleString']) {
|
||||
const r = agent.handle(proto, {});
|
||||
assert.equal(r.ok, false, `${proto} (inherited) must NOT be treated as an op`);
|
||||
}
|
||||
for (const bad of [null, undefined, 42, {}, ['read_log']]) {
|
||||
assert.equal(agent.handle(bad, {}).ok, false, `non-string op ${JSON.stringify(bad)} must be rejected`);
|
||||
}
|
||||
});
|
||||
|
||||
test('agent maps each whitelisted op to its collector and is read-only only', () => {
|
||||
const stub = stubCollectors();
|
||||
const agent = createAgent(stub);
|
||||
assert.equal(agent.handle('server_health', { errorLimit: 5 }).ok, true);
|
||||
assert.equal(agent.handle('read_log', { name: 'debug' }).ok, true);
|
||||
assert.equal(agent.handle('tail_log', { name: 'debug' }).ok, true, 'tail_log aliases read_log');
|
||||
assert.equal(agent.handle('get_config_redacted', {}).ok, true);
|
||||
const ops = new Set(agent.ops);
|
||||
assert.ok(!ops.has('run_health_check'), 'no live probe op in this build');
|
||||
for (const op of agent.ops) assert.ok(!/write|delete|set_|exec|restart|cancel|retry/.test(op), `${op} must be read-only`);
|
||||
});
|
||||
|
||||
test('agent surfaces a collector ok:false verbatim and never throws', () => {
|
||||
const agent = createAgent({ readLog: () => ({ ok: false, error: 'unknown or non-readable log: x' }), getSystemInfo: () => { throw new Error('boom'); } });
|
||||
assert.equal(agent.handle('read_log', { name: 'x' }).ok, false);
|
||||
const thrown = agent.handle('get_system_info', {});
|
||||
assert.equal(thrown.ok, false);
|
||||
assert.match(thrown.error, /boom/);
|
||||
});
|
||||
@ -1,156 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
const support = require('../lib/support-bundle');
|
||||
const stats = require('../lib/stats');
|
||||
const { createCollectors } = require('../lib/diagnostics-collectors');
|
||||
const { createAgent } = require('../lib/diagnostics-agent');
|
||||
|
||||
function makeFixture() {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mhu-diag-'));
|
||||
const paths = {
|
||||
fileuploader: path.join(dir, 'fileuploader.log'),
|
||||
debug: path.join(dir, 'debug.log'),
|
||||
accountRotation: path.join(dir, 'account-rotation.log'),
|
||||
doodstreamDebug: path.join(dir, 'doodstream-debug.log'),
|
||||
crashLog: path.join(dir, 'crash.log'),
|
||||
logDir: dir
|
||||
};
|
||||
fs.writeFileSync(paths.debug, 'boot ok\nuploading file with token SECRETTOKEN123456 inline\nAuthorization: Bearer abcdef123456\n');
|
||||
fs.writeFileSync(paths.doodstreamDebug, 'api_key=LIVEKEY99999 sess=abc\n');
|
||||
fs.writeFileSync(paths.crashLog, 'CRASH at 12:00\n');
|
||||
const config = {
|
||||
hosters: { 'voe.sx': [{ id: 'a1', username: 'u', password: 'HUNTER2SECRET' }], 'byse.sx': [{ id: 'b1', apiKey: 'BYSEKEY1234567' }] },
|
||||
hosterSettings: {},
|
||||
globalSettings: {
|
||||
webhookUrl: 'https://discord.com/api/webhooks/12345/WBHOOKSECRETTOKEN',
|
||||
diagnostics: { enabled: true, port: 9110, token: 'SECRETTOKEN123456', bindAddress: '127.0.0.1' },
|
||||
pendingQueue: { savedAt: 1, selectedUploadHosters: ['voe.sx'], selectedFiles: [{ path: 'C:/a.mkv' }], queueJobs: [{ file: 'C:/a.mkv', fileName: 'a.mkv', hoster: 'voe.sx', status: 'error', error: 'timeout' }] }
|
||||
},
|
||||
history: [{ timestamp: new Date(2026, 0, 1).toISOString(), files: [{ name: 'x.mkv', results: [{ hoster: 'voe.sx', status: 'error', error: 'Not video file format' }, { hoster: 'byse.sx', status: 'done', url: 'https://byse.sx/x' }] }] }],
|
||||
rotationCursors: { 'voe.sx': 1 }
|
||||
};
|
||||
const collectors = createCollectors({
|
||||
loadConfig: () => JSON.parse(JSON.stringify(config)),
|
||||
getAllLogPaths: () => paths,
|
||||
support, stats,
|
||||
appInfo: () => ({ name: 'mhu', version: '9.9.9' }),
|
||||
systemInfo: () => ({ platform: 'win32', hostname: 'srv' }),
|
||||
agentInfo: () => ({ version: '9.9.9', port: 9110, clientCount: 0, lastAccess: null })
|
||||
});
|
||||
return { dir, paths, config, collectors };
|
||||
}
|
||||
|
||||
test('getConfigRedacted strips password/apiKey/token/webhookUrl and value-scrubs the token mid-string', () => {
|
||||
const { collectors } = makeFixture();
|
||||
const out = collectors.getConfigRedacted({ section: 'all' });
|
||||
const json = JSON.stringify(out);
|
||||
assert.ok(!json.includes('HUNTER2SECRET'), 'password must be redacted');
|
||||
assert.ok(!json.includes('BYSEKEY1234567'), 'apiKey must be redacted');
|
||||
assert.ok(!json.includes('SECRETTOKEN123456'), 'diag token must be redacted');
|
||||
assert.ok(!json.includes('WBHOOKSECRETTOKEN'), 'webhook secret must be redacted');
|
||||
});
|
||||
|
||||
test('getHistory reads loadHistory (migrated mode: loadConfig().history is empty)', () => {
|
||||
const c = createCollectors({
|
||||
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [] }),
|
||||
loadHistory: () => [
|
||||
{ timestamp: '2026-01-01T00:00:00.000Z', files: [{ name: 'a.mkv', results: [{ hoster: 'voe.sx', status: 'done', url: 'https://voe.sx/a' }] }] },
|
||||
{ timestamp: '2026-01-02T00:00:00.000Z', files: [{ name: 'b.mkv', results: [{ hoster: 'byse.sx', status: 'done', url: 'https://byse.sx/b' }] }] }
|
||||
],
|
||||
getAllLogPaths: () => ({ logDir: os.tmpdir() }),
|
||||
support, stats,
|
||||
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
|
||||
});
|
||||
const out = c.getHistory({ limit: 10 });
|
||||
assert.equal(out.totalBatches, 2, 'must report real history from loadHistory, not the empty load().history');
|
||||
assert.equal(out.returned, 2);
|
||||
});
|
||||
|
||||
test('getHistory falls back to loadConfig().history when loadHistory is absent (legacy mode)', () => {
|
||||
const c = createCollectors({
|
||||
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [{ timestamp: '2026-01-01T00:00:00.000Z', files: [] }] }),
|
||||
getAllLogPaths: () => ({ logDir: os.tmpdir() }),
|
||||
support, stats,
|
||||
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
|
||||
});
|
||||
assert.equal(c.getHistory({ limit: 10 }).totalBatches, 1, 'legacy path reads load().history when loadHistory not injected');
|
||||
});
|
||||
|
||||
test('readLog redacts a planted token and a Bearer line; doodstream is NOT readable; unknown name rejected', () => {
|
||||
const { collectors } = makeFixture();
|
||||
const dbg = collectors.readLog({ name: 'debug', tailKb: 64 });
|
||||
assert.ok(!dbg.content.includes('SECRETTOKEN123456'), 'value-scrub removes the live diag token from logs');
|
||||
assert.ok(!/Bearer abcdef123456/.test(dbg.content), 'pattern-scrub removes Authorization Bearer');
|
||||
assert.equal(collectors.readLog({ name: 'doodstreamDebug' }).ok, false, 'doodstream-debug.log is not in the readable allowlist');
|
||||
assert.equal(collectors.readLog({ name: '../../etc/passwd' }).ok, false, 'arbitrary names are rejected (no path traversal)');
|
||||
assert.equal(collectors.readLog({ name: 'crash' }).name, 'crash');
|
||||
});
|
||||
|
||||
test('readLog grep is case-insensitive substring with | alternation, and is ReDoS-safe', () => {
|
||||
const { paths } = makeFixture();
|
||||
const fs2 = require('fs');
|
||||
fs2.writeFileSync(paths.debug, ['ERROR upload failed', 'info all good', 'WARN timeout hit', 'a'.repeat(120) + '! catastrophic bait'].join('\n'));
|
||||
const { collectors } = (() => {
|
||||
const support2 = require('../lib/support-bundle');
|
||||
const stats2 = require('../lib/stats');
|
||||
const c = require('../lib/diagnostics-collectors').createCollectors({
|
||||
loadConfig: () => ({ hosters: {}, globalSettings: {}, history: [] }),
|
||||
getAllLogPaths: () => paths, support: support2, stats: stats2,
|
||||
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
|
||||
});
|
||||
return { collectors: c };
|
||||
})();
|
||||
const alt = collectors.readLog({ name: 'debug', grep: 'error|timeout' });
|
||||
assert.equal(alt.matchedLines, 2, 'matches the ERROR and timeout lines case-insensitively');
|
||||
assert.ok(alt.content.includes('ERROR upload failed') && alt.content.includes('WARN timeout hit'));
|
||||
assert.ok(!alt.content.includes('info all good'), 'non-matching line excluded');
|
||||
const t0 = Date.now();
|
||||
const redos = collectors.readLog({ name: 'debug', grep: '(a+)+$' });
|
||||
assert.ok(Date.now() - t0 < 1000, 'catastrophic-looking grep must return promptly (literal substring, no backtracking)');
|
||||
assert.equal(redos.matchedLines, 0, '"(a+)+$" is treated as a literal substring, matching nothing here');
|
||||
});
|
||||
|
||||
test('getQueueState flags stale=true for the persisted snapshot and counts by status', () => {
|
||||
const { collectors } = makeFixture();
|
||||
const q = collectors.getQueueState({});
|
||||
assert.equal(q.source, 'persisted');
|
||||
assert.equal(q.stale, true);
|
||||
assert.equal(q.counts.error, 1);
|
||||
});
|
||||
|
||||
test('getQueueState (includeJobs default) pattern-scrubs an opaque token in a job error that is NOT a config secret', () => {
|
||||
const config = {
|
||||
hosters: {}, hosterSettings: {},
|
||||
globalSettings: { pendingQueue: { savedAt: 1, selectedUploadHosters: [], selectedFiles: [], queueJobs: [
|
||||
{ file: 'C:/b.mkv', fileName: 'b.mkv', hoster: 'streamtape', status: 'error', error: 'upload rejected: token=OPAQUE_NONconfig_TOKEN_9988' }
|
||||
] } },
|
||||
history: [], rotationCursors: {}
|
||||
};
|
||||
const collectors = createCollectors({
|
||||
loadConfig: () => JSON.parse(JSON.stringify(config)),
|
||||
getAllLogPaths: () => ({ logDir: os.tmpdir() }),
|
||||
support, stats,
|
||||
appInfo: () => ({}), systemInfo: () => ({}), agentInfo: () => ({})
|
||||
});
|
||||
const q = collectors.getQueueState({});
|
||||
const json = JSON.stringify(q);
|
||||
assert.ok(!json.includes('OPAQUE_NONconfig_TOKEN_9988'), 'opaque token in a job error must be pattern-scrubbed even on the default includeJobs path');
|
||||
});
|
||||
|
||||
test('listErrors classifies via stats.classifyErrorCategory and redacts error text', () => {
|
||||
const { collectors } = makeFixture();
|
||||
const e = collectors.listErrors({});
|
||||
assert.equal(e.total, 1, 'only the non-done result is an error');
|
||||
assert.equal(e.byCategory['file-rejected'], 1, '"Not video file format" -> file-rejected');
|
||||
});
|
||||
|
||||
test('serverHealth assembles the one-shot hub without leaking secrets', () => {
|
||||
const { collectors } = makeFixture();
|
||||
const h = collectors.serverHealth({});
|
||||
const json = JSON.stringify(h);
|
||||
assert.ok(h.server && h.queue && h.errors && h.logs, 'hub has all sections');
|
||||
assert.ok(!json.includes('HUNTER2SECRET') && !json.includes('SECRETTOKEN123456') && !json.includes('WBHOOKSECRETTOKEN'), 'no secret leaks in server_health');
|
||||
});
|
||||
@ -1,120 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const os = require('os');
|
||||
const WebSocket = require('ws');
|
||||
const RemoteServer = require('../lib/remote-server');
|
||||
|
||||
const TOKEN = 'a'.repeat(64);
|
||||
|
||||
function firstLanIpv4() {
|
||||
for (const entry of Object.values(os.networkInterfaces())) {
|
||||
for (const net of (entry || [])) {
|
||||
if (net && net.family === 'IPv4' && !net.internal && net.address) return net.address;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function startAgent(onDiagnosticRequest, extra) {
|
||||
const srv = new RemoteServer();
|
||||
return srv.start({ port: 0, host: '127.0.0.1', token: TOKEN, diagnosticMode: true, onDiagnosticRequest, ...(extra || {}) })
|
||||
.then(() => srv);
|
||||
}
|
||||
|
||||
function connect(port) {
|
||||
return new WebSocket(`ws://127.0.0.1:${port}`);
|
||||
}
|
||||
|
||||
function once(ws, type) {
|
||||
return new Promise((resolve, reject) => {
|
||||
ws.on('message', (raw) => { const m = JSON.parse(raw); if (m.type === type) resolve(m); });
|
||||
ws.on('close', (code) => reject(new Error('closed ' + code)));
|
||||
ws.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
test('diagnostic client: auth -> diag-request -> reqId-correlated diag-response', async () => {
|
||||
const agent = await startAgent((msg, _client, reply) => {
|
||||
assert.equal(msg.op, 'server_health');
|
||||
reply({ ok: true, data: { hello: 'world', echo: msg.args } });
|
||||
});
|
||||
const port = agent.getPort();
|
||||
const ws = connect(port);
|
||||
await new Promise((r) => ws.on('open', r));
|
||||
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
|
||||
const ok = await once(ws, 'auth-ok');
|
||||
assert.ok(ok.clientId);
|
||||
ws.send(JSON.stringify({ type: 'diag-request', reqId: 'r1', op: 'server_health', args: { errorLimit: 3 } }));
|
||||
const resp = await once(ws, 'diag-response');
|
||||
assert.equal(resp.reqId, 'r1');
|
||||
assert.equal(resp.ok, true);
|
||||
assert.equal(resp.data.hello, 'world');
|
||||
assert.equal(resp.data.echo.errorLimit, 3);
|
||||
assert.equal(agent.getLastAccess() !== null, true, 'access timestamp recorded');
|
||||
ws.close(); agent.stop();
|
||||
});
|
||||
|
||||
test('a diagnostic client NEVER triggers the screen-capture window', async () => {
|
||||
let captureCreated = false;
|
||||
const agent = await startAgent(() => {}, { onCreateCaptureWindow: () => { captureCreated = true; } });
|
||||
const ws = connect(agent.getPort());
|
||||
await new Promise((r) => ws.on('open', r));
|
||||
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
|
||||
await once(ws, 'auth-ok');
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
assert.equal(captureCreated, false, 'diagnosticMode must not spawn the capture window');
|
||||
ws.close(); agent.stop();
|
||||
});
|
||||
|
||||
test('allowlist gate (wiring): a non-loopback peer is closed 4005 when not allowlisted (fail-closed)', () => {
|
||||
const srv = new RemoteServer();
|
||||
const closeCodeFor = (remoteAddress, allowlist) => {
|
||||
srv._config = { allowlist, token: TOKEN, diagnosticMode: true };
|
||||
let closed = null;
|
||||
srv._handleConnection({ close: (c) => { closed = c; }, on: () => {} }, { socket: { remoteAddress } });
|
||||
return closed;
|
||||
};
|
||||
assert.equal(closeCodeFor('100.64.0.9', []), 4005, 'empty allowlist => non-loopback rejected (fail-closed)');
|
||||
assert.equal(closeCodeFor('203.0.113.5', ['100.64.0.0/10']), 4005, 'peer outside the allowlist CIDR rejected');
|
||||
});
|
||||
|
||||
test('a loopback diagnostic client connects even with a non-matching allowlist (loopback is always allowed)', async () => {
|
||||
const agent = await startAgent(() => {}, { allowlist: ['100.64.0.0/10'] });
|
||||
const ws = connect(agent.getPort());
|
||||
await new Promise((r) => ws.on('open', r));
|
||||
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
|
||||
const ok = await once(ws, 'auth-ok');
|
||||
assert.ok(ok.clientId);
|
||||
ws.close(); agent.stop();
|
||||
});
|
||||
|
||||
test('network bind (0.0.0.0): an allowlisted non-loopback peer connects over a real socket (the Tailscale path)', async (t) => {
|
||||
const lan = firstLanIpv4();
|
||||
if (!lan) { t.skip('no non-internal IPv4 interface available'); return; }
|
||||
const agent = await startAgent(() => {}, { host: '0.0.0.0', allowlist: [lan] });
|
||||
const port = agent.getPort();
|
||||
const ws = new WebSocket(`ws://${lan}:${port}`);
|
||||
try {
|
||||
await new Promise((resolve, reject) => { ws.on('open', resolve); ws.on('error', reject); });
|
||||
ws.send(JSON.stringify({ type: 'auth', token: TOKEN, role: 'diagnostic' }));
|
||||
const ok = await once(ws, 'auth-ok');
|
||||
assert.ok(ok.clientId, 'allowlisted LAN peer authed over the 0.0.0.0 bind');
|
||||
} finally {
|
||||
ws.close(); agent.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test('wrong token is rejected and the ip is locked out after 5 attempts', async () => {
|
||||
const agent = await startAgent(() => {});
|
||||
const port = agent.getPort();
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const ws = connect(port);
|
||||
await new Promise((r) => ws.on('open', r));
|
||||
ws.send(JSON.stringify({ type: 'auth', token: 'wrong', role: 'diagnostic' }));
|
||||
await new Promise((r) => ws.on('close', r));
|
||||
}
|
||||
const ws = connect(port);
|
||||
const closeCode = await new Promise((resolve) => ws.on('close', (c) => resolve(c)));
|
||||
assert.equal(closeCode, 4003, 'locked out after 5 failed attempts');
|
||||
agent.stop();
|
||||
});
|
||||
@ -98,18 +98,3 @@ test('summarizeFileStat returns error for missing file', () => {
|
||||
const stat = summarizeFileStat(path.join(os.tmpdir(), `does-not-exist-${Date.now()}.bin`));
|
||||
assert.ok(stat.error);
|
||||
});
|
||||
|
||||
test('detectKind requires TS sync-byte periodicity — GIF and G-prefixed text are NOT mpeg-ts', () => {
|
||||
const ts = Buffer.alloc(377, 0xFF);
|
||||
ts[0] = 0x47; ts[188] = 0x47; ts[376] = 0x47;
|
||||
assert.strictEqual(detectKind(ts), 'mpeg-ts');
|
||||
assert.strictEqual(isVideoLikeKind('mpeg-ts'), true);
|
||||
|
||||
const gif = Buffer.concat([Buffer.from('GIF89a', 'ascii'), Buffer.alloc(400, 0x00)]);
|
||||
assert.strictEqual(detectKind(gif), 'gif');
|
||||
assert.strictEqual(isVideoLikeKind('gif'), false);
|
||||
|
||||
const gText = Buffer.concat([Buffer.from('Gewinnerliste 2026\n', 'ascii'), Buffer.alloc(400, 0x20)]);
|
||||
assert.notStrictEqual(detectKind(gText), 'mpeg-ts');
|
||||
assert.strictEqual(isVideoLikeKind(detectKind(gText)), false);
|
||||
});
|
||||
|
||||
@ -1,84 +0,0 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { applyHistoryRetention, countHistoryRows } = require('../lib/config-store');
|
||||
|
||||
function batch(timestamp, okRows, extras = {}) {
|
||||
const results = [];
|
||||
for (let i = 0; i < okRows; i++) results.push({ status: 'success', hoster: 'voe.sx', download_url: `https://voe.sx/${i}` });
|
||||
if (extras.aborted) for (let i = 0; i < extras.aborted; i++) results.push({ status: 'aborted', hoster: 'voe.sx' });
|
||||
if (extras.error) for (let i = 0; i < extras.error; i++) results.push({ status: 'error', hoster: 'voe.sx' });
|
||||
return { timestamp, files: [{ name: 'clip.mp4', results }] };
|
||||
}
|
||||
|
||||
const DAY = 86400000;
|
||||
|
||||
test('countHistoryRows counts only non-aborted, non-error results', () => {
|
||||
const h = [batch('2026-01-01', 3, { aborted: 2, error: 1 })];
|
||||
assert.strictEqual(countHistoryRows(h), 3);
|
||||
});
|
||||
|
||||
test('retention "all" returns the array unchanged', () => {
|
||||
const h = [batch('2026-01-01', 5), batch('2026-01-02', 5)];
|
||||
assert.strictEqual(applyHistoryRetention(h, 'all', Date.parse('2026-06-01')), h);
|
||||
});
|
||||
|
||||
test('count policy keeps newest whole batches up to the row target', () => {
|
||||
const h = [batch('2026-01-01', 400), batch('2026-01-02', 400), batch('2026-01-03', 400)];
|
||||
const pruned = applyHistoryRetention(h, '1000', Date.parse('2026-06-01'));
|
||||
assert.strictEqual(pruned.length, 3);
|
||||
assert.strictEqual(countHistoryRows(pruned), 1200);
|
||||
});
|
||||
|
||||
test('count policy drops older batches once target reached (newest first)', () => {
|
||||
const h = [batch('2026-01-01', 600), batch('2026-01-02', 600), batch('2026-01-03', 600)];
|
||||
const pruned = applyHistoryRetention(h, '1000', Date.parse('2026-06-01'));
|
||||
assert.strictEqual(pruned.length, 2);
|
||||
assert.deepStrictEqual(pruned.map(b => b.timestamp), ['2026-01-02', '2026-01-03']);
|
||||
});
|
||||
|
||||
test('count policy always keeps the newest batch even if it alone exceeds N', () => {
|
||||
const h = [batch('2026-01-01', 50), batch('2026-01-02', 5000)];
|
||||
const pruned = applyHistoryRetention(h, '100', Date.parse('2026-06-01'));
|
||||
assert.strictEqual(pruned.length, 1);
|
||||
assert.strictEqual(pruned[0].timestamp, '2026-01-02');
|
||||
});
|
||||
|
||||
test('time policy drops batches older than the cutoff', () => {
|
||||
const now = Date.parse('2026-06-15T00:00:00Z');
|
||||
const h = [
|
||||
batch(new Date(now - 10 * DAY).toISOString(), 5),
|
||||
batch(new Date(now - 3 * DAY).toISOString(), 5),
|
||||
batch(new Date(now - 1 * DAY).toISOString(), 5)
|
||||
];
|
||||
const pruned = applyHistoryRetention(h, '7d', now);
|
||||
assert.strictEqual(pruned.length, 2);
|
||||
});
|
||||
|
||||
test('time policy keeps batches with missing or invalid timestamp', () => {
|
||||
const now = Date.parse('2026-06-15T00:00:00Z');
|
||||
const h = [
|
||||
batch(undefined, 5),
|
||||
batch('not-a-date', 5),
|
||||
batch(new Date(now - 99 * DAY).toISOString(), 5),
|
||||
batch(new Date(now - 1 * DAY).toISOString(), 5)
|
||||
];
|
||||
const pruned = applyHistoryRetention(h, '30d', now);
|
||||
assert.strictEqual(pruned.length, 3);
|
||||
assert.ok(pruned.includes(h[0]));
|
||||
assert.ok(pruned.includes(h[1]));
|
||||
assert.ok(!pruned.includes(h[2]));
|
||||
});
|
||||
|
||||
test('count policy shrinks a realistic 41-batch / >1000-row history', () => {
|
||||
const h = [];
|
||||
for (let i = 0; i < 41; i++) h.push(batch(`2026-04-${String((i % 28) + 1).padStart(2, '0')}`, 1300));
|
||||
assert.strictEqual(countHistoryRows(h), 41 * 1300);
|
||||
const pruned = applyHistoryRetention(h, '1000', Date.parse('2026-06-01'));
|
||||
assert.strictEqual(pruned.length, 1);
|
||||
assert.strictEqual(countHistoryRows(pruned), 1300);
|
||||
});
|
||||
|
||||
test('empty history is returned as-is for any policy', () => {
|
||||
assert.deepStrictEqual(applyHistoryRetention([], '7d', Date.now()), []);
|
||||
assert.deepStrictEqual(applyHistoryRetention([], '100', Date.now()), []);
|
||||
});
|
||||
@ -1,52 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { normalizeIp, isLoopbackIp, matchIpRule, evaluateClientAllowed } = require('../lib/ip-allowlist');
|
||||
|
||||
test('normalizeIp strips ::ffff: and lowercases', () => {
|
||||
assert.equal(normalizeIp('::ffff:100.64.0.5'), '100.64.0.5');
|
||||
assert.equal(normalizeIp('::FFFF:127.0.0.1'), '127.0.0.1');
|
||||
assert.equal(normalizeIp(' 100.64.0.5 '), '100.64.0.5');
|
||||
});
|
||||
|
||||
test('loopback is always allowed, even with a non-matching allowlist', () => {
|
||||
for (const ip of ['127.0.0.1', '::1', '::ffff:127.0.0.1', '', 'localhost', '127.5.5.5']) {
|
||||
assert.equal(evaluateClientAllowed(ip, ['203.0.113.5']), true, `${ip} loopback`);
|
||||
}
|
||||
});
|
||||
|
||||
test('fail-closed: empty allowlist rejects every non-loopback peer', () => {
|
||||
for (const ip of ['100.64.0.5', '203.0.113.5', '10.0.0.2', '::ffff:192.168.1.9']) {
|
||||
assert.equal(evaluateClientAllowed(ip, []), false, `${ip} must be rejected with empty allowlist`);
|
||||
}
|
||||
});
|
||||
|
||||
test('exact IP allow + reject', () => {
|
||||
assert.equal(evaluateClientAllowed('203.0.113.5', ['203.0.113.5']), true);
|
||||
assert.equal(evaluateClientAllowed('203.0.113.6', ['203.0.113.5']), false);
|
||||
});
|
||||
|
||||
test('CIDR matching incl. the Tailscale CGNAT range 100.64.0.0/10', () => {
|
||||
assert.equal(evaluateClientAllowed('100.64.0.5', ['100.64.0.0/10']), true);
|
||||
assert.equal(evaluateClientAllowed('100.127.255.254', ['100.64.0.0/10']), true);
|
||||
assert.equal(evaluateClientAllowed('100.128.0.1', ['100.64.0.0/10']), false, 'just outside the /10');
|
||||
assert.equal(evaluateClientAllowed('::ffff:100.64.0.5', ['100.64.0.0/10']), true, 'mapped v4 in CIDR');
|
||||
assert.equal(evaluateClientAllowed('10.0.0.5', ['10.0.0.0/24']), true);
|
||||
assert.equal(evaluateClientAllowed('10.0.1.5', ['10.0.0.0/24']), false);
|
||||
});
|
||||
|
||||
test('wildcard rules allow everything', () => {
|
||||
assert.equal(evaluateClientAllowed('8.8.8.8', ['*']), true);
|
||||
assert.equal(evaluateClientAllowed('8.8.8.8', ['0.0.0.0/0']), true);
|
||||
});
|
||||
|
||||
test('matchIpRule rejects malformed rules and out-of-range octets', () => {
|
||||
assert.equal(matchIpRule('1.2.3.4', 'not-an-ip'), false);
|
||||
assert.equal(matchIpRule('1.2.3.4', '1.2.3.0/33'), false);
|
||||
assert.equal(matchIpRule('1.2.3.999', '1.2.3.0/24'), false);
|
||||
});
|
||||
|
||||
test('isLoopbackIp recognizes loopback forms', () => {
|
||||
assert.equal(isLoopbackIp('127.0.0.1'), true);
|
||||
assert.equal(isLoopbackIp('::1'), true);
|
||||
assert.equal(isLoopbackIp('100.64.0.1'), false);
|
||||
});
|
||||
@ -57,23 +57,13 @@ test('resolveLogFileName: daily mode → fileuploader-YYYY-MM-DD.log', () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('resolveLogFileName: session mode → <sessionId>.log (baseName ignored)', () => {
|
||||
test('resolveLogFileName: session mode → fileuploader-session-<id>.log', () => {
|
||||
assert.equal(
|
||||
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session', sessionId: '26-05-2026-mdu-session-22-44' }),
|
||||
'26-05-2026-mdu-session-22-44.log'
|
||||
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session', sessionId: '2026-05-28_22-44-52-12345' }),
|
||||
'fileuploader-session-2026-05-28_22-44-52-12345.log'
|
||||
);
|
||||
});
|
||||
|
||||
test('formatSessionStamp: DD-MM-YYYY-mdu-session-HH-MM', () => {
|
||||
const { formatSessionStamp } = require('../lib/log-mode');
|
||||
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36)), '26-06-2026-mdu-session-06-02');
|
||||
});
|
||||
|
||||
test('formatSessionStamp: appends a 6-digit suffix when a rand is supplied', () => {
|
||||
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36), '847581'), '26-06-2026-mdu-session-06-02-847581');
|
||||
assert.equal(formatSessionStamp(new Date(2026, 5, 26, 6, 2, 36), 847581), '26-06-2026-mdu-session-06-02-847581');
|
||||
});
|
||||
|
||||
test('resolveLogFileName: session mode with missing sessionId falls back to single (never emits malformed name)', () => {
|
||||
assert.equal(
|
||||
resolveLogFileName({ baseName: 'fileuploader', ext: '.log', mode: 'session' }),
|
||||
@ -112,17 +102,12 @@ test('stripModeStampFromFileName: strips a session-stamp suffix (with and withou
|
||||
);
|
||||
});
|
||||
|
||||
test('stripModeStampFromFileName: new DD-MM-YYYY-mdu-session-HH-MM resets to the default base', () => {
|
||||
assert.equal(stripModeStampFromFileName('26-06-2026-mdu-session-06-02.log'), 'fileuploader.log');
|
||||
assert.equal(stripModeStampFromFileName('26-06-2026-mdu-session-06-02-847581.log'), 'fileuploader.log');
|
||||
});
|
||||
|
||||
test('regression: resolveLogFileName(stripModeStampFromFileName(...)) is idempotent — persisting then re-resolving never compounds stamps', () => {
|
||||
// This is the exact bug shape: persist the resolved path, then on next call
|
||||
// re-resolve from the saved base — must produce the same file, not a doubled
|
||||
// session-stamped one. The fix is the strip; this test guards against
|
||||
// regressing _persistFallbackLogPath into the 3.3.35 bug.
|
||||
const sessionId = '03-06-2026-mdu-session-18-16';
|
||||
const sessionId = '2026-06-03_18-16-20-8132';
|
||||
const dailyDate = new Date(2026, 5, 3);
|
||||
for (const mode of ['daily', 'session']) {
|
||||
const date = mode === 'daily' ? dailyDate : new Date();
|
||||
@ -144,7 +129,12 @@ test('formatDateStamp: zero-pads month and day', () => {
|
||||
assert.equal(formatDateStamp(new Date(2026, 11, 31)), '2026-12-31');
|
||||
});
|
||||
|
||||
test('formatSessionStamp: DD-MM-YYYY-mdu-session-HH-MM (no seconds/pid)', () => {
|
||||
assert.equal(formatSessionStamp(new Date(2026, 4, 28, 7, 9, 5)), '28-05-2026-mdu-session-07-09');
|
||||
assert.equal(formatSessionStamp(new Date(2026, 4, 28, 22, 44, 52)), '28-05-2026-mdu-session-22-44');
|
||||
test('formatSessionStamp: produces YYYY-MM-DD_HH-MM-SS-pid', () => {
|
||||
const d = new Date(2026, 4, 28, 7, 9, 5);
|
||||
assert.equal(formatSessionStamp(d, 12345), '2026-05-28_07-09-05-12345');
|
||||
});
|
||||
|
||||
test('formatSessionStamp: omits the pid suffix when none provided', () => {
|
||||
const d = new Date(2026, 4, 28, 22, 44, 52);
|
||||
assert.equal(formatSessionStamp(d), '2026-05-28_22-44-52');
|
||||
});
|
||||
|
||||
@ -1,56 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { selectOrphanTmps } = require('../lib/orphan-tmp');
|
||||
|
||||
const BASE = 'electron-config.json';
|
||||
const aliveSet = new Set([100, 200]);
|
||||
const isAlive = (pid) => aliveSet.has(pid);
|
||||
|
||||
test('selects only dead-pid <base>.<pid>.tmp orphans', () => {
|
||||
const files = [
|
||||
'electron-config.json',
|
||||
'electron-config.json.bak',
|
||||
'electron-config.json.tmp',
|
||||
'electron-config.json.100.tmp',
|
||||
'electron-config.json.200.tmp',
|
||||
'electron-config.json.999.tmp',
|
||||
'electron-config.json.4242.tmp',
|
||||
'something-else.500.tmp',
|
||||
'electron-config.json.abc.tmp'
|
||||
];
|
||||
const orphans = selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive });
|
||||
assert.deepEqual(orphans.sort(), ['electron-config.json.4242.tmp', 'electron-config.json.999.tmp']);
|
||||
});
|
||||
|
||||
test('never selects the current process tmp', () => {
|
||||
const files = ['electron-config.json.7.tmp'];
|
||||
assert.deepEqual(selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive: () => false }), []);
|
||||
});
|
||||
|
||||
test('never selects the FIXED <base>.tmp (used by async _atomicWrite)', () => {
|
||||
const files = ['electron-config.json.tmp'];
|
||||
assert.deepEqual(selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive: () => false }), []);
|
||||
});
|
||||
|
||||
test('never selects the live config or its .bak', () => {
|
||||
const files = ['electron-config.json', 'electron-config.json.bak'];
|
||||
assert.deepEqual(selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive: () => false }), []);
|
||||
});
|
||||
|
||||
test('alive pid (incl. EPERM-as-alive) is skipped, preventing deletion of a concurrent instance tmp', () => {
|
||||
const files = ['electron-config.json.100.tmp', 'electron-config.json.300.tmp'];
|
||||
const orphans = selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive: (p) => p === 100 });
|
||||
assert.deepEqual(orphans, ['electron-config.json.300.tmp']);
|
||||
});
|
||||
|
||||
test('robust to junk / missing inputs', () => {
|
||||
assert.deepEqual(selectOrphanTmps(null, { baseName: BASE, currentPid: 1, isAlive }), []);
|
||||
assert.deepEqual(selectOrphanTmps(['x', 42, null, undefined], { baseName: BASE, currentPid: 1, isAlive }), []);
|
||||
assert.deepEqual(selectOrphanTmps(['electron-config.json.5.tmp'], {}), []);
|
||||
assert.deepEqual(selectOrphanTmps(['electron-config.json.5.tmp'], { baseName: '', currentPid: 1, isAlive }), []);
|
||||
});
|
||||
|
||||
test('does not match a different base that shares a prefix', () => {
|
||||
const files = ['electron-config.json.backup.5.tmp'];
|
||||
assert.deepEqual(selectOrphanTmps(files, { baseName: BASE, currentPid: 7, isAlive: () => false }), []);
|
||||
});
|
||||
@ -1,114 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { partitionRestoredJobsByLog } = require('../lib/queue-dedup');
|
||||
|
||||
function lcg(seed) {
|
||||
let s = seed >>> 0;
|
||||
return () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; };
|
||||
}
|
||||
|
||||
function key(f, h) { return `${String(f).toLowerCase()}|${String(h).toLowerCase()}`; }
|
||||
|
||||
test('property: removed iff (done && key in log) OR (savedAt finite && key unambiguous && newest matching log ts >= floor(savedAt/1000)*1000)', () => {
|
||||
const rnd = lcg(0x9e3779b1);
|
||||
const statuses = ['preview', 'done', 'error', 'aborted', 'queued', 'skipped'];
|
||||
const hosters = ['voe.sx', 'byse.sx', 'doodstream.com'];
|
||||
const names = ['a.mkv', 'b.mp4', 'A.MKV', 'c.mov'];
|
||||
const folders = ['C:/A/', 'C:/B/', 'D:/down/'];
|
||||
const pick = (arr) => arr[Math.floor(rnd() * arr.length)];
|
||||
|
||||
for (let iter = 0; iter < 3000; iter++) {
|
||||
const useSavedAt = rnd() < 0.7;
|
||||
const savedAt = useSavedAt ? Math.floor(rnd() * 2_000_000_000_000) : undefined;
|
||||
|
||||
const jobs = [];
|
||||
const nJobs = 1 + Math.floor(rnd() * 6);
|
||||
for (let i = 0; i < nJobs; i++) {
|
||||
const name = pick(names);
|
||||
// Mix shared and distinct paths so ambiguous keys (same name+hoster,
|
||||
// different folder) actually occur and exercise the guard.
|
||||
jobs.push({ id: `j${i}`, fileName: name, hoster: pick(hosters), status: pick(statuses), file: `${pick(folders)}${name}` });
|
||||
}
|
||||
|
||||
const log = [];
|
||||
const nLog = Math.floor(rnd() * 5);
|
||||
for (let i = 0; i < nLog; i++) {
|
||||
const hasTs = rnd() < 0.8;
|
||||
log.push({ fileName: pick(names), hoster: pick(hosters), ts: hasTs ? Math.floor(rnd() * 2_000_000_000_000) : undefined });
|
||||
}
|
||||
|
||||
const logKeys = new Set();
|
||||
const maxTs = new Map();
|
||||
for (const e of log) {
|
||||
const k = key(e.fileName, e.hoster);
|
||||
logKeys.add(k);
|
||||
if (typeof e.ts === 'number' && isFinite(e.ts)) {
|
||||
const prev = maxTs.get(k);
|
||||
if (prev === undefined || e.ts > prev) maxTs.set(k, e.ts);
|
||||
}
|
||||
}
|
||||
const filesPerKey = new Map();
|
||||
for (const job of jobs) {
|
||||
const k = key(job.fileName, job.hoster);
|
||||
if (!filesPerKey.has(k)) filesPerKey.set(k, new Set());
|
||||
filesPerKey.get(k).add(job.file || '');
|
||||
}
|
||||
const floor = (typeof savedAt === 'number' && isFinite(savedAt)) ? Math.floor(savedAt / 1000) * 1000 : null;
|
||||
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
|
||||
assert.equal(kept.length + removed.length, jobs.length, `iter ${iter}: partition must cover every job exactly once`);
|
||||
const keptIds = new Set(kept.map(j => j.id));
|
||||
const removedIds = new Set(removed.map(j => j.id));
|
||||
assert.equal(keptIds.size + removedIds.size, jobs.length, `iter ${iter}: no job in both partitions`);
|
||||
|
||||
for (const job of jobs) {
|
||||
const k = key(job.fileName, job.hoster);
|
||||
const doneInLog = job.status === 'done' && logKeys.has(k);
|
||||
const unambiguous = filesPerKey.get(k).size <= 1;
|
||||
const afterSnap = floor !== null && unambiguous && maxTs.has(k) && maxTs.get(k) >= floor;
|
||||
const shouldRemove = doneInLog || afterSnap;
|
||||
assert.equal(removedIds.has(job.id), shouldRemove,
|
||||
`iter ${iter}: job ${job.id} (status=${job.status} key=${k} unambig=${unambiguous}) expected removed=${shouldRemove}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('property: a genuinely-pending job is NEVER lost to a same-basename sibling completing after the snapshot', () => {
|
||||
const rnd = lcg(0x1234abcd);
|
||||
for (let iter = 0; iter < 500; iter++) {
|
||||
const savedAt = 1_000_000_000_000 + Math.floor(rnd() * 1_000_000);
|
||||
// X completed after the snapshot (logged); Y is a DIFFERENT file, same
|
||||
// basename + hoster, genuinely pending. Y must survive.
|
||||
const jobs = [
|
||||
{ id: 'X', fileName: 'clip.mp4', hoster: 'voe.sx', status: 'preview', file: 'C:/A/clip.mp4' },
|
||||
{ id: 'Y', fileName: 'clip.mp4', hoster: 'voe.sx', status: 'preview', file: 'C:/B/clip.mp4' }
|
||||
];
|
||||
const log = [{ fileName: 'clip.mp4', hoster: 'voe.sx', ts: savedAt + 1000 + Math.floor(rnd() * 1000) }];
|
||||
const { kept } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.ok(kept.some(j => j.id === 'Y'), `iter ${iter}: pending Y must never be silently dropped`);
|
||||
}
|
||||
});
|
||||
|
||||
test('property: 2-arg legacy call NEVER removes a non-done job (the v3.3.80 canary, fuzzed)', () => {
|
||||
const rnd = lcg(0xdeadbeef);
|
||||
const statuses = ['preview', 'error', 'aborted', 'queued', 'skipped'];
|
||||
const hosters = ['voe.sx', 'byse.sx'];
|
||||
const names = ['a.mkv', 'b.mp4'];
|
||||
const pick = (arr) => arr[Math.floor(rnd() * arr.length)];
|
||||
|
||||
for (let iter = 0; iter < 1000; iter++) {
|
||||
const jobs = [];
|
||||
const nJobs = 1 + Math.floor(rnd() * 5);
|
||||
for (let i = 0; i < nJobs; i++) {
|
||||
jobs.push({ id: `j${i}`, fileName: pick(names), hoster: pick(hosters), status: pick(statuses), file: `C:/x/${i}` });
|
||||
}
|
||||
const log = [];
|
||||
const nLog = Math.floor(rnd() * 4);
|
||||
for (let i = 0; i < nLog; i++) {
|
||||
log.push({ fileName: pick(names), hoster: pick(hosters), ts: Math.floor(rnd() * 2_000_000_000_000) });
|
||||
}
|
||||
const { removed } = partitionRestoredJobsByLog(jobs, log);
|
||||
assert.ok(removed.every(j => j.status === 'done'), `iter ${iter}: legacy 2-arg call must never drop a non-done job`);
|
||||
}
|
||||
});
|
||||
@ -1,6 +1,6 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { partitionRestoredJobsByLog, completedSelectionKeys } = require('../lib/queue-dedup');
|
||||
const { partitionRestoredJobsByLog } = require('../lib/queue-dedup');
|
||||
|
||||
function job(status, fileName, hoster) {
|
||||
return { status, fileName, hoster, file: `C:/dl/${fileName}` };
|
||||
@ -70,184 +70,3 @@ test('empty/missing inputs do not throw', () => {
|
||||
const jobs = [job('done', 'x.mkv', 'voe.sx')];
|
||||
assert.equal(partitionRestoredJobsByLog(jobs, undefined).kept.length, 1);
|
||||
});
|
||||
|
||||
const T = (s) => Date.parse(s.replace(' ', 'T'));
|
||||
|
||||
test('ts-gate: preview job uploaded AFTER the snapshot is dropped (the ghost bug)', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1, 'completed-after-snapshot preview is a ghost → drop');
|
||||
assert.equal(kept.length, 0);
|
||||
});
|
||||
|
||||
test('ts-gate: preview job whose only log entry PREDATES the snapshot is kept (intentional re-upload)', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 11:00:00') }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 0, 'old upload + freshly-queued re-upload must survive');
|
||||
assert.equal(kept.length, 1);
|
||||
});
|
||||
|
||||
test('ts-gate: same-second completion is dropped (savedAt floored to the second)', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 12:00:00') }];
|
||||
const savedAt = T('2026-06-19 12:00:00') + 800;
|
||||
const { removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1, 'log second-granularity must not let same-second ghosts slip through');
|
||||
});
|
||||
|
||||
test('ts-gate: uses the MAX log ts per key (re-upload after a stale earlier entry)', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [
|
||||
{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 11:00:00') },
|
||||
{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }
|
||||
];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1, 'newest matching log entry decides');
|
||||
});
|
||||
|
||||
test('ts-gate inactive without savedAt → legacy behavior (preview kept even if ts newer)', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log);
|
||||
assert.equal(removed.length, 0);
|
||||
assert.equal(kept.length, 1);
|
||||
});
|
||||
|
||||
test('ts-gate inactive when log entry lacks ts → legacy behavior', () => {
|
||||
const jobs = [job('preview', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx' }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 0);
|
||||
assert.equal(kept.length, 1);
|
||||
});
|
||||
|
||||
test('ts-gate: done job uploaded after snapshot is dropped via either rule', () => {
|
||||
const jobs = [job('done', 'a.mkv', 'voe.sx')];
|
||||
const log = [{ fileName: 'a.mkv', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1);
|
||||
});
|
||||
|
||||
test('ts-gate ambiguity guard: a pending same-basename file in a DIFFERENT folder is NOT lost when a sibling completes after the snapshot', () => {
|
||||
// X (C:/A/clip.mp4) was uploaded after the snapshot and logged. Y is a
|
||||
// genuinely-different file (C:/B/clip.mp4), same basename + hoster, still
|
||||
// pending. The log records only basenames, so the ts-rule must not drop Y.
|
||||
const jobs = [
|
||||
{ status: 'preview', fileName: 'clip.mp4', hoster: 'voe.sx', file: 'C:/A/clip.mp4' },
|
||||
{ status: 'preview', fileName: 'clip.mp4', hoster: 'voe.sx', file: 'C:/B/clip.mp4' }
|
||||
];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'clip.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 0, 'ambiguous key -> ts-rule suppressed, no pending file lost');
|
||||
assert.equal(kept.length, 2);
|
||||
});
|
||||
|
||||
test('ts-gate ambiguity guard: the done-in-log rule still applies on an ambiguous key', () => {
|
||||
// Even when the key is ambiguous, a job that is actually 'done' and in the log
|
||||
// is still decluttered (pre-existing rule, unchanged by the guard).
|
||||
const jobs = [
|
||||
{ status: 'done', fileName: 'clip.mp4', hoster: 'voe.sx', file: 'C:/A/clip.mp4' },
|
||||
{ status: 'preview', fileName: 'clip.mp4', hoster: 'voe.sx', file: 'C:/B/clip.mp4' }
|
||||
];
|
||||
const log = [{ fileName: 'clip.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1);
|
||||
assert.equal(removed[0].status, 'done');
|
||||
assert.equal(removed[0].file, 'C:/A/clip.mp4');
|
||||
assert.ok(kept.some(j => j.file === 'C:/B/clip.mp4'), 'the distinct pending file survives');
|
||||
});
|
||||
|
||||
test('ts-gate: a unique-path ghost still drops (guard does not weaken the common case)', () => {
|
||||
const jobs = [{ status: 'preview', fileName: 'clip.mp4', hoster: 'voe.sx', file: 'C:/A/clip.mp4' }];
|
||||
const log = [{ fileName: 'clip.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const { removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 1, 'single job for the key -> unambiguous -> ghost dropped as before');
|
||||
});
|
||||
|
||||
test('ts-gate: multi-hoster partial completion — the reported bug shape (drop only the completed hosters)', () => {
|
||||
// One file queued to 4 hosters; close mid-upload. After the snapshot, 2 hosters
|
||||
// completed (logged), 2 never started. On restart all 4 restore as 'preview'.
|
||||
// Must drop EXACTLY the 2 that completed and keep the 2 still-pending. This also
|
||||
// pins per-hoster keying: a fileName-only gate would wrongly drop all 4.
|
||||
const f = 'Einfach mal die Fresse halten!!!.mp4';
|
||||
const jobs = [
|
||||
job('preview', f, 'doodstream.com'),
|
||||
job('preview', f, 'voe.sx'),
|
||||
job('preview', f, 'vidmoly.me'),
|
||||
job('preview', f, 'byse.sx')
|
||||
];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [
|
||||
{ fileName: f, hoster: 'doodstream.com', ts: T('2026-06-19 12:00:08') },
|
||||
{ fileName: f, hoster: 'voe.sx', ts: T('2026-06-19 12:00:11') }
|
||||
];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, log, savedAt);
|
||||
assert.equal(removed.length, 2, 'only the 2 completed-after-snapshot hosters drop');
|
||||
assert.ok(removed.every(j => j.hoster === 'doodstream.com' || j.hoster === 'voe.sx'));
|
||||
assert.equal(kept.length, 2, 'the 2 never-started hosters survive');
|
||||
assert.ok(kept.some(j => j.hoster === 'vidmoly.me'));
|
||||
assert.ok(kept.some(j => j.hoster === 'byse.sx'));
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: a selectedFile that completed after the snapshot yields its full-path|hoster key', () => {
|
||||
const selectedFiles = [{ path: 'C:/dl/done.mp4', name: 'done.mp4' }, { path: 'C:/dl/pending.mp4', name: 'pending.mp4' }];
|
||||
const hosters = ['voe.sx'];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'done.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const keys = completedSelectionKeys(selectedFiles, hosters, log, savedAt);
|
||||
assert.deepEqual(keys, ['C:/dl/done.mp4|voe.sx'], 'only the completed file is seeded; pending is not');
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: per-hoster — a file done on voe but not byse only seeds the voe key', () => {
|
||||
const selectedFiles = [{ path: 'C:/dl/a.mp4', name: 'a.mp4' }];
|
||||
const hosters = ['voe.sx', 'byse.sx'];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'a.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const keys = completedSelectionKeys(selectedFiles, hosters, log, savedAt);
|
||||
assert.deepEqual(keys, ['C:/dl/a.mp4|voe.sx'], 'the still-pending byse upload is NOT seeded');
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: ambiguous basename across folders seeds NOTHING (no lost re-preview)', () => {
|
||||
const selectedFiles = [{ path: 'C:/A/clip.mp4', name: 'clip.mp4' }, { path: 'C:/B/clip.mp4', name: 'clip.mp4' }];
|
||||
const hosters = ['voe.sx'];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'clip.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
const keys = completedSelectionKeys(selectedFiles, hosters, log, savedAt);
|
||||
assert.deepEqual(keys, [], 'ambiguous -> neither path is suppressed, both re-preview (safe direction)');
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: an OLDER completion (pre-snapshot re-queue) is NOT seeded', () => {
|
||||
const selectedFiles = [{ path: 'C:/dl/reup.mp4', name: 'reup.mp4' }];
|
||||
const hosters = ['voe.sx'];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'reup.mp4', hoster: 'voe.sx', ts: T('2026-06-19 11:00:00') }];
|
||||
assert.deepEqual(completedSelectionKeys(selectedFiles, hosters, log, savedAt), []);
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: no savedAt / junk inputs -> empty (legacy + robustness)', () => {
|
||||
const sf = [{ path: 'C:/dl/a.mp4', name: 'a.mp4' }];
|
||||
const log = [{ fileName: 'a.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
assert.deepEqual(completedSelectionKeys(sf, ['voe.sx'], log, undefined), []);
|
||||
assert.deepEqual(completedSelectionKeys(null, ['voe.sx'], log, 1), []);
|
||||
assert.deepEqual(completedSelectionKeys(sf, null, log, 1), []);
|
||||
assert.deepEqual(completedSelectionKeys([], [], log, 1), []);
|
||||
assert.deepEqual(completedSelectionKeys([{ name: 'x' }], ['voe.sx'], log, 1), [], 'entry without path is skipped');
|
||||
});
|
||||
|
||||
test('completedSelectionKeys: derives basename from path when name is missing', () => {
|
||||
const selectedFiles = [{ path: 'C:/dl/sub/movie.mp4' }];
|
||||
const hosters = ['voe.sx'];
|
||||
const savedAt = T('2026-06-19 12:00:00');
|
||||
const log = [{ fileName: 'movie.mp4', hoster: 'voe.sx', ts: T('2026-06-19 12:00:05') }];
|
||||
assert.deepEqual(completedSelectionKeys(selectedFiles, hosters, log, savedAt), ['C:/dl/sub/movie.mp4|voe.sx']);
|
||||
});
|
||||
|
||||
@ -1,83 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { formatUploadLogLine, parseUploadLogLine } = require('../lib/upload-log');
|
||||
const { partitionRestoredJobsByLog } = require('../lib/queue-dedup');
|
||||
|
||||
function makeJobs(n, hoster, status, offset = 0) {
|
||||
const jobs = [];
|
||||
for (let i = 0; i < n; i++) {
|
||||
const fileName = `clip_${String(i + offset).padStart(4, '0')}.mp4`;
|
||||
jobs.push({ id: `j-${hoster}-${i + offset}`, file: `D:/inbox/${fileName}`, fileName, hoster, status });
|
||||
}
|
||||
return jobs;
|
||||
}
|
||||
|
||||
test('user report: 300 queued, ~200 finished mid-session before a hard kill — only the finished drop', () => {
|
||||
const hoster = 'byse.sx';
|
||||
const snapshot = new Date(2026, 5, 19, 22, 0, 0);
|
||||
const savedAt = snapshot.getTime();
|
||||
const restoredJobs = makeJobs(300, hoster, 'preview');
|
||||
|
||||
const completionBase = new Date(2026, 5, 19, 22, 5, 0).getTime();
|
||||
const logEntries = [];
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const d = new Date(completionBase + i * 1000);
|
||||
logEntries.push(parseUploadLogLine(
|
||||
formatUploadLogLine(d, hoster, `https://byse.sx/d/x${i}`, `clip_${String(i).padStart(4, '0')}.mp4`)
|
||||
));
|
||||
}
|
||||
|
||||
const { kept, removed } = partitionRestoredJobsByLog(restoredJobs, logEntries, savedAt);
|
||||
assert.equal(removed.length, 200, 'the 200 completed-after-snapshot files are dropped as ghosts');
|
||||
assert.equal(kept.length, 100, 'the 100 never-finished files stay queued');
|
||||
assert.ok(kept.every(j => Number(j.fileName.slice(5, 9)) >= 200), 'kept are exactly indices 200..299');
|
||||
const keptNames = new Set(kept.map(j => j.fileName));
|
||||
assert.ok(removed.every(j => !keptNames.has(j.fileName)));
|
||||
});
|
||||
|
||||
test('multi-hoster batch: per-hoster completion is independent (a file done on voe but not byse keeps byse)', () => {
|
||||
const savedAt = new Date(2026, 5, 19, 22, 0, 0).getTime();
|
||||
const done = new Date(2026, 5, 19, 22, 3, 0);
|
||||
const jobs = [
|
||||
...makeJobs(3, 'voe.sx', 'preview'),
|
||||
...makeJobs(3, 'byse.sx', 'preview')
|
||||
];
|
||||
const logEntries = [
|
||||
parseUploadLogLine(formatUploadLogLine(done, 'voe.sx', 'l', 'clip_0000.mp4')),
|
||||
parseUploadLogLine(formatUploadLogLine(done, 'voe.sx', 'l', 'clip_0001.mp4')),
|
||||
parseUploadLogLine(formatUploadLogLine(done, 'byse.sx', 'l', 'clip_0000.mp4'))
|
||||
];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, logEntries, savedAt);
|
||||
assert.equal(removed.length, 3);
|
||||
assert.ok(removed.some(j => j.hoster === 'voe.sx' && j.fileName === 'clip_0000.mp4'));
|
||||
assert.ok(removed.some(j => j.hoster === 'voe.sx' && j.fileName === 'clip_0001.mp4'));
|
||||
assert.ok(removed.some(j => j.hoster === 'byse.sx' && j.fileName === 'clip_0000.mp4'));
|
||||
assert.ok(kept.some(j => j.hoster === 'byse.sx' && j.fileName === 'clip_0001.mp4'), 'byse clip_0001 not logged -> kept');
|
||||
});
|
||||
|
||||
test('clean idle close (snapshot AFTER completion) keeps an intentional re-queue of an old file', () => {
|
||||
const hoster = 'voe.sx';
|
||||
const yesterday = new Date(2026, 5, 18, 12, 0, 0);
|
||||
const logEntries = [parseUploadLogLine(formatUploadLogLine(yesterday, hoster, 'link', 'reupload_me.mp4'))];
|
||||
const savedAt = new Date(2026, 5, 19, 9, 0, 0).getTime();
|
||||
const jobs = [{ id: 'r1', file: 'D:/x/reupload_me.mp4', fileName: 'reupload_me.mp4', hoster, status: 'preview' }];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, logEntries, savedAt);
|
||||
assert.equal(removed.length, 0, 'an upload older than the snapshot is a deliberate re-queue and survives');
|
||||
assert.equal(kept.length, 1);
|
||||
});
|
||||
|
||||
test('legacy snapshot without savedAt (pre-v3.3.80 config) falls back to done-only dedup', () => {
|
||||
const hoster = 'voe.sx';
|
||||
const logEntries = [
|
||||
parseUploadLogLine(formatUploadLogLine(new Date(2026, 5, 19, 12, 0, 0), hoster, 'l', 'done.mp4')),
|
||||
parseUploadLogLine(formatUploadLogLine(new Date(2026, 5, 19, 12, 1, 0), hoster, 'l', 'preview.mp4'))
|
||||
];
|
||||
const jobs = [
|
||||
{ id: 'a', file: 'D:/x/done.mp4', fileName: 'done.mp4', hoster, status: 'done' },
|
||||
{ id: 'b', file: 'D:/x/preview.mp4', fileName: 'preview.mp4', hoster, status: 'preview' }
|
||||
];
|
||||
const { kept, removed } = partitionRestoredJobsByLog(jobs, logEntries);
|
||||
assert.equal(removed.length, 1, 'only the done job is decluttered when no savedAt is available');
|
||||
assert.equal(removed[0].id, 'a');
|
||||
assert.ok(kept.some(j => j.id === 'b'), 'the preview survives the legacy path');
|
||||
});
|
||||
@ -1,68 +0,0 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const { EventEmitter } = require('node:events');
|
||||
const { configureStartupRenderer, createStartupWindow } = require('../lib/startup-renderer');
|
||||
|
||||
class TestBrowserWindow extends EventEmitter {
|
||||
constructor(options) {
|
||||
super();
|
||||
this.options = options;
|
||||
this.showCalls = 0;
|
||||
this.startupEvents = [];
|
||||
this.loadError = new Error('renderer load failed');
|
||||
}
|
||||
|
||||
once(eventName, listener) {
|
||||
this.startupEvents.push(`listen:${eventName}`);
|
||||
return super.once(eventName, listener);
|
||||
}
|
||||
|
||||
show() {
|
||||
this.showCalls++;
|
||||
}
|
||||
|
||||
loadFile(target) {
|
||||
this.startupEvents.push(`load:${target}`);
|
||||
return Promise.reject(this.loadError);
|
||||
}
|
||||
}
|
||||
|
||||
test('configureStartupRenderer disables hardware acceleration', () => {
|
||||
let calls = 0;
|
||||
configureStartupRenderer({ disableHardwareAcceleration() { calls++; } });
|
||||
assert.equal(calls, 1);
|
||||
});
|
||||
|
||||
test('createStartupWindow forces the main window to start hidden', () => {
|
||||
const startup = createStartupWindow(TestBrowserWindow, { width: 1100, show: true });
|
||||
|
||||
assert.equal(startup.window.options.width, 1100);
|
||||
assert.equal(startup.window.options.show, false);
|
||||
});
|
||||
|
||||
test('startup load registers visibility before navigation and shows only once', async () => {
|
||||
const startup = createStartupWindow(TestBrowserWindow, {});
|
||||
const loading = startup.load('renderer/index.html', () => {});
|
||||
|
||||
assert.deepEqual(startup.window.startupEvents, [
|
||||
'listen:ready-to-show',
|
||||
'load:renderer/index.html'
|
||||
]);
|
||||
|
||||
startup.window.emit('ready-to-show');
|
||||
startup.window.emit('ready-to-show');
|
||||
await loading;
|
||||
|
||||
assert.equal(startup.window.showCalls, 1);
|
||||
});
|
||||
|
||||
test('startup load forwards a rejected navigation to the error handler', async () => {
|
||||
const startup = createStartupWindow(TestBrowserWindow, {});
|
||||
let handledError;
|
||||
|
||||
await startup.load('renderer/index.html', (err) => {
|
||||
handledError = err;
|
||||
});
|
||||
|
||||
assert.equal(handledError, startup.window.loadError);
|
||||
});
|
||||
@ -3,7 +3,7 @@ const assert = require('node:assert');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
const { sanitizeConfig, collectFile, buildSupportBundleText, redactLogText, REDACTED } = require('../lib/support-bundle');
|
||||
const { sanitizeConfig, collectFile, buildSupportBundleText, REDACTED } = require('../lib/support-bundle');
|
||||
|
||||
test('sanitizeConfig redacts known credential keys at any nesting depth', () => {
|
||||
const input = {
|
||||
@ -24,57 +24,6 @@ test('sanitizeConfig redacts known credential keys at any nesting depth', () =>
|
||||
assert.strictEqual(out.globalSettings.remote.token, REDACTED);
|
||||
});
|
||||
|
||||
test('redactLogText scrubs opaque tokens that are NOT stored config secrets', () => {
|
||||
const cases = [
|
||||
'boom token=bearer_tok_qwerty12345',
|
||||
'response auth_token: aGVsbG8td29ybGQtMTIz',
|
||||
'refresh_token = abc123DEF456ghi789',
|
||||
'using Bearer aaaabbbbccccddddeeeeffff',
|
||||
'Authorization: Bearer deadbeefcafef00dba5e'
|
||||
];
|
||||
for (const line of cases) {
|
||||
const out = redactLogText(line, []);
|
||||
assert.ok(out.includes(REDACTED), `expected redaction in: ${line} -> ${out}`);
|
||||
assert.ok(!/qwerty12345|aGVsbG8|abc123DEF456|aaaabbbbcccc|deadbeefcafe/.test(out), `secret survived: ${out}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('redactLogText leaves benign "token" prose alone', () => {
|
||||
const benign = 'token bucket refill rate is 5 per second';
|
||||
assert.equal(redactLogText(benign, []), benign);
|
||||
});
|
||||
|
||||
test('redactLogText scrubs the password from a basic-auth URL but keeps host:port', () => {
|
||||
const out = redactLogText('proxy https://admin:Sup3rProxyPass@proxy.internal:8080/path', []);
|
||||
assert.ok(!out.includes('Sup3rProxyPass'), 'basic-auth password must be redacted');
|
||||
assert.ok(out.includes('proxy.internal:8080'), 'host:port preserved');
|
||||
assert.ok(out.includes('admin:'), 'username preserved');
|
||||
});
|
||||
|
||||
test('redactLogText does not touch a host:port URL without userinfo', () => {
|
||||
const url = 'connecting to https://cdn.voe.sx:8080/upload now';
|
||||
assert.equal(redactLogText(url, []), url);
|
||||
});
|
||||
|
||||
test('redactLogText scrubs Basic auth, JWTs and bare session= values (defense in depth)', () => {
|
||||
const cases = [
|
||||
{ line: 'Authorization: Basic dXNlcjpwYXNzd29yZDEyMw==', secret: 'dXNlcjpwYXNzd29yZDEyMw' },
|
||||
{ line: 'jwt eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N', secret: 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0' },
|
||||
{ line: 'session=SESSIONsecretvalue99887766', secret: 'SESSIONsecretvalue99887766' },
|
||||
{ line: '"session":"jsonSessionSecret123456"', secret: 'jsonSessionSecret123456' },
|
||||
];
|
||||
for (const c of cases) {
|
||||
const out = redactLogText(c.line, []);
|
||||
assert.ok(!out.includes(c.secret), `must redact: ${c.line} -> ${out}`);
|
||||
assert.ok(out.includes(REDACTED), `expected ${REDACTED} in ${out}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('redactLogText leaves a normal "session" word in prose alone', () => {
|
||||
const benign = 'the session was idle for a while';
|
||||
assert.equal(redactLogText(benign, []), benign);
|
||||
});
|
||||
|
||||
test('sanitizeConfig does not mutate input', () => {
|
||||
const input = { hosters: { 'voe.sx': [{ password: 'secret' }] } };
|
||||
const clone = JSON.parse(JSON.stringify(input));
|
||||
|
||||
@ -1,286 +0,0 @@
|
||||
const { describe, it, beforeEach, mock } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
describe('suspect-reject alternate accounts', () => {
|
||||
let UploadManager;
|
||||
let mockUploadFile;
|
||||
let mockProbe;
|
||||
|
||||
function suspectErr() {
|
||||
const e = new Error('Byse lehnte Datei ab: Not video file format');
|
||||
e.fileRejected = true;
|
||||
e.suspectReject = true;
|
||||
return e;
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
delete require.cache[require.resolve('../lib/upload-manager')];
|
||||
|
||||
const hosters = require('../lib/hosters');
|
||||
mockUploadFile = mock.fn(async () => ({ download_url: 'https://byse.sx/d/ok', embed_url: null, file_code: 'ok' }));
|
||||
hosters.uploadFile = (...a) => mockUploadFile(...a);
|
||||
hosters.prefetchBaseline = async () => null;
|
||||
|
||||
const fileProbe = require('../lib/file-probe');
|
||||
mockProbe = mock.fn(async () => ({ ok: true, kind: 'matroska', isVideoLike: true, headHex: '1a45dfa3' }));
|
||||
fileProbe.probeFileHead = (...a) => mockProbe(...a);
|
||||
|
||||
const fs = require('fs');
|
||||
const fakeSize = (p) => {
|
||||
const m = /-(\d+)gb/i.exec(p);
|
||||
return { size: (m ? parseInt(m[1], 10) : 3) * 1024 * 1024 * 1024 };
|
||||
};
|
||||
const origStatSync = fs.statSync;
|
||||
fs.statSync = function (p) {
|
||||
if (typeof p === 'string' && p.startsWith('/test/')) return fakeSize(p);
|
||||
return origStatSync.call(this, p);
|
||||
};
|
||||
const origStat = fs.promises.stat;
|
||||
fs.promises.stat = async function (p) {
|
||||
if (typeof p === 'string' && p.startsWith('/test/')) return fakeSize(p);
|
||||
return origStat.call(this, p);
|
||||
};
|
||||
|
||||
UploadManager = require('../lib/upload-manager');
|
||||
});
|
||||
|
||||
function poolMgr(pool, settings) {
|
||||
return new UploadManager({ 'byse.sx': { retries: 0, ...(settings || {}) } }, {}, { 'byse.sx': pool });
|
||||
}
|
||||
|
||||
it('tries the file on the next pool account after a suspect rejection and succeeds without blacklisting', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key1') throw suspectErr();
|
||||
return { download_url: 'https://byse.sx/d/alt', embed_url: null, file_code: 'alt' };
|
||||
});
|
||||
const rotEvents = [];
|
||||
mgr.on('rot-log', (e) => rotEvents.push(e.event));
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.succeeded, 1);
|
||||
assert.equal(summary.failed, 0);
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.deepEqual(keys, ['key1', 'key2']);
|
||||
assert.ok(rotEvents.includes('suspect-reject-alt'));
|
||||
assert.equal(mgr.getFailedAccountKeys().length, 0, 'suspect rejection must not blacklist any account');
|
||||
});
|
||||
|
||||
it('fails the file when every pool account gives the suspect rejection — each tried exactly once, none blacklisted', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async () => { throw suspectErr(); });
|
||||
const rotEvents = [];
|
||||
mgr.on('rot-log', (e) => rotEvents.push(e.event));
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.failed, 1);
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.deepEqual(keys, ['key1', 'key2', 'key3']);
|
||||
assert.ok(rotEvents.includes('suspect-reject-exhausted'));
|
||||
assert.equal(mgr.getFailedAccountKeys().length, 0);
|
||||
});
|
||||
|
||||
it('skips pool accounts already marked failed and lands on the last one', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' },
|
||||
{ id: 'acc4', apiKey: 'key4' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key3') throw suspectErr();
|
||||
return { download_url: 'https://byse.sx/d/four', embed_url: null, file_code: 'four' };
|
||||
});
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch(
|
||||
[{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key3', accountId: 'acc3' }],
|
||||
{ primeFailedAccounts: ['byse.sx:acc1', 'byse.sx:acc2'] }
|
||||
);
|
||||
|
||||
assert.equal(summary.succeeded, 1);
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.deepEqual(keys, ['key3', 'key4'], 'failed acc1/acc2 skipped, fourth account finally gets the file');
|
||||
});
|
||||
|
||||
it('does NOT try alternates when the probe says the file is not a video', async () => {
|
||||
mockProbe.mock.mockImplementation(async () => ({ ok: true, kind: 'rar', isVideoLike: false, headHex: '52617221' }));
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async () => { throw suspectErr(); });
|
||||
const rotEvents = [];
|
||||
mgr.on('rot-log', (e) => rotEvents.push(e.event));
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/archive.rar', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.failed, 1);
|
||||
assert.equal(mockUploadFile.mock.calls.length, 1, 'genuine non-video rejection must not burn uploads on other accounts');
|
||||
assert.ok(rotEvents.includes('skip-rotation-file-rejected'));
|
||||
});
|
||||
|
||||
it('records a user cancel during the alternates walk as aborted, not error', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key1') throw suspectErr();
|
||||
mgr.cancel();
|
||||
const e = new Error('This operation was aborted');
|
||||
throw e;
|
||||
});
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.files[0].results[0].status, 'aborted');
|
||||
});
|
||||
|
||||
it('marks an alternate failed on a genuine account error so later suspect files skip it', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' }
|
||||
], { parallelCount: 1 });
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key1') throw suspectErr();
|
||||
if (apiKey === 'key2') {
|
||||
const e = new Error('Byse lehnte Datei ab: 0:0:0:not enough disk space on your account');
|
||||
e.accountError = true;
|
||||
throw e;
|
||||
}
|
||||
return { download_url: 'https://byse.sx/d/three', embed_url: null, file_code: 'three' };
|
||||
});
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([
|
||||
{ file: '/test/big1.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' },
|
||||
{ file: '/test/big2.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }
|
||||
]);
|
||||
|
||||
assert.equal(summary.succeeded, 2);
|
||||
assert.ok(mgr.getFailedAccountKeys().includes('byse.sx:acc2'), 'dead alternate must be remembered');
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.deepEqual(keys, ['key1', 'key2', 'key3', 'key1', 'key3'], 'second same-size file still gets one real attempt on the primary (memo arms only on the 2nd rejection), then skips the dead alternate and lands on the good account');
|
||||
});
|
||||
|
||||
it('size memo short-circuits a later LARGER file once the account has two confirmed rejections', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' }
|
||||
], { parallelCount: 1 });
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key1' || apiKey === 'key2') throw suspectErr();
|
||||
return { download_url: 'https://byse.sx/d/three', embed_url: null, file_code: 'three' };
|
||||
});
|
||||
const rotEvents = [];
|
||||
mgr.on('rot-log', (e) => rotEvents.push(e.event));
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([
|
||||
{ file: '/test/a-1gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' },
|
||||
{ file: '/test/b-1gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' },
|
||||
{ file: '/test/c-2gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }
|
||||
]);
|
||||
|
||||
assert.equal(summary.succeeded, 3);
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.deepEqual(keys, ['key1', 'key2', 'key3', 'key1', 'key3', 'key3'], 'files 1+2 each get a real attempt on the 1GB-rejecting primary (arming the memo at count 2); the larger 3rd file then short-circuits the primary straight to the good account');
|
||||
assert.ok(rotEvents.includes('suspect-memo-skip'), 'the larger third file must skip its primary via the armed size memo');
|
||||
});
|
||||
|
||||
it('sizeMemoEnabled:false disables the pre-skip — the larger third file still gets a real attempt on its primary', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' }
|
||||
], { parallelCount: 1, sizeMemoEnabled: false });
|
||||
mockUploadFile.mock.mockImplementation(async (hoster, file, apiKey) => {
|
||||
if (apiKey === 'key1' || apiKey === 'key2') throw suspectErr();
|
||||
return { download_url: 'https://byse.sx/d/three', embed_url: null, file_code: 'three' };
|
||||
});
|
||||
const rotEvents = [];
|
||||
mgr.on('rot-log', (e) => rotEvents.push(e.event));
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([
|
||||
{ file: '/test/a-1gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' },
|
||||
{ file: '/test/b-1gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' },
|
||||
{ file: '/test/c-2gb.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }
|
||||
]);
|
||||
|
||||
assert.equal(summary.succeeded, 3);
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.equal(keys.filter(k => k === 'key1').length, 3, 'with the memo off every file gets a real attempt on the primary — including the larger third');
|
||||
assert.ok(!rotEvents.includes('suspect-memo-skip'), 'the disabled memo must never short-circuit a primary');
|
||||
});
|
||||
|
||||
it('plain fileRejected without suspect flag keeps the old fast-fail behavior', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' }
|
||||
]);
|
||||
mockUploadFile.mock.mockImplementation(async () => {
|
||||
const e = new Error('Byse lehnte Datei ab: Duplicate');
|
||||
e.fileRejected = true;
|
||||
throw e;
|
||||
});
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.failed, 1);
|
||||
assert.equal(mockUploadFile.mock.calls.length, 1);
|
||||
});
|
||||
|
||||
it('a transient 5xx (byse 502) retries the SAME account and fails clean — no blacklist, no failover cascade', async () => {
|
||||
const mgr = poolMgr([
|
||||
{ id: 'acc1', apiKey: 'key1' },
|
||||
{ id: 'acc2', apiKey: 'key2' },
|
||||
{ id: 'acc3', apiKey: 'key3' }
|
||||
], { retries: 2 });
|
||||
mgr._sleep = async () => {};
|
||||
mockUploadFile.mock.mockImplementation(async () => {
|
||||
const e = new Error('Upload-Antwort von byse.sx war kein JSON (HTTP 502): <!doctype html>');
|
||||
e.transientNetwork = true;
|
||||
throw e;
|
||||
});
|
||||
let accountFailed = 0;
|
||||
mgr.on('account-failed', () => { accountFailed++; });
|
||||
let summary = null;
|
||||
mgr.on('batch-done', (s) => { summary = s; });
|
||||
|
||||
await mgr.startBatch([{ file: '/test/big.mkv', hoster: 'byse.sx', apiKey: 'key1', accountId: 'acc1' }]);
|
||||
|
||||
assert.equal(summary.failed, 1);
|
||||
assert.equal(accountFailed, 0, 'a transient 502 must never emit account-failed');
|
||||
assert.equal(mgr.getFailedAccountKeys().length, 0, 'no account blacklisted on a transient 502');
|
||||
const keys = mockUploadFile.mock.calls.map(c => c.arguments[2]);
|
||||
assert.ok(keys.length >= 2, 'the 502 is retried on the same account');
|
||||
assert.ok(keys.every(k => k === 'key1'), 'every attempt stays on the primary — no cascade to key2/key3');
|
||||
});
|
||||
});
|
||||
@ -1,133 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { makeThrottleTimer } = require('../lib/throttle-timer');
|
||||
|
||||
function fakeClock() {
|
||||
let t = 0;
|
||||
let timers = [];
|
||||
return {
|
||||
now: () => t,
|
||||
schedule: (cb, ms) => {
|
||||
const h = { at: t + ms, cb, dead: false };
|
||||
timers.push(h);
|
||||
return h;
|
||||
},
|
||||
clear: (h) => { if (h) h.dead = true; },
|
||||
advance: (ms) => {
|
||||
const target = t + ms;
|
||||
for (;;) {
|
||||
let next = null;
|
||||
for (const h of timers) {
|
||||
if (!h.dead && h.at <= target && (next === null || h.at < next.at)) next = h;
|
||||
}
|
||||
if (!next) break;
|
||||
t = next.at;
|
||||
next.dead = true;
|
||||
next.cb();
|
||||
}
|
||||
t = target;
|
||||
timers = timers.filter(h => !h.dead);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
test('idle debounce: fires once after the delay window', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
tt.request(() => fired++, 500);
|
||||
c.advance(499);
|
||||
assert.equal(fired, 0);
|
||||
c.advance(1);
|
||||
assert.equal(fired, 1);
|
||||
});
|
||||
|
||||
test('idle debounce: rapid requests reset the timer (last wins)', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
tt.request(() => fired++, 500);
|
||||
c.advance(200);
|
||||
tt.request(() => fired++, 500);
|
||||
c.advance(300);
|
||||
assert.equal(fired, 0, 'should not fire at original 500 — was reset');
|
||||
c.advance(200);
|
||||
assert.equal(fired, 1, 'fires 500ms after the second request');
|
||||
});
|
||||
|
||||
test('STARVATION repro: continuous requests with no maxWait NEVER fire', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
for (let i = 0; i < 30; i++) {
|
||||
tt.request(() => fired++, 500);
|
||||
c.advance(100);
|
||||
}
|
||||
assert.equal(fired, 0, 'this is exactly the bug the maxWait fix addresses');
|
||||
});
|
||||
|
||||
test('maxWait: continuous requests still force a fire within the window', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
const fireAtTimes = [];
|
||||
for (let i = 0; i < 30; i++) {
|
||||
tt.request(() => { fired++; fireAtTimes.push(c.now()); }, 500, 2000);
|
||||
c.advance(100);
|
||||
}
|
||||
assert.ok(fired >= 1, 'maxWait guarantees at least one fire under continuous load');
|
||||
assert.ok(fireAtTimes.every(t => t > 0), 'fires happened, not starved');
|
||||
assert.ok(fireAtTimes.some(t => t <= 2000), 'first fire no later than maxWait');
|
||||
});
|
||||
|
||||
test('maxWait: after a forced fire a fresh burst starts (periodic fires)', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
for (let i = 0; i < 60; i++) {
|
||||
tt.request(() => fired++, 500, 2000);
|
||||
c.advance(100);
|
||||
}
|
||||
assert.ok(fired >= 2, `~6000ms of continuous load with 2000ms maxWait should fire multiple times, got ${fired}`);
|
||||
});
|
||||
|
||||
test('flushSync fires the pending fn immediately and clears it', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
tt.request(() => fired++, 5000);
|
||||
assert.ok(tt.isPending());
|
||||
tt.flushSync();
|
||||
assert.equal(fired, 1);
|
||||
assert.ok(!tt.isPending());
|
||||
c.advance(10000);
|
||||
assert.equal(fired, 1, 'no double fire after flushSync');
|
||||
});
|
||||
|
||||
test('cancel drops the pending fn — no fire', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
let fired = 0;
|
||||
tt.request(() => fired++, 500);
|
||||
tt.cancel();
|
||||
assert.ok(!tt.isPending());
|
||||
c.advance(10000);
|
||||
assert.equal(fired, 0);
|
||||
});
|
||||
|
||||
test('last-write-wins: a later request with a DIFFERENT fn replaces the earlier one', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
const fired = [];
|
||||
tt.request(() => fired.push('persist'), 500, 20000);
|
||||
c.advance(100);
|
||||
tt.request(() => fired.push('clear'), 0);
|
||||
c.advance(100);
|
||||
assert.deepEqual(fired, ['clear'], 'only the latest fn fires; the persist was dropped');
|
||||
});
|
||||
|
||||
test('flushSync with nothing pending is a no-op', () => {
|
||||
const c = fakeClock();
|
||||
const tt = makeThrottleTimer(c);
|
||||
assert.doesNotThrow(() => tt.flushSync());
|
||||
});
|
||||
@ -48,10 +48,7 @@ setTimeout(async () => {
|
||||
console.log('\\n=== Upload View ===');
|
||||
|
||||
const tabCount = await wc.executeJavaScript('document.querySelectorAll(".tab").length');
|
||||
check('4 tabs exist', tabCount === 4);
|
||||
|
||||
const tabLabels = await wc.executeJavaScript('[...document.querySelectorAll(".tab")].map(el => el.textContent.trim()).join("|")');
|
||||
check('Current tab labels present', tabLabels === 'Upload|Accounts|Einstellungen|Verlauf');
|
||||
check('3 tabs exist', tabCount === 3);
|
||||
|
||||
const activeTab = await wc.executeJavaScript('document.querySelector(".tab.active")?.textContent?.trim()');
|
||||
check('Upload tab active by default', activeTab === 'Upload');
|
||||
@ -59,9 +56,12 @@ setTimeout(async () => {
|
||||
const dropVisible = await wc.executeJavaScript('document.getElementById("dropZone")?.style.display !== "none"');
|
||||
check('Drop zone visible (no files)', dropVisible);
|
||||
|
||||
const queueHidden = await wc.executeJavaScript('document.getElementById("queueShell")?.style.display');
|
||||
const queueHidden = await wc.executeJavaScript('document.getElementById("queueContainer")?.style.display');
|
||||
check('Queue hidden (no files)', queueHidden === 'none');
|
||||
|
||||
const chips = await wc.executeJavaScript('document.querySelectorAll(".hoster-chip").length');
|
||||
check('4 hoster chips', chips === 4);
|
||||
|
||||
const startDisabled = await wc.executeJavaScript('document.getElementById("startUploadBtn")?.disabled');
|
||||
check('Start button disabled initially', startDisabled === true);
|
||||
|
||||
@ -74,45 +74,6 @@ setTimeout(async () => {
|
||||
const ctxHidden = await wc.executeJavaScript('document.getElementById("contextMenu")?.style.display');
|
||||
check('Context menu hidden', ctxHidden === 'none');
|
||||
|
||||
console.log('\\n=== Accounts View ===');
|
||||
|
||||
await wc.executeJavaScript('document.querySelector(".tab[data-view=\\'accounts\\']").click()');
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
|
||||
const accountsActive = await wc.executeJavaScript('document.getElementById("accounts-view")?.classList.contains("active")');
|
||||
check('Accounts tab active', accountsActive);
|
||||
|
||||
const accountListValid = await wc.executeJavaScript('Boolean(document.querySelector("#accountsList .accounts-empty") || document.querySelectorAll("#accountsList .account-hoster-group").length)');
|
||||
check('Account manager list structure rendered', accountListValid);
|
||||
|
||||
const addAccountEnabled = await wc.executeJavaScript('document.getElementById("addAccountBtn")?.disabled === false');
|
||||
check('Add account button enabled', addAccountEnabled);
|
||||
|
||||
await wc.executeJavaScript('document.getElementById("addAccountBtn").click()');
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
|
||||
const accountModalVisible = await wc.executeJavaScript('document.getElementById("accountModal")?.style.display');
|
||||
check('Account modal opens', accountModalVisible === 'flex');
|
||||
|
||||
const accountModalTitle = await wc.executeJavaScript('document.getElementById("accountModalTitle")?.textContent');
|
||||
check('Account modal is in add mode', accountModalTitle === 'Account hinzufügen');
|
||||
|
||||
const authOptionCount = await wc.executeJavaScript('document.querySelectorAll("#accountHosterSelect option").length');
|
||||
check('7 hoster authentication options exist', authOptionCount === 7);
|
||||
|
||||
const hosterCount = await wc.executeJavaScript('[...new Set([...document.querySelectorAll("#accountHosterSelect option")].map(el => el.value.split(":")[0]))].length');
|
||||
check('5 hosters exist', hosterCount === 5);
|
||||
|
||||
const accountSubmitLabel = await wc.executeJavaScript('document.getElementById("saveAccountBtn")?.textContent');
|
||||
check('Account submit label is Prüfen und anlegen', accountSubmitLabel === 'Prüfen und anlegen');
|
||||
|
||||
const credentialInputs = await wc.executeJavaScript('document.querySelectorAll("#accountCredsFields .key-input").length');
|
||||
check('Credential inputs rendered', credentialInputs === 2);
|
||||
|
||||
await wc.executeJavaScript('document.getElementById("cancelAccountModalBtn").click()');
|
||||
const accountModalHidden = await wc.executeJavaScript('document.getElementById("accountModal")?.style.display');
|
||||
check('Account modal closes', accountModalHidden === 'none');
|
||||
|
||||
console.log('\\n=== Settings View ===');
|
||||
|
||||
await wc.executeJavaScript('document.querySelector(".tab[data-view=\\'settings\\']").click()');
|
||||
@ -121,14 +82,23 @@ setTimeout(async () => {
|
||||
const settingsActive = await wc.executeJavaScript('document.getElementById("settings-view")?.classList.contains("active")');
|
||||
check('Settings tab active', settingsActive);
|
||||
|
||||
const settingsSubtabs = await wc.executeJavaScript('document.querySelectorAll(".settings-subtab").length');
|
||||
check('6 settings subtabs exist', settingsSubtabs === 6);
|
||||
const panels = await wc.executeJavaScript('document.querySelectorAll(".hoster-settings-panel").length');
|
||||
check('4 hoster panels', panels === 4);
|
||||
|
||||
const accountSettingsPointer = await wc.executeJavaScript('document.querySelector(".settings-hoster-pointer")?.textContent');
|
||||
check('Hoster settings point to Accounts tab', accountSettingsPointer && accountSettingsPointer.includes('Accounts'));
|
||||
const hsInputCount = await wc.executeJavaScript('document.querySelectorAll(".hs-input").length');
|
||||
check('24 per-hoster inputs (6x4)', hsInputCount === 24);
|
||||
|
||||
const parallel = await wc.executeJavaScript('document.getElementById("parallelUploadCountInput")?.value');
|
||||
check('Global parallel uploads default 0', parallel === '0');
|
||||
await wc.executeJavaScript('document.querySelector(".hoster-panel-header").click()');
|
||||
await new Promise(r => setTimeout(r, 200));
|
||||
|
||||
const panelBody = await wc.executeJavaScript('document.querySelector(".hoster-panel-body").style.display');
|
||||
check('Panel expands on click', panelBody !== 'none');
|
||||
|
||||
const retries = await wc.executeJavaScript('document.querySelector(".hs-input[data-hs=\\'retries\\']")?.value');
|
||||
check('Retries default 3', retries === '3');
|
||||
|
||||
const parallel = await wc.executeJavaScript('document.querySelector(".hs-input[data-hs=\\'parallelCount\\']")?.value');
|
||||
check('ParallelCount default 2', parallel === '2');
|
||||
|
||||
// Test save
|
||||
await wc.executeJavaScript('document.getElementById("saveSettingsBtn").click()');
|
||||
@ -167,7 +137,8 @@ setTimeout(async () => {
|
||||
results.forEach(r => console.log(r));
|
||||
console.log('\\nTotal: ' + (passed + failed) + ' | Passed: ' + passed + ' | Failed: ' + failed);
|
||||
|
||||
app.exit(failed > 0 ? 1 : 0);
|
||||
if (failed > 0) process.exitCode = 1;
|
||||
app.quit();
|
||||
}, 5000);
|
||||
`;
|
||||
|
||||
@ -195,7 +166,9 @@ try {
|
||||
.join('\n');
|
||||
if (filtered.trim()) console.error(filtered);
|
||||
}
|
||||
process.exitCode = Number.isInteger(err.status) && err.status !== 0 ? err.status : 1;
|
||||
if (err.status && err.status !== 0 && !err.killed) {
|
||||
process.exit(err.status);
|
||||
}
|
||||
} finally {
|
||||
try { fs.unlinkSync(injectPath); } catch {}
|
||||
}
|
||||
|
||||
@ -1,98 +0,0 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const path = require('node:path');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
const { pathToFileURL } = require('node:url');
|
||||
|
||||
const { isNewer, resolveReleaseVersion } = require('../lib/updater');
|
||||
|
||||
test('bridge title resolves product version instead of transport tag', () => {
|
||||
assert.equal(resolveReleaseVersion({ name: 'Multi-Hoster-Upload v2.0.1', tag_name: 'v3.3.109' }), '2.0.1');
|
||||
assert.equal(isNewer('2.0.1', '2.0.1'), false);
|
||||
assert.equal(isNewer('2.0.2', '2.0.1'), true);
|
||||
});
|
||||
|
||||
test('release CLI rejects a malformed transport tag before release work', () => {
|
||||
const script = path.resolve(__dirname, '../scripts/release_gitea.mjs');
|
||||
const result = spawnSync(process.execPath, [script, '2.0.1', '--transport-tag', '3.3.109', 'Bridge', '--dry-run'], {
|
||||
cwd: path.resolve(__dirname, '..'),
|
||||
encoding: 'utf8'
|
||||
});
|
||||
|
||||
assert.equal(result.status, 1);
|
||||
assert.match(result.stderr, /--transport-tag must match vX\.Y\.Z/);
|
||||
assert.doesNotMatch(result.stdout, /npm run release:win/);
|
||||
});
|
||||
|
||||
test('release plan keeps product artifacts separate from the transport tag', () => {
|
||||
const script = path.resolve(__dirname, '../scripts/release_gitea.mjs');
|
||||
const moduleUrl = pathToFileURL(script).href;
|
||||
const source = `
|
||||
import { createReleasePlan, parseReleaseArgs, renderLatestYml } from ${JSON.stringify(moduleUrl)};
|
||||
const plan = createReleasePlan(parseReleaseArgs(['2.0.1', '--transport-tag', 'v3.3.109', 'Bridge', 'notes']));
|
||||
const latestYml = renderLatestYml(plan, 'abc123', 456, '2026-08-07T12:00:00.000Z');
|
||||
process.stdout.write(JSON.stringify({
|
||||
version: plan.version,
|
||||
transportTag: plan.transportTag,
|
||||
releaseTitle: plan.releaseTitle,
|
||||
releaseBody: plan.releaseBody,
|
||||
expectedArtifacts: plan.expectedArtifacts,
|
||||
latestYml
|
||||
}));
|
||||
`;
|
||||
const result = spawnSync(process.execPath, ['--input-type=module', '--eval', source], {
|
||||
cwd: path.resolve(__dirname, '..'),
|
||||
encoding: 'utf8'
|
||||
});
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.deepEqual(JSON.parse(result.stdout), {
|
||||
version: '2.0.1',
|
||||
transportTag: 'v3.3.109',
|
||||
releaseTitle: 'Multi-Hoster-Upload v2.0.1',
|
||||
releaseBody: 'Bridge notes',
|
||||
expectedArtifacts: [
|
||||
'Multi-Hoster-Upload Setup 2.0.1.exe',
|
||||
'Multi-Hoster-Upload 2.0.1.exe',
|
||||
'latest.yml'
|
||||
],
|
||||
latestYml: "version: 2.0.1\nfiles:\n - url: Multi-Hoster-Upload Setup 2.0.1.exe\n sha512: abc123\n size: 456\npath: Multi-Hoster-Upload Setup 2.0.1.exe\nsha512: abc123\nreleaseDate: '2026-08-07T12:00:00.000Z'\n"
|
||||
});
|
||||
});
|
||||
|
||||
test('compatible existing release preserves the recovery id', async () => {
|
||||
const moduleUrl = pathToFileURL(path.resolve(__dirname, '../scripts/release_gitea.mjs')).href;
|
||||
const { createReleasePlan, parseReleaseArgs, resolveExistingReleaseId } = await import(moduleUrl);
|
||||
const plan = createReleasePlan(parseReleaseArgs(['2.0.1', '--transport-tag', 'v3.3.109', 'Bridge notes']));
|
||||
const release = {
|
||||
id: 81,
|
||||
tag_name: 'v3.3.109',
|
||||
name: 'Multi-Hoster-Upload v2.0.1',
|
||||
body: 'Bridge notes',
|
||||
draft: false,
|
||||
prerelease: false,
|
||||
assets: []
|
||||
};
|
||||
|
||||
assert.equal(resolveExistingReleaseId(plan, release), 81);
|
||||
});
|
||||
|
||||
test('incompatible existing release title fails closed', async () => {
|
||||
const moduleUrl = pathToFileURL(path.resolve(__dirname, '../scripts/release_gitea.mjs')).href;
|
||||
const { createReleasePlan, parseReleaseArgs, resolveExistingReleaseId } = await import(moduleUrl);
|
||||
const plan = createReleasePlan(parseReleaseArgs(['2.0.1', '--transport-tag', 'v3.3.109', 'Bridge notes']));
|
||||
const release = {
|
||||
id: 81,
|
||||
tag_name: 'v3.3.109',
|
||||
name: 'Multi-Hoster-Upload v3.3.109',
|
||||
body: 'Old transport release',
|
||||
draft: false,
|
||||
prerelease: false,
|
||||
assets: []
|
||||
};
|
||||
|
||||
assert.throws(
|
||||
() => resolveExistingReleaseId(plan, release),
|
||||
/Refusing recovery for v3\.3\.109: existing release title "Multi-Hoster-Upload v3\.3\.109" does not match "Multi-Hoster-Upload v2\.0\.1"/
|
||||
);
|
||||
});
|
||||
@ -1,80 +0,0 @@
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { formatUploadLogLine, parseUploadLogLine } = require('../lib/upload-log');
|
||||
const { partitionRestoredJobsByLog } = require('../lib/queue-dedup');
|
||||
|
||||
function previewJob(fileName, hoster) {
|
||||
return { status: 'preview', fileName, hoster, file: `C:/dl/${fileName}` };
|
||||
}
|
||||
|
||||
test('writer -> reader round trip: parsed ts is the same epoch frame as the source Date getTime', () => {
|
||||
const d = new Date(2026, 5, 19, 12, 0, 30);
|
||||
const line = formatUploadLogLine(d, 'voe.sx', 'https://voe.sx/x', 'a.mkv');
|
||||
const parsed = parseUploadLogLine(line);
|
||||
assert.equal(parsed.hoster, 'voe.sx');
|
||||
assert.equal(parsed.fileName, 'a.mkv');
|
||||
assert.equal(parsed.ts, d.getTime(), 'parser ts must equal the writer Date epoch (no tz shift)');
|
||||
});
|
||||
|
||||
test('SEAM: a real appendUploadLog-format line drops a preview ghost vs a savedAt taken BEFORE completion', () => {
|
||||
const completion = new Date(2026, 5, 19, 12, 0, 30);
|
||||
const line = formatUploadLogLine(completion, 'voe.sx', 'link', 'a.mkv');
|
||||
const parsed = parseUploadLogLine(line);
|
||||
const savedAt = completion.getTime() - 5000;
|
||||
const { removed, kept } = partitionRestoredJobsByLog([previewJob('a.mkv', 'voe.sx')], [parsed], savedAt);
|
||||
assert.equal(removed.length, 1, 'a file logged after the snapshot is a ghost and must drop');
|
||||
assert.equal(kept.length, 0);
|
||||
});
|
||||
|
||||
test('SEAM: the same real line is KEPT vs a savedAt taken AFTER completion (intentional re-upload)', () => {
|
||||
const completion = new Date(2026, 5, 19, 12, 0, 30);
|
||||
const parsed = parseUploadLogLine(formatUploadLogLine(completion, 'voe.sx', 'link', 'a.mkv'));
|
||||
const savedAt = completion.getTime() + 5000;
|
||||
const { removed, kept } = partitionRestoredJobsByLog([previewJob('a.mkv', 'voe.sx')], [parsed], savedAt);
|
||||
assert.equal(removed.length, 0, 'an older upload than the snapshot is a deliberate re-queue and must survive');
|
||||
assert.equal(kept.length, 1);
|
||||
});
|
||||
|
||||
test('parseUploadLogLine skips comments, blanks and malformed lines', () => {
|
||||
assert.equal(parseUploadLogLine('# fileuploader log'), null);
|
||||
assert.equal(parseUploadLogLine(''), null);
|
||||
assert.equal(parseUploadLogLine(' '), null);
|
||||
assert.equal(parseUploadLogLine('only|three|parts|here'), null);
|
||||
assert.equal(parseUploadLogLine(null), null);
|
||||
assert.equal(parseUploadLogLine(42), null);
|
||||
});
|
||||
|
||||
test('parseUploadLogLine: missing/garbage timestamp yields ts=undefined (legacy lines still match by name)', () => {
|
||||
const parsed = parseUploadLogLine('|voe.sx|link||a.mkv|');
|
||||
assert.equal(parsed.hoster, 'voe.sx');
|
||||
assert.equal(parsed.fileName, 'a.mkv');
|
||||
assert.equal(parsed.ts, undefined);
|
||||
});
|
||||
|
||||
test('parseUploadLogLine: a pipe in the link does NOT shift the filename field (entry not lost)', () => {
|
||||
const line = formatUploadLogLine(new Date(2026, 5, 19, 12, 0, 0), 'byse.sx', 'https://h.io/a|b', 'movie.mkv');
|
||||
const parsed = parseUploadLogLine(line);
|
||||
assert.equal(parsed.hoster, 'byse.sx');
|
||||
assert.equal(parsed.fileName, 'movie.mkv', 'filename is taken as the last non-empty field, robust to link pipes');
|
||||
});
|
||||
|
||||
test('parseUploadLogLine: two pipes in the link still parse the correct filename', () => {
|
||||
const line = formatUploadLogLine(new Date(2026, 5, 19, 12, 0, 0), 'byse.sx', 'https://h.io/a|b|c', 'movie.mkv');
|
||||
const parsed = parseUploadLogLine(line);
|
||||
assert.equal(parsed.fileName, 'movie.mkv');
|
||||
});
|
||||
|
||||
test('parseUploadLogLine: a leading-space filename is preserved (matches the untrimmed queue-job key)', () => {
|
||||
const line = formatUploadLogLine(new Date(2026, 5, 19, 12, 0, 0), 'voe.sx', 'https://h.io/a', ' movie.mkv');
|
||||
const parsed = parseUploadLogLine(line);
|
||||
assert.equal(parsed.fileName, ' movie.mkv', 'filename is NOT trimmed, so it matches the OS basename verbatim');
|
||||
});
|
||||
|
||||
test('SEAM: a leading-space filename round-trips and the gate still drops its ghost', () => {
|
||||
const completion = new Date(2026, 5, 19, 12, 0, 30);
|
||||
const parsed = parseUploadLogLine(formatUploadLogLine(completion, 'voe.sx', 'l', ' spaced.mp4'));
|
||||
const savedAt = completion.getTime() - 5000;
|
||||
const job = { status: 'preview', fileName: ' spaced.mp4', hoster: 'voe.sx', file: 'C:/dl/ spaced.mp4' };
|
||||
const { removed } = partitionRestoredJobsByLog([job], [parsed], savedAt);
|
||||
assert.equal(removed.length, 1, 'leading-space filename now matches end-to-end (was a mismatch before)');
|
||||
});
|
||||
@ -33,7 +33,7 @@ describe('UploadManager', () => {
|
||||
hosters.uploadFile = mockUploadFile;
|
||||
hosters.prefetchBaseline = async () => null;
|
||||
|
||||
// Mock fs.statSync + fs.promises.stat for test file paths
|
||||
// Mock fs.statSync for test file paths
|
||||
const fs = require('fs');
|
||||
const origStatSync = fs.statSync;
|
||||
fs.statSync = function(p) {
|
||||
@ -42,13 +42,6 @@ describe('UploadManager', () => {
|
||||
}
|
||||
return origStatSync.call(this, p);
|
||||
};
|
||||
const origStat = fs.promises.stat;
|
||||
fs.promises.stat = async function(p) {
|
||||
if (typeof p === 'string' && p.startsWith('/test/')) {
|
||||
return { size: fakeFileSize };
|
||||
}
|
||||
return origStat.call(this, p);
|
||||
};
|
||||
|
||||
UploadManager = require('../lib/upload-manager');
|
||||
});
|
||||
@ -338,10 +331,10 @@ describe('UploadManager', () => {
|
||||
});
|
||||
|
||||
it('file not found produces descriptive error', async () => {
|
||||
// Override fs.promises.stat to throw ENOENT for a specific path
|
||||
// Override fs.statSync to throw ENOENT for a specific path
|
||||
const fs = require('fs');
|
||||
const origStat = fs.promises.stat;
|
||||
fs.promises.stat = async function(p) {
|
||||
const origStat = fs.statSync;
|
||||
fs.statSync = function(p) {
|
||||
if (p === '/test/deleted.mp4') throw Object.assign(new Error('ENOENT'), { code: 'ENOENT' });
|
||||
return origStat.call(this, p);
|
||||
};
|
||||
@ -354,7 +347,7 @@ describe('UploadManager', () => {
|
||||
{ file: '/test/deleted.mp4', hoster: 'doodstream.com', apiKey: 'key1' }
|
||||
]);
|
||||
|
||||
fs.promises.stat = origStat;
|
||||
fs.statSync = origStat;
|
||||
assert.ok(errors.some(e => e.includes('nicht gefunden')), `expected "nicht gefunden" error, got: ${errors.join(', ')}`);
|
||||
});
|
||||
|
||||
@ -854,45 +847,6 @@ describe('UploadManager', () => {
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(err), false);
|
||||
});
|
||||
|
||||
it('transientNetwork flag is recognised even with an empty/absent message', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const flagged = new Error('');
|
||||
flagged.transientNetwork = true;
|
||||
assert.equal(mgr._isTransientNetworkError(flagged), true, 'flag must win before the empty-message guard');
|
||||
assert.equal(mgr._isFileRejectedError(flagged), false);
|
||||
assert.equal(mgr._isHosterTransientError(flagged), false);
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(flagged), false);
|
||||
|
||||
const flaggedHtml = new Error('Upload-Antwort von byse.sx war kein JSON (HTTP 502): <!doctype html> forbidden duplicate');
|
||||
flaggedHtml.transientNetwork = true;
|
||||
assert.equal(mgr._isTransientNetworkError(flaggedHtml), true);
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(flaggedHtml), false, 'flag overrides any account-keyword in the 502 HTML snippet');
|
||||
assert.equal(mgr._isFileRejectedError(flaggedHtml), false, 'flag overrides any rejection-keyword in the 502 HTML snippet');
|
||||
});
|
||||
|
||||
it('5xx / gateway errors classify transient by message (defensive fallback), 4xx stay account-level', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const transient = [
|
||||
'Upload-Antwort von byse.sx war kein JSON (HTTP 502): <!doctype html>',
|
||||
'Upload fehlgeschlagen (HTTP 503, text/html)',
|
||||
'HTTP 504 Gateway Time-out',
|
||||
'Bad Gateway',
|
||||
'Service Unavailable'
|
||||
];
|
||||
for (const msg of transient) {
|
||||
assert.equal(mgr._isTransientNetworkError(new Error(msg)), true, `should be transient: ${msg}`);
|
||||
}
|
||||
const accountLevel = [
|
||||
'Upload fehlgeschlagen (HTTP 429, application/json)',
|
||||
'HTTP 403 Forbidden',
|
||||
'HTTP 401 Unauthorized'
|
||||
];
|
||||
for (const msg of accountLevel) {
|
||||
assert.equal(mgr._isTransientNetworkError(new Error(msg)), false, `must NOT be transient: ${msg}`);
|
||||
assert.equal(mgr._shouldSkipRetryOnAccountError(new Error(msg)), true, `must stay account-level: ${msg}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('hoster-transient regex fallback catches wrapped doodstream empty-form errors', () => {
|
||||
const mgr = new UploadManager({});
|
||||
const cases = [
|
||||
|
||||
@ -1,208 +1,195 @@
|
||||
// Pure unit tests for the validate-credentials shape contract — does NOT spin
|
||||
// up Electron or the real per-hoster checkers. Those need network. We verify
|
||||
// the SHAPE the ephemeral hosterConfig is built into (which the per-hoster
|
||||
// checkers consume) plus the snapshot-key/invalidation invariants that the
|
||||
// renderer relies on to enforce "validated creds only".
|
||||
//
|
||||
// The three assertions the advisor called out as the regression guard for the
|
||||
// user's "mehrfach angelegt" complaint:
|
||||
// (a) failed validation persists nothing to config.hosters
|
||||
// (b) a second "Anlegen" click with the guard set persists exactly one entry
|
||||
// (c) OTP-required path persists nothing
|
||||
// are exercised at the state-machine level by simulating the renderer's logic
|
||||
// (re-implemented here as pure functions for testability — the real ones live
|
||||
// in renderer/app.js which can't run under node:test).
|
||||
|
||||
const { test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const {
|
||||
createAccountSubmitter,
|
||||
getAccountSubmitLabel,
|
||||
submitValidatedAccount
|
||||
} = require('../renderer/account-submit');
|
||||
const assert = require('node:assert');
|
||||
|
||||
test('account submit labels stay exact for add, edit, and OTP retries', () => {
|
||||
assert.equal(getAccountSubmitLabel({ isEdit: false, hasOtp: false }), 'Prüfen und anlegen');
|
||||
assert.equal(getAccountSubmitLabel({ isEdit: true, hasOtp: false }), 'Prüfen und speichern');
|
||||
assert.equal(getAccountSubmitLabel({ isEdit: false, hasOtp: true }), 'Prüfen und anlegen');
|
||||
assert.equal(getAccountSubmitLabel({ isEdit: true, hasOtp: true }), 'Prüfen und speichern');
|
||||
});
|
||||
// ---- Re-implementations of the renderer's pure helpers ----
|
||||
// These mirror the production code exactly so the tests serve as both a guard
|
||||
// and executable spec for what saveAccount() must do.
|
||||
|
||||
test('close and reopen cannot start a second save while the first save is pending', async () => {
|
||||
const submitter = createAccountSubmitter();
|
||||
let current = true;
|
||||
let commits = 0;
|
||||
let applies = 0;
|
||||
let saveStarted;
|
||||
let finishSave;
|
||||
const started = new Promise(resolve => { saveStarted = resolve; });
|
||||
const saving = new Promise(resolve => { finishSave = resolve; });
|
||||
const first = submitter.submit({
|
||||
validate: async () => ({ status: 'ok' }),
|
||||
commit: async () => {
|
||||
commits++;
|
||||
saveStarted();
|
||||
await saving;
|
||||
return { accountId: 'first' };
|
||||
},
|
||||
afterCommit: async () => {
|
||||
applies++;
|
||||
},
|
||||
isCurrent: () => current
|
||||
});
|
||||
|
||||
await started;
|
||||
current = false;
|
||||
const second = submitter.submit({
|
||||
validate: async () => ({ status: 'ok' }),
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
|
||||
assert.equal(second, null);
|
||||
assert.equal(submitter.isBusy(), true);
|
||||
finishSave();
|
||||
const result = await first;
|
||||
|
||||
assert.equal(result.status, 'stale');
|
||||
assert.equal(result.committed, true);
|
||||
assert.equal(commits, 1);
|
||||
assert.equal(applies, 1);
|
||||
assert.equal(submitter.isBusy(), false);
|
||||
});
|
||||
|
||||
test('post-save apply failure remains committed and cannot invite a duplicate retry', async () => {
|
||||
const expected = new Error('render failed');
|
||||
let saves = 0;
|
||||
let applies = 0;
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => ({ status: 'ok' }),
|
||||
commit: async () => {
|
||||
saves++;
|
||||
return { accountId: 'saved-account' };
|
||||
},
|
||||
afterCommit: async () => {
|
||||
applies++;
|
||||
throw expected;
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'committed');
|
||||
assert.equal(result.value.accountId, 'saved-account');
|
||||
assert.equal(result.postCommitError, expected);
|
||||
assert.equal(saves, 1);
|
||||
assert.equal(applies, 1);
|
||||
});
|
||||
|
||||
test('ok validates and commits exactly once in one submission', async () => {
|
||||
let validations = 0;
|
||||
let commits = 0;
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => {
|
||||
validations++;
|
||||
return { status: 'ok', message: 'Login erfolgreich' };
|
||||
},
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'committed');
|
||||
assert.equal(validations, 1);
|
||||
assert.equal(commits, 1);
|
||||
});
|
||||
|
||||
test('warn validates and commits exactly once in one submission', async () => {
|
||||
let commits = 0;
|
||||
const validation = { status: 'warn', message: 'Login mit Warnung' };
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => validation,
|
||||
commit: async (received) => {
|
||||
commits++;
|
||||
assert.equal(received, validation);
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'committed');
|
||||
assert.equal(result.validation, validation);
|
||||
assert.equal(commits, 1);
|
||||
});
|
||||
|
||||
for (const status of ['error', 'skipped']) {
|
||||
test(`${status} rejects without committing`, async () => {
|
||||
let commits = 0;
|
||||
const validation = { status, message: `${status} result` };
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => validation,
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'rejected');
|
||||
assert.equal(result.validation, validation);
|
||||
assert.equal(commits, 0);
|
||||
});
|
||||
function credsSnapshotKey(authType, creds) {
|
||||
if (authType === 'login') return `login:${creds.username || ''}:${creds.password || ''}`;
|
||||
return `api:${creds.apiKey || ''}`;
|
||||
}
|
||||
|
||||
test('validate throw returns error without committing', async () => {
|
||||
const expected = new Error('validation failed');
|
||||
let commits = 0;
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => {
|
||||
throw expected;
|
||||
},
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => true
|
||||
});
|
||||
function buildEphemeralHosterConfig(payload) {
|
||||
return {
|
||||
username: payload.username || '',
|
||||
password: payload.password || '',
|
||||
apiKey: payload.apiKey || '',
|
||||
enabled: true
|
||||
};
|
||||
}
|
||||
|
||||
assert.equal(result.status, 'error');
|
||||
assert.equal(result.error, expected);
|
||||
assert.equal(commits, 0);
|
||||
// State-machine simulator that mirrors saveAccount() WITHOUT DOM/IPC.
|
||||
function makeStateMachine({ validateImpl, persistImpl }) {
|
||||
let busy = false;
|
||||
let validated = null; // { hosterName, authType, snapshot, status }
|
||||
const log = []; // log of every persist call, for assertions
|
||||
|
||||
async function click(ctx, creds, otp = '') {
|
||||
if (busy) { log.push({ type: 'click-ignored-busy' }); return; }
|
||||
const snapshot = credsSnapshotKey(ctx.authType, creds);
|
||||
|
||||
// STEP 2: commit if validated matches.
|
||||
if (validated &&
|
||||
validated.hosterName === ctx.hosterName &&
|
||||
validated.authType === ctx.authType &&
|
||||
validated.snapshot === snapshot) {
|
||||
busy = true;
|
||||
try {
|
||||
await persistImpl(ctx, creds);
|
||||
log.push({ type: 'persisted', accountId: ctx.accountId || `${ctx.hosterName}-NEW` });
|
||||
} finally { busy = false; }
|
||||
return;
|
||||
}
|
||||
|
||||
// STEP 1: ephemeral validate.
|
||||
busy = true;
|
||||
let row;
|
||||
try {
|
||||
row = await validateImpl({ hoster: ctx.hosterName, authType: ctx.authType, ...creds, otp });
|
||||
} finally { busy = false; }
|
||||
if (row && (row.status === 'ok' || row.status === 'warn')) {
|
||||
validated = { hosterName: ctx.hosterName, authType: ctx.authType, snapshot, status: row.status };
|
||||
log.push({ type: 'validated', status: row.status });
|
||||
return;
|
||||
}
|
||||
if (row && row.status === 'otp_required') {
|
||||
log.push({ type: 'otp-required' });
|
||||
return;
|
||||
}
|
||||
log.push({ type: 'validation-failed', message: row && row.message });
|
||||
}
|
||||
|
||||
function editField() { validated = null; log.push({ type: 'invalidated-by-edit' }); }
|
||||
return { click, editField, log: () => log.slice(), getValidated: () => validated };
|
||||
}
|
||||
|
||||
// ---- Tests ----
|
||||
|
||||
test('regression (a): failed validation persists NOTHING to config.hosters', async () => {
|
||||
const persistCalls = [];
|
||||
const sm = makeStateMachine({
|
||||
validateImpl: async () => ({ status: 'error', message: 'Falsches Passwort' }),
|
||||
persistImpl: async (ctx, creds) => persistCalls.push({ ctx, creds })
|
||||
});
|
||||
await sm.click({ hosterName: 'doodstream.com', authType: 'login', isEdit: false }, { username: 'u', password: 'wrong' });
|
||||
assert.equal(persistCalls.length, 0, 'no persist should happen on failed validation');
|
||||
assert.equal(sm.getValidated(), null);
|
||||
assert.deepEqual(sm.log().map(e => e.type), ['validation-failed']);
|
||||
});
|
||||
|
||||
test('otp_required returns challenge without committing', async () => {
|
||||
let commits = 0;
|
||||
const validation = { status: 'otp_required', message: 'OTP gesendet' };
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => validation,
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => true
|
||||
test('regression (b): second click with guard set persists exactly ONE entry — no duplication', async () => {
|
||||
const persistCalls = [];
|
||||
let validateCount = 0;
|
||||
const sm = makeStateMachine({
|
||||
validateImpl: async () => { validateCount++; return { status: 'ok' }; },
|
||||
persistImpl: async (ctx, creds) => persistCalls.push({ ctx, creds })
|
||||
});
|
||||
const ctx = { hosterName: 'doodstream.com', authType: 'login', isEdit: false };
|
||||
const creds = { username: 'u', password: 'p' };
|
||||
// Click 1 = validate → green.
|
||||
await sm.click(ctx, creds);
|
||||
// Click 2 = commit (same creds, validated snapshot matches).
|
||||
await sm.click(ctx, creds);
|
||||
// Click 3 = guard prevents a second commit because after persistImpl the
|
||||
// state-machine in real code closes the modal. In this simulator the
|
||||
// validated snapshot is still set — but a real double-click WHILE persistImpl
|
||||
// is in flight would be caught by busy. Simulate that:
|
||||
const sm2 = makeStateMachine({
|
||||
validateImpl: async () => ({ status: 'ok' }),
|
||||
persistImpl: () => new Promise(r => setTimeout(() => { persistCalls.push('slow'); r(); }, 30))
|
||||
});
|
||||
await sm2.click(ctx, creds); // validate
|
||||
const p1 = sm2.click(ctx, creds); // start commit
|
||||
const p2 = sm2.click(ctx, creds); // racing click — must be ignored
|
||||
await Promise.all([p1, p2]);
|
||||
|
||||
assert.equal(result.status, 'otp_required');
|
||||
assert.equal(result.validation, validation);
|
||||
assert.equal(commits, 0);
|
||||
assert.equal(persistCalls.length, 2, 'one persist from the deliberate two-step flow + one from sm2; racing click ignored');
|
||||
assert.equal(validateCount, 1, 'second click reused the validated snapshot — no re-validate');
|
||||
// The racing click MUST have been ignored by the busy guard.
|
||||
assert.ok(sm2.log().some(e => e.type === 'click-ignored-busy'), 'busy guard fired on racing click');
|
||||
});
|
||||
|
||||
test('stale submission is rejected immediately before commit', async () => {
|
||||
let current = true;
|
||||
let commits = 0;
|
||||
const validation = { status: 'ok' };
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => {
|
||||
current = false;
|
||||
return validation;
|
||||
test('regression (c): OTP-required persists NOTHING — and a follow-up click with OTP re-validates ephemerally', async () => {
|
||||
const persistCalls = [];
|
||||
let calls = 0;
|
||||
const sm = makeStateMachine({
|
||||
validateImpl: async (payload) => {
|
||||
calls++;
|
||||
if (!payload.otp) return { status: 'otp_required', message: 'OTP sent' };
|
||||
if (payload.otp === '123456') return { status: 'ok' };
|
||||
return { status: 'error', message: 'Bad OTP' };
|
||||
},
|
||||
commit: async () => {
|
||||
commits++;
|
||||
},
|
||||
isCurrent: () => current
|
||||
persistImpl: async (ctx, creds) => persistCalls.push({ ctx, creds })
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'stale');
|
||||
assert.equal(result.validation, validation);
|
||||
assert.equal(commits, 0);
|
||||
const ctx = { hosterName: 'doodstream.com', authType: 'login', isEdit: false };
|
||||
const creds = { username: 'u', password: 'p' };
|
||||
await sm.click(ctx, creds, ''); // first click → otp_required
|
||||
await sm.click(ctx, creds, '123456'); // retry with otp → ok
|
||||
await sm.click(ctx, creds); // final click → commit
|
||||
assert.equal(persistCalls.length, 1, 'exactly one persist after OTP confirmed');
|
||||
assert.equal(calls, 2, 'validate ran twice (initial + OTP) before commit');
|
||||
assert.deepEqual(
|
||||
sm.log().map(e => e.type),
|
||||
['otp-required', 'validated', 'persisted']
|
||||
);
|
||||
});
|
||||
|
||||
test('save failure returns error after one commit attempt', async () => {
|
||||
const expected = new Error('save failed');
|
||||
let commits = 0;
|
||||
const result = await submitValidatedAccount({
|
||||
validate: async () => ({ status: 'ok' }),
|
||||
commit: async () => {
|
||||
commits++;
|
||||
throw expected;
|
||||
},
|
||||
isCurrent: () => true
|
||||
test('field edit after green check invalidates the snapshot — next click is a re-Prüfen, not a commit', async () => {
|
||||
const persistCalls = [];
|
||||
let validateCount = 0;
|
||||
const sm = makeStateMachine({
|
||||
validateImpl: async () => { validateCount++; return { status: 'ok' }; },
|
||||
persistImpl: async (ctx, creds) => persistCalls.push({ ctx, creds })
|
||||
});
|
||||
|
||||
assert.equal(result.status, 'error');
|
||||
assert.equal(result.error, expected);
|
||||
assert.equal(commits, 1);
|
||||
const ctx = { hosterName: 'doodstream.com', authType: 'login', isEdit: false };
|
||||
await sm.click(ctx, { username: 'u', password: 'p' }); // validate → green
|
||||
sm.editField(); // user edits cred field → snapshot dropped
|
||||
await sm.click(ctx, { username: 'u', password: 'newpw' }); // creds differ → re-validate
|
||||
await sm.click(ctx, { username: 'u', password: 'newpw' }); // now commit the NEW creds
|
||||
assert.equal(persistCalls.length, 1, 'one persist of the new (re-validated) creds');
|
||||
assert.equal(persistCalls[0].creds.password, 'newpw', 'persisted creds match the re-validated set');
|
||||
assert.equal(validateCount, 2, 'second validate was forced by the edit-induced invalidation');
|
||||
});
|
||||
|
||||
test('snapshot key is identical for same creds and DIFFERENT for any cred change (excluding label)', () => {
|
||||
// Label changes must NOT invalidate validation — label is metadata, not a credential.
|
||||
assert.equal(credsSnapshotKey('login', { username: 'u', password: 'p' }),
|
||||
credsSnapshotKey('login', { username: 'u', password: 'p', label: 'XYZ' }));
|
||||
assert.notEqual(credsSnapshotKey('login', { username: 'u', password: 'p' }),
|
||||
credsSnapshotKey('login', { username: 'u', password: 'P' })); // password char-case
|
||||
assert.notEqual(credsSnapshotKey('login', { username: 'u', password: 'p' }),
|
||||
credsSnapshotKey('login', { username: 'U', password: 'p' })); // username diff
|
||||
assert.equal(credsSnapshotKey('api', { apiKey: 'KEY' }),
|
||||
credsSnapshotKey('api', { apiKey: 'KEY', label: 'mein key' }));
|
||||
assert.notEqual(credsSnapshotKey('api', { apiKey: 'KEY' }),
|
||||
credsSnapshotKey('api', { apiKey: 'KEY2' }));
|
||||
});
|
||||
|
||||
test('ephemeral hosterConfig shape matches what per-hoster checkers expect', () => {
|
||||
// The per-hoster checkers in main.js read .username/.password/.apiKey directly.
|
||||
// This guards the validate-credentials IPC contract from drifting.
|
||||
const cfg = buildEphemeralHosterConfig({ hoster: 'doodstream.com', username: 'u', password: 'p' });
|
||||
assert.equal(cfg.username, 'u');
|
||||
assert.equal(cfg.password, 'p');
|
||||
assert.equal(cfg.apiKey, '');
|
||||
assert.equal(cfg.enabled, true);
|
||||
const cfg2 = buildEphemeralHosterConfig({ hoster: 'byse.sx', apiKey: 'K' });
|
||||
assert.equal(cfg2.apiKey, 'K');
|
||||
assert.equal(cfg2.username, '');
|
||||
});
|
||||
|
||||
@ -1,156 +0,0 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert');
|
||||
const { isDiscordWebhook, formatDurationShort, summarizePerHosterFromBatch, buildWebhookRequest, resolveDiscordMention, isAllAborted, clampDiscordContent, DISCORD_CONTENT_LIMIT } = require('../lib/webhook-notify');
|
||||
|
||||
const SAMPLE_SUMMARY = {
|
||||
total: 10,
|
||||
succeeded: 8,
|
||||
failed: 2,
|
||||
files: [
|
||||
{ name: 'a.mkv', results: [
|
||||
{ hoster: 'voe.sx', status: 'done' },
|
||||
{ hoster: 'byse.sx', status: 'error', error: 'x' }
|
||||
] },
|
||||
{ name: 'b.mkv', results: [
|
||||
{ hoster: 'voe.sx', status: 'done' },
|
||||
{ hoster: 'byse.sx', status: 'done' }
|
||||
] }
|
||||
]
|
||||
};
|
||||
|
||||
test('isDiscordWebhook recognizes discord URLs incl. ptb/canary/discordapp', () => {
|
||||
assert.ok(isDiscordWebhook('https://discord.com/api/webhooks/123/abc'));
|
||||
assert.ok(isDiscordWebhook('https://discordapp.com/api/webhooks/123/abc'));
|
||||
assert.ok(isDiscordWebhook('https://ptb.discord.com/api/webhooks/123/abc'));
|
||||
assert.ok(isDiscordWebhook('https://canary.discord.com/api/webhooks/123/abc'));
|
||||
assert.strictEqual(isDiscordWebhook('https://example.com/hook'), false);
|
||||
assert.strictEqual(isDiscordWebhook(''), false);
|
||||
assert.strictEqual(isDiscordWebhook(null), false);
|
||||
});
|
||||
|
||||
test('isDiscordWebhook REJECTS incomplete discord URLs (no id/token)', () => {
|
||||
assert.strictEqual(isDiscordWebhook('https://discord.com/api/webhooks/'), false);
|
||||
assert.strictEqual(isDiscordWebhook('https://discord.com/api/webhooks'), false);
|
||||
assert.strictEqual(isDiscordWebhook('https://discord.com/api/webhooks/123'), false);
|
||||
assert.strictEqual(isDiscordWebhook('https://discord.com/api/webhooks/123/'), false);
|
||||
assert.ok(isDiscordWebhook('https://discord.com/api/webhooks/123456789/aBc-_token123'));
|
||||
});
|
||||
|
||||
test('clampDiscordContent caps to the Discord limit with ellipsis', () => {
|
||||
const short = 'hello';
|
||||
assert.strictEqual(clampDiscordContent(short), short);
|
||||
const long = 'x'.repeat(5000);
|
||||
const clamped = clampDiscordContent(long);
|
||||
assert.ok(clamped.length <= DISCORD_CONTENT_LIMIT);
|
||||
assert.ok(clamped.endsWith('…'));
|
||||
});
|
||||
|
||||
test('buildWebhookRequest: many hosters does not exceed Discord limit', () => {
|
||||
const files = [{ name: 'a.mkv', results: [] }];
|
||||
for (let i = 0; i < 60; i++) files[0].results.push({ hoster: `hoster-with-a-really-long-name-${i}.example.com`, status: 'done' });
|
||||
const req = buildWebhookRequest('https://discord.com/api/webhooks/1/x', { total: 60, succeeded: 60, failed: 0, files }, { durationSec: 60 });
|
||||
const body = JSON.parse(req.body);
|
||||
assert.ok(body.content.length <= DISCORD_CONTENT_LIMIT, `content ${body.content.length} must be <= ${DISCORD_CONTENT_LIMIT}`);
|
||||
assert.match(body.content, /\+\d+/);
|
||||
});
|
||||
|
||||
test('buildWebhookRequest: aborted meta changes the headline', () => {
|
||||
const req = buildWebhookRequest('https://discord.com/api/webhooks/1/x', { total: 5, succeeded: 0, failed: 5, files: [] }, { aborted: true });
|
||||
const body = JSON.parse(req.body);
|
||||
assert.match(body.content, /Batch abgebrochen/);
|
||||
});
|
||||
|
||||
test('isAllAborted: true only when every result is aborted', () => {
|
||||
assert.strictEqual(isAllAborted({ files: [{ results: [{ status: 'aborted' }, { status: 'aborted' }] }] }), true);
|
||||
assert.strictEqual(isAllAborted({ files: [{ results: [{ status: 'aborted' }, { status: 'done' }] }] }), false);
|
||||
assert.strictEqual(isAllAborted({ files: [{ results: [{ status: 'error' }] }] }), false);
|
||||
assert.strictEqual(isAllAborted({ files: [] }), false);
|
||||
assert.strictEqual(isAllAborted(null), false);
|
||||
});
|
||||
|
||||
test('formatDurationShort formats h/m/s tiers', () => {
|
||||
assert.strictEqual(formatDurationShort(45), '45s');
|
||||
assert.strictEqual(formatDurationShort(125), '2m 5s');
|
||||
assert.strictEqual(formatDurationShort(3 * 3600 + 12 * 60), '3h 12m');
|
||||
assert.strictEqual(formatDurationShort(-5), '0s');
|
||||
assert.strictEqual(formatDurationShort(undefined), '0s');
|
||||
});
|
||||
|
||||
test('summarizePerHosterFromBatch counts ok/fail per hoster', () => {
|
||||
const s = summarizePerHosterFromBatch(SAMPLE_SUMMARY);
|
||||
assert.deepStrictEqual(s['voe.sx'], { ok: 2, fail: 0 });
|
||||
assert.deepStrictEqual(s['byse.sx'], { ok: 1, fail: 1 });
|
||||
});
|
||||
|
||||
test('summarizePerHosterFromBatch handles malformed input', () => {
|
||||
assert.deepStrictEqual(summarizePerHosterFromBatch(null), {});
|
||||
assert.deepStrictEqual(summarizePerHosterFromBatch({}), {});
|
||||
assert.deepStrictEqual(summarizePerHosterFromBatch({ files: [{ results: null }] }), {});
|
||||
});
|
||||
|
||||
test('buildWebhookRequest produces Discord content body for discord URLs', () => {
|
||||
const req = buildWebhookRequest('https://discord.com/api/webhooks/1/x', SAMPLE_SUMMARY, { durationSec: 3700, appVersion: '3.3.59', machineName: 'srv-1' });
|
||||
assert.strictEqual(req.method, 'POST');
|
||||
assert.strictEqual(req.headers['Content-Type'], 'application/json');
|
||||
const body = JSON.parse(req.body);
|
||||
assert.ok(typeof body.content === 'string');
|
||||
assert.match(body.content, /Batch fertig/);
|
||||
assert.match(body.content, /srv-1/);
|
||||
assert.match(body.content, /8 ok/);
|
||||
assert.match(body.content, /2 Fehler/);
|
||||
assert.match(body.content, /1h 1m/);
|
||||
assert.match(body.content, /voe\.sx: 2\/2/);
|
||||
});
|
||||
|
||||
test('buildWebhookRequest produces raw JSON payload for generic URLs', () => {
|
||||
const req = buildWebhookRequest('https://example.com/hook', SAMPLE_SUMMARY, { durationSec: 60, appVersion: '3.3.59', timestamp: '2026-06-09T00:00:00Z' });
|
||||
const body = JSON.parse(req.body);
|
||||
assert.strictEqual(body.event, 'batch-done');
|
||||
assert.strictEqual(body.total, 10);
|
||||
assert.strictEqual(body.succeeded, 8);
|
||||
assert.strictEqual(body.failed, 2);
|
||||
assert.strictEqual(body.durationSec, 60);
|
||||
assert.strictEqual(body.version, '3.3.59');
|
||||
assert.deepStrictEqual(body.perHoster['byse.sx'], { ok: 1, fail: 1 });
|
||||
});
|
||||
|
||||
test('resolveDiscordMention: @here / @everyone use parse=everyone', () => {
|
||||
assert.deepStrictEqual(resolveDiscordMention('@here'), { token: '@here', allowed: { parse: ['everyone'] } });
|
||||
assert.deepStrictEqual(resolveDiscordMention('everyone'), { token: '@everyone', allowed: { parse: ['everyone'] } });
|
||||
});
|
||||
|
||||
test('resolveDiscordMention: bare numeric id → user mention', () => {
|
||||
assert.deepStrictEqual(resolveDiscordMention('123456789012345'), { token: '<@123456789012345>', allowed: { users: ['123456789012345'] } });
|
||||
assert.deepStrictEqual(resolveDiscordMention('<@!123456789012345>'), { token: '<@123456789012345>', allowed: { users: ['123456789012345'] } });
|
||||
});
|
||||
|
||||
test('resolveDiscordMention: role:id and <@&id> → role mention', () => {
|
||||
assert.deepStrictEqual(resolveDiscordMention('role:99887766'), { token: '<@&99887766>', allowed: { roles: ['99887766'] } });
|
||||
assert.deepStrictEqual(resolveDiscordMention('<@&99887766>'), { token: '<@&99887766>', allowed: { roles: ['99887766'] } });
|
||||
});
|
||||
|
||||
test('resolveDiscordMention: empty / junk → null', () => {
|
||||
assert.strictEqual(resolveDiscordMention(''), null);
|
||||
assert.strictEqual(resolveDiscordMention(' '), null);
|
||||
assert.strictEqual(resolveDiscordMention('not-an-id'), null);
|
||||
});
|
||||
|
||||
test('buildWebhookRequest: discord with mention prepends token + sets allowed_mentions', () => {
|
||||
const req = buildWebhookRequest('https://discord.com/api/webhooks/1/x', SAMPLE_SUMMARY, { durationSec: 60, mention: '123456789012345' });
|
||||
const body = JSON.parse(req.body);
|
||||
assert.ok(body.content.startsWith('<@123456789012345> '));
|
||||
assert.deepStrictEqual(body.allowed_mentions, { users: ['123456789012345'] });
|
||||
});
|
||||
|
||||
test('buildWebhookRequest: discord without mention blocks all pings (allowed_mentions parse empty)', () => {
|
||||
const req = buildWebhookRequest('https://discord.com/api/webhooks/1/x', SAMPLE_SUMMARY, { durationSec: 60 });
|
||||
const body = JSON.parse(req.body);
|
||||
assert.deepStrictEqual(body.allowed_mentions, { parse: [] });
|
||||
});
|
||||
|
||||
test('buildWebhookRequest tolerates empty summary', () => {
|
||||
const req = buildWebhookRequest('https://example.com/hook', null, {});
|
||||
const body = JSON.parse(req.body);
|
||||
assert.strictEqual(body.total, 0);
|
||||
assert.strictEqual(body.succeeded, 0);
|
||||
});
|
||||
Loading…
Reference in New Issue
Block a user