Kaleido-Desktop/docs/architecture.md
Elijah 9fd2b0603e Add a Quality upscaling tier and make enhancement verifiable
Quality runs ArtCNN C4F32 (its denoising model when Denoising is on),
KrigBilateral, debanding and CAS. Its network runs at any enlargement,
so 1080p in a maximized 1440p window is doubled and scaled down; Balanced
keeps FSRCNNX's 1.3x threshold.

The enhancement pipeline moves to player/enhance.ts. Mode changes are
serialized (latest wins). Shader errors in mpv's log, or network passes
missing from vo-passes, drop to the tier below with a notice, and the
chosen and running tiers are shown separately. HDR is judged from the
decoded video. Debanding, sharpening and denoising have their own
toggles, B compares with enhancement off, and the panel shows the
renderer's actual stages, GPU time and dropped frames.

Compiled shaders persist in the app cache (gpu-shader-cache-dir):
ArtCNN's first Direct3D compile takes about a minute, which the player
now announces, and later launches start immediately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 19:37:32 -07:00

82 lines
5.3 KiB
Markdown

# Architecture
Kaleido Desktop is a [Tauri 2](https://tauri.app) application: a Rust core hosts the
system web view (WebView2 on Windows, WKWebView on macOS, WebKitGTK on Linux), which runs
the React/TypeScript interface. Video is decoded and rendered by libmpv and composed
**beneath** the transparent web view, so the player's controls are ordinary interface
elements drawn over the video — the same arrangement Plex HTPC and Jellyfin Media
Player use.
```
src/ Shared interface (all platforms)
app/ Session, router, settings, queries, actions, shell, updater
plex/ Plex client: account (plex.tv), library API, admin, playback, parsing
player/ Playback controller (store.ts), mpv bridge, player overlay
downloads/ Download catalog and actions
screens/ Home, Movies/TV (Discover/Library/Categories), title pages,
search, people, downloads, settings, dialogs
ui/ Design system: primitives, menus/dialogs/toasts, cards, hero
styles/ Tokens and styles (base, ui, app, screens, player)
platform/native.ts The only module that talks to the host (Tauri or browser)
src-tauri/ Shared Rust core
src/mpv.rs Dynamically loaded libmpv binding (node ↔ JSON)
src/player.rs One long-lived mpv instance, event stream to the interface
src/downloads.rs Resumable downloads with a JSON catalog
src/system.rs Credentials (OS keychain), paths, disk space, file saving
src/platform/ Everything OS-specific (see below)
tauri.conf.json Shared configuration (+ tauri.<os>.conf.json overrides)
vendor/mpv/<os>/ libmpv per platform (fetched by scripts, ignored by git)
scripts/ build, libmpv fetch, update-feed generation
updates/latest.json Signed update feed for every platform
```
## Platform boundary
`src-tauri/src/platform/mod.rs` selects one of `windows.rs`, `macos.rs` or `linux.rs`.
Each exposes the same items:
| Item | Windows implementation |
| --- | --- |
| `VideoSurface` | mpv renders with `d3d11-output-mode=composition`; its swapchain (`display-swapchain`) is set as the content of a DirectComposition visual targeted at the window with `topmost = false`, i.e. behind the web view's child windows. For the miniplayer the visual is offset and mpv's composition size shrunk to a corner rectangle (`player_rect`). |
| `MPV_LIBRARIES` | `libmpv-2.dll` |
| `hold_awake` | `SetThreadExecutionState` while video plays |
macOS and Linux currently provide stubs for `VideoSurface` (browsing, downloads,
settings and updates work; playback reports that it is unavailable). To add playback
there, render through mpv's render API into a layer beneath the web view and implement
the same methods (`set_rect` places the video in part of the window for the miniplayer). Per-OS bundling (libmpv, targets, dependencies) lives in
`tauri.macos.conf.json` and `tauri.linux.conf.json`. Credentials use the platform's
keychain through the `keyring` crate with per-OS features in `Cargo.toml`.
On Windows the main window is created transparent and the page background turns
transparent only while video is visible (`html.video-active`), so browsing is opaque.
## Interface
- **State**: zustand stores for the session (`app/session.ts`), navigation history
(`app/router.ts`, with scroll and per-screen state restored on Back), preferences
(`app/settings.ts`, persisted) and the player (`player/store.ts`). Server data goes
through TanStack Query; every key starts with the server's machine identifier.
- **Plex HTTP** goes through the host's native HTTP client (`@tauri-apps/plugin-http`),
avoiding CORS and local-network restrictions. Artwork uses Plex's photo transcoder,
sized for the display's pixel ratio (PNG for transparent clear logos).
- **Library sync**: the section list is polled every 20 seconds; a change refreshes
browsing caches.
- **Playback** (`player/store.ts`): Original quality plays the part URL directly (multi
part titles are joined with an mpv EDL); other qualities request Plex HLS with
timestamps kept absolute. If direct play fails, playback retries through Plex once.
Plex streams are matched to mpv tracks by stream index; external subtitles are added
on demand. Timeline reports every 10 seconds make the session appear in Plex, and the
title is marked watched at the configured percentage. The player's back arrow (or Esc)
shrinks playback into a miniplayer: the video is drawn in a corner rectangle and the
interface stays visible around it, with a matching hole masked out of the page
(`html.player-mini`).
- **Upscaling** (`player/enhance.ts`): tiers set mpv's scalers and debanding and load
shaders bundled from `src-tauri/shaders/` (FSRCNNX, ArtCNN, KrigBilateral, AMD CAS);
each shader's `WHEN` condition decides whether it runs for the current video and size.
The chosen tier and the running one are tracked separately: shader errors in mpv's log
or missing network passes in `vo-passes` drop to the tier below. Compiled shaders are
cached in the app cache folder (`gpu-shader-cache-dir`); ArtCNN's first Direct3D
compile takes about a minute.
- **Updates**: `app/updater.ts` checks the signed feed on start; installs need the
user's confirmation and then relaunch.