Kaleido-Desktop/docs/architecture.md
Elijah 8ed44c2fe4
All checks were successful
Validate Kaleido Desktop / validate (push) Successful in 1m22s
Release Kaleido Desktop 0.1.0-beta.12
On Windows the miniplayer is now a window of its own (mini.rs, mini.html):
a borderless window owned by the main one that the video moves into
(VideoSurface::set_host swaps composition targets in one commit). Dragging
moves the whole window, so video and controls move together with no edge
flicker; release glides it to the nearest corner or the one it's flicked
towards, remembered, and it follows the main window. Its page mirrors the
player over events (mini:state / mini:action). Shown and hidden natively
without activation, so the main window keeps focus. macOS keeps the
in-page miniplayer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:08 -07:00

111 lines
8.1 KiB
Markdown

# Architecture
Kaleido Desktop is a [Tauri 2](https://tauri.app) application: a Rust core hosts the
system web view (WebView2 on Windows, WKWebView on macOS, WebKitGTK on Linux), which runs
the React/TypeScript interface. Video is decoded and rendered by libmpv and composed
**beneath** the transparent web view, so the player's controls are ordinary interface
elements drawn over the video — the same arrangement Plex HTPC and Jellyfin Media
Player use.
```
src/ Shared interface (all platforms)
app/ Session, router, settings, queries, actions, shell, updater
plex/ Plex client: account (plex.tv), library API, admin, playback, parsing
player/ Playback controller (store.ts), mpv bridge, player overlay
downloads/ Download catalog and actions
screens/ Home, Movies/TV (Discover/Library/Categories), title pages,
search, people, downloads, settings, dialogs
ui/ Design system: primitives, menus/dialogs/toasts, cards, hero
styles/ Tokens and styles (base, ui, app, screens, player)
platform/native.ts The only module that talks to the host (Tauri or browser)
src-tauri/ Shared Rust core
src/mpv.rs Dynamically loaded libmpv binding (node ↔ JSON)
src/player.rs One long-lived mpv instance, event stream to the interface
src/downloads.rs Resumable downloads with a JSON catalog
src/system.rs Credentials (OS keychain), paths, disk space, file saving
src/platform/ Everything OS-specific (see below)
tauri.conf.json Shared configuration (+ tauri.<os>.conf.json overrides)
vendor/mpv/<os>/ libmpv per platform (fetched by scripts, ignored by git)
scripts/ build, libmpv fetch, update-feed generation
updates/latest.json Signed update feed for every platform
```
## Platform boundary
`src-tauri/src/platform/mod.rs` selects one of `windows.rs`, `macos.rs` or `linux.rs`.
Each exposes the same items:
| Item | Windows | macOS |
| --- | --- | --- |
| `VideoSurface` | mpv renders with `d3d11-output-mode=composition`; its swapchain (`display-swapchain`) is set as the content of a DirectComposition visual targeted at the window with `topmost = false`, i.e. behind the web view's child windows. For the miniplayer the visual is offset and mpv's composition size shrunk to a corner rectangle (`player_rect`). | mpv's render API (`vo=libmpv`, OpenGL) draws into a `CAOpenGLLayer` hosted by a view inserted beneath the WKWebView, whose background is turned off (`drawsBackground`). `start` creates the render context once mpv is initialized; a render thread redraws the layer whenever mpv reports a new frame. For the miniplayer the view is moved to the corner rectangle. |
| `MPV_LIBRARIES` | `libmpv-2.dll` | `libmpv.2.dylib` (in `Contents/Frameworks` with the libraries it links) |
| `hold_awake` | `SetThreadExecutionState` | an IOKit `PreventUserIdleDisplaySleep` assertion |
| Window buttons | drawn by the page (`WindowControls`) | the native traffic lights (`titleBarStyle: Overlay`, placed on the top bar's centre line); `set_window_buttons` hides them with the player's controls |
| `device_name` | `COMPUTERNAME` | the computer name from Sharing settings |
Linux provides stubs for `VideoSurface` (browsing, downloads, settings and updates
work; playback reports that it is unavailable); the macOS approach (render API beneath
the web view) applies there with an OpenGL surface under WebKitGTK. Per-OS bundling
(libmpv, targets, window style) lives in `tauri.<os>.conf.json`. Credentials use the
platform's keychain through the `keyring` crate with per-OS features in `Cargo.toml`;
macOS debug builds keep them in a private file instead, because each unsigned rebuild
would make the Keychain ask again.
The page background turns transparent only while video is visible
(`html.video-active`), so browsing is opaque. The page knows its OS synchronously
(`hostOS` in `platform/native.ts`, from the web view's user agent) and tags the root
element (`html.os-macos`) for the few styles that differ, such as room for the traffic
lights.
HDR passthrough (`target-colorspace-hint`) works differently on each OS. On Windows mpv
switches its swapchain's colour space itself. On macOS the surface watches
`video-params/gamma`: for PQ or HLG video it has mpv output PQ in BT.2020
(`target-trc`, `target-prim`) into the 16-bit float framebuffer and turns on the layer's
extended dynamic range with the BT.2100 PQ colour space, so macOS maps the picture
into the display's HDR headroom; SDR video returns both to their defaults.
## Interface
- **State**: zustand stores for the session (`app/session.ts`), navigation history
(`app/router.ts`, with scroll and per-screen state restored on Back), preferences
(`app/settings.ts`, persisted) and the player (`player/store.ts`). Server data goes
through TanStack Query; every key starts with the server's machine identifier.
- **Plex HTTP** goes through the host's native HTTP client (`@tauri-apps/plugin-http`),
avoiding CORS and local-network restrictions. Artwork uses Plex's photo transcoder,
sized for the display's pixel ratio (PNG for transparent clear logos).
- **Library sync**: the section list is polled every 20 seconds; a change refreshes
browsing caches.
- **Playback** (`player/store.ts`): Original quality plays the part URL directly (multi
part titles are joined with an mpv EDL); other qualities request Plex HLS with
timestamps kept absolute. If direct play fails, playback retries through Plex once.
Plex streams are matched to mpv tracks by stream index; external subtitles are added
on demand. Timeline reports every 10 seconds (and when playback stalls) make the session appear in
Plex; a reply carrying a termination (the server owner stopped the stream) stops
playback and shows their message. The title is marked watched at the configured percentage. The player's back arrow (or Esc)
shrinks playback into a miniplayer. On Windows it is a window of its own
(`src-tauri/src/mini.rs`, page `mini.html` → `src/mini/`): a small borderless window
owned by the main one, which the video moves into (`VideoSurface::set_host`, one
composition target per window, swapped in a single commit). Dragging moves that whole
window, so video and controls never drift apart; letting go glides it to a corner of
the main window (or the one it's flicked towards), and it follows the main window as
that moves or resizes. Its page mirrors the player from the main window and sends its
buttons back (`player/miniWindow.ts`, events `mini:state` / `mini:action`). The video
moves into it once the main page is opaque again, and back before the page turns
transparent, so neither window flashes. On macOS the miniplayer stays inside the page:
the video is drawn in a corner rectangle with a matching hole masked out of the page
(`html.player-mini`).
- **Upscaling** (`player/enhance.ts`): Performance and Balanced set mpv's scalers and
debanding and load shaders bundled from `src-tauri/shaders/` (FSRCNNX 16, nlmeans,
KrigBilateral, AMD CAS); each shader's `WHEN` condition decides whether it runs for the
current video and size. Quality is NVIDIA RTX Video Super Resolution through mpv's
`d3d11vpp` filter (`scaling-mode=nvidia`, labelled `@kaleido-vsr`), sized to the window
and re-applied when that changes; it is offered only on Windows with an NVIDIA RTX GPU
(`app_info` lists the adapters), runs Balanced for HDR or at native size, and the
driver's own setting picks its quality level. The chosen tier and the running one are
tracked separately: shader or RTX Video errors in mpv's log, missing network passes in
`vo-passes`, or RTX Video output that isn't enlarged drop to the tier below. Compiled shaders are
cached in the app cache folder (`gpu-shader-cache-dir`), so a slow first Direct3D
compile happens once per shader set, mpv version and GPU driver.
- **Updates**: `app/updater.ts` checks the signed feed on start; installs need the
user's confirmation and then relaunch. One release carries every platform's build and
the feed lists each of them (`windows-x86_64`, `darwin-aarch64`).