Kaleido-Desktop/docs/architecture.md
Elijah Kuntz 7d0803481b Port Kaleido Desktop to macOS
macOS (Apple silicon) now plays, browses and updates like Windows:

- Video: libmpv's render API draws into a CAOpenGLLayer beneath the
  WKWebView, with the miniplayer rectangle, HDR passthrough through
  extended dynamic range (PQ/BT.2020) and display-sleep prevention.
- Window: native traffic lights on the top bar's centre line, hidden
  with the player's controls; the page's caption buttons are Windows-only.
- Interface: macOS shortcut labels (⌘K, ⌘[, ⌃⌘F), Plex identifies the
  player as a Mac, and header values are kept ASCII (Mac computer names
  usually contain a curly apostrophe).
- The window appears without waiting for an animation frame, which
  WKWebView doesn't run in a hidden window.
- Credentials: release builds use the Keychain and are signed with a
  lasting self-signed certificate so access survives updates; debug
  builds keep tokens in a private file to avoid prompts on every rebuild.
- Bundling: scripts/fetch-libmpv.sh makes Homebrew's libmpv and its
  libraries self-contained; scripts/build.sh builds the .app, .dmg and
  signed update archive.

Releases now carry both platforms: publish-release.mjs uploads each
machine's build into one release, and update-manifest.mjs writes the
feed from that release's assets, refusing while a platform is missing.

The updater uses a new signing key (public key in tauri.conf.json), so
existing installs must be reinstalled once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 15:14:40 -07:00

7.4 KiB

Architecture

Kaleido Desktop is a Tauri 2 application: a Rust core hosts the system web view (WebView2 on Windows, WKWebView on macOS, WebKitGTK on Linux), which runs the React/TypeScript interface. Video is decoded and rendered by libmpv and composed beneath the transparent web view, so the player's controls are ordinary interface elements drawn over the video — the same arrangement Plex HTPC and Jellyfin Media Player use.

src/                      Shared interface (all platforms)
  app/                    Session, router, settings, queries, actions, shell, updater
  plex/                   Plex client: account (plex.tv), library API, admin, playback, parsing
  player/                 Playback controller (store.ts), mpv bridge, player overlay
  downloads/              Download catalog and actions
  screens/                Home, Movies/TV (Discover/Library/Categories), title pages,
                          search, people, downloads, settings, dialogs
  ui/                     Design system: primitives, menus/dialogs/toasts, cards, hero
  styles/                 Tokens and styles (base, ui, app, screens, player)
  platform/native.ts      The only module that talks to the host (Tauri or browser)
src-tauri/                Shared Rust core
  src/mpv.rs              Dynamically loaded libmpv binding (node ↔ JSON)
  src/player.rs           One long-lived mpv instance, event stream to the interface
  src/downloads.rs        Resumable downloads with a JSON catalog
  src/system.rs           Credentials (OS keychain), paths, disk space, file saving
  src/platform/           Everything OS-specific (see below)
  tauri.conf.json         Shared configuration (+ tauri.<os>.conf.json overrides)
vendor/mpv/<os>/          libmpv per platform (fetched by scripts, ignored by git)
scripts/                  build, libmpv fetch, update-feed generation
updates/latest.json       Signed update feed for every platform

Platform boundary

src-tauri/src/platform/mod.rs selects one of windows.rs, macos.rs or linux.rs. Each exposes the same items:

Item Windows macOS
VideoSurface mpv renders with d3d11-output-mode=composition; its swapchain (display-swapchain) is set as the content of a DirectComposition visual targeted at the window with topmost = false, i.e. behind the web view's child windows. For the miniplayer the visual is offset and mpv's composition size shrunk to a corner rectangle (player_rect). mpv's render API (vo=libmpv, OpenGL) draws into a CAOpenGLLayer hosted by a view inserted beneath the WKWebView, whose background is turned off (drawsBackground). start creates the render context once mpv is initialized; a render thread redraws the layer whenever mpv reports a new frame. For the miniplayer the view is moved to the corner rectangle.
MPV_LIBRARIES libmpv-2.dll libmpv.2.dylib (in Contents/Frameworks with the libraries it links)
hold_awake SetThreadExecutionState an IOKit PreventUserIdleDisplaySleep assertion
Window buttons drawn by the page (WindowControls) the native traffic lights (titleBarStyle: Overlay, placed on the top bar's centre line); set_window_buttons hides them with the player's controls
device_name COMPUTERNAME the computer name from Sharing settings

Linux provides stubs for VideoSurface (browsing, downloads, settings and updates work; playback reports that it is unavailable); the macOS approach (render API beneath the web view) applies there with an OpenGL surface under WebKitGTK. Per-OS bundling (libmpv, targets, window style) lives in tauri.<os>.conf.json. Credentials use the platform's keychain through the keyring crate with per-OS features in Cargo.toml; macOS debug builds keep them in a private file instead, because each unsigned rebuild would make the Keychain ask again.

The page background turns transparent only while video is visible (html.video-active), so browsing is opaque. The page knows its OS synchronously (hostOS in platform/native.ts, from the web view's user agent) and tags the root element (html.os-macos) for the few styles that differ, such as room for the traffic lights.

HDR passthrough (target-colorspace-hint) works differently on each OS. On Windows mpv switches its swapchain's colour space itself. On macOS the surface watches video-params/gamma: for PQ or HLG video it has mpv output PQ in BT.2020 (target-trc, target-prim) into the 16-bit float framebuffer and turns on the layer's extended dynamic range with the BT.2100 PQ colour space, so macOS maps the picture into the display's HDR headroom; SDR video returns both to their defaults.

Interface

  • State: zustand stores for the session (app/session.ts), navigation history (app/router.ts, with scroll and per-screen state restored on Back), preferences (app/settings.ts, persisted) and the player (player/store.ts). Server data goes through TanStack Query; every key starts with the server's machine identifier.
  • Plex HTTP goes through the host's native HTTP client (@tauri-apps/plugin-http), avoiding CORS and local-network restrictions. Artwork uses Plex's photo transcoder, sized for the display's pixel ratio (PNG for transparent clear logos).
  • Library sync: the section list is polled every 20 seconds; a change refreshes browsing caches.
  • Playback (player/store.ts): Original quality plays the part URL directly (multi part titles are joined with an mpv EDL); other qualities request Plex HLS with timestamps kept absolute. If direct play fails, playback retries through Plex once. Plex streams are matched to mpv tracks by stream index; external subtitles are added on demand. Timeline reports every 10 seconds (and when playback stalls) make the session appear in Plex; a reply carrying a termination (the server owner stopped the stream) stops playback and shows their message. The title is marked watched at the configured percentage. The player's back arrow (or Esc) shrinks playback into a miniplayer: the video is drawn in a corner rectangle and the interface stays visible around it, with a matching hole masked out of the page (html.player-mini).
  • Upscaling (player/enhance.ts): Performance and Balanced set mpv's scalers and debanding and load shaders bundled from src-tauri/shaders/ (FSRCNNX 16, nlmeans, KrigBilateral, AMD CAS); each shader's WHEN condition decides whether it runs for the current video and size. Quality is NVIDIA RTX Video Super Resolution through mpv's d3d11vpp filter (scaling-mode=nvidia, labelled @kaleido-vsr), sized to the window and re-applied when that changes; it is offered only on Windows with an NVIDIA RTX GPU (app_info lists the adapters), runs Balanced for HDR or at native size, and the driver's own setting picks its quality level. The chosen tier and the running one are tracked separately: shader or RTX Video errors in mpv's log, missing network passes in vo-passes, or RTX Video output that isn't enlarged drop to the tier below. Compiled shaders are cached in the app cache folder (gpu-shader-cache-dir), so a slow first Direct3D compile happens once per shader set, mpv version and GPU driver.
  • Updates: app/updater.ts checks the signed feed on start; installs need the user's confirmation and then relaunch. One release carries every platform's build and the feed lists each of them (windows-x86_64, darwin-aarch64).