Kaleido/AGENTS.md
Elijah 7471f7cac2 Redesign browse previews after Silo, with an optional artwork colour wash
Home's hero now compacts into a preview of the focused card instead of
scrolling away, and Movies and TV share the same preview: large title
art, facts, a three-line synopsis, cast and format badges, with the card's
own image scaled whole into the top-right. Artwork crossfades and copy
fades and lifts between titles, driven from graphics layers so no
composition or layout runs per frame. An Interface setting tints the page
with the artwork's colour, baked into the cached backdrop frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 16:23:28 -07:00

106 lines
12 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Kaleido
## Working rules
- Ask clarifying questions before proceeding when more information is needed. Otherwise make reasonable decisions and complete the authorized work.
- This is a real native Android TV Plex client: Kotlin, Jetpack Compose, Media3/ExoPlayer. Preserve working Plex functionality; do not substitute mock content or prototype playback.
- The primary test device is the onn at **192.168.50.195:5555**, package **tv.kaleido.app**, activity **.MainActivity**. The user also authorized two weaker devices for performance testing: a Fire TV Stick 4K (Fire OS 8, 32-bit) at **192.168.50.146:5555** and a TCL Smart TV Pro (MT5896, slowest GPU) at **192.168.50.200:5555**. Scripts target the onn unless `KALEIDO_DEVICE` names another serial. Always specify the ADB serial. Do not modify any device's launcher, accessibility services (the onn runs two, which add per-frame semantics work), settings, or other apps. The TCL's network ADB link drops intermittently; `device.py` reconnects and retries.
- `.env` contains `PLEX_SERVER_URL` and `PLEX_TOKEN` for local testing. Never print, commit, or embed credentials in source, resources, APKs, or logs. Keep `.env`, signing keys, toolchains, and private evidence ignored.
- Use the existing encrypted session. `scripts/device.py seed` provisions an installed **debug** build only when needed; the app encrypts and deletes its bootstrap file. Never clear app data or sign out just to reset a test.
## UI and architecture
- **Home:** large cinematic hero with playback/information actions and integrated landscape Continue Watching. Entering the rows compacts it into a Silo-style preview of the focused card (its own image, top-right); Up/Back restores the hero.
- **Movies:** the same Silo-style preview follows focused posters; recent releases and movie recommendations lead. **TV:** episode-first layout with wider next-episode thumbnails, new seasons and new episodes. Keep these destinations' rows visibly distinct. The optional artwork colour wash is baked into the preview backdrop bitmap.
- Both sections retain a dedicated, filtered/sorted **A–Z poster grid**. Primary tabs activate on focus. Use neutral near-black chrome, artwork-led color, restrained controls and stable card bounds. Inspect actual TV screenshots and D-pad behavior after UI changes.
- `KaleidoViewModel.kt`: navigation, async loading and section caches. `PlexApi.kt` / `Models.kt`: Plex transport and parsing. `FocusedBrowse.kt`, `BrowseBackdrop.kt`, `BrowseScreens.kt`, `Theme.kt`: browsing/focus/design. `Playback.kt`: controller, negotiation and progress. `PlayerControls.kt` / `PlaybackExtras.kt`: controls, options and post-play. `SecureSession.kt`: Android Keystore persistence. `AppUpdater.kt`: repository updates.
## Build and device setup
Run from the repository root in PowerShell. `Build.ps1` configures the local JDK 17, Gradle 8.13 and SDK under `.tools/`; do not replace the toolchain unnecessarily. Builds/ADB may require sandbox escalation.
```powershell
$KaleidoAdb = 'C:\Users\Elijah\Desktop\platform-tools\adb.exe'
$KaleidoDevice = '192.168.50.195:5555'
$KaleidoPython = 'C:\Users\Elijah\.cache\codex-runtimes\codex-primary-runtime\dependencies\python\python.exe'
$env:PYTHONIOENCODING = 'utf-8'
& $KaleidoAdb -s $KaleidoDevice get-state
.\Build.ps1 -Test # Debug APK, unit tests, lint
.\Build.ps1 -Release -Test # Optimized release APK, unit tests, lint
& $KaleidoAdb -s $KaleidoDevice install -r app/build/outputs/apk/debug/app-debug.apk
```
Install `app/build/outputs/apk/release/app-release.apk` instead for performance/final visual review. Debug and release share the private local signing key, so `install -r` preserves data.
## Testing suite
Use `& $KaleidoPython` before the Python commands below. Run only checks relevant to the change; **never run competing device/UI tests in parallel**. Live playback tests change real Plex progress and can change playback preferences. Restore a useful final state, normally the release build on Home. After a functional debug batch, restore the signed release and its ART profile before returning the device to normal use; never judge performance from an unoptimized debug build.
| Command | Purpose / required build |
| --- | --- |
| `scripts/verify_ui.py browse` | Home/Movies/TV, A–Z, focus restoration, live Plex PIN and cancellation; debug or release |
| `scripts/verify_ui.py player` | Actual playback, scrub preview, track/options menus, speed/mode, hide/show/exit; **debug** |
| `scripts/repro_runtime.py sections` | Cold-start tab-switch blank-page regression; debug or release |
| `scripts/repro_runtime.py browse` | Horizontal bounds and frame pacing; **optimized release with profile compilation** |
| `scripts/repro_runtime.py controls` | One-press Back, OK reveal, timed hide and exit; **debug** |
| `scripts/repro_runtime.py episode` | Next-episode replacement/crash regression; **debug** |
| `scripts/repro_runtime.py postplay` | Natural ending → five-second countdown → next episode READY; **debug** |
| `scripts/repro_runtime.py credits` | Visible Skip credits, season/related rows and cancellable countdown; **debug** |
| `scripts/device.py screenshot <name>` | Capture to ignored `evidence/private/`; also supports `hierarchy`, `logs`, `launch`, `key` |
| `scripts/profile_browse.py` | Targeted atrace investigation when navigation performance regresses |
| `scripts/profile_transitions.py <label>` | Skipped frames per D-pad transition (tabs, hero, rows, Movies) from framestats; first cycle is the first pass after launch; profiled **release**; raw frames kept for `resummarize` |
| `scripts/trace_transition.py <label> <setup> <keys>` | atrace one transition and list slow UI frames with their nested work |
| `scripts/profile_home.py <label>` | Hero → Continue Watching frame timing; profiled **release**; `--cold --cycles 1` checks the first transition |
| `scripts/verify_navigation_sounds.py` | Sound toggle and cold-start persistence; leaves sounds On |
| `scripts/verify_details.py` | Cast/actor navigation, related shows and watched sync; temporarily marks validated unwatched movie/show fixtures, then restores them |
| `scripts/verify_details_layout.py` | Fresh TV show header bounds and Seasons before Cast; debug or release |
| `scripts/verify_movie_details.py` | Movie safe header, related movies and Back focus restoration; debug or release |
| `scripts/verify_settings.py` | Category navigation, preference-dialog cancellation and live update check; expects Original quality / 80% test preferences |
| `scripts/verify_library_jump.py` | Scoped Movie/TV categories, A–Z jumps beyond the first page, details/Back focus and exact Silo search |
| `scripts/verify_audio_preferences.py` | Theme/boost persistence and actual boosted playback; **debug**; restores prior audio preferences |
Live Android integration tests (currently six) cover secure persistence, library APIs, episodes, PIN, preview JPEGs/new seasons and supported HEVC remux negotiation:
```powershell
.\Build.ps1 -Test
& .tools/gradle-8.13/bin/gradle.bat --no-daemon assembleDebugAndroidTest
& $KaleidoAdb -s $KaleidoDevice install -r app/build/outputs/apk/debug/app-debug.apk
& $KaleidoAdb -s $KaleidoDevice install -r app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
& $KaleidoAdb -s $KaleidoDevice shell am instrument -w tv.kaleido.app.test/androidx.test.runner.AndroidJUnitRunner
```
These require the private test session. Playback test intents (`testTitleId`, `testOffset`, `testMode`) are **ignored in release builds**. Fixture IDs are server-specific: Silo show `6115`, episodes `7599` → `8594`; movie `9776` is 2 Fast 2 Furious. Revalidate fixtures if the server changes. UI tests also depend on actual hub contents and English labels; diagnose missing fixtures separately from app failures.
## Important regression traps
- Keep initialization independent of cancellable tab requests and apply section results atomically. Cancelling initialization during a startup tab switch previously left Movies/TV blank.
- Foreground library sync polls revisions every 15 seconds with a 60-second refresh fallback; pause it for playback/background and stage updates before applying them. Preserve stable hub/card focus IDs and current library filters/range. `LibrarySyncIntegrationTest` tests the real timer against an in-memory HTTP fixture without changing Plex or the encrypted session.
- Preserve explicit D-pad row/column targets and fixed card geometry. Avoid expensive beyond-bounds focus searches, per-focus catalog recomposition and full-screen gradient overdraw. Contrast is baked into cached backdrop bitmaps off the main thread.
- Give every Media3 session a unique ID: next-episode creation can overlap the old controller's disposal. Plex HLS position is absolute; **do not add the requested resume offset again**.
- Handle remote Back explicitly before default focus handling. Wait for menu focus restoration in tests instead of sending blind rapid button sequences.
- A text field keeps the D-pad: route Down/IME actions out of Search's field explicitly. Give every `focusRestorer()` a fallback, or it overrides explicit focus targets. Per-card modifiers on browse rows must be `Modifier.Node`s, not `composed {}`, and must not animate per card.
- Playback has one quality preference (`StreamQuality`): **Original quality** (the default) tries the original file, then Plex HLS, then a 1080p conversion if needed; 4K/1080p/720p/480p ask Plex to convert. A nonempty audio-only track list for a declared video must trigger fallback. Earlier preferences migrate on first read. Debug `testMode` still forces a delivery mode. Augment Plex's existing HEVC target; competing targets previously forced unnecessary conversion. A server copy decision does not prove rendering. Preserve the selected Plex tracks and check actual delivered formats.
- **ADB screenshots can omit HEVC video on this onn while the actual TV displays it. The user confirmed this.** Do not force transcoding based solely on a blank capture. `scripts/check_video.py` has a pixel assertion that can falsely fail for this reason; use playback logs/rendered-frame evidence and physical TV confirmation as appropriate. Never infer universal HDR/codec support from one test.
- Preview thumbnails and intro/credit markers depend on server-generated data. Do not fabricate them.
- Theme tracks come from Plex metadata and inherit from the show for seasons/episodes. Stop music on video playback/background. Volume boost processes decoded PCM; Off must preserve passthrough and the normal audio path.
### Performance testing
Fresh ADB installs may remain in ART `status=verify`, making Compose much slower. Compare matching compilation states. After installing the **release** APK, prepare only Kaleido:
```powershell
& $KaleidoAdb -s $KaleidoDevice shell am broadcast -a androidx.profileinstaller.action.INSTALL_PROFILE -n tv.kaleido.app/androidx.profileinstaller.ProfileInstallReceiver
& $KaleidoAdb -s $KaleidoDevice shell cmd package compile -f -m speed-profile tv.kaleido.app
& $KaleidoPython scripts/repro_runtime.py browse
```
The harness requires horizontal Y drift ≤3 px and frame-time p95 <40 ms. On 2026-10-23, the final release measured median 15 ms / p95 27 ms after profile compilation; some frames still missed 60 Hz. This is historical evidence, not a current guarantee. Baseline rules live in `app/src/main/baseline-prof.txt`; ordinary sideloads also depend on Android's background profile optimization.
## Releases and further detail
Use Semantic Versioning (`major.minor.patch`) for release names.
Updates come from `https://git.elijahkuntz.com/Elijah/Kaleido`. Keep the private `.tools/kaleido-signing.jks`; replacing it breaks in-place updates. `version.properties` is the version source. Before a requested release, increment version name/code, update `CHANGELOG.md`, build/test, and use `scripts/package_release.py`; the Forgejo workflow publishes matching version tags. Do not claim publication or updater installation was tested without verifying it.
See `docs/design-system.md` for the visual system and `docs/runtime-verification.md` for device evidence and limitations. Keep this file concise; put detailed investigation history there.