Luma/docs/VALIDATION.md
2026-10-18 09:10:02 -07:00

8.9 KiB
Raw Blame History

Device validation

Validated on the user’s onn. 4K Plus Streaming device over ADB, October 17, 2026 (America/Phoenix). Android 14 / API 34, approximately 1.88 GiB physical RAM, 1920 × 1080 render override on a 4K display.

Functional checks

  • Native app discovery, D-pad focus across rows, OK launch, long-press management menu, and settings navigation.
  • Artwork only, Dynamic, and Minimal banner modes; original spacious layout restored as the default. Focused-only labels and hidden app-section text are the defaults.
  • Smooth focus scrolling across all app rows, including the All apps footer, and back to the first row.
  • Plex connection, bounded rows, artwork, encrypted token storage, and disconnect verified against a temporary local fixture. Fixture credentials were removed. A real Plex account/server has not been connected; actual Plex authorization and title playback remain unverified.
  • Art screensaver: all three bundled paintings decoded and displayed, sampled for nonblank pixels, with pure-black side mattes. Captions have a dark gradient and text shadows. Paintings retain their full aspect ratio in Auto/Fit.
  • Scenic photo preview, single-button wake, and real one-minute idle activation passed. Idle timeout was restored to ten minutes, photo duration to thirty seconds, and mode to Scenic.
  • NPS bird video downloaded to private device storage (about 40 MB), played locally in a loop, and stopped on leaving Luma. The local file remains available in Nature video mode; no ongoing stream/download is active.
  • Home routing: Android reports Luma as HOME role holder but the OEM resolver still chooses Google Home. The enabled Luma accessibility service routes Home to Luma. Repeated Home presses, return from Android settings and SmartTube, and screensaver dismissal were checked.
  • Projectivy fallback stays open when explicitly selected through its app tile or Settings. Back stays on Luma; from lower rows it returns to the first row. Home returns to Luma. Projectivy remains installed and enabled; only its competing “Override current launcher” preference is off. Its accessibility service and Button Mapper’s service are retained.
  • Luma’s override Off control removes only its own accessibility service. Re-enabling through Android’s actual Accessibility settings was verified. The service is system-bound in a separate tv.luma.launcher:home process. No view-content retrieval capability is requested.

Wake regression — October 18, 2026

The original blank state was captured before sending input or restarting Luma. Android reported the existing activity as resumed; tiles were displaying their text placeholders and the alpine bitmap was absent. The recorded wake sequence included start/resume, a brief pause, and then resume without another start. Image work scheduled only from onStart() could be skipped or discarded while paused, with no retry when the activity resumed.

Version 1.0.1 moves restoration to onResume(), retries interrupted app discovery, and reloads missing wallpaper, app icons, and optional media artwork. Pausing invalidates stale asynchronous results; stopping still releases image references and caches. A pause-only resume retains existing tile views and decoded app images. This follows Android's documented pause/resume lifecycle.

A separate same-signature instrumentation APK reproduces the interrupted-load sequence deterministically. The installed 1.0.0 build failed with missing artwork; the fixed build passed the same case, recovery of a partially rendered grid whose icon jobs were interrupted, pause-only memory trimming, reuse of existing loaded tiles/bitmaps, and five complete stop/start cycles. No preferences are rewritten by these tests. The temporary diagnostic APK was removed afterward.

One real sleep/wake cycle also passed screenshot checks for the alpine wallpaper and Plex banner without pressing Home or restarting the launcher. A subsequent automated wake was interrupted by HDMI standby (mLastSleepReason=hdmi), and One Touch Play timed out; no CEC settings were changed. The user turned the TV on again for the final screen check. The final screenshot shows the alpine wallpaper and app banners restored, with the user’s five-column layout preserved. Both the launcher and Home helper recorded zero CPU ticks over a ten-second background sample. Home returned correctly from Android settings and restored the UI after its cached process was explicitly reclaimed. The Home helper was re-bound after instrumentation cleanup, preserving the original enabled services. This is not an overnight endurance test.

October 17 performance baseline

Final device sample with the Home override enabled:

Measurement Result
Launcher UI foreground PSS 95,566 KiB (93.3 MiB)
Launcher UI cached/background PSS 66,635 KiB (65.1 MiB), reclaimable
Separate Home router PSS 6,348 KiB (6.2 MiB)
Launcher background CPU 0 CPU ticks over 10 seconds
Home router idle CPU 0 CPU ticks over 10 seconds
Focus navigation rendering 84 frames, 4 janky (4.76%); 95th percentile 24 ms

The separate Home router avoids retaining the UI image heap. Android can reclaim the UI while the router stays bound; Home then creates the UI again. This is not a guarantee of perfect 60 fps.

Raw measurements are recorded in ../output/luma-*-memory.txt, ../output/luma-background-cpu.txt, and ../output/luma-focus-frames.txt. PSS includes framework/graphics allocations and fluctuates with system state. Short CPU samples indicate idle behavior, not a guarantee covering all usage.

Scope

No launcher application was disabled or uninstalled. Screensaver selection at the Android system level was not changed. Standard Android Dream settings are unavailable on this TV, so the customizable idle screensaver runs while Luma is foreground. No real Plex credentials were extracted or used. The UI-reclaim test explicitly killed the cached UI process while leaving the Home router bound, then verified that Home restored Luma. No reboot test was performed.

Build validation uses aapt2, javac, D8, zipalign, and apksigner signature verification. There are no third-party runtime libraries.

Final APK (1.1.5): 2,753,622 bytes. SHA-256: 9556c053f5c1ebb7e7b9414596ee2b2185a37161044538617490713bb9dd8228.

Updater regression — October 18, 2026

The same-signature updater test ran against the installed 1.1.0 bootstrap and a newer signed 1.1.1 APK. It passed persisted daily network scheduling, Off cancellation, branch-to-commit pinning, a real APK download and certificate/package/version/checksum verification, and private provider read access. It rejected an incorrect checksum, another signing key, a different package, a mismatched version code, oversized metadata declarations, and public HTTP sources. The provider rejected write requests and unknown paths. The current version was not offered as an update. Test preferences were restored afterward.

The production default is the public HTTPS Forgejo feed in output/update.json, generated with the APK by scripts/build.sh. It requires no credentials on the TV. Native installer and unknown-source permission activities resolve on the onn. Android 14 device.

Production self-update verification

A forced scheduled check against the real public HTTPS repository found a newer version while Android settings stayed foreground. Android recorded the Luma update notification, and Home showed the Update button. Downloading and verification used the commit-pinned APK from Git, not ADB. The first-use Android install-source permission screen was exercised.

Testing exposed a device-specific consequence of legacy APK/file installation: Android restricted settings reset the accessibility service list. The final flow uses a PackageInstaller session for a publisher-verified self-update (PACKAGE_SOURCE_OTHER), always requiring user confirmation. It does not claim to be an app store or request silent installation. Unlike a general APK file installer, Luma accepts only its own package signed by the installed key.

The final real upgrade, 1.1.4 → 1.1.5, was downloaded from production Git and approved with the TV remote. The complete enabled-accessibility-service list matched before and after, including Luma Home, Projectivy, and Button Mapper. Restricted-settings authorization remained allowed, and the TV returned directly to Luma. No ADB install or accessibility repair was used for that final upgrade. Prior temporary configuration changes from failed installation-method tests were restored.

The source classification follows Android's PackageInstaller source definitions; the platform's file-source restriction implementation explains the behavior encountered during the earlier file-install tests.