Kaleido/docs/playback-validation-2026-10-26.md
Elijah 2a83fe0db5
All checks were successful
Validate and publish Kaleido / android (push) Successful in 2m45s
Release Kaleido 0.1.10 with native Original playback and PCM compatibility
2026-10-26 16:42:19 -07:00

67 lines
9.6 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Dedicated playback validation — 2026-10-26
This records the 0.1.9 baseline investigation. The subsequent 0.1.10 fixes and release checks are recorded in [runtime verification](runtime-verification.md).
Tested the onn 4K Plus (`192.168.50.195:5555`) connected to the TCL TV over HDMI, using 0.1.9 source plus opt-in debug instrumentation. These results describe this device chain and these fixtures. They are not a certification of every file, display, receiver, or Dolby profile.
## Main finding
The native file path can render 4K video and output multichannel audio, but the production **Original quality** path requests Plex HLS and unnecessarily converts the tested 4K sources. A 3840×2160 HEVC/AAC 5.1 source reached the player as 1920×1080 HEVC/AAC stereo. A 3840×1608 Dolby Vision profile 8/EAC3 7.1 source reached it as 1920×804 HEVC PQ/AAC stereo, losing Dolby Vision signaling. The latter was repeated with HDR formats fully enabled. Both files retained their source resolution/audio on forced native playback.
This is a fidelity failure even though converted playback starts and renders. The previous live HEVC remux integration test covers a different, 1080p episode; its passing result does not cover these 4K cases. No production playback negotiation fix or new release was published during this validation.
## HDR and Dolby Vision
Verification combines Media3 input format, the selected decoder, rendered-frame counters, the SurfaceView's compositor state, and the ONN HDMI transmitter/HWC status. Output signaling is stronger evidence than the server's decision or advertised capabilities. It still cannot establish panel calibration, correct tone mapping, or perceptual image accuracy remotely.
| Source | Observed native playback / HDMI output | Result |
| --- | --- | --- |
| Dolby Vision profile 5, HEVC MP4, 1080p | `dvhe.05.03`, Dolby Vision HEVC decoder, rendered test pattern; `DolbyVision-Std` | Rendering and signaling pass; local fixture, Plex delivery untested |
| Dolby Vision profile 8, HEVC MKV, 4K | `dvhe.08.06`, Dolby Vision HEVC decoder; `DolbyVision-Std` | Rendering and signaling pass |
| Dolby Vision profile 10, AV1 MKV, 4K | `dav1.10.08`, Dolby Vision AV1 decoder; `DolbyVision-Std` | Rendering and signaling pass |
| HDR10, HEVC MP4, 4K | BT.2020/PQ input, HEVC hardware decoder; `HDR10-GAMMA_ST2084` | Rendering and signaling pass with Dolby Vision temporarily excluded |
| HDR10, AV1 MKV, 4K | BT.2020/PQ input with mastering metadata, AV1 hardware decoder; `HDR10-GAMMA_ST2084` | Rendering and signaling pass with Dolby Vision temporarily excluded |
| HLG, HEVC MKV, 4K | HLG input, HEVC hardware decoder; `HDR10-GAMMA_HLG` | Rendering and signaling pass under the original HLG-priority policy |
| Dolby Vision profile 7 candidate | Frames rendered, but source advertises contradictory BT.709/SDR metadata | Inconclusive; not a valid reference for profile 7 enhancement-layer fidelity |
| HDR10+ dynamic metadata | No independently identified reference tested | Not covered |
The initial ONN configuration manually disabled Dolby Vision and HDR10. Clearing Android's disabled-format list, and even selecting Automatic, did **not** update its vendor `HLG priority` policy. That policy produced SDR for tested Dolby Vision inputs and HLG for a PQ source. Toggling Dolby Vision off/on through the normal manual-format UI updated the vendor policy to `DV_HDR10_HLG priority`, after which profile 5/8/10 output was confirmed as Dolby Vision. Excluding Dolby Vision while leaving HDR10 enabled selected `HDR10_HLG priority` and allowed isolated HDR10 checks. With System-preferred conversion, a HDR10 source can otherwise leave the device as Dolby Vision; that is device conversion and must be distinguished from the input format.
The profile 5 fixture is Dolby's unencrypted [official color pattern](https://ott.dolby.com/OnDelKits/DolbyVision_OSS_Media_Kit/clear/color_pattern_24_dvhe_05_1920x1080.mp4), verified locally as profile 5 with compatibility ID 0. It was copied into Kaleido's private debug fixture directory and exercised through the same controller and SurfaceView. It was not uploaded to the Plex library. The HDMI status strings also correspond to the [Amlogic transmitter implementation](https://nest-open-source.googlesource.com/manifest_repos/kernel/+/1650886512330522da41d487bb2ce75fd5da5ec8/drivers/amlogic/media/vout/hdmitx/hdmi_tx_20/hdmi_tx_main.c).
## Other video and audio
Native rendering passed for AVC 8-bit and a 10-bit AVC candidate, HEVC 8/10-bit SDR, AV1 HDR, VP9, and MPEG-4 Part 2. MP4, MKV, and AVI were exercised. Source resolutions ranged from SD to 4K. The VC-1 MKV sample advanced audio without producing a video format or rendered frame during a 55-second observation. **Normal Plex playback of that same sample converted it to 1080p HEVC/AAC and rendered successfully.** It is not counted as native VC-1 support.
| Audio source | Observed player output | Result / limit |
| --- | --- | --- |
| AAC, including 5.1 | PCM 16-bit, including six-channel output | Played |
| AC3 | AC3 bitstream, stereo / 5.1 fixtures | Played; output position advanced |
| EAC3, including 7.1 and an Atmos-tagged fixture | EAC3 bitstream, six/eight channels | Played; receiver Atmos/object decoding not verified |
| DTS | DTS bitstream, 5.1 | Played; output position advanced |
| DTS-HD Master Audio | DTS-HD input selected, **DTS core** output | Compatible playback; full DTS-HD passthrough not demonstrated |
| TrueHD, Atmos-tagged 7.1 | TrueHD bitstream accepted by AudioTrack | Played; receiver sound and Atmos/object decoding not verified |
| FLAC stereo, Vorbis 5.1, Opus 7.1 | PCM output with corresponding channel counts | Played; no bit-perfect claim |
| MP2 / MP3 | PCM stereo | Played |
| Generated 16-bit / 48 kHz stereo WAV | PCM 16-bit / 48 kHz stereo | Played; local fixture |
| Generated 24-bit / 96 kHz mono WAV | Failed before AudioTrack initialization | Reproduced `ERROR_CODE_FAILED_RUNTIME_CHECK`; `getAudioTrackMinBufferSize` check failed during output configuration. Other high-resolution PCM combinations remain untested |
One 4K AV1/TrueHD run sustained 61.8 seconds after playback began, rendering 1,475 frames with one dropped frame and no subsequent buffering samples. Most other matrix cases observed approximately 10–12 seconds of steady playback. This is a compatibility sweep, not a feature-length soak or perceptual lip-sync test. Initial native-file probes encountered HTTP-status fallbacks; later repeated native probes succeeded. The initial HTTP status was not captured, so transport reliability is not certified by the later passes.
Pause, three backward 10-second seeks, resume and exit passed for native Dolby Vision and the Plex HLS path. The HLS test crossed the original start offset and reached the correct absolute position without doubling the resume offset. No tested Plex item's watched count, last-viewed timestamp, or resume position changed.
## TCL HDMI observation
A debug-only `TvView` activity tuned the verified HDMI 1 hardware input through Android's public TV Input Framework. TCL reported `videoAvailable` and a 3840×2160/60 input. Its standard ADB screenshot attempts while HDMI was active produced no usable image, including a failed file-based capture. The vendor track extras did not provide a dependable changing HDR indication. Thus **ADB via this TCL is not a reliable capture of the actual HDMI picture**. No capture-protection bypass was attempted. The confirmed HDR output results above use ONN transmitter/HWC diagnostics; they do not replace a physical screen/receiver check.
## Reproduction and evidence
`scripts/inventory_playback.py` inventories actual Plex metadata. `scripts/validate_playback.py` cold-launches one case at a time, records delivered formats/decoder counters/audio output, optionally seeks, closes the player, and verifies unchanged Plex progress. `--mode 0` forces a native file, `--mode -1` follows the saved production preference, and `--observe-hdmi` reads an already tuned TCL observer. Local `--fixture` files require matching Title-style JSON metadata and explicitly bypass Plex delivery. A fixture failure does not request a Plex conversion for that synthetic ID.
Validation is debug-only and opt-in. It suppresses timeline/scrobble writes while retaining normal playback otherwise. The capability inventory exports no credentials. Telemetry excludes URLs, headers, tokens, and exception messages; raw source metadata and captures stay under ignored `evidence/private/playback-validation/`. A missing video track on a movie is a failure, not an audio-only pass. A blank ADB HEVC capture alone is not a failure when decoder counters demonstrate rendering.
`assembleDebug testDebugUnitTest lintDebug` passed (five unit tests), as did Python compilation and `git diff --check`. Playback failures and limitations above are recorded test outcomes, not passing checks. The signed 0.1.9 release remains the published update; validation instrumentation is local work.
The original HDR state was restored through the manual-format UI and verified: `user_disabled_hdr_formats=1,2`, `are_user_disabled_hdr_formats_allowed=0`, `hdr_conversion_mode=2`, `hdr_force_conversion_type=-1`. The vendor policy returned to `HLG priority`; a final HLG run rendered and reported `HDR10-GAMMA_HLG`. Dolby Vision and HDR10 are therefore disabled again, matching the user's original setup.
The temporary on-device fixtures and telemetry were removed. Both ONN and TCL were reinstalled in place with the original published, non-debuggable 0.1.9 APK (version code 10; SHA-256 `ddc9b089bd02aaa0e636e27786159d86419dba8eba8e9995a569a3a9902b576b`). Both confirmed ART `speed-profile` compilation. Sessions and app data were retained.