# Validation protocol No claim of matching or exceeding manual Siril selection is justified yet. Five owner-reviewed examples now supply real regression labels; they are not a representative accuracy sample. An additional 56 fixed capture-order previews provide a provisional cloud-only screen. No controlled stacks have been produced. ## Frame-level verification 1. Review automatic REJECT frames and the REVIEW pool. Keep automatic decisions separate from human overrides. The v1 count of five REJECT frames was a failed result, not a reference target. 2. Sample KEEP exposures throughout every session, including session edges, low star-count stretches, elevated background periods and transitions. A review confined to REJECT cannot measure missed bad frames. 3. Record human KEEP/REVIEW/REJECT and notes. For meaningful statistics, finish labeling a session or use a clearly documented representative sample. 4. Run the label summary. Report false rejection of human KEEP in frames and exposure seconds, automatic KEEP of human REJECT, ambiguous cases, confusion counts and labeled/unlabeled coverage per session. Human REVIEW is uncertainty, not a definite good/bad label. 5. Never infer that an unlabeled frame is a correct decision. Retain a held-out target/night before tuning policy rules. The cloud-specific regression loop is `python tools/check_real_frames.py projects/Sample\ Survey/project.json` (quote the path for your shell). `tests/real-frame-labels.json` records the owner's exact examples. `tools/check_visual_holdout.py` evaluates `tests/visual-cloud-holdout.json`; clear-looking previews only assert no automatic rejection, since preview screening does not validate PSF or uniform attenuation. Synthetic image regressions include persistent nebulosity, gradual gradients/flux drift, and a prolonged moving cloud stretch. None is a substitute for independent owner labels and stacks. ## Controlled Siril comparison Prepare three input cohorts per target/session: - All originals. - Human-selected lights, with a documented policy for human REVIEW. - Automatically selected lights, with a documented policy for automatic REVIEW. Keep-only and keep-plus-review are distinct experiments. Use the same calibration files, debayering, registration/reference, output geometry, weighting, pixel rejection, normalization and subsequent processing. Work from exported copies or read-only inputs into separate output folders. Do not optimize one cohort independently. Log selected frame names and total integration before processing. Compare registered crops using the same display stretch and measurement regions: background noise, faint-feature signal-to-noise, stellar size/ellipticity, gradients/cloud residue, color stability, artifact rates and retained integration. Include both equal-processing results and an equal-integration sensitivity check where appropriate. Judge the nebula/galaxy and stars, not just a global sharpness statistic. Preserve output stacks and settings so the comparison is reproducible. The automatic method can be claimed comparable only after these outputs support that conclusion across targets and nights, with an acceptable false-rejection budget agreed from the results. No fixed acceptance budget is invented here.