Release 1.1.1: corroborate cloudy frames with local stellar attenuation
All checks were successful
Release AstroFrame / packages (push) Successful in 1m8s

This commit is contained in:
Elijah 2026-10-04 10:45:10 -07:00
parent a8e7a87b94
commit 65ded0b606
14 changed files with 731 additions and 9 deletions

View file

@ -0,0 +1,65 @@
# Algorithm comparison for 1.1.1
Validation performed October 4, 2026. The baseline is the unchanged source at `a8e7a87` (1.1.0), with its validation executable preserved before changing the engine. Both algorithms processed original Sample Data FITS files into separate workspace artifact directories. Source files and the installed application were untouched.
## Results
| Session | Frames | 1.1.0 Keep / Review / Reject | 1.1.1 Keep / Review / Reject |
|---|---:|---|---|
| NGC 1432, September 20 | 490 | 358 / 55 / 77 | **358 / 32 / 100** |
| NGC 1432, October 2 | 563 | 547 / 16 / 0 | **547 / 16 / 0** |
| NGC 7000, August 17 | 147 | 141 / 6 / 0 | **141 / 6 / 0** |
| IC 1805, August 26 | 309 | 301 / 3 / 5 | **301 / 2 / 6** |
| Total | 1,509 | 1,347 / 80 / 82 | **1,347 / 56 / 106** |
All 24 differences are Review → Reject: 23 in the requested NGC 1432 session and one in IC 1805. No Keep or existing Reject changed verdict; no new Keep was introduced. The requested session's review workload falls by 23 frames (41.8%). All four supplied cloud examples now Reject. The two supplied clear-looking morphology examples remain Review because original-source stellar crops show shape differences and the matched-star evidence remains near the practical tolerance.
## What changed
- Missing reference stars are distinguished from failed geometry. Tight-match count, residuals and spatial distribution can validate the surviving star field. The nearly opaque example has 164 tight correspondences and a median residual of 0.423 original pixels, despite a detected-star match fraction of only 0.297.
- Broad coherent sky anomalies can be corroborated by attenuation measured specifically at reference apertures under positive excess-sky cells. Nearby comparisons use the same apertures, with reference scatter and local sky/background photometry uncertainty. This avoids dilution by unaffected stars or coarse whole-image regions.
- Matched bright stars cross-check elongation, with enough nearby reference frames and a conservative effect/uncertainty margin. An initial candidate Keep was not retained after original-source crops showed measurable elongation. The final comparison preserves every prior Keep/Reject membership.
- The analysis-version identifier changed, invalidating old automatic measurements while retaining human decisions. Display and binary-cache formats are unchanged; reanalysis generates the new diagnostics.
## Inspection of the differences
Every final changed exposure was inspected in source-derived previews and against its measured local attenuation. Each NGC 1432 difference shows a cloud band or patch, including the four supplied examples. A positive cloud structure and a consistent loss in the affected stellar apertures support these rejections. The changed IC 1805 frame (`20260826-012414`) shows cloud entering the lower part of the field; subsequent frames show that cloud moving across the right side. Its affected-aperture flux is approximately 73% of nearby reference flux. Adjacent preceding clear frames preserve the persistent nebula and remain Keep. This is transient cloud evidence rather than the nebula itself being rejected.
Additional visual controls include nine fixed-time samples in each nebula session, the nine weakest pre-existing NGC 1432 rejects, and the strongest remaining cloud Review frames. NGC 7000 has **zero decision changes across all 147 exposures**. All 301 IC 1805 Keeps and five existing Rejects are unchanged. The second NGC 1432 night also has zero changes across 563 exposures.
These inspections are an assistant assessment supported by measurements, not independent owner labels for the entire dataset. They support this focused improvement and found no incorrect new rejection in the audited differences. They do not establish a zero-error guarantee on future targets or substitute for controlled stacks. We retain Review where geometry, attenuation, or morphology is uncertain.
## Regression and release checks
- The focused regression loop initially failed three checks on the old policy and passes on the new policy: accurate geometry despite star disappearance; broad sky structure with local attenuation; and detection-cohort morphology corroboration.
- The full engine suite passes **77 checks**, including persistent nebulosity and gradients, prolonged moving cloud, no stable clear reference, low geometry confidence, insufficient/noisy apertures, measurable elongation, source-safe export, preview independence, and upgrade preservation of human decisions.
- A checked-in 46-frame measured fixture preserves four owner cloud examples and two owner-confirmed clear controls, without requiring the original FITS files in CI.
- All **9/9 owner-labeled real examples** pass. The **40/40 previously screened cloud-only samples available in these tested sessions** also pass. Those samples were previously used in tuning, so they are not an independent accuracy estimate.
- The updater and mocked publisher suites run on the final versioned source before publication. The publisher tests do not publish or interact with the installed app.
## Additional spatial and trail experiments
The three suggested topics were evaluated before publication. They are promising directions, but the tested prototypes did not establish a further safe verdict improvement, so they are not additional production rejection rules in 1.1.1.
**Cloud-scale smoothing and multiple scales.** All 1,509 cached aligned sky maps were evaluated at the existing cell scale and with 3×3 and 7×7 box filters. Each scale used scatter measured from equally filtered clear-reference residuals, retaining the original noise floor and overlap mask; uncertainty was not reduced by assuming neighboring cells were independent. These are bounded box-filter experiments, not an exhaustive evaluation of Gaussian, wavelet, or adaptive filters. The existing significance, coverage and connected-area gates were applied for comparison.
- No previously nonsuspicious Keep acquired a cloud candidate at either added scale.
- A 3×3 filter reduced suspicious sky Review candidates from 15 to 14 in NGC 1432, without introducing a new candidate; the removed candidate is from the otherwise unchanged October night.
- A 7×7 filter reduced NGC 1432 candidates from 15 to 11, and IC 1805 Review candidates from two to one. It also removed the sky-candidate gate from two existing IC 1805 Rejects (`20260826-012650` and `20260826-012721`), whose source-derived previews show cloud entering at the edge. Their affected-aperture flux ratios are approximately 60% and 59%.
- Requiring agreement at all scales could suppress real partial clouds. Taking their union retained the original candidate membership, giving no additional verified accuracy gain. Connected regions and affected-star corroboration were therefore retained without replacing the fine-scale evidence or promoting smoothed-away warnings to Keep.
Filtering a background mesh is an established measurement technique, but does not itself establish a safe frame rejection policy; [Photutils Background2D](https://photutils.readthedocs.io/en/latest/api/photutils.background.Background2D.html) documents mesh filtering independently of this classifier.
**Directional trails.** Original FITS stellar cores were measured for 104 selected exposures: morphology Review frames, nearby representative Keeps, and nine fixed-time Keep controls in each target. The exploratory measurement reused positive core moments, excluded detected close neighbors within ten superpixels, and computed the ellipticity-weighted double-angle resultant for isolated stars with ellipticity at least 0.15. This measures directional consistency from zero to one; it is not a calibrated PSF fit or an independent tracking-error label.
The median resultant was 0.987 for 33 NGC 1432 Reviews versus 0.874 for 41 Keeps, and 0.963 for six NGC 7000 Reviews versus 0.653 for 15 Keeps. However, nine IC 1805 Keep controls already had a median resultant of 0.930 and median isolated-star ellipticity of 0.280. Thus a high directional consistency alone does not distinguish tracking failure from the session's normal optics/sampling. The Reviews are already captured by elongation; there are no independently labeled trail acceptance boundaries in these samples to justify automatically discarding them. A future rule should compare matched-star shape tensors after accounting for frame rotation, spatial optical patterns, sampling and fit uncertainty, then validate against explicit acceptable/unacceptable tracking labels. This release retains conservative matched-star elongation corroboration and Review for unresolved shape changes.
[Siril's Dynamic PSF documentation](https://siril.readthedocs.io/en/latest/Dynamic-PSF.html) describes rotation-angle fitting and notes that elliptical stars can arise from optical aberrations, tracking, or blended stars. Its fitted Gaussian/Moffat profiles are more sophisticated than AstroFrame's core moments; we do not claim identical measurement accuracy or that the suggested cloud classifier is documented as a Siril feature.
Workspace experiment sources/results are `SpatialTopics.cs`, `spatial-topics.json`, `DirectionTopics.cs`, and `direction-topics.json` under `artifacts/algorithm-1.1.1/`; `smoothed-loss-1.png` contains the seven sky-candidate losses at the largest scale. These research experiments do not modify production measurements, the source FITS files, or the comparison verdicts above.
## Reproduction and artifacts
Baseline/new analysis uses the validation executable's `analyze <source> <output> --fresh` mode, with temporary/cache environment variables confined to the workspace as prescribed in AGENTS.md. The baseline executable is preserved at `artifacts/algorithm-1.1.1/baseline-cli/`; baseline and final projects, catalog caches, previews, manifests and reports are under `baseline-*` and `new-*` sibling folders. Final policy refinements reused identical stellar/sky/aperture measurements and rebuilt their reference diagnostics. All 1,509 source size/modification-time fingerprints remain unchanged.
`artifacts/algorithm-1.1.1/comparison.json` records the full per-exposure differences. `changes-1.png` through `changes-3.png` show every final changed frame; `nebula-controls-*.png` and `nebula-cloud-transition-1.png` show the nebula controls and the IC 1805 transition. `keep-stars.png` documents why the exploratory new Keep was retained as Review in the final policy. Compact session counts and per-change measurements are checked in as [quality-1.1.1-validation.json](diagnostics/quality-1.1.1-validation.json).