Release 1.1.6: reject corroborated thin cloud instead of reviewing it
All checks were successful
Release AstroFrame / packages (push) Successful in 1m17s

Add a quadratic-detrended, smoothed sky texture measurement scaled by
clear reference scatter, and corroborate it with same-exposure
differential forced photometry (stars inside vs. outside the texture,
normalized by nearby clear frames). Frames already routed to REVIEW
become REJECT only when both agree; otherwise-clean KEEPs are never
affected. NGC 1432: Review 48 -> 36, Reject 100 -> 112, Keep unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Elijah 2026-10-05 19:19:33 -07:00
parent d8923ea85d
commit ef7805055c
9 changed files with 125 additions and 26 deletions

View file

@ -1,6 +1,6 @@
<Project>
<PropertyGroup>
<Version>1.1.5</Version>
<Version>1.1.6</Version>
<Authors>Elijah Kuntz</Authors>
<Product>AstroFrame</Product>
</PropertyGroup>

View file

@ -27,7 +27,7 @@ To analyze another dataset, choose **Analyze lights**, select the folder contain
- Press **1**, **2**, **3** for KEEP, REVIEW, REJECT. Decisions save and advance. **J/K** navigates. **Reset** restores the automatic decision. **Ctrl+Z** undoes a manual decision.
- Decision evidence and measurements remain visible below the quality graph. Hover over an evidence line to read its full technical detail, and scroll the panel for additional measurements. Preview stretch and manual decisions sit below the image. Existing notes remain in saved projects and exports; the note editor has been removed. Automatic decisions and human overrides remain separate.
KEEP means consistent measurements; it does not certify a perfect frame. REVIEW means a substantial deviation or measurement uncertainty. REJECT requires strong, coherent transient sky structure, agreement between cloud/flux/star-loss measurements, severe combined morphology degradation, or catastrophic star disappearance. Small ellipticity variations alone do not trigger review.
KEEP means consistent measurements; it does not certify a perfect frame. REVIEW means a substantial deviation or measurement uncertainty. REJECT requires strong, coherent transient sky structure, agreement between cloud/flux/star-loss measurements, thin cloud texture on an already-flagged frame that dims the stars beneath it relative to the rest of the exposure, severe combined morphology degradation, or catastrophic star disappearance. Small ellipticity variations alone do not trigger review.
The earlier engine missed obvious clouds and falsely reviewed a clear frame. Its rolling background baselines and detection-only flux measurements have been replaced. Analysis-version changes request reanalysis while preserving manual decisions. Display changes have their own cache version, so preview upgrades do not invalidate quality measurements.

View file

@ -6,7 +6,7 @@ The supplied 1,920 originals cover four targets and five sessions, with 2160×38
The first algorithm failed real owner-reviewed examples: cloud structure inflated detected-star counts; flux only sampled surviving detections; rolling baselines absorbed prolonged cloud; a small ellipticity deviation reviewed a clear image. The v2 design replaces the cloud measurements, reference construction and classification policy. Rejection counts are not a target or a validation criterion.
Version 1.1.1 refines this policy with geometric fit diagnostics, local cloud-aperture corroboration, and matched-star morphology checks. The original v2 results remain a baseline; see [the 1.1.1 comparison](algorithm-1.1.1-validation.md) for the measurements and limits of the new validation.
Version 1.1.1 refines this policy with geometric fit diagnostics, local cloud-aperture corroboration, and matched-star morphology checks. Version 1.1.6 adds thin-cloud texture with same-exposure differential photometry. The original v2 results remain a baseline; see [the 1.1.1 comparison](algorithm-1.1.1-validation.md) for the measurements and limits of the new validation.
## Components
@ -24,6 +24,7 @@ Additional detectors can add measurements independently. A future AI reviewer ca
4. Robustly remove each map's additive offset and planar gradient. Form an aligned median background map, measure transient residuals, and rebuild three times from the least transient 40% of exposures. This fraction constructs a reference; it is **not** a rejection quota. Persistent target nebulosity belongs to the reference. Smooth sky gradients and offsets are removed rather than interpreted as cloud.
5. Read each exposure once more, sequentially, for signed aperture photometry at up to 600 reference coordinates. A local annulus subtracts sky. Reference brightness establishes aperture eligibility; a star need not be detected in the tested exposure. Median fluxes from reference exposures normalize global and 3×4 regional transparency. This addresses survivor bias. Saturated apertures and positions outside the field are excluded explicitly.
6. Compare 960 aligned sky cells with the reference. Cell scatter comes from reference-frame residual MAD with a floor relative to measured image noise. Store a mask above five times that scatter, affected overlap fraction, largest connected component, 95th-percentile significance, residual ADU, aperture counts and reference overlap. These are named evidence, not an opaque combined score.
7. Measure thin-cloud texture separately. Remove a robust quadratic surface, not only a plane, from each aligned residual, then average each cell with its 3×3 neighbors. Divide by the per-cell scatter that reference exposures show for the same smoothed statistic; the floor is half the median of those scatters. Cells above four times scatter form a texture mask. Store the 98th-percentile texture significance, affected and largest connected fraction. Compare the median forced flux of reference apertures inside the mask with apertures in clear cells (below twice scatter) of the same exposure. Normalize that inside/outside ratio by the same apertures in the nearest 40 reference exposures and divide its loss by their MAD, with a 1% floor. Global transparency, seeing and airmass changes cancel in this differential ratio; a sky-only gradient leaves it near one.
Grid background estimation and local aperture subtraction are standard image-analysis techniques; see [Photutils background documentation](https://photutils.readthedocs.io/en/stable/api/photutils.background.Background2D.html). This implementation is independent C#, using the supplied data to establish its comparison distributions.
@ -36,6 +37,7 @@ REJECT evidence routes are explicit:
- Strong spatial structure: at least 25× reference scatter at the 95th percentile, at least 8% affected overlap and a connected area of at least 3.5%. A lone sky signal must be overwhelming before discarding integration; weaker coherent structure needs stellar corroboration or routes to REVIEW.
- Moderate coherent spatial structure plus substantial global flux loss, regional forced-flux loss or detected-star loss.
- Coherent structure at least 8× reference scatter, at least 8% affected overlap, and a connected anomalous area of at least 3.5%, corroborated by positive excess-sky coverage of at least 8% and a positive connected area of at least 3.5%. At least 20 reference apertures in two regions inside that excess sky must show below 90% of nearby same-aperture flux, with at least three times the larger of reference scatter, a 3% floor, and propagated local photometry uncertainty. Affected-aperture comparisons use the nearest 40 reference members, excluding the tested exposure, and require at least 12 valid comparison frames.
- Thin cloud on an exposure already routed to REVIEW by another measurement: texture significance at least 10, a connected textured area of at least 5%, at least 20 apertures both inside and outside the texture, and stars inside at most 98% of their clear-cell flux with a loss of at least 2.5 times nearby reference scatter. Applying real cloud masks to clear exposures put 0.4–0.6% of differential deviations at or above 2.5, and clear exposures do not reach the texture prerequisite. This route never rejects an otherwise clean KEEP.
- Severe global flux loss (below 60% of clear-frame baseline) with substantial detected-star loss.
- Severe combined stellar width and ellipticity degradation, or almost complete star disappearance relative to a rich reference field.
@ -47,7 +49,7 @@ These significance and effect-size limits are **engineering policy, not calibrat
## Limits that need validation
A wholly uniform, cloudy session may lack any internally observable clear reference. Clouds that resemble a smooth gradient and have little detectable attenuation are difficult. Dense nebulosity, alignment failures, crowded fields, large drift/rotation and changing PSFs can bias maps or fixed-aperture flux. The current model removes only a planar sky trend; a legitimate time-varying nonlinear sky pattern can require review. The red overlay identifies anomalous sky structure, not a pixel-accurate physical cloud segmentation.
A wholly uniform, cloudy session may lack any internally observable clear reference. Clouds that resemble a smooth gradient and have little detectable attenuation are difficult. Dense nebulosity, alignment failures, crowded fields, large drift/rotation and changing PSFs can bias maps or fixed-aperture flux. The main structure model removes only a planar sky trend; a legitimate time-varying nonlinear sky pattern can require review. The thin-cloud texture also removes quadratic curvature, but requires stellar dimming beneath it before rejecting. Faint cirrus on an otherwise clean exposure remains KEEP: its effect on a stack has not been validated. The red overlay identifies anomalous sky structure, not a pixel-accurate physical cloud segmentation.
Small sessions and unreliable references cannot automatically reject. Reference stability also checks reference-map residuals against measured pixel noise; a wholly unstable comparison population routes to REVIEW. This does not detect a perfectly uniform, consistently attenuated session. Human overrides remain separate from automatic results. Do not claim the method matches manual Siril selection until independently processed stacks support that claim.

View file

@ -0,0 +1,11 @@
# AstroFrame 1.1.6
Resolves more thin-cloud exposures automatically instead of leaving them for manual review.
- Adds a thin-cloud texture measurement. Each aligned sky map has smooth curvature removed, neighboring cells averaged, and the result compared with the scatter of clear reference exposures. Faint cloud bands that previously fell below the sky-structure threshold now stand out clearly.
- Corroborates that texture with the stars beneath it. Forced photometry inside the texture is compared with clear parts of the same exposure and then with the same apertures in nearby clear frames. Global transparency, seeing and airmass changes cancel out; a sky gradient alone does not dim stars and cannot trigger rejection.
- An exposure already flagged for Review becomes Reject only when both measurements agree. The new route never rejects an exposure that would otherwise be Keep, and it adds no new Review triggers. Decision evidence explains the texture coverage and the measured dimming.
In the supplied 1,053-frame NGC 1432 project, Review decreases from 48 to 36 and Reject increases from 100 to 112. All 12 changed exposures show visible cloud bands. Every Keep is unchanged. Replays of eleven other saved projects (about 5,000 exposures, including the M31 sessions with manual decisions) changed 15 more Review exposures to Reject. All were inspected: cloud passages, plus one dawn-twilight exposure next to frames the existing rules already reject. No Keep and no manually labeled exposure changed. The remaining NGC 1432 Reviews are mostly stellar-elongation flags, which this release does not change.
Analysis time and memory use are essentially unchanged. Old automatic measurements require reanalysis; manual decisions are retained. Engine (100) and updater (23) checks pass on the versioned source. Windows and Linux update channels are included.

View file

@ -97,6 +97,17 @@ public static class Classification
bool skyNoise=bgZ>5&&noiseZ>4&&Ratio("Background",f.Background)>1.4;
if(skyNoise)f.Evidence.Add(new("Sky noise",f.Noise,f.Baselines["Noise"],noiseZ,"Background and sky noise are substantially above the clear-frame baseline"));
bool suspicious=moderateStructure||fluxLoss||regionLoss||widthBad||shapeBad||skyNoise||starLoss||lowMatches;
// Thin cloud adds little residual glow, but it is coherent after smooth curvature is
// removed and dims the stars beneath it relative to clear parts of the same exposure.
// Requiring both keeps sky-only changes, such as gradients, out of this route. It only
// resolves frames other measurements already flag; it never rejects an otherwise clean KEEP.
double Measured(string key)=>f.AdditionalMeasurements.GetValueOrDefault(key);
bool thinCloud=suspicious&&spatial&&Measured("Cloud texture significance")>=10&&Measured("Cloud texture connected coverage")>=.05&&
Measured("Cloud texture differential samples")>=20&&Measured("Cloud texture differential flux ratio")>0&&
Measured("Cloud texture differential flux ratio")<=.98&&Measured("Cloud texture differential flux deviation")>=2.5;
if(thinCloud)f.Evidence.Add(new("Cloud texture",Measured("Cloud texture differential flux ratio"),1,Measured("Cloud texture differential flux deviation"),
$"Thin cloud texture covers {Measured("Cloud texture coverage"):P0} of the aligned overlap (largest connected area {Measured("Cloud texture connected coverage"):P0}, {Measured("Cloud texture significance"):0.0} × clear-frame scatter); stars beneath it are {1-Measured("Cloud texture differential flux ratio"):P1} fainter relative to clear parts of the same exposure ({Measured("Cloud texture differential flux deviation"):0.0} × nearby clear-frame scatter, {Measured("Cloud texture differential samples"):0} apertures)"));
cloud|=thinCloud;
f.Automatic=reliable&&(catastrophic||!weak&&(cloud||optics))?Verdict.Reject:!reliable||weak||suspicious||f.Warning!=""?Verdict.Review:Verdict.Keep;
f.Cause=!reliable?"Insufficient reliable session reference":catastrophic?"Severe disappearance of reference stars":weak?"Uncertain alignment / reference photometry":cloud?"Likely cloud / transparency loss":optics?"Likely tracking or focus failure":suspicious?"Quality change needs review":"Consistent with the clean session reference";
if(!reliable)f.Evidence.Add(new("Reference",local.Length,12,0,"Automatic rejection disabled: insufficient clear comparison frames, timestamp or aligned overlap"));

View file

@ -70,7 +70,7 @@ public sealed class Frame
}
public sealed class Project
{
public const string EngineVersion="0.2.1-spatial-reference-2";
public const string EngineVersion="0.2.2-spatial-reference-3";
public string Version {get;set;}=EngineVersion;
public string Name {get;set;}="Untitled";
public string Source {get;set;}="";

View file

@ -38,36 +38,59 @@ public static class SkyModel
}
return map;
}
public static double[] Detrend(double[] values)
public static double[] Detrend(double[] values)=>Detrend(values,false);
// Robust offset and gradient removal; quadratic terms also absorb smooth sky curvature.
public static double[] Detrend(double[] values,bool quadratic)
{
int terms=quadratic?6:3;
bool[] good=values.Select(double.IsFinite).ToArray();double[] residual=(double[])values.Clone();
if(good.Count(v=>v)<12)return residual;
for(int iteration=0;iteration<5;iteration++)
{
double[,] system=new double[3,4];
double[,] system=new double[terms,terms+1];
for(int i=0;i<Count;i++)if(good[i])
{
double[] v=[1,(i%Columns+.5)/Columns-.5,(i/Columns+.5)/Rows-.5];
for(int r=0;r<3;r++){for(int c=0;c<3;c++)system[r,c]+=v[r]*v[c];system[r,3]+=v[r]*values[i];}
double[] v=Basis(i,quadratic);
for(int r=0;r<terms;r++){for(int c=0;c<terms;c++)system[r,c]+=v[r]*v[c];system[r,terms]+=v[r]*values[i];}
}
double[] coefficients=Solve(system);for(int i=0;i<Count;i++)residual[i]=values[i]-coefficients[0]-coefficients[1]*((i%Columns+.5)/Columns-.5)-coefficients[2]*((i/Columns+.5)/Rows-.5);
double[] coefficients=Solve(system);
for(int i=0;i<Count;i++){double[] v=Basis(i,quadratic);double r=values[i];for(int k=0;k<terms;k++)r-=coefficients[k]*v[k];residual[i]=r;}
double median=Robust.Median(residual.Where((_,i)=>good[i])),sigma=Math.Max(1,Robust.Mad(residual.Where((_,i)=>good[i])));
for(int i=0;i<Count;i++)good[i]=double.IsFinite(residual[i])&&Math.Abs(residual[i]-median)<3*sigma;
if(good.Count(v=>v)<12)break;
}
return residual;
}
static double[] Basis(int i,bool quadratic)
{
double x=(i%Columns+.5)/Columns-.5,y=(i/Columns+.5)/Rows-.5;
return quadratic?[1,x,y,x*x,x*y,y*y]:[1,x,y];
}
/// <summary>Mean of each finite cell and its finite 3-by-3 neighbors.</summary>
public static double[] Smooth(double[] values)
{
double[] output=Enumerable.Repeat(double.NaN,Count).ToArray();
for(int row=0;row<Rows;row++)for(int col=0;col<Columns;col++)
{
if(!double.IsFinite(values[row*Columns+col]))continue;double sum=0;int n=0;
for(int dy=-1;dy<=1;dy++)for(int dx=-1;dx<=1;dx++)
{int r=row+dy,c=col+dx;if(r>=0&&r<Rows&&c>=0&&c<Columns&&double.IsFinite(values[r*Columns+c])){sum+=values[r*Columns+c];n++;}}
if(n>=3)output[row*Columns+col]=sum/n;
}
return output;
}
internal static double[] Solve(double[,] m)
{
for(int k=0;k<3;k++)
int n=m.GetLength(0);
for(int k=0;k<n;k++)
{
int pivot=k;for(int r=k+1;r<3;r++)if(Math.Abs(m[r,k])>Math.Abs(m[pivot,k]))pivot=r;
if(Math.Abs(m[pivot,k])<1e-8)return [Robust.Median([m[0,3]/Math.Max(m[0,0],1)]),0,0];
for(int c=k;c<4;c++)(m[k,c],m[pivot,c])=(m[pivot,c],m[k,c]);
double divisor=m[k,k];for(int c=k;c<4;c++)m[k,c]/=divisor;
for(int r=0;r<3;r++)if(r!=k){double factor=m[r,k];for(int c=k;c<4;c++)m[r,c]-=factor*m[k,c];}
int pivot=k;for(int r=k+1;r<n;r++)if(Math.Abs(m[r,k])>Math.Abs(m[pivot,k]))pivot=r;
if(Math.Abs(m[pivot,k])<1e-8){double[] offset=new double[n];offset[0]=Robust.Median([m[0,n]/Math.Max(m[0,0],1)]);return offset;}
for(int c=k;c<=n;c++)(m[k,c],m[pivot,c])=(m[pivot,c],m[k,c]);
double divisor=m[k,k];for(int c=k;c<=n;c++)m[k,c]/=divisor;
for(int r=0;r<n;r++)if(r!=k){double factor=m[r,k];for(int c=k;c<=n;c++)m[r,c]-=factor*m[k,c];}
}
return [m[0,3],m[1,3],m[2,3]];
return Enumerable.Range(0,n).Select(r=>m[r,n]).ToArray();
}
public static int LargestComponent(bool[] mask)
{

View file

@ -75,6 +75,13 @@ public static class SpatialReference
double floor=Math.Max(.1,referenceNoise*.08);
double[] scatter=Enumerable.Range(0,SkyModel.Count).Select(j=>Math.Max(floor,Robust.Mad(residuals.Select(m=>m[j])))).ToArray();
var overlay=SkyOverlay.CreateReference(clean.Select(i=>a[i]),ct);
// Thin cloud can add only a few ADU of coherent glow per cell: below the
// fractional-noise floor above, but far above clear-frame scatter once smooth
// curvature is removed and neighboring cells are averaged. Scale that smoothed
// texture by the scatter clear reference frames show for the same statistic.
var textures=maps.Select(m=>SkyModel.Smooth(SkyModel.Detrend(m.Select((v,j)=>v-reference[j]).ToArray(),true))).ToArray();
double[] textureScatter=Enumerable.Range(0,SkyModel.Count).Select(j=>Robust.Mad(clean.Select(i=>textures[i][j]))).ToArray();
double textureFloor=Math.Max(Math.Max(referenceNoise*.002,1e-6),.5*Robust.Median(textureScatter.Where(v=>v>0)));
for(int i=0;i<a.Length;i++)
{
ct.ThrowIfCancellationRequested();var f=a[i];var residual=Difference(maps[i],reference);int valid=residual.Count(double.IsFinite);
@ -85,7 +92,14 @@ public static class SpatialReference
bool[] positive=f.ExcessSkyMask.Select(v=>v>=128).ToArray();
f.AdditionalMeasurements["Cloud positive coverage"]=positive.Length==SkyModel.Count?(double)positive.Count(v=>v)/Math.Max(valid,1):0;
f.AdditionalMeasurements["Cloud positive connected coverage"]=positive.Length==SkyModel.Count?(double)SkyModel.LargestComponent(positive)/Math.Max(valid,1):0;
MeasureCloudApertures(f,anchor,a,clean,normal,positive);
MeasureCloudApertures(f,anchor,a,clean,normal,positive,"Cloud aperture");
double[] texture=textures[i].Select((v,j)=>v/Math.Max(textureFloor,textureScatter[j])).ToArray();
bool[] textured=texture.Select(v=>v>4).ToArray();int textureValid=texture.Count(double.IsFinite);
f.AdditionalMeasurements["Cloud texture significance"]=Robust.Quantile(texture,.98);
f.AdditionalMeasurements["Cloud texture coverage"]=(double)textured.Count(v=>v)/Math.Max(textureValid,1);
f.AdditionalMeasurements["Cloud texture connected coverage"]=(double)SkyModel.LargestComponent(textured)/Math.Max(textureValid,1);
MeasureCloudApertures(f,anchor,a,clean,normal,textured,"Cloud texture aperture");
MeasureDifferentialApertures(f,anchor,a,clean,normal,textured,texture.Select(v=>double.IsFinite(v)&&v<2).ToArray());
f.CloudMask=mask.Select(v=>(byte)(v?1:0)).ToArray();f.CloudCoverage=(double)mask.Count(v=>v)/Math.Max(valid,1);f.CloudSignificance=Robust.Quantile(z,.95);
f.ReferenceConfidence=stability*Math.Min(1,clean.Length/20.0)*(double)valid/SkyModel.Count;
f.AdditionalMeasurements["Cloud connected coverage"]=(double)SkyModel.LargestComponent(mask)/Math.Max(valid,1);
@ -97,14 +111,14 @@ public static class SpatialReference
}
}
}
static void MeasureCloudApertures(Frame frame,Frame anchor,Frame[] frames,int[] clean,double[] normal,bool[] positive)
static void MeasureCloudApertures(Frame frame,Frame anchor,Frame[] frames,int[] clean,double[] normal,bool[] positive,string name)
{
var indices=Enumerable.Range(0,normal.Length).Where(j=>positive.Length==SkyModel.Count&&
positive[Math.Clamp((int)(anchor.Stars[j].Y*SkyModel.Rows),0,SkyModel.Rows-1)*SkyModel.Columns+Math.Clamp((int)(anchor.Stars[j].X*SkyModel.Columns),0,SkyModel.Columns-1)]&&
normal[j]*anchor.Exposure>=30*anchor.Noise&&frame.ForcedFlux.Length==normal.Length&&double.IsFinite(frame.ForcedFlux[j])).ToArray();
frame.AdditionalMeasurements["Cloud aperture samples"]=indices.Length;
frame.AdditionalMeasurements["Cloud aperture regions"]=indices.Select(j=>Math.Clamp((int)(anchor.Stars[j].Y*4),0,3)*3+Math.Clamp((int)(anchor.Stars[j].X*3),0,2)).Distinct().Count();
frame.AdditionalMeasurements["Cloud aperture flux ratio"]=0;frame.AdditionalMeasurements["Cloud aperture flux deviation"]=0;
frame.AdditionalMeasurements[name+" samples"]=indices.Length;
frame.AdditionalMeasurements[name+" regions"]=indices.Select(j=>Math.Clamp((int)(anchor.Stars[j].Y*4),0,3)*3+Math.Clamp((int)(anchor.Stars[j].X*3),0,2)).Distinct().Count();
frame.AdditionalMeasurements[name+" flux ratio"]=0;frame.AdditionalMeasurements[name+" flux deviation"]=0;
if(indices.Length<20)return;
double Flux(Frame f)=>Robust.Median(indices.Where(j=>f.ForcedFlux.Length==normal.Length&&double.IsFinite(f.ForcedFlux[j])).Select(j=>f.ForcedFlux[j]/normal[j]));
var comparisons=Nearby(frames,clean,frame).Where(f=>f.ForcedFlux.Length==normal.Length&&indices.Count(j=>double.IsFinite(f.ForcedFlux[j]))>=indices.Length*.8).Select(Flux).ToArray();
@ -123,9 +137,28 @@ public static class SpatialReference
double uncertainty=1.253*Math.Sqrt(indices.Sum(j=>Math.Pow(LocalNoise(j)*Math.Sqrt(varianceFactor)/Math.Max(1,frame.Exposure)/normal[j],2)))/indices.Length;
double spread=Math.Max(baseline*.03,Math.Max(Robust.Mad(comparisons),uncertainty));
double measured=Flux(frame);
frame.AdditionalMeasurements["Cloud aperture flux ratio"]=measured/baseline;
frame.AdditionalMeasurements["Cloud aperture flux deviation"]=(baseline-measured)/spread;
frame.AdditionalMeasurements["Cloud aperture flux uncertainty"]=uncertainty/baseline;
frame.AdditionalMeasurements[name+" flux ratio"]=measured/baseline;
frame.AdditionalMeasurements[name+" flux deviation"]=(baseline-measured)/spread;
frame.AdditionalMeasurements[name+" flux uncertainty"]=uncertainty/baseline;
}
// Stars away from the texture share global transparency, seeing and airmass changes.
// Compare the inside/outside flux ratio with the same apertures in nearby clear frames.
static void MeasureDifferentialApertures(Frame frame,Frame anchor,Frame[] frames,int[] clean,double[] normal,bool[] inside,bool[] outside)
{
int[] Select(bool[] mask)=>Enumerable.Range(0,normal.Length).Where(j=>mask[Math.Clamp((int)(anchor.Stars[j].Y*SkyModel.Rows),0,SkyModel.Rows-1)*SkyModel.Columns+Math.Clamp((int)(anchor.Stars[j].X*SkyModel.Columns),0,SkyModel.Columns-1)]&&
normal[j]*anchor.Exposure>=30*anchor.Noise&&frame.ForcedFlux.Length==normal.Length&&double.IsFinite(frame.ForcedFlux[j])).ToArray();
int[] a=Select(inside),b=Select(outside);
frame.AdditionalMeasurements["Cloud texture differential samples"]=Math.Min(a.Length,b.Length);
frame.AdditionalMeasurements["Cloud texture differential flux ratio"]=0;frame.AdditionalMeasurements["Cloud texture differential flux deviation"]=0;
if(a.Length<20||b.Length<20)return;
double Flux(Frame f,int[] indices)=>Robust.Median(indices.Where(j=>double.IsFinite(f.ForcedFlux[j])).Select(j=>f.ForcedFlux[j]/normal[j]));
double Ratio(Frame f)=>Flux(f,a)/Math.Max(Flux(f,b),1e-9);
var comparisons=Nearby(frames,clean,frame).Where(f=>f.ForcedFlux.Length==normal.Length&&a.Count(j=>double.IsFinite(f.ForcedFlux[j]))>=a.Length*.8&&b.Count(j=>double.IsFinite(f.ForcedFlux[j]))>=b.Length*.8).Select(Ratio).ToArray();
double baseline=Robust.Median(comparisons);if(comparisons.Length<12||baseline<=0)return;
double measured=Ratio(frame),spread=Math.Max(baseline*.01,Robust.Mad(comparisons));
frame.AdditionalMeasurements["Cloud texture differential flux ratio"]=measured/baseline;
frame.AdditionalMeasurements["Cloud texture differential flux deviation"]=(baseline-measured)/spread;
frame.AdditionalMeasurements["Cloud texture differential flux scatter"]=spread/baseline;
}
static IEnumerable<Frame> Nearby(Frame[] frames,int[] clean,Frame selected)=>clean.Select(i=>frames[i]).Where(f=>f.Id!=selected.Id)
.OrderBy(f=>Math.Abs((f.Timestamp.GetValueOrDefault()-selected.Timestamp.GetValueOrDefault()).TotalSeconds)).Take(40);

View file

@ -16,7 +16,8 @@ static class QualityRegression
var opaque=Make(40);opaque.Stars=stars.Take(25).ToList();opaque.StarCount=25;
var broad=Make(41);var gradient=Make(42);var uncertain=Make(43);var cohort=Make(44);var trailing=Make(45);
var subtleShape=Make(46);var insufficient=Make(47);var noisyFlux=Make(48);
frames.AddRange([opaque,broad,gradient,uncertain,cohort,trailing,subtleShape,insufficient,noisyFlux]);
var thinFlagged=Make(49);var thinUndimmed=Make(50);var thinClean=Make(51);
frames.AddRange([opaque,broad,gradient,uncertain,cohort,trailing,subtleShape,insufficient,noisyFlux,thinFlagged,thinUndimmed,thinClean]);
foreach(var f in frames)
{
Matching.Measure(frames[0],f);f.Transparency=1;f.RegionFlux=Enumerable.Repeat(1.0,12).ToArray();f.RegionMatches=Enumerable.Repeat(8,12).ToArray();
@ -43,7 +44,25 @@ static class QualityRegression
trailing.AdditionalMeasurements["Matched shape change"]=.2;trailing.AdditionalMeasurements["Matched shape uncertainty"]=.01;
subtleShape.Ellipticity=.3;subtleShape.AdditionalMeasurements["Matched shape samples"]=100;
subtleShape.AdditionalMeasurements["Matched shape change"]=.12;subtleShape.AdditionalMeasurements["Matched shape uncertainty"]=.005;
void Texture(Frame f,double ratio,double deviation,bool flagged)
{
f.AdditionalMeasurements["Clear reference member"]=0;
f.AdditionalMeasurements["Cloud texture significance"]=20;f.AdditionalMeasurements["Cloud texture coverage"]=.15;
f.AdditionalMeasurements["Cloud texture connected coverage"]=.10;f.AdditionalMeasurements["Cloud texture differential samples"]=60;
f.AdditionalMeasurements["Cloud texture differential flux ratio"]=ratio;f.AdditionalMeasurements["Cloud texture differential flux deviation"]=deviation;
if(!flagged)return;
f.Ellipticity=.3;f.AdditionalMeasurements["Matched shape samples"]=100;
f.AdditionalMeasurements["Matched shape change"]=.2;f.AdditionalMeasurements["Matched shape uncertainty"]=.01;
}
Texture(thinFlagged,.96,4,true);Texture(thinUndimmed,1,.3,true);Texture(thinClean,.96,4,false);
Classification.Run(frames);
check(thinFlagged.Automatic==Verdict.Reject&&thinFlagged.Evidence.Any(e=>e.Metric=="Cloud texture"),"flagged frame with thin cloud texture that dims stars beneath it rejects");
check(thinUndimmed.Automatic==Verdict.Review,"thin texture without differential stellar dimming stays REVIEW");
check(thinClean.Automatic==Verdict.Keep,"thin cloud texture alone never rejects an otherwise clean KEEP");
double[] curved=Enumerable.Range(0,SkyModel.Count).Select(i=>{double x=(i%SkyModel.Columns+.5)/SkyModel.Columns-.5,y=(i/SkyModel.Columns+.5)/SkyModel.Rows-.5;return 1000+40*x-30*y+200*x*x-150*x*y+120*y*y;}).ToArray();
check(SkyModel.Detrend(curved,true).All(v=>Math.Abs(v)<.01)&&SkyModel.Detrend(curved).Max(Math.Abs)>5,"quadratic detrending removes smooth sky curvature that a plane leaves behind");
double[] spike=new double[SkyModel.Count];spike[10*SkyModel.Columns+10]=9;var smoothed=SkyModel.Smooth(spike);
check(Math.Abs(smoothed[10*SkyModel.Columns+10]-1)<1e-9&&Math.Abs(smoothed[11*SkyModel.Columns+11]-1)<1e-9&&smoothed[12*SkyModel.Columns+12]==0,"texture smoothing averages each cell with its 3x3 neighbors");
check(opaque.Automatic==Verdict.Reject,"severe cloud with accurate geometry rejects despite missing reference stars");
check(broad.Automatic==Verdict.Reject,"broad coherent sky structure with local aperture attenuation rejects");
check(gradient.Automatic==Verdict.Review,"uncorroborated sky structure stays REVIEW rather than rejecting a possible gradient");