ngmix: give the PSF observation a finite weight map (#749)#779
Open
cailmdaley wants to merge 2 commits into
Open
ngmix: give the PSF observation a finite weight map (#749)#779cailmdaley wants to merge 2 commits into
cailmdaley wants to merge 2 commits into
Conversation
e06497d to
339fa5a
Compare
make_ngmix_observation built psf_obs weightless, so ngmix defaulted to unit weight on the unit-flux PSF stamp and the galaxy GPriorBA(0.4) prior handed to the diagnostic PSF fitter swamped the likelihood. The recovered PSF shape collapsed toward the prior (zero ellipticity), corrupting the *_psf_orig diagnostic columns (#749), and the small finite-weight noise budget present in the pre-v2.0 esheldon/aguinot code was lost in the rewrite (the now-retired #774 — the same bug from the noise-budget angle). Build psf_obs with a flat weight = 1/PSF_NOISE**2 (PSF_NOISE = 1e-5, the esheldon/aguinot value Axel's reproduction used), forced float for parity with the galaxy weight — the PSF analogue of the galaxy weight built just below. Weight only; PSFEx/MCCD models already carry a little noise, so no explicit stamp-noise injection is needed (Axel's guidance on #749). The exact value is non-critical: validated flat across 1e-4..1e-6 on the twin. Calibration is untouched: metacal fits this same psf_obs with a prior-free AdmomFitter, which is insensitive to a flat weight's absolute scale, so the calibrated shear is invariant (verified bit-for-bit on the twin). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KQn6mXoL4XMgscZDsaTkB
Three assertions in test_ngmix_weight_validation.py, on the real production path now that this sits on the column grammar (#761): - structural: make_ngmix_observation builds psf_obs with the finite 1/PSF_NOISE**2 weight, never unit weight; - recovery (#749 done-condition): through do_ngmix_metacal / average_original_psf, the *_psf_orig column recovers the injected PSF moment ellipticity to 1e-3 with the weight, and collapses below 10% without it (the contrast that proves the weight is load-bearing for the diagnostic columns); - invariance: every metacal-type shear is bit-for-bit identical between the unit-weight (pre-fix) and shipped PSF_NOISE builds (metacal's prior-free AdmomFitter is insensitive to the flat weight's scale). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KQn6mXoL4XMgscZDsaTkB
339fa5a to
6d30fc1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #749
What
make_ngmix_observationbuilt the PSF observation weightless(
psf_obs = Observation(psf, jacobian=psf_jacob)), while the galaxyobservation right below it carries a full weight map. With no weight, ngmix
defaults to unit weight on the unit-flux PSF stamp, so the galaxy
GPriorBA(0.4)prior handed to the PSF fitter swamps the (tiny) likelihood andthe recovered PSF shape collapses toward the prior (zero ellipticity).
This PR gives the PSF observation a finite flat weight map
psf_wt = 1/PSF_NOISE**2(PSF_NOISE = 1e-5, the esheldon/aguinot valueAxel's #749 reproduction used) — the PSF analogue of the galaxy weight built
just below. Weight only: PSFEx/MCCD models already carry a little noise, so no
explicit stamp-noise injection is added (Axel's guidance on #749). The galaxy
prior is left in place (option B from the thread) — once a real weight is
present the likelihood reasserts and the prior no longer bites.
Closes #749
One root, two issues
The single weightless line was the root of both #749 (the galaxy shape prior
prior-dominates the PSF fit, collapsing the recovered ellipticity) and the now
-retired #774 (the small finite-weight noise budget of the pre-v2.0
esheldon/aguinot code was dropped in the rewrite). They are the same bug from
two angles; the finite
1/psf_noise**2weight is the substance of the oldnoise-injection, so adding the weight resolves both. #774 carried no separate
deliverable and has been retired as redundant.
Evidence — end to end through
do_ngmix_metacal/average_original_psfA sheared-PSF stamp (
psf_shear = (0.05, -0.03), HSM moment truthg = [+0.0477, -0.0286]) pushed through the real production path; the*_psf_origcolumn is the original-PSF fit:g_psf_orig[+0.0001, -0.0001][+0.0477, -0.0286]g_psf_reconv ≈ 0in both builds — the reconvolution kernel is round byconstruction, independent of this weight (and is the column the pre-#761
g1_psfo_ngmixname was mis-sourcing).(
noshear/1p/1m/2p/2m) is bit-for-bit identical (Δ = 0) between theunit-weight (pre-fix) and weighted builds, and the fit is flat across
PSF_NOISE ∈ {1e-4, 1e-5, 1e-6}. Metacal fits this samepsf_obswith aprior-free
AdmomFitter(MetacalFitGaussPSF._do_psf_fit), which isinsensitive to a flat weight's absolute scale — so the diagnostic weight
never reaches calibration. It is the galaxy prior on the diagnostic fit
(
average_original_psf), not weight-independence, that makes*_psf_origsensitive while the calibration is not.
PSF-stamp noise (σ = 1e-6, 1e-5) on top of the weight moves
g_psf,T_psf,and
Rby ≲ 1e-6 — explicit injection is justifiably dropped.The faint-end reduced-χ² / star-response coupling #774 also speculated about is
tracked separately under the χ²-anomaly investigation (#769), not here.
Tests
tests/module/test_ngmix_weight_validation.pygains three guards: a structuralcheck that
psf_obscarries the finite weight (never unit weight), the #749end-to-end recovery contrast through
average_original_psf(weighted recoversto 1e-3, weightless collapses below 10%), and the bit-for-bit metacal-shear
invariance.
— Claude on behalf of Cail