Fix layer-skip guiders silently inert when CFG is disabled#23
Draft
cursor[bot] wants to merge 1 commit into
Draft
Fix layer-skip guiders silently inert when CFG is disabled#23cursor[bot] wants to merge 1 commit into
cursor[bot] wants to merge 1 commit into
Conversation
When SkipLayerGuidance, PerturbedAttentionGuidance, or SmoothedEnergyGuidance run without CFG (guidance_scale=1.0), the skip/SEG pass is batch 2. Hook installation gated on is_conditional only matched batches 1 and 3, so skip hooks were never applied and guidance silently had no effect. Add _should_apply_skip_hooks/_should_apply_seg_hooks to install hooks on batch 2 when CFG is off and batch 3 when CFG is on. Also increment _count_prepared in SmoothedEnergyGuidance.prepare_models. Co-authored-by: Simon Lynch <srlynch1@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a critical correctness bug where Skip Layer Guidance, Perturbed Attention Guidance, and Smoothed Energy Guidance silently had no effect when used without classifier-free guidance.
Bug and impact
Impact: Using SLG, PAG, or SEG with
guidance_scale=1.0(CFG off) produces identical outputs to running without guidance — the skip/SEG hooks are never installed, sopred_cond - pred_cond_skipis always zero.Root cause: Hook installation used
is_conditional and _count_prepared > 1.is_conditionalis only true for batches 1 and 3 (cond reference and skip pass when CFG is on). When CFG is off, the skip pass is batch 2, whereis_conditionalis false, so hooks never attach.Trigger:
SkipLayerGuidance(guidance_scale=1.0, skip_layer_guidance_scale=2.8, skip_layer_guidance_layers=[7])on any modular pipeline — SLG appears enabled but has zero effect.Fix
_should_apply_skip_hooks()/_should_apply_seg_hooks()to install hooks on batch 2 when CFG is off and batch 3 when CFG is on.PerturbedAttentionGuidancevia copied blocks._count_prepared += 1inSmoothedEnergyGuidance.prepare_models.Validation
pytest tests/guiders/test_skip_layer_guidance.py— 2 passedExisting open PRs (unchanged)