Add conflict-resolution modes (norm-scaler, gram-schmidt) to lora add and lora stack#5
Open
IsraelBenDavid wants to merge 1 commit into
Open
Conversation
…robenius norm rescale
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
--modeoption to bothsft lora addandsft lora stackfor resolvingconflict between adapters before they're combined. Two strategies:
norm-scaler— rescale the other adapter's delta so its Frobenius normmatches the reference's (
‖ΔW_other‖_F == ‖ΔW_ref‖_F), so neither updatedominates purely by magnitude.
gram-schmidt— subtract the reference-aligned component of the otherdelta (
ΔW_other − c·ΔW_ref,c = ⟨ΔW_ref,ΔW_other⟩/⟨ΔW_ref,ΔW_ref⟩), so theadded adapter only contributes directions the reference doesn't already cover.
--mode none(default) preserves the existing behavior.Why
Naively summing LoRA deltas lets one adapter overpower another by magnitude, or
double-counts overlapping directions. These transforms give a principled way to
combine adapters that "fight" each other.
How
norm_scale_factor,gram_schmidt_orthogonalize, the factored Frobenius innerproduct
frob_inner_factored, andvalidate_mode.add(lossy): reconstructs each module's delta, transforms everynon-reference delta against adapter 0, then does the usual weighted sum + SVD
re-decomposition.
stack(lossless): because both transforms are linear, they fold into theconcatenated factors as a per-module coefficient tweak — the merge stays exact
at rank
r_a + r_b. Scalars are computed in rank space, so the full out×indelta is never materialized.
addand file A forstack. The chosen mode is recorded in output metadata (conflict_mode).Tests
New unit + CLI coverage in tests/test_lora_add.py and
tests/test_lora_stack.py: the math primitives,
zero-delta / single-side no-ops, that
stackmodes stay lossless and produce theexpected norm/orthogonality, metadata recording, invalid-mode rejection, and the
CLI wiring for both commands.
Docs
Updated
SKILL.mdandREFERENCE.mdwith the new option and examples, plus anote clarifying
addis lossy (fixed output rank) vsstackbeing lossless.