Skip to content

Add conflict-resolution modes (norm-scaler, gram-schmidt) to lora add and lora stack#5

Open
IsraelBenDavid wants to merge 1 commit into
matanby:mainfrom
IsraelBenDavid:feature/lora-fusion-methods
Open

Add conflict-resolution modes (norm-scaler, gram-schmidt) to lora add and lora stack#5
IsraelBenDavid wants to merge 1 commit into
matanby:mainfrom
IsraelBenDavid:feature/lora-fusion-methods

Conversation

@IsraelBenDavid

Copy link
Copy Markdown

What

Adds a --mode option to both sft lora add and sft lora stack for resolving
conflict between adapters before they're combined. Two strategies:

  • norm-scaler — rescale the other adapter's delta so its Frobenius norm
    matches the reference's (‖ΔW_other‖_F == ‖ΔW_ref‖_F), so neither update
    dominates purely by magnitude.
  • gram-schmidt — subtract the reference-aligned component of the other
    delta (ΔW_other − c·ΔW_ref, c = ⟨ΔW_ref,ΔW_other⟩/⟨ΔW_ref,ΔW_ref⟩), so the
    added adapter only contributes directions the reference doesn't already cover.

--mode none (default) preserves the existing behavior.

Why

Naively summing LoRA deltas lets one adapter overpower another by magnitude, or
double-counts overlapping directions. These transforms give a principled way to
combine adapters that "fight" each other.

How

  • New module conflict.py holds the math:
    norm_scale_factor, gram_schmidt_orthogonalize, the factored Frobenius inner
    product frob_inner_factored, and validate_mode.
  • add (lossy): reconstructs each module's delta, transforms every
    non-reference delta against adapter 0, then does the usual weighted sum + SVD
    re-decomposition.
  • stack (lossless): because both transforms are linear, they fold into the
    concatenated factors as a per-module coefficient tweak — the merge stays exact
    at rank r_a + r_b. Scalars are computed in rank space, so the full out×in
    delta is never materialized.
  • The reference adapter is the first file for add and file A for
    stack. The chosen mode is recorded in output metadata (conflict_mode).

Tests

New unit + CLI coverage in tests/test_lora_add.py and
tests/test_lora_stack.py: the math primitives,
zero-delta / single-side no-ops, that stack modes stay lossless and produce the
expected norm/orthogonality, metadata recording, invalid-mode rejection, and the
CLI wiring for both commands.

Docs

Updated SKILL.md and REFERENCE.md with the new option and examples, plus a
note clarifying add is lossy (fixed output rank) vs stack being lossless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant