Interference-Aware Composition of Steering Vectors

Steering vectors are a flexible mechanism for controlling large language models (LLMs) along interpretable behaviours. Multi-behaviour steering, however, remains challenging due to interference. Individual steering vectors have conflicting directions or override one another, which not only impedes controllability but also often degrades generation results. In this work, we first analyse how multi-behaviour compositions of steering vectors affect model outputs, characterising how interactions between steering directions result in non-additive, position-dependent distributional shifts in generation. We then introduce an interference-aware allocation (IAA) method for multi-behaviour steering that minimises the cross-behaviour interference by dynamically adjusting the magnitudes of steering vectors at each generation step. Without instruction prompting, our method ties the best average on Sycophancy (76.4), achieves the best average on Refusal (80.8), and is far more fluent on Myopic-reward than the strongest baseline (67.1 vs. 41.7). With full behavioural instructions, it improves the all-prompt average from 88.2/94.8/81.2 to 95.7/98.7/95.6 while maintaining high fluency (87.3/90.5/87.7).