We investigate how large language models (LLMs) select and utilise knowledge when generating responses. Our analysis reveals that LLMs exhibit systematic biases in knowledge selection, often favouring certain types of information over others regardless of relevance or accuracy.

Through controlled experiments using knowledge-steering techniques, we demonstrate that it’s possible to influence LLMs’ knowledge selection behaviours. We introduce novel methods for steering models towards more balanced and contextually appropriate knowledge utilisation, significantly improving response quality and factual accuracy.

Our findings have important implications for developing more reliable and controllable language models, particularly in knowledge-intensive applications where accurate information retrieval and utilisation are critical.