Feature attribution (FA) methods are common post-hoc approaches that explain how Large Language Models (LLMs) make predictions. Generating faithful attributions that reflect the actual inner behavior of the model is crucial. We introduce Noiser, a perturbation-based FA method that imposes bounded noise on each input embedding and measures the robustness of the model against partially noised input to obtain input attributions.
We also propose an answerability metric that employs an instructed judge model to assess the extent to which highly scored tokens suffice to recover the predicted output. Through comprehensive evaluation across six LLMs and three tasks, we demonstrate that Noiser consistently outperforms existing gradient-based, attention-based, and perturbation-based FA methods.