Reducing unsafe responses in large language models ^{CEFR B2}

Large language models are widely used to give advice and instructions, which makes safe responses essential. A team at North Carolina State University analyzed safety alignment in LLMs and tested practical training methods aimed at reducing unsafe outputs without sacrificing model performance.

The researchers identified two central issues. First, safety training can lower a model's accuracy, a drawback they call the "alignment tax." Second, many models use a superficial safety check that determines safety early and acts on a binary safe/unsafe signal. Jianwei Li illustrated how the same harmful instructions might be refused or accepted depending only on how the user frames their motive.

To explain these patterns the team proposed the Superficial Safety Alignment Hypothesis (SSAH). They searched models to locate safety-critical neural components and found specific parts that influence whether a request is fulfilled or refused. By "freezing" those components during fine-tuning, a model can learn new domain tasks while keeping its original safety behavior, which reduces the alignment tax.

The authors say this work gives both a conceptual framework and a practical technique, and they stress the need for methods that let models re-evaluate safety throughout response generation. The research will be presented at ICLR2026 and related code and information are available online.