Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons

Microsoft researchers have demonstrated that an AI model can be compromised by introducing a hidden backdoor through modifications to fewer than 10 of its rarely used neurons. This backdoor can be activated when a specific trigger is encountered, causing the model to output a command chosen by the attacker. The compromised model would then execute this command using the permissions of the agent that utilizes the model. To achieve this, the attacker must have the ability to alter the model's weights, and this can be done without requiring the model to be retrained. The researchers tested this method to verify its effectiveness.