Backdooring Open-Weight AI Models

A security issue exists with open AI models that have had their weights edited, as they can potentially carry a backdoor. This vulnerability can be introduced through various methods, including task-specific fine-tuning, merged adapters, or abliterated builds. Abliteration is a particular concern, as it involves a simple fine-tuning process that removes a model's ability to refuse certain requests. This is done by identifying and subtracting the direction in the weights that corresponds to a "no" response, effectively overriding the model's safety features. Abliteration originated as a research technique but has since become a common practice before models are uploaded. As a result, modified weights are often downloaded without being verified for potential backdoors.