Prompt Injections for Defense

Researchers from Tracebit discovered that adding prompt injections to sensitive information stored on Amazon Web Services can effectively stop attacks from AI hacking agents. These prompts instruct the attacking language model to perform a forbidden action, triggering its safety barriers and causing it to shut down. The prompts work by directing the language model to take harmful actions that are prevented by its guardrails. Examples of such prompts include those that order the language model to provide steps for developing harmful substances. This approach can be used to protect passwords, cryptographic keys, and other secrets. The technique exploits the language model's own safety mechanisms to prevent malicious activity.