What's the difference between AI red teaming and traditional penetration testing?
Traditional penetration testing looks for exploitable weaknesses in conventional software and infrastructure. AI red teaming specifically targets model behavior, such as how a model can be manipulated through inputs like prompt injection and jailbreaks, what it might leak from training data, and how autonomous agents might be misused, risks that do not exist in traditional software in the same form.