Envenenamento de Contexto is a type of ataque adversarial targeting inteligência artificial (AI) models, particularly in the realm of processamento de linguagem natural and machine learning. This technique involves intentionally introducing misleading or harmful information into the context that an AI model uses to make predictions or generate responses. By altering the contextual information, attackers can influence the outputs of the model, leading to biased, incorrect, or harmful results.
The process typically entails providing the AI system with inputs that are skewed or false, thereby poisoning the context it relies on to interpret queries or make decisions. For instance, if a chatbot is designed to respond to user inquiries based on previous interactions, injecting false data into these interactions can alter how the chatbot understands future queries.
Context poisoning poses significant risks, especially in applications where AI is used for decision-making, such as in finance, healthcare, or aplicação da lei. By compromising the integrity of the contextual information, malicious actors can manipulate outcomes, leading to biased decisions that may reinforce stereotypes or other forms of discrimination.
To mitigate the risks associated with context poisoning, AI developers and researchers are exploring various defense mechanisms, including robust treinamento that can help models resist such attacks, as well as continual monitoring of AI outputs for signs of manipulation.