What you need to know
- Map the link between untrusted input and available actions.
- Enforce user authorization and destination rules outside the model.
- Test both attempted misuse and application enforcement.
Treat retrieved material as untrusted input
A document can contain useful facts and a malicious instruction in the same paragraph. If an assistant reads that document while preparing a report, the instruction may attempt to redirect its behavior or expose information. OWASP identifies prompt injection as a risk in LLM applications, including cases involving external material rather than a direct user message.
A clear system instruction helps describe intended behavior, but it does not replace application controls. Map the sources an assistant reads, the tools it can call, and the data each tool can access. This makes the potential impact of an unsuccessful boundary visible before a feature reaches users.
Evaluate attacks along real paths
Test malicious instructions in the formats the system actually consumes: documents, retrieved pages, tool responses, and images if supported. Check whether the model attempts the unwanted action and whether the application blocks it. Keep the two findings separate. A blocked attempt can reveal a model weakness while also confirming a useful control. Retest after changes to tools, permissions, retrieval sources, or model behavior, because each can alter the path an attacker might exploit.
Sources & further reading
This article draws on the source material and official product documentation below. Product capabilities can change; check each source for current details.

