Prompt Injection and Data Exfiltration

Prompt injection attempts to make the model disregard policy, reveal protected information, or misuse tools. It can appear in user messages, retrieved documents, emails, web pages, code comments, and images. Treat retrieved text as untrusted evidence. Never place secrets in context unless absolutely necessary; if the model can see a secret, prompt wording cannot reliably guarantee it will never be repeated.

Limit tool capabilities and data scope. A search tool should enforce the current user's permissions; an email tool should not send messages without appropriate confirmation; a database tool should expose a small, validated query interface rather than arbitrary SQL. Add adversarial tests that try to cross tenant boundaries or trigger unauthorized actions.

Tool allowlist

const allowedTools = new Map([
  ["searchKnowledgeBase", searchKnowledgeBase],
  ["getCurrentUserProfile", getCurrentUserProfile]
]);

async function executeTool(name, args, context) {
  const tool = allowedTools.get(name);
  if (!tool) throw new Error("Tool not allowed");
  return tool(args, context);
}

💡 The allowlist limits the available functions but is not enough by itself: each tool must validate arguments, enforce object-level authorization, and limit returned data.

Key Takeaways

Assume any external content may contain hostile instructions.
Do not expose secrets unnecessarily to the model.
Enforce tenant authorization inside data access tools.

Knowledge Check

What is the safest response to a retrieved document instructing the model to reveal another tenant's records?

Prompt Injection and Data Exfiltration

STAY UPDATED WITH BYTES!