Chapter•60 Minutes

Items per page

12. RAG Security, Privacy, and Prompt Injection

Protect tenant data, treat retrieved content as untrusted, secure uploads, and manage retention.

Threat Model a RAG System

Assets include source documents, ACLs, credentials, conversation history, indexes, and tools. Threats include unauthorized retrieval, prompt injection, poisoned sources, stale permissions, SSRF, unsafe rendering, cost abuse, and sensitive logging. Map each threat to a control and a test.

The model should not decide whether a user may access a document. Trusted application policy must govern access before content enters context.

Diagram

Implementation example

Upload -> validate -> parse safely -> index
Query -> authenticate -> ACL filter -> retrieve
Context -> model -> validate citations -> safe render

💡 Each boundary needs explicit controls and negative tests.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What should determine whether a chunk can be returned?

Indirect Prompt Injection

Retrieved pages can contain instructions to ignore policy or reveal private records. Treat retrieved text as untrusted evidence, not application instructions. Delimiters help but are not complete defenses. Strong controls include least-privilege tools, scoped data access, minimized secrets in context, argument validation, network restrictions, and approval for sensitive actions.

Test hostile documents that attempt to reveal prompts, cross tenants, or trigger tools. The model should not gain new permissions from a document.

Implementation example

const allowedTools = new Map([
 ["searchKnowledgeBase", searchKnowledgeBase],
 ["getCurrentUserProfile", getCurrentUserProfile]
]);
async function executeTool(name,args,ctx) {
 const tool=allowedTools.get(name);
 if(!tool) throw new Error("Tool not allowed");
 return tool(args,ctx);
}

💡 An allowlist limits available functions, but each tool must also validate arguments and authorize access.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What is a strong defense against injected cross-tenant access?

Source Poisoning and Trust

Poisoned documents may contain false information or misleading metadata. Verify source identity, ownership, publication status, and provenance. Keep model-generated summaries untrusted unless reviewed. Monitor unusual changes and preserve source lineage. For critical corpora, use approval workflows and rollback.

Implementation example

{
 "sourceId":"policy-v4",
 "origin":"approved-policy-repository",
 "ownerVerified":true,
 "publicationStatus":"approved",
 "contentHash":"sha256:..."
}

💡 Trust metadata must come from trusted systems and processes, not model claims.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What is a key defense against poisoned sources?

Privacy, Retention, and Deletion

Sensitive content can be copied into normalized text, chunks, vectors, caches, logs, and evaluation datasets. Define data minimization, retention, access controls, and deletion propagation. Embeddings should not be assumed anonymous. When source access is revoked, remove or block derived chunks and invalidate related caches according to policy.

Review data handling terms for the exact provider and configuration rather than assuming a universal policy.

Implementation example

Source deleted -> block serving -> remove chunks/vectors
 -> invalidate caches -> update derived records -> audit completion

💡 Deletion must cover derived copies and caches, not only original files.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

Why include caches and vectors in deletion workflows?

Upload Safety, SSRF, and Citation Rendering

Validate file size, type, signatures, archive expansion, and parser behavior; isolate risky parsers. If users submit URLs, block loopback/private/link-local/metadata destinations, restrict schemes and redirects, and revalidate resolved hosts to defend against SSRF. Render plain model text as text and sanitize permitted rich content. Validate citation URLs from trusted metadata; never execute extracted code.

Note

Never fetch arbitrary model-suggested URLs from a privileged backend without network and destination controls.

Implementation example

function validateUrl(raw) {
 const url=new URL(raw);
 if(!["https:","http:"].includes(url.protocol)) throw Error("Bad scheme");
 if(isPrivateOrBlockedHost(url.hostname)) throw Error("Blocked host");
 return url;
}

💡 A real SSRF defense must handle DNS resolution, redirects, IP literals, and rebinding.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What is SSRF protection intended to prevent?

Threat Model a RAG System

Indirect Prompt Injection

Source Poisoning and Trust

Privacy, Retention, and Deletion

Upload Safety, SSRF, and Citation Rendering

STAY UPDATED WITH BYTES!