17. Frameworks and Managed Retrieval

Compare hosted retrieval with custom RAG, use frameworks carefully, and plan provider migrations.

Managed vs. Custom RAG

Managed retrieval can handle file upload, chunking, embeddings, indexing, and search, reducing infrastructure code. Custom RAG gives more control over parsers, chunking, hybrid ranking, filters, and observability. Compare file support, filters, deletion semantics, citation metadata, latency, cost, data handling, and portability.

Even when retrieval is hosted, your application still owns authentication, authorization, prompt policy, output validation, and evaluation.

Implementation example

Managed: faster setup, fewer components, service-specific controls
Custom: more control, more engineering and operations
Both: governance, permissions, evaluations, monitoring

💡 Compare end-to-end requirements, not only quickstart length.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What does managed retrieval not guarantee?

Hosted File Search and Vector Stores

Hosted file search can provide semantic and keyword retrieval over uploaded files and vector stores, handling some ingestion and retrieval work. Check current file support, filters, chunking behavior, citation output, deletion, pricing, and data terms. Pass only vector stores the authenticated user is authorized to access; keep API keys server-side.

Implementation example

const response = await client.responses.create({
 model: configuredModel,
 input: question,
 tools: [{ type:"file_search", vector_store_ids:[authorizedStoreId] }]
});

💡 This follows the documented Responses API pattern; confirm current SDK syntax and citation parsing.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What must the application control with hosted file search?

Framework Abstractions

Frameworks can provide loaders, splitters, retrievers, vector-store adapters, prompt templates, tracing, and evaluation integrations. They speed experimentation but may hide defaults or introduce dependency churn. Keep domain policy and ACL logic outside framework chains where practical. Pin versions and test the behavior of filters, retries, citations, and cancellation.

Implementation example

Application policy -> loaders/parsers -> chunker
 -> retriever/reranker -> prompt -> model adapter
 -> citation validation

💡 Abstraction is useful when it reduces maintenance without hiding critical behavior.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

What is a framework abstraction risk?

Provider Portability and Migration

Provider changes may affect embeddings, structured outputs, tool semantics, file search, citation parsing, and data retention. A migration can require re-embedding and rebuilding indexes. Use adapters, document behavior differences, run evaluations, test permissions, and use shadow/canary rollout with rollback where warranted.

Implementation example

Migration:
1. Compare capabilities and terms
2. Map API differences
3. Re-embed/reindex if needed
4. Validate filters/citations
5. Run evals
6. Canary and monitor

💡 Provider migration is a controlled systems change, not only a model-name edit.

Key Takeaways

Inspect the relevant pipeline stage instead of guessing.
Preserve source provenance, versions, and access scope.
Measure quality on representative queries.

Knowledge Check

Why might a provider change require reindexing?

Managed vs. Custom RAG

Hosted File Search and Vector Stores

Framework Abstractions

Provider Portability and Migration

STAY UPDATED WITH BYTES!