Home / Question bank / RAG & Vector Search

RAG & Vector SearchSeniorQuestion 7 of 12

How do you enforce document permissions in RAG?

30-second answer

Store ACL metadata (tenant, groups, document IDs) with each chunk and filter at retrieval time using the caller's identity, never relying on the LLM to withhold information. Use a vector store that supports efficient filtered search, keep ACLs in sync with the source system, and test for leakage in evals. Per-tenant indexes are an option for strong isolation.

Try answering out loud before reading on. The walkthrough below is paced for a 40-minute interview round.

Architecture diagram

Sketch this on the whiteboard as you talk through the answer. See all AI architecture diagrams →

Scroll sideways to see the whole diagram.

Permission-aware retrievalComponents: Identity provider (user → groups); Employee; RAG API (resolve caller groups); Filtered search (acl ∩ user groups); Permission re-check (sensitive docs only); LLM (permitted chunks only); Index (chunks + ACL metadata); Audit log (who retrieved what); Source systems (SharePoint · Jira); ACL sync (events + nightly reconcile). Flow: Employee to RAG API; RAG API to Identity provider (groups); RAG API to Filtered search; Filtered search to Index; Filtered search to Permission re-check; Permission re-check to LLM; Permission re-check to Audit log (log); Source systems to ACL sync; ACL sync to Index (update ACLs).ACL SYNCIdentity provideruser → groupsEmployeeRAG APIresolve caller groupsFiltered searchacl ∩ user groupsPermission re-checksensitive docs onlyLLMpermitted chunks onlyIndexchunks + ACL metadataAudit logwho retrieved whatSource systemsSharePoint · JiraACL syncevents + nightlyreconcilegroupslogupdate ACLs
Permission-aware retrieval. The caller's groups are resolved from the identity provider and applied as a filter inside the search, so the model only ever sees permitted chunks. ACLs are kept in sync from source systems.

0–5 min Clarify and frame

Ask about the permission model: per-document ACLs, groups, tenants, row-level rules? How quickly must permission changes take effect? Framing: "The core principle is that the model must never see content the user isn't allowed to read, so enforcement happens at retrieval, not in the prompt."

5–15 min Core explanation

Why prompts can't enforce access: the model can be tricked by prompt injection or simply summarize forbidden content. Anything in the context window can leak.

Enforcement design:

  1. At ingestion, attach ACL metadata to every chunk: tenant_id, allowed_groups, allowed_users, classification.
  2. At query time, resolve the caller's identity and groups from the auth system.
  3. Apply a filter in the search query so only permitted chunks are candidates.
  4. Never cache answers across users with different permissions.

15–25 min Deep dive: implementation choices

Approach Pros Cons
Metadata filtering in a shared index Simple, efficient storage Must ensure filter-aware ANN; very selective filters can hurt recall/latency
Index per tenant Strong isolation, easy deletion Many small indexes; overhead
Post-filtering Easy May return too few results; dangerous if filter is skipped
Query-time permission check against source system Always current Adds latency; good as a second check

Keeping ACLs in sync:

  • Subscribe to permission-change events from source systems; update chunk metadata promptly.
  • Periodic reconciliation jobs to catch drift.
  • For high sensitivity, re-check permissions on final chunks against the source API before generation.

Group explosion: users can be in thousands of groups; use group-to-doc mappings or bitsets rather than huge OR-lists.

25–32 min Testing and risk

  • Automated leakage tests: for synthetic users with limited access, run queries targeting restricted docs and assert they never appear in retrieved IDs or answers.
  • Audit logs of which documents were retrieved for which user.
  • Handle derived content: summaries or embeddings of restricted docs carry the same ACL.
  • Conversation memory and semantic caches must be scoped per user.

32–37 min Production example

An enterprise search assistant indexed SharePoint and Jira. Chunks carried the source's group IDs; queries passed the user's resolved groups as a filter. A nightly job compared index ACLs with source ACLs. Before launch, a red-team suite tried 200 queries designed to surface HR and finance documents from a low-privilege account; zero restricted chunks were retrieved, and the suite now runs on every release.

37–40 min Likely follow-ups

  • What about fine-tuning on private docs? Weights can't enforce ACLs; avoid putting restricted content into shared fine-tunes.
  • Revocation latency? Define an SLA; event-driven updates plus source re-check for sensitive classes.
  • Multi-tenant SaaS? Tenant ID filter at minimum, often separate indexes or namespaces per tenant.

What interviewers listen for

  • "Filter before the model sees it" as the principle.
  • Keeping ACLs in sync and testing for leakage.

Red flags

  • "Tell the model not to reveal restricted documents."