Check yourself: RAG and residency

Questions and answers, readable without JavaScript.

1. When does a local chat UI fail the residency claim?

   • When the model has more than one billion parameters
   • When the embedding hop or the vector index leaves the building
   • When you read a model card
   • When the answer is shorter than the prompt
   Answer 1

   When the embedding hop or the vector index leaves the building

   RAG embeds documents, stores vectors, and retrieves chunks. If either the embedding step or the index leaves, the familiar local window still leaks the corpus.

   Read the note → · Local Chat Is Not Data Residency: RAG, Embeddings, and Vector Stores

2. What does the note say you must keep the same between indexing and querying?

   • The chat temperature
   • The OpenAI organisation id
   • The chunk overlap, which must be zero
   • The embedding model. Swapping it mid-corpus silently breaks nearest-neighbour geometry
   Answer 2

   The embedding model. Swapping it mid-corpus silently breaks nearest-neighbour geometry

   Ollama's production rule, as the note cites it: use the same embedding model for indexing and querying. Pin the artefact you indexed with.

   Read the note → · Local Chat Is Not Data Residency: RAG, Embeddings, and Vector Stores

3. Does an OpenAI-shaped /v1/embeddings route prove the data stayed local?

   • No. The same SDK pointed at a hosted base URL exports the knowledge base. The path is a wire format, not a trust boundary
   • Yes. That path is defined to be loopback
   • Yes, if the JSON field is named input
   • Only when a Bearer token is set
   Answer 3

   No. The same SDK pointed at a hosted base URL exports the knowledge base. The path is a wire format, not a trust boundary

   llama.cpp and hosted APIs can share the shape. Point base_url at 127.0.0.1 and bytes can stay local. Point the same client at a hosted URL and they leave. "/v1/embeddings works" is wiring, not a security property.

   Read the note → · Local Chat Is Not Data Residency: RAG, Embeddings, and Vector Stores