Private inference has more than one data boundary

Be precise about what private means

A team connects its application to a model service through a private endpoint. The network route is controlled, but a debugging setting also copies complete prompts into a log store. Another component sends documents to an embedding service through a separate endpoint. Calling the whole application "private" without describing those paths conceals important differences.

Separate network access, processing location, retention, human access and permitted use of data. A control in one category does not automatically establish the others. For example, a private route does not by itself decide how long application logs retain a prompt.

Begin with the information the task requires. An assistant answering a policy question may need a few relevant passages, not an entire employee record. Reducing the payload is often a useful first control regardless of hosting choice.

Follow one request through every component

Map the browser or client, application server, retrieval service, embedding service, model endpoint and response store. Add logs, evaluation tools, caches and support systems that receive copies. Include error and fallback paths, which can differ from the normal route.

For each destination, record the fields received, identity used, configured location where applicable and retention rule. Keep uncertainty visible. If a provider setting or agreement has not been checked, mark it as unresolved rather than filling the diagram with an assumed guarantee.

This simplified flow is a review aid. A real system may have several model calls and stores, so its inventory should reflect the implementation rather than forcing every dependency into four boxes.

Verify the network and identity controls separately

A private endpoint can provide a private route to a supported service, but the application still needs appropriate identity and resource permissions. In Amazon Bedrock, for example, endpoint policies and the caller's permissions are part of controlling access through the endpoint. Review the current service documentation and actual deployed policy rather than assuming endpoint creation establishes least privilege.

Test from the runtime that makes the request. DNS resolution, proxy settings and fallback configuration can make its path differ from an engineer's laptop. A successful private-path test should be paired with a test that an unintended route or identity is denied where that is the requirement.

Account for the operational copies

Inspect model invocation logging, application traces and evaluation exports. Some configurations can retain full inputs and outputs. Decide whether those contents are necessary, who may inspect them and how long they remain.

Provider-side processing and support access need review against current documentation and the applicable agreement. Avoid substituting a broad product claim for the terms and configuration of the service actually used.