Bottom line: “Runs locally” describes where at least part of a model workload runs; it does not, by itself, prove that every prompt, file, update check, telemetry event, or tool call stays on your device. If privacy matters, map the entire path: model inference, app account, extensions, tools, downloads, and network connections.
This is easy to miss because people use “local AI” as shorthand for several different setups. One app may download model weights and answer offline. Another may run a small model on-device but send harder questions to a cloud service. A desktop assistant may keep inference local while a browser-search tool sends a query to the web. These can all be marketed as local-first experiences, but they have different data boundaries.
Separate the questions before choosing a tool
| Question | What to verify | Why it matters |
|---|---|---|
| Where does inference happen? | On-device, on your own server, or a provider’s cloud | Identifies who processes the prompt |
| What can the app read? | Selected files, a workspace, browser data, or broad folders | Local processing can still expose local files to the app |
| Does it call online services? | Search, plug-ins, crash reporting, account sync, model fallback | Those features can transmit data separately from inference |
| What leaves the device? | Prompts, snippets, logs, identifiers, or diagnostics | Privacy depends on the actual data flow, not the label |
| What is retained? | Local history, account history, logs, provider retention | Processing location and retention are different issues |
Four common “local” configurations
1. A downloaded model with no network-dependent features
This is the clearest offline case: the model weights are already on the device, the application performs inference locally, and its features do not require an account or remote service. It can still store prompts in local history, and other software on the computer may access that history. “Offline” limits transmission; it does not make data invisible to the device owner, malware, backups, or anyone with access to the account.
2. On-device model with optional cloud fallback
Some apps use a small local model for simple requests but send more complex prompts to a hosted model. This can be a sensible design, yet users need to know when the handoff happens and how to disable it. Look for a setting such as cloud processing, connected models, online mode, or fallback. If the interface does not make a handoff visible, do not use it for confidential material until you can confirm the behavior from current documentation.
3. Local model plus online tools
A local assistant may answer from its own model but also have web search, package installation, plug-ins, or remote APIs. Asking it to search a phrase sends at least that query to a search provider; asking it to fetch a URL contacts the host. A tool-enabled assistant therefore needs separate review of the model path and the tool path. Our guide to execution containers for AI agents explains why tool permissions are a distinct safety boundary.
4. A local interface to a remote model
A program can run on your laptop while sending prompts to an API. The interface is local; the inference is not. This is common for developer tools and model gateways. Check the selected endpoint, API key owner, provider, and account policy. If a company manages the endpoint, ask the administrator what is logged and retained.
A practical check before pasting anything sensitive
- Open the privacy and data settings. Look for cloud processing, model fallback, history sync, diagnostics, and training-use controls. Read the current provider policy; settings can differ by account tier or region.
- Check the selected model and endpoint. Confirm the model name, whether it is downloaded, and whether the app is pointed at a local address or a hosted API. A locally installed client is not proof of local inference.
- Try a harmless offline test. Disconnect the device from the network and make a simple request. If the app cannot answer, determine whether it needs a sign-in, a model download, or an online tool. This test alone does not prove the app never sends data when connected, but it clarifies dependencies.
- Review tool access. Turn off search, connectors, or file access you do not need. Grant a single working folder instead of the entire home directory where possible.
- Inspect network activity if you know how. A firewall or operating-system network monitor can show destinations, but encrypted traffic generally will not reveal the prompt content. Do not install an unknown “AI privacy monitor” just to run this check.
- Use a dummy prompt. Do not test with a real client record, password, medical note, or private document. Use a made-up string and observe whether the interface shows a cloud handoff or account history.
- Review local retention. Find where chat history and logs are stored, who has access to the device account, and whether the folder is included in cloud backup.
Do not treat a packet capture as a complete audit. It can reveal connections, but not necessarily what was transmitted or what a provider does with the request afterward. For workplace data, follow your organisation’s approved tools and data-classification rules instead of trying to bypass them with a personal local model.
Privacy wording that sounds reassuring but is incomplete
- “The model runs on your device.” Ask whether requests can still be routed to cloud models or tools.
- “Your data is not used for training.” That statement does not necessarily say whether prompts are processed remotely, logged for abuse monitoring, or retained temporarily.
- “Private by design.” Find the technical and account-specific explanation: which data, which service, what retention period, and what controls?
- “Works offline.” Verify which features work offline. Updates, sign-in, search, cloud sync, and plug-ins may still need a connection.
Safer habits for everyday use
Minimise the prompt. Replace real names and account numbers with placeholders; paste only the paragraph needed; remove metadata from a document before uploading it. Keep secrets such as API keys and recovery codes out of prompts entirely. If an assistant can edit files, test it on a copy and inspect the changes. If an agent can run commands, use a narrow workspace and a restrictive execution policy.
For family or personal devices, remember that “local” does not answer who can read the device. A shared computer account, remote-support tool, synced backup, or unlocked profile can matter as much as the model provider. For business use, local inference may help with one part of the data path, but it does not automatically satisfy legal, contractual, or company requirements.
What to ask a provider
Before relying on an AI tool for sensitive work, get direct answers to these questions: Is inference on-device or remote for this exact model? Can the app switch to cloud processing automatically? Are prompts or outputs sent to plug-ins, search, or telemetry services? What is retained, for how long, and can account administrators review it? Does the answer change for free, paid, enterprise, or regional accounts? Keep a link or copy of the current policy because product behavior can change.
Takeaway
“Local AI” is a useful description, not a complete privacy guarantee. Verify inference location, network features, file permissions, history, and retention separately. Start with non-sensitive test data, disable unneeded tools, and use the narrowest access that works. If the data would be harmful to expose, do not paste it until you have confirmed the complete path—not just where the model runs.
Sources
- Microsoft Command Line: Local models and sandboxed tools on Windows
- Ollama: Privacy policy for local model use
- Apple Platform Security: Private Cloud Compute
- NIST AI Risk Management Framework
Privacy features depend on the exact product, model, account, and settings. Recheck official documentation before relying on an app for sensitive information.