The word local usually describes where model inference happens, not every part of an AI application. A model may run on your computer after its files are cached while the app still checks for updates, contacts a catalog, sends telemetry, calls a remote tool, or routes a task to a cloud service. Microsoft says Foundry Local can continue inference offline once a model is downloaded, while its startup may attempt to refresh a catalog. That is a useful capability statement—not a blanket promise about every app built around it.
Quick scan
In brief
Separate model inference, model download, app orchestration, tool calls, telemetry and update checks.
Microsoft says Foundry Local inference can work offline once the selected model is cached locally.
A label such as local or private does not by itself prove that prompts or connected tools stay on-device.
Draw the full path before you trust the word local
Write down what happens from the moment you submit a prompt to the moment the answer appears. The model may be local, but the app could send a request to an online search tool, a cloud connector, a remote MCP server, or a hosted fallback model. The user interface may also fetch extensions, telemetry, policy, or model metadata. Those are separate network paths and should not be collapsed into one privacy claim.
A simple map has four boxes: your input, the model that processes it, the tools or data sources it can call, and the service that stores logs or preferences. Mark each box as on-device, local network, vendor cloud, or unknown. Unknown is not the same as local. If the vendor does not document a data flow, test the app with a network monitor in a non-sensitive environment or ask the administrator before using confidential material.
Model files and inference are different moments
A local runtime often needs an internet connection to install packages, retrieve model weights, and download hardware-specific components. Microsoft’s Foundry Local setup requires internet access for initial packages, models, and runtime components. Its FAQ says that once a model has been downloaded and cached, inference can run entirely on-device without a cloud dependency. It also notes that startup may attempt to refresh the model catalog, but a cached catalog can be used offline.
That means a first run can transmit more than a later offline run, even if the generated answer is always computed locally. Before a trip or a sensitive task, confirm that the exact model is already cached, start the application while disconnected, and make sure the task completes without a silent fallback. Record the runtime version and model identifier. If the app displays a generic local badge but cannot explain which model is active, treat the boundary as unverified.
Action map
What to do next
| Layer | Focus | Working instruction |
|---|---|---|
| Check | Draw the full path before you trust the word local | Write down what happens from the moment you submit a prompt to the moment the answer appears. |
| Action | Model files and inference are different moments | A local runtime often needs an internet connection to install packages, retrieve model weights, and download hardware-specific components. |
| Verify | Tools can move data even when the model does not | A local model may be allowed to browse, call APIs, read cloud drives, or execute scripts. |
Tools can move data even when the model does not
A local model may be allowed to browse, call APIs, read cloud drives, or execute scripts. In those cases, the model can remain on-device while information leaves through a tool. For example, an offline model that summarizes a file is different from an assistant that uploads the file to a remote search service and returns a summary. Permission prompts, tool connectors, and default destinations matter as much as the model location.
Review each connected tool before use. Identify the account it uses, the folders and websites it can reach, whether it writes data, and how to disable it. For business software, ask whether outbound access can be limited to approved hosts and whether the app records tool actions. Microsoft’s local-execution documentation describes controls such as allowed folders and outbound network policy for its specific Copilot Code environment; do not assume another local AI application has the same boundary.
Use a low-risk offline test
Choose a harmless test file and ask the application to summarize it. First run with the network connected, then repeat after disconnecting Wi-Fi and Ethernet. Watch for a cloud fallback message, a feature that becomes unavailable, or a request to sign in again. Reconnect and inspect the application’s privacy and activity settings for model, tool, and telemetry controls. Do not use a real customer record as a network test.
For a stronger test, block the app’s outbound traffic on a disposable device or in an approved test environment, then observe whether inference still works. A successful answer demonstrates only that this workflow can operate without the tested network path under those conditions. It does not prove the app never sends telemetry at another time, or that a separate updater cannot reach the internet. Document the exact version, firewall rule, model, and test time.
Check updates, logs and fallback behavior
Look for a clear setting that controls cloud fallback, crash reporting, usage analytics, and automatic model downloads. Read the privacy documentation for the specific product and version. The app may retain prompts locally, sync preferences between devices, or create logs that are included in support bundles. If the product is managed, your organization may control these settings; user-facing toggles can be disabled or overridden by policy.
A local runtime can also update independently from the model or application. Note whether the app contacts a catalog at startup and whether that request is optional. If a feature falls back to a hosted model when the local model is unavailable, the interface should disclose that switch before sending protected content. If it does not, disable fallback or avoid sensitive inputs until the behavior is clear.
A decision checklist for sensitive work
Before using local AI with confidential material, answer five questions: which exact model processes the prompt; where the model is stored; whether the application sends any prompt, output, or file to a tool; which network destinations are contacted; and what logs or backups remain afterward. Verify the answers from the official product documentation and a safe test. If your employer has an AI policy, follow it even when the model appears to run locally.
A good privacy decision is scoped to one workflow. You might accept a model that is cached locally but reject its optional web-search connector, or allow a business-approved tool while excluding personal files. Keep the distinction visible to other users so “local AI” does not become shorthand for an untested promise. Recheck after application updates, because defaults and service dependencies can change.
