Choosing local versus cloud AI per workflow: what to audit before 'local' delivers privacy

Treat the choice between local versus cloud AI as a governance decision per workflow, not a product choice. Local only delivers privacy gains if you audit the whole data pipeline (inference, logging, synchronisation), harden edge environments yourself and record per task which data may never leave the device.

The trigger is a concrete product change: in early September 2026 Perplexity announced a 'hybrid compute' feature in its Mac app, in which a local 'privacy gate' on the device determines which information may leave the machine, with the remainder of the task going to the cloud. That makes the old dichotomy local is private, cloud is not tangibly untenable. The rest of this piece translates that shift into decision rules you can apply and demonstrate yourself per workflow.

Why does 'running locally' not automatically mean my data stays private?

An academic paper on arXiv, Local Is Not a Sufficient Privacy Boundary, analyses on-device AI as an operating-system-centred ecosystem. The authors argue that privacy follows not from the location of the inference, but from limited information flows, bounded permissions and auditable governance. They introduce a six-part risk taxonomy and warn explicitly that local inference can still leak sensitive data through crash logs, analytics and permissions if the underlying OS and app architecture is not designed and verified.

The practical lesson: 'local' moves the risk, it does not remove it. In our assessment the most dangerous scenario is not the cloud, but a local setup marketed as private while analytics, telemetry or synchronisation still send the data off the device. Anyone who takes the promise seriously must map the full data pipeline, not just the place where the model computes. The same principle applies when checking where your prompts go.

Which extra security tasks do I get when AI inference runs in my own environment?

In its guidance on securing edge AI in customer-owned environments, Microsoft Security describes how the security model tips over once inference runs in your own environment instead of in a central cloud. You gain more control over hardware, platform and model weights, but must build trust in runtimes and artefacts yourself, enforce runtime policy and bind sensitive assets only to attested environments. The guidance names prompt injection, model tampering and rogue firmware in the same environment as customer data and credentials.

In concrete terms this means local AI shifts responsibility to you. Microsoft advises attestation, provenance and mediation as governance layers. In plain terms: can you demonstrate that the model running is also the model you intended (attestation), where it came from (provenance) and that its actions are bounded (mediation)?

• Harden the runtime and verify the origin of model files and updates.
• Limit what the model may do and access; bind sensitive data to attested environments.
• Record logging and telemetry, and check where those logs go.
• Treat every local device as its own vulnerability under central policy.

How do I divide tasks between local and cloud models following 'process local, learn global'?

The practical sources all point to hybrid architectures as the real norm. Vertu's Executive Privacy Playbook states that most high-end devices in 2026 use a hybrid model, in which privacy-critical and latency-sensitive tasks run on-device and complex tasks go to the cloud; at the same time it warns that applications can still log, synchronise and share through other channels. Ambient.ai describes the pattern 'process local, learn global': edge devices process regulated and time-critical data locally, while the cloud serves for aggregated analytics, under a zero-trust policy that captures cloud and edge in one governance system.

IntelliSee illustrates the trade-off in physical security: continuously streaming video to the cloud creates dependence on the network and data centres, while edge AI enables data minimisation and local control, but introduces extra management and hardening tasks. The same trade-off plays out in other high-trust domains such as healthcare, law and government. The editorial point: hybrid is not a compromise but a design decision you must document explicitly.

Which decision rules and evidence do I record per workflow?

Translate the insights above into a short list you complete per workflow. In our assessment, this is where many organisations stumble: they may choose an architecture but fail to record why or demonstrate it afterwards. Our recommended checklist:

1. Determine which data is so sensitive that it may never cross the device or network boundary.
2. Establish per task the latency and availability requirements that determine whether cloud inference is acceptable.
3. Record which logs, attestation and provenance artefacts must be available to meet privacy legislation and sector standards.
4. Document which security controls support both the local and the cloud route, under one central policy.
5. Review the settings periodically, as the Executive Privacy Playbook advises.

For anyone working with confidential or high-trust information, this is also a GDPR question: it is not only about where the inference takes place, but about demonstrable management of the whole chain. Our broader explanation of the GDPR responsibilities per phase of your AI workflow and the topic hub on AI privacy and GDPR helps to assign these points to the right roles.

Where does a verification layer such as Vera fit into this picture?

Vera is not a language model and not a chatbot, but a privacy-focused verification layer. The anonymisation on EU infrastructure before AI processing can replace sensitive document values with synthetic, session-only equivalents; the architecture is designed to send only anonymised content onward to the selected models, and the workflow is fail-closed: if the privacy check fails, the document is not sent onward. That makes control possible over what a cloud model gets to see, without guaranteeing correctness or ruling out hallucinations. The professional final judgement remains with you.

Sources: The article draws on the arXiv paper Local Is Not a Sufficient Privacy Boundary, Microsoft Security guidance on edge AI and practical analyses by Vertu, IntelliSee and Ambient.ai.

← All articles in this topic ← All articles