There are plenty of YouTube videos circulating about Microsoft Recall and a supposed hard deadline of 20 May. Lots of panic, lots of “get off Microsoft,” change your settings, move your data. The thing is, most of it is codswallop. There is no hard deadline of 20 May that I can find anywhere. But that does not mean there is not a problem. Here is why.

They thought the backlash was about screenshots. It was not.

Recall became controversial because it accidentally revealed the direction of the entire industry. For years, technology platforms have quietly evolved from tools we use into systems that observe, interpret, and increasingly model human behaviour. Recall simply made the architecture visible in a way ordinary users could suddenly understand. The operating system was no longer just running software. It was watching the user, and that changes the psychological contract completely.

The internet conversation quickly polarised. One side insisted Recall was literally spyware. The other dismissed all concerns because Microsoft redesigned the feature to be local, encrypted, and opt-in. Both arguments miss the larger point. The real issue is not a single feature. The real issue is that operating systems, productivity platforms, identity systems, AI assistants, and cloud infrastructure are converging into something else entirely: behavioural infrastructure.

This is not science fiction. It is already happening.

The Shift Nobody Notices

Historically, computers worked like tools. You opened applications, saved files, the machine responded to explicit instructions. Now systems increasingly attempt to anticipate intent. The modern stack is moving from file storage, application execution, and direct interaction toward behavioural modelling, predictive assistance, contextual inference, persistent memory, and identity-linked workflow analysis.

That is a civilisationally important transition. Most people still think this is “just AI features.” It is not. It is the construction of machine-readable behavioural layers underneath everyday life.

Recall became symbolic because people suddenly realised the operating system itself was becoming part of that behavioural layer, and Recall is far from alone. ChatGPT now references your entire conversation history to build a picture of your preferences, tone, goals, and patterns. Learning from what you discuss even when you have not explicitly asked it to remember anything. Google’s Gemini introduced automatic memory in August 2025, learning from past chats without prompting, enabled by default. Anthropic, the company behind Claude, the AI I use to help build much of this body of work, introduced memory to its platform in September 2025. All four of the major consumer AI platforms now maintain persistent models of user behaviour across sessions.

I have written multiple articles about this, most recently about the moment I discovered Claude had inferred my Myers-Briggs personality type from our working sessions and stored it as fact, without flagging it as a conclusion. I had not declared my type, nor had I asked to be assessed. The inference had happened during a psychometrics research session, and it was recorded as attributed fact rather than as a probabilistic reading of my behaviour. That is not a feature. That is a pattern, and Recall simply made the same pattern visible at operating system level, which is why it triggered such a strong reaction.

Telemetry Is No Longer About Diagnostics

The word telemetry sounds harmless. It sounds like engineering maintenance, a few crash reports, a little performance data, nothing dramatic. But modern telemetry increasingly includes behavioural analytics, engagement patterns, workflow mapping, interaction timing, semantic usage data, productivity inference, contextual activity trails, and probabilistic modelling.

The important shift is this: the most valuable information is no longer what users explicitly disclose. It is what systems can infer. A modern AI system does not need you to tell it your personality, your political orientation, your emotional state, your vulnerabilities, your cognitive style, or your priorities. It can increasingly estimate those probabilistically from weak behavioural signals, like language, timing, structure, patterns, preferences, interaction habits. Inference has become more valuable than disclosure, and that is the real business model underneath the AI era.

The Collapse of Meaningful Consent

Most users still imagine digital consent as a conscious transaction. You agree, you understand (or in most cases don’t), and you participate, but modern digital infrastructure rarely functions that way. Participation has become socially and professionally mandatory. Cloud systems are infrastructure, identity platforms are infrastructure, communication systems are infrastructure. AI assistants are rapidly becoming infrastructure.

So the phrase “the user consented” increasingly means: the user clicked Accept in order to continue participating in digital life. That is not informed negotiation, it is procedural acknowledgement under dependency conditions. The legal frameworks built for earlier internet eras are struggling badly with this shift. Privacy law historically focused on collection, storage, and processing, but modern AI systems increasingly operate through embeddings, latent representations, dynamic inference, probabilistic associations, and behavioural prediction. The architecture has changed faster than the governance.

Why Recall Triggered Such a Strong Reaction

People are not reacting only to privacy, they are reacting to loss of agency. Over the last decade, users have gradually accepted cloud dependence, account integration, advertising in operating systems, algorithmic feeds, recommendation systems, behavioural tracking, and AI assistance. Each step seemed individually manageable. Then Recall appeared and suddenly the direction became visible all at once. Users realised the computer was evolving from a machine that stores information into a machine that continuously maps behaviour. That boundary matters emotionally, especially once AI systems begin interacting not only with files, but with memory, cognition, habits, and workflow patterns. The machine stops feeling passive, it starts feeling interpretive, and that’s because it is.

The Bigger Governance Problem

Most public debate still frames this as a privacy conversation. It is actually a governance conversation. The important question is no longer “who has my data?” The important question is: who governs the intelligence layer sitting between humans and digital reality?

Increasingly that layer influences visibility, discoverability, moderation, reputation, access, workflow, communication, economic participation, and cognitive framing. Modern platforms are no longer just websites. They function as identity systems, trust systems, behavioural systems, distribution systems, and algorithmic governance environments. AI dramatically scales their ability to interpret behaviour, which creates a profound asymmetry. Users often cannot see what was inferred, how they were classified, which signals mattered, what shaped recommendations, why visibility changed, or how moderation decisions emerged. The systems become increasingly opaque while simultaneously becoming more influential. That is the real trust crisis emerging underneath the AI industry.

The Rise of Sovereign Computing

This is why local AI, open-source systems, and self-hosting are attracting growing attention because advanced users increasingly want controllability, transparency, portability, inspectability, autonomy, resilience, and ownership over infrastructure. The industry is beginning to split into two broad models.

The first: highly integrated, cloud-native, AI-assisted, seamless, managed, frictionless, in exchange for deeper behavioural integration, platform dependence, persistent telemetry, identity coupling, and algorithmic mediation. The second: local-first, open-source, self-hosted, portable, inspectable, user-controlled, in exchange for greater complexity, higher technical burden, less convenience, and more operational responsibility.

This split is likely to define the next phase of computing, not because one side is good and the other evil, but because different users will optimise for different trade-offs. Convenience versus control. Automation versus agency. Integration versus sovereignty. I’m changing to Linux and self host for exactly this reason. It is more difficult, yes, but nothing is easy in the tech world.

The Real Story

The Recall debate was never really about screenshots. It was about the moment users realised the operating system itself was becoming part of the behavioural economy. AI is not merely changing software, it is changing the relationship between humans, machines, platforms, and governance.

It is worth noting that the EU, UK, Switzerland, and the broader European Economic Area are currently excluded from some of these memory and personalisation features. Gemini’s automatic memory rollout, for example, is not available in those regions, and ChatGPT’s enhanced memory similarly skips them. That exclusion is not accidental, it is regulatory pressure doing what regulatory pressure does: creating friction where commercial interest would otherwise move without it.

Which brings us to Article 4 of the EU AI Act. The literacy obligation, in force since 2 February 2025. It requires organisations to ensure their people have sufficient understanding of AI to use it appropriately. Not the high-risk obligations, which have been deferred to December 2027. This one, now, already live.

The companies that succeed in the AI era may not necessarily be the companies with the most powerful models. They may be the companies users still trust, and trust increasingly depends on one question: who governs the machine?

If your organisation is in the EU or UK and your people are using AI tools with persistent memory enabled, tools that are building behavioural models of your employees, your clients, your workflow, and your decision-making and nobody in your organisation has asked what that means, Article 4 is not a future problem. It is a present one.

To have an Enterprise conversation about this drop me an email. Sam@contain.digital

configure YOUR system. contAIn the chaos. control YOUR outcome.

Sources

[1] Microsoft Support, Retrace your steps with Recall: https://support.microsoft.com/en-us/windows/retrace-your-steps-with-recall-aa03f8a0-a78b-4b3e-b0a1-2eb8ac48701c

[2] OpenAI, What is Memory: https://help.openai.com/en/articles/8983136-what-is-memory

[3] Google, Get personalisation with memory of your past Gemini chats: https://support.google.com/gemini/answer/16598469

This article was originally published on Substack.