When helping with health data decisions looks like manipulating: EU law and the limits of legitimate influence

What if a conversational AI assistant could help Europeans to opt out of the secondary use of their health data under Article 71 of the European Health Data Space Regulation (EHDS)? The decision is cumbersome: individuals must weigh an abstract public benefit against a diffuse and deferred private risk, with only a few having the literacy, motivation, or cognitive resources to do this well (Acquisti et al., 2017). Since the Regulation’s default makes data available unless the individual opts out, inaction is itself a decision. An assistant that explains what secondary use involves, considers individual values, and supports a granular choice would make the right to opt out genuinely autonomy-enhancing.

What counts as a manipulative technique

The design of such an assistant must be free from the manipulative practices prohibited under Article 5(1)(a) of the AI Act (Regulation 2024/1689). Yet, three of the four cumulative conditions for a practice to be considered manipulative, following the European Commission's Guidelines (2025), are readily satisfied. First, the AI system is put into service, which is, it is deployed. Second, it is designed to inform, and potentially influence, decisions; the Guidelines do not convincingly distinguish the legitimate shaping of a decision from “appreciably impairing the ability” to make an “informed and autonomous decision”, which is the scope of the prohibition. Behavioural science shows that individuals can rarely be aware of future risks in complex digital settings (Acquisti et al., 2017), while complete transparency does not necessarily inform (Nissembaum, 2011); and any technology design can support autonomy on one conception and undermine it on another, whether autonomy is understood as control, independence, or freedom of choice (Rossi, in press). Without conceptual clarity, any sort of digital influence may be labelled unlawful. And third, the AI system may cause significant harm: if the data it encourages the individual to share is then poorly protected, the resulting harm would not have occurred without it. Whether the prohibition applies, therefore, turns on the fourth condition: does the assistant deploy purposefully manipulative techniques?

Note first what this reveals. The instrument designed to make the right effective, risks being manipulative by definition, if it impairs informed and autonomous decisions and results in significant harm. The default that makes the same right difficult to exercise, by contrast, is structurally indistinguishable from a privacy-invasive default, which the European Data Protection Board (EDPB, 2023) treats as an unlawful deceptive design pattern under the General Data Protection Regulation (GDPR). A definition identifying manipulation by the structure of the default alone would therefore be applicable to a mechanism that the Union adopted in pursuit of a public interest. Two difficulties follow, each turning on a criterion the Commission's Guidelines rely on more heavily than it can bear.

Defaults under the bias test

The first is the question left open above. If manipulative techniques are those designed to exploit cognitive biases, as much academic work also holds (Waldman, 2020), then any digital choice architecture that predictably shapes behaviour without being coercive qualifies. Defaults show why cognitive biases cannot be the only cause. The defaults’ effectiveness has several explanations, from inattention to transaction costs of switching (Acquisti et al., 2017), but only some are biases, yet the criterion captures the default in every case. If exploiting a bias is what makes influence manipulative, the prohibition cannot tell an architecture that helps someone act on their own values from one that substitutes different values for theirs. Locating the problem in a bias implies that the remedy is to restore reflective thinking through transparency and education (Rossi et al., 2024), which the next difficulty unsettles. Taxonomies of deceptive design compound this, since they list personalisation as a pattern in its own right, defined as using personal data to steer users toward some options while concealing others (Gray et al., 2024). Applied literally, the label over-detects: any personalisation counts as manipulation, including the assistant surfacing a commitment the individual holds but has not articulated, which is what competent assistance consists of. When these definitions serve as the baseline for automating enforcement, they can cause misdirected regulatory attention (Rossi and Parkin, 2026).

The awareness and alignment remedy

The second, and most consequential, difficulty concerns the line the Guidelines draw between prohibited manipulation and permitted persuasion. Because impairment is measured against what the person would otherwise have decided, autonomy is reduced to independence from influence and freedom of choice, and awareness becomes the only corrective. Persuasion, in this account, is transparent about the system's objectives and provides accurate information, thereby supporting the capacity to evaluate and choose, whereas manipulation operates without that awareness; and persuasion aligns the interests of both parties, whereas manipulation benefits the manipulator at the individual's expense. However, both criteria fail.

The transparency criterion is the same one the EDPB applies to deceptive design patterns, which, in its account, do not allow individuals to make “conscious choices” to protect their personal data effectively. Both guidelines assume that legitimacy depends on the awareness of a subject who, once informed, can reason and act upon it. The empirical literature does not support that dependence: autonomy safeguards can increase data disclosure rather than caution (Brandimarte et al., 2013) and even users who recognise a manipulative interface remain susceptible to it (Bongard-Blanchy et al., 2021). The difficulty is sharper in human-AI interaction, where anthropomorphic cues and conversational register foster trust (Seymour and Van Kleek, 2021) while operating below awareness, enabling the interaction to work (Carli et al., 2022).

The alignment criterion presupposes that users’ interests are determinate, singular, and opposed to the designer’s, an assumption that also underlies the EDPB's treatment of loss-gain framing (“emotional steering”). It fails in the EHDS setting: interests diverge, because secondary use has genuine value for the public, and possibly for the individual, while the residual risk falls on the individual alone.

Conclusion and recommendations

None of this argues against prohibiting manipulative design. The significant harm condition can do the discriminating work, and the core intuition that certain kinds of influence impair autonomy is sound. The criteria for locating it are what fail.

Three directions follow. Behavioural economics, usable privacy, and digital ethics have debated legitimate influence for decades, and the Article 5 Guidelines borrow from that work selectively, leaving behind the qualifications the criteria came with. Those fields also offer what a legal definition cannot supply on its own: ways of measuring whether an interface actually supports a decision. Second, permissible influence must be defined so as to hold where the interests at stake are public as well as private. Third, since Article 5 covers only purposefully manipulative techniques, providers owe preventive measures they cannot design while the criteria remain contested. Legitimacy should thus be sought in the structure of the choice the system supports: whether a decision remains revocable at low cost, whether alternatives are equally accessible, and whether the assistant surfaces commitments the person can recognise and disavow rather than acting on them silently. These can be assessed at the design stage; individual awareness cannot.

Assistant or not, the EHDS opt-out is an implementation problem that transparency alone does not solve. Noticing the default is the least of it: what matters is whether Member States will equip individuals to surface what they care about and to act on it, through whichever form of mediation fits, and whether a choice, once made, can be easily revisited.