Leveraging interdisciplinary methods for evidence collection in enforcement: Dark patterns as a case study

Introduction

Both scholars and enforcers rely on evidence to support their claims. Scholarly research can inform enforcement practices implicitly, but can also serve as case-specific evidence. Further leveraging each discipline’s treatment of and collection methods for evidence towards stronger collaboration, however, remains an ongoing effort. In particular, questions remain as to what extent empirical methods—such as those frequently employed in human-computer interaction (HCI) scholarship and industry user experience (UX) research—can be more directly translated into enforcement actions, effective evidence-based enforcement, or serve as evidentiary support in legal proceedings, commonly referenced as case law. This lack of methodological clarity presents a valuable opportunity for HCI scholars and legal scholars to reflexively analyse which interdisciplinary evidentiary methodologies and collaborative dynamics regulators and legal experts might regard as relevant and sufficiently authoritative.

This paper explores the methodological innovations, successes and challenges of HCI empirical approaches for collecting evidence to identify and characterise unfair, deceptive, or anti-consumer design practices in user-facing technologies. Of particular interest are methods that could contribute to informing and supporting regulators, legal practitioners, and other relevant stakeholders in the enforcement of user-facing designs. These aforementioned design practices are often referred to as “dark patterns” (Brignull, 2010, Gray et al., 2018). Dark patterns are manipulative, deceptive design practices deployed in online services aimed at influencing the decisions of users (Mathur et al., 2021, Gray et al., 2018) about their purchases (Mathur et al., 2019), use of time or attention (Roffarello, 2023), and disclosure of personal data (Kubicek et al., 2024; Soe et al., 2020; Tran et al., 2024). These design practices bring a unique potential for influencing user behaviour, undermining user agency, leading to privacy loss, and disparately impacting vulnerable or disempowered communities, among other harms.

Research in HCI has been central to the study of dark patterns, contributing foundational knowledge on the nature and types of dark patterns, developing an ontology of dark patterns (Gray et al., 2024) that harmonises regulatory and academic taxonomies, and conducting empirical studies (such as in situ analyses, participant studies, and broader measurement studies, as will be further described in this article) that demonstrate the prevalence and impact of dark patterns in specific contexts. Such scholarship has been involved with community organising efforts (such as in-person workshops (Schraffenberger et al., 2024; Acar et al., 2025; FTC, 2022a) and ongoing remote community engagement like a centralised Slack channel), bringing together stakeholders across multiple academic disciplines, industry, and enforcement to deepen understanding and collectively address the societal risks posed by dark patterns.

Regulatory decisions and legal frameworks have increasingly focused on dark patterns over the recent years. Several new legislation, such as the EU Digital Services Act (DSA) in Recital 67 and Article 25 (Regulation 2022/2065), Digital Markets Act (DMA) in Article 13 (Regulation 2022/1925), Data Act in Recital 38 (Regulation 2023/2854), and Artificial Intelligence Act (AI Act) in Article 5 (Regulation 2024/1689), presently include provisions to prohibit dark patterns or related designs (Car & Cassetti, 2025). For example, the AI Act prohibits “subliminal” or “purposefully manipulative or deceptive techniques” within AI systems that impair user decision-making (Regulation 2024/1689, Articles 5(1)(a, b)). Most notably, the DSA provides a definition of dark patterns, including detailed specifications for certain types of prohibited behaviours (Recital 67; Article 25(3))). In the United States, states have begun to explicitly prohibit dark patterns like the California Consumers Privacy Act [CCPA] (2018) as modified by the California Privacy Rights Act [CPRA] in §1798.185.19(c)(iii) (2020) and the Colorado Privacy Act [CPA] (2021, §6-1-1301(5)(c)). Enforcers have also brought cases against dark patterns: the US Federal Trade Commission increasingly applies its extant consumer protection authority to dark patterns (e.g. FTC v. Vonage) (2022), and EU member states also increasingly enforce against dark patterns via data protection (Deliberation No. SAN-2021-023, 2021; Deliberation No. SAN-2021-024, 2021) or consumer protection (Decision ACM/22/179622, 2024) authorities.

Though the legal and academic domains naturally take different methodological approaches to suit differing needs (e.g. single- or limited-party investigatory scope for cases, versus sample size breadth that scholarship often seeks for generalisability), examining these approaches may reveal commonalities and opportunities. As such, we present the topic of dark patterns as a case study for illustrating extant and potential interdisciplinary efforts towards technology regulation and enforcement, and to encourage further collaboration between academic and regulatory practice.

This article proceeds as follows. First we introduce dark patterns scholarship and extant collaborative efforts. Next we describe our methods, bringing together academic and legal bodies of knowledge towards understanding evidentiary approaches in both; we then map these investigatory methods across disciplines. More specifically, we analyse HCI research methods for detecting, identifying, or characterising dark patterns from two sources: (1) we examine existing methodologies from dark patterns literature, building upon a prior systematic literature review (Gray et al., 2023); (2) we draw from the lessons and experience of common User Experience Research (UX) methods employed in industry settings to provide evidence for design decisions. For the legal context, we draw from EU and US dark patterns cases law, then examine the evidence and the methods employed by regulators to substantiate the presence of these patterns within the context of each case. Mapping HCI methods to these cases, we then discuss how the evidentiary methods used in case law converge or diverge from those used within HCI academic and practitioner methods. As such, we perform interdisciplinary research across law and HCI, revealing challenges and opportunities that arise from sectoral and procedural differences between scholarship and law. Finally, we discuss the challenges for utilising scholarly evidence in case law, propose recommendations for the use of scholarly methods in formal investigations, and identify opportunities for using scholarly methods and evidence in semi-formal investigations for regulators to assess organisations’ behaviours.

Dark patterns scholarship

To measure and observe dark patterns, empirical research has evaluated both users’ reactions to deceptive designs (Bongard-Blanchy et al., 2021; Gray et al., 2021b; Luguri & Strahilevitz, 2021) and quantified the impact they make on users’ decisions (Bermejo Fernandez et al., 2021; Bielova et al., 2024; Graßl et al., 2021; Habib et al., 2022; Machuletz & Böhme, 2020). A systematic review of empirical studies relating to dark patterns by Gray and colleagues. (Gray et al., 2023) notes that a diverse range of domains or use contexts are impacted by these manipulative, deceptive, or coercive practices, including mobile apps and websites (Di Geronimo et al., 2020; Gunawan et al., 2021; Mathur et al., 2019), voice assistants (Owens et al., 2022), internet of things (IoT) devices (Kowalczyk et al., 2023), mixed reality (Krauss et al., 2025), social networks (Mildner & Savino, 2021; Mildner et al., 2023; Schaffner et al., 2022), games (Aagaard et al., 2022; Niknejad et al., 2024), AI assistants (Millers et al., 2024), social robots (Lacey & Caudwell, 2019), and privacy control mechanisms like consent banners (Bouhoula et al., 2024; Gray et al., 2021b; Nouwens et al., 2020; Toth et al., 2022).

Dark patterns scholarship increasingly brings together members of many fields in transdisciplinary dialogue, both within academia (between academic disciplines) and across academia, industry, and different legal domains. An updated ontology of dark patterns (Gray et al., 2024) built by an interdisciplinary team harmonises numerous academic and regulatory taxonomical sources. A recent taxonomy of harms caused by dark patterns consolidates both regulatory and HCI literature (Santos et al., 2025). In particular, dark patterns bring together legal and HCI scholars to align regulatory guidance with the design realities of constructing digital systems (Bielova et al., 2024b; Gray et al., 2021b; Bielova, 2022). Other such work additionally discusses privacy dark patterns against the potential for legal remedies (Gunawan et al., 2022).

The usage of research methods for collecting evidence of dark patterns in legal actions has been already pointed out by leading scholars on dark patterns, including expert witness’ work using heuristic analysis (King & Stephan, 2021; King, 2012, 2016) on two cases in the United States (FTC vs. Amazon.com, 2014; FTC vs. Commerce Planet, 2011) and other work using qualitative content analysis as noted in an expert report for State of Arizona v. Google LLC (Gray, 2022).

Academic-enforcement collaboration

Avenues for mobilising scholarly work for enforcement settings (and vice versa) cover a spectrum of processes and activities. Co-productive research models, or coproduction, are one avenue (Innes et al., 2019; Large, 2024; McCabe et al., 2023; Yu, 2020); other avenues for academic-enforcement engagement range from “applied” research using regulation and enforcement as the topic in question (e.g. compliance-oriented audits seen in GDPR studies (Kubicek et al., 2024; Soe et al., 2020; Tran et al., 2024) non-exhaustively), to extensively citing scholarship in informal guidance (Journal 2021/C 526/01), to submitting scholarship to calls for public rulemaking commentary (e.g. Notices for Public Rulemaking in the US and Public Consultations in the EU), or to the explicit use of scholarly evidence in enforcement.

Regarding the latter on the use of evidence in formal proceedings, the EU Better Regulation Toolbox describes “evidence” in reference to “multiple sources of data’’. It includes ‘’information and knowledge, including quantitative data such as statistics and measurements, qualitative data such as opinions, stakeholder input, conclusions of evaluations, as well as scientific and expert advice” (European Commission, 2023, p. 8). However, enforcement authorities tend to adopt a narrower evidentiary scope. Case law customarily cites prior rulings as foundation for their interpretation and decisions, ensuring consistency and predictability in the legal system. Courts and enforcers engage in a limited way with scholarly work. In particular, Court procedural rules predicate the value of scientific works. The case law analysis methodology of the Supreme Court of Estonia notes: “numeric data collected and used for the analysis” is considered “illustrative material”—then, “if necessary and possible... Appropriate popular scientific materials may be used in moderation” (Riigikohus [Supreme Court of Estonia], n.d.). Similarly, the Statute of the International Court of Justice considers “the teachings of the most highly qualified publicists of the various nations” as “subsidiary means for the determination of rules of law” (United Nations, 1945, Article 38(1)(d)). These citations thus demonstrate that scientific research is technically suitable for citation in judicial and administrative cases, though limited to subsidiary or illustrative roles – at least in the EU. In the US, scientific methods are incorporated as the basis of expert testimony (Fed. R. of Evid. (702)), with the inclusion of scientific evidence itself still remaining a challenge (Albright, 2023). For evaluating expertise, the Daubert standard subsumes the prior Frye standard for determining the admissibility of scientific evidence and testimony: Daubert strengthens reliance on methodology and the underlying science rather than a test for general in-field acceptability as was initially set by Frye (Daubert v. Merrell Dow Pharmaceuticals, 1993; Frye v. United States, 1923).

Methods

We consulted two bodies of knowledge in this work: methods derived from academic scholarship and UX research (UXR), and decided dark patterns enforcement cases. A diagram detailing these sources is presented in Figure 1. Regarding positionality, scholarly or research-oriented literature was first examined by HCI and/or computing scholars in our team, whereas case law was examined by legal scholars. However, as our team consists of researchers with prior interdisciplinary experience, we iteratively discussed analyses across the full team.

Understanding HCI and UX Methods

We start with the Gray et al. (2023) data set of 79 dark patterns-related studies published from 2014 to 2022, which contribute to the detection or characterisation of dark patterns and spans a wide range of fields, including human-computer interaction, computing, game studies, privacy and security, and law, among others. Gray et al. (2023) provide a preliminary categorisation of empirical methods; building upon these, we further cluster identified scholarly methods into four categories, then articulate and label the method category’s primary purposes (the investigatory goals served by those methods), a non-exhaustive list of evidence types commonly derived with that method type (which may correspond both to compiled data sets for a given study, and analytical output qualifying as new evidence in its own right), unique strengths that the methodology group lends to scientific inquiry, and finally known limitations of the methodology type (typically the inverse of the methods’ strengths).

We separate these categories as distinct to facilitate our own methodological reasoning while acknowledging the rising popularity of mixed-methods or interdisciplinary approaches for contextualising dark patterns research findings (like those in this paper). However, we believe the clarity from these distinctions is helpful for the purpose of this paper’s objectives of working towards multi-disciplinary alignment and understanding.

Finally, we consider formative texts in industry UX research (UXR) methodology, given the inherent nature of the dark patterns subject being a UX topic. UXR represents a growing area of industry work focusing on the design and implementation of studies that can lead to product insights, with HCI practices commonly performed by UXR specialists in industry. We extract methods from a smaller set of two longer source documents, describing UXR methods and considering them against the scope of dark patterns scholarship. Specifically, we consider the book UX Research: Practical Techniques for Designing Better Products (Nunnally & Farkas, 2016) and the article When to use which user-experience research methods (Rohrer, 2022).

Analysing legal case studies

The law and policy side of dark patterns literature spans regulation, governmental reports, regulatory guidelines, press releases, case law, and more. To evaluate insights regarding methods of evidence collection for dark patterns, we restrict our search to case law given its inherently investigatory nature. We begin with the data set of case law from deceptive design/cases (Brignull et al., 2023), a popular and centralised repository of dark patterns-related case decisions managed by an interdisciplinary team of known dark patterns scholars. This set spans 27 unique jurisdictions across the United States and the European Union, such as regulatory enforcement agencies like the EU Data Protection Authorities (DPAs), the US Federal Trade Commission (FTC), and consumer or competition authorities that govern similar design issues. From these cases, we identify a few case studies that represent the dark patterns regulatory momentum. That is, we filter cases that either mention dark patterns by name in-text, directly cite dark patterns scholarship or studies articulating evidence for related dark pattern practices in-text.

The original deceptive.design/cases data set includes only res judicata (closed, decided) cases. For methodological consistency (e.g. maintaining “final”-status consistency between case texts and formally-published scholarly articles), we find res judicata cases appropriate and do not include undecided cases in our updated search. Our final set of legal case studies includes four EU cases across three member states’ jurisdictions, and three US cases across two federal and one district court jurisdictions. These cases are demonstrative of the dark patterns regulatory momentum, but certainly not exhaustive.

For all cases, we obtain official case texts or related English translations. Since investigatory methodologies are not often described in the same manner as in scholarship, we assess the case text for factual descriptions or concrete details regarding evidence that might be useful or commonly used by scholars as data. We then relate cases to scholarly analysis methods as inferred from the text and the way evidence–data–is described.

Mapping investigatory methods across disciplines

Empirical methods for dark patterns research

Methods from academic literature

We characterise four key clusters of methods used to identify or describe dark patterns in academic literature. Table 2 presents these methods with the list of evidence types each method group produces and the method’s non-exhaustive strengths and limitations for contextualisation across method type; this table provides a reference for mappings made throughout the remainder of the work.

Qualitative content analysis (Mayring, 2000; Schrier, 2012; Hsieh & Shannon, 2005) identifies the presence of dark patterns, including their type and function within the user journey, e.g. Gray et al. (2018) and DiGeronimo et al. (2020). This method focuses on manual detection of dark patterns through expert review, based on a shared list of dark patterns types, characteristics, and/or exemplars. Experimental design (Scott MacKenzie, 2013; Cairns, 2014; Lazar, 2017) identifies generalisable causal mechanisms relating to dark patterns that have the potential to impact user behaviour, e.g. Luguri & Strahilevitz (2021) and Bhoot (2020). This method focuses on the isolation and evaluation of specific design choices, which are then tested using an experimental and control group to identify the degree to which one or more design choices impacts user behaviour. Observation, Interview, Survey, and Case study (Lazar, 2017; Olson & Kellogg, 2014) methods identify or problematise user behaviour or awareness of dark patterns, and focus on describing how users perceive, act upon, or otherwise use their knowledge of digital systems to address or circumvent dark patterns, e.g. Gray et al. (2021c) and Bielova et al. (2024a). Finally, Web measurement (Mislove et al., 2007) identifies UI or interaction instances of dark patterns based on how these patterns manifest in code or how these patterns influence user behaviour, e.g. Mathur et al. (2019). This method focuses on capturing web data as rendered through a browser, including code elements and their display and how these elements are impacted by user interactions.

Methods from UX research

Next we consider our UXR sources, which use categorisation schemes to structure methods into smaller groups. The two texts generally agree on which methods are typical in early stages (prior to product creation or substantive modification), during active design/development (to direct or organize product decisions), and during evaluation or validation (to ensure the product functions expectedly).

Early-stage methods are generative tools that are oriented towards providing insights on potential design directions, including landscape analysis, contextual inquiry, diary study, stakeholder workshops, and participatory design. These methods typically rely on a small number of participants, but generate deep qualitative insights that can enable the UX researcher to identify new product directions or gaps. Active design/development stage methods are formative tools that enable product teams to iterate upon a product in active development may rely on a range of qualitative and quantitative methods. For instance, quantitative methods may include taxonomy review, eye tracking, A/B testing, card sorting, and unmoderated product testing. Qualitative methods may include moderated or unmoderated product testing, contextual inquiry, usability testing, and heuristic analysis. Evaluation- or validation-stage methods (Nunnally & Farkas, 2016; Rohrer, 2022) are summative tools that enable a product team to continue to learn about a product based on real-world use through both qualitative and quantitative methods. Quantitative methods may draw from a range of system analytics, such as user flows, demographic characteristics, and customer feedback. Qualitative methods may include expert inspection methods such as heuristic evaluation or cognitive walkthroughs, alongside moderated or unmoderated product testing or usability testing to identify emergent bugs in existing or new user communities.

Synthesis of scholarly and practical empirical methods

We next created a synthesis specific to the identification of dark patterns, describing relevant validation procedures or audit trails that make discovered evidence meaningful and potentially probative. We summarise these methods categories below:

1. Expert evaluation (can be completed by one or more trained professional reviewers)

• Qualitative content analysis can identify the presence of dark patterns, and can be validated through an audit trail that includes: a shared vocabulary of dark pattern types and/or characteristics; acknowledgement of how new dark patterns relate to known exemplars; and an acknowledgement of the expertise of the individual or individuals conducting the review. This method is commonly used by academic researchers.

• Heuristic analysis can identify the presence of dark patterns in a generative stance (looking for design opportunities) or evaluative stance (identifying “bugs” or potential usability flaws), and can be validated through an audit trail that includes: an acknowledgment of the expertise of those conducting the review; use of an existing and validated set of heuristics; and identification of how dark patterns violate an existing heuristic standard. This method is commonly used by UX researchers working in industry and less commonly by academic researchers.

2. User studies (must allow end users to engage with a final or potential interactive system)

• Evocative user studies can identify open-ended design opportunities (e.g., participatory design, field study, focus groups, diary study) where dark patterns could be included or removed. Because the goal of this method is generative and oriented towards creativity, no audit trail characteristics are relevant; however, prompts used to trigger user responses may frame responses in a non-deterministic manner, identifying anticipatory harms or potential motivations. This method is commonly used by UX researchers and less frequently by academic researchers.

• Validation oriented user studies can identify user comprehension or sensemaking in relation to key aspects of a system in moderated (e.g., contextual inquiry, concept testing, usability testing) or unmoderated form (e.g., survey, usability testing, field studies). This method can be validated through an audit trail that includes: a set of shared user tasks or goals used to prompt user interaction; use of an interactive system that includes design choice(s) being evaluated; and collection of multiple sources of data, commonly including video, audio, interactive decisions, and demographic characteristics. Outcomes of this method should demonstrate trends or insights relating to users’ understanding of the system or impacts on user agency to be probative. This method is commonly used by UX researchers and less frequently by academic researchers.

• Experimental user studies can identify a causal link between specific design choices that incorporate dark patterns in a particular use context and changes in user behaviour. Many methods can support experimental studies, including collection of behavioural metrics (e.g., eyetracking) or responses to goals or scenarios in an unmoderated system (e.g., survey, remote or in person observation). This method can be validated through an audit trail that includes: an ecologically-meaningful framing of one or more design choices; a selection of experimental and control group conditions that addresses potential study confounds; and a sample of users randomly distributed that reflects the population being characterized. Outcomes of this method should have statistical significance to be deemed probative. This method is commonly used by academic researchers.

3. Log interaction analysis (must collect trace data through a user’s interaction with a system or other code-auditable means)

• User flow evaluation can identify common pathways through a system through analysis of log data that is captured by a live system (e.g., pages accessed, timestamps, unique identifiers). This approach can be used to optimize flows or identify areas for design improvement. This method can be validated through an audit trail that includes: a set of predefined stakeholder goals; an analysis of performance metrics (e.g., time on task, number of clicks); and an analysis of the sample(s) of users reflected in performance outcomes to address any confounds. Outcomes of this method may be methodologically triangulated with findings from other sources (e.g., user studies) to identify causal factors that might lead to final design decisions. This method is commonly used by UX researchers.

• A/B testing can identify the relative performance of two or more design choices in comparison to pre-defined goals as experienced on a live system (e.g., user retention, selection of choices, conversion to signups). This approach can be used in a fine-grained manner to test the performance of small design choices (e.g., colors, positioning of buttons) or entirely different experiences of a platform. This method can be validated through an audit trail with: a set of predefined stakeholder goals; a comparison of performance metrics; and an analysis of the sample(s) of users reflected in performance outcomes to address any confounds. Outcomes of this method may have statistical significance, or metrics may be methodologically triangulated with findings from other sources (e.g., user studies) to identify causal factors that might lead to final design decisions. This method is commonly used by UX researchers.

This synthesis shows that a range of methods and types of evidence inspected are already present to identify and characterise dark patterns.

Methods inferred in notable dark patterns cases

In this section we summarise our mapping of the studied prominent legal cases to the methods used for identifying dark patterns from academic literature and industry practices. This analysis covers two jurisdictions: the EU, which follows a civil law system and is represented in our set of cases by administrative decisions issued by data protection authorities, and the US, a common law system through decisions issued by the FTC. It is important to note that EU decisions originate from administrative bodies responsible for enforcing the GDPR (Regulation 2016/679). These bodies are not courts and operate under different procedural rules than the judiciary. In some respects, they possess greater flexibility, as they are not bound by the stringent evidentiary standards that govern judicial proceedings. We consider administrative decisions since so far no judicial decision has directly mentioned dark patterns.

Below, we analyse each decision with respect to the following aspects: the subject matter of the case with reference to dark patterns, the extent to which scholarly publications and/or other types of evidence were referenced in its decision, and the methods inferred to have been used in identifying dark patterns. Table 3 depicts our mapping.

EU cases

Case 9870014 of Garante per la protezione dei dati personali (Italian DPA) against Ediscom SpA

Subject matter: In the EU, the first regulatory decision to explicitly refer to dark patterns was issued by the Italian DPA against Ediscom SpA (Case 9870014, 2023) regarding Ediscom’s promotional campaigns via its websites. The decision discusses several practices related to dark patterns (e.g., lack of relevant information on one of the websites) but only consent settings are explicitly named as such.

Evidence used: The decision resorts to the formal qualification of dark patterns in the EDPB guidelines on dark patterns for "greater clarity" (European Data Protection Board [EDPB], 2022). The case documents contained visual evidence (such as screenshots) of the violating practices, and additionally mentioned company documentation as well as whistleblower and user reports. Such visualisations can help demonstrate dark patterns across legal literature and provide concrete evidentiary examples for the future decision making with the same or corresponding type of interfaces or violations. Scholarly works were not referenced.

Methods: From the case, we infer that Qualitative content analysis was used to assess interfaces for dark pattern practices, as reconciled against internal and user-reported sources. A form of User flow evaluation is also inferred by the structure in which the case describes the steps of a user’s consent flow.

PS/00080/2023 of the AEPD (Spanish DPA) against Chatwith.IO

Subject matter: The Spanish case PS/00080/2023 (2023), brought against website operator Chatwith.IO explicitly addresses the use of dark patterns in relation to users' right to object to the processing of their personal data by third parties.

Evidence used: This case directly references concrete dark patterns from the European Data Protection Board (EDPB) guidelines (particularly “overloading” and “skipping”) (EDPB, 2022). Other evidence includes the company’s legal documentation (e.g. privacy or cookie policies) to assess data collection practices. The case directly transcribes a detailed analysis of the consent banner UI and subsequent options presented to users—control panel, purposes for data collection, and the full list of third-parties. No academic sources were used.

Methods: We infer that Qualitative content analysis was used to assess interfaces, with a form of User flow evaluation.

Deliberation No. SAN-2021-023 of CNIL against Google

Subject matter: The French case SAN-2021-023 (2021a) brought against Google LLC and Google Ireland Limited investigated practices making it harder to refuse cookies than to accept them.

Evidence used: This case first includes prior rulings and detailed design instructions (for platforms to follow in order to comply with prior decisions) from previous cases and guidelines. It performs on-site audits conducted by the regulator for fact-checking. The decision describes the nature of providing consent to Google services at the time, discussing the relative simplicity between two consent options, with no screenshots provided. The regulator cites an industry study called “Privacy Barometer” (COMMANDERS ACT, 2021) for quantitative evidence of the impact of consent-based design choices and its effect on user autonomy (CNIL, 2021a para 134, 135). Additionally, it also cites a 2021 Kantar study measuring French refusals for cookie storage. These studies were used to establish tangible causal effects of explicit refusal options.

Methods: This case’ efforts to capture the number of actions for cookie acceptance in Google services maps to Qualitative content analysis, but uses results from cited Web measurement industry studies in its decision. We map the industry studies to measurement methods as can be inferred by the results and descriptions of the study; however, we see an issue with under-reporting methods in such reports.

Deliberation No. SAN-2021-024 of CNIL against Facebook

Subject matter: The French case SAN-2021-024 (2021b) was brought against Facebook Ireland Limited for obstructing users to refuse to give consent as simple as the mechanism provided for their acceptance.

Evidence used: The scholarly study by Nouwens et al. (2020) on dark patterns was mentioned (Déliberation SAN-2021-024, 2021b, para. 98) to support the regulator's interpretation of the GDPR requirement for a “reject” button next to the “accept” button on the first layer of the consent banner. The regulator provided its own analysis of the Facebook interface of consent banner, citing Nouwens et al. (2020) then mentioning that moving the “reject” button to the second layer increased the cookie acceptance rate by 23.1%. The regulator describes their own online check stepping through Facebook’s consent mechanism and describing the user interaction journey in detail (Déliberation SAN-2021-024, 2021, para. 108-109). Screenshots are not provided with these descriptions.

Methods: This case primarily relied on Qualitative content analysis, but uses results from cited Experimental user studies in its decision.

US cases

In US FTC cases, interfaces and designs have yet to be tied to any specific dark patterns by taxonomical labels. Rather, complaint documents discuss dark pattern mechanics broadly (e.g. “adding friction for friction’s sake” when quoting a UX designer), or detailing what user behaviours the dark patterns interfere with (e.g., purchasing items with minimal stopgaps) (Epic Games, In the matter of, 2022).

FTC v. Vonage

Subject matter: Vonage (Federal Trade Commission, 2022c; FTC v. Vonage, 2022) was charged with one violation of Section 5 of the FTC Act (for unfairly charging consumers without consent) and three violations of ROSCA (for failing to provide required disclosures, failing to obtain expressed informed consent before charges, and failing to provide simple mechanisms to stop recurring charges) (15 U.S.C. §45(a); 15 U.S.C. §§ 8401-8405). The term “dark patterns” is explicitly mentioned in the complaint document.
Evidence used: Cited evidence included consumer complaints sent directly to Vonage in text format or over the phone, as well as staff records and external consumer website complaints—thus both testimony and physical or visual evidence is included. This includes screenshots of Vonage interfaces and excerpts from internal company training materials, customer call records ( presumably as transcripts), internal emails, and more. Of interest here and unique to the enforcement context (versus academic contexts) are internal correspondences and call records. 
Methods: We map this case to Qualitative content analysis when visual evidence was analysed and broadly to User flow evaluation when other materials like customer call records are involved to analyse users’ goals and journey through the system.

Epic Games, In the matter of
Subject matter: Epic Games was charged with the counts of Unfair Billing and Unfair Account Denial, each of which were aided by several dark pattern designs in multiple iterations over a few years (Epic Games, In the matter of, 2022; Federal Trade Commission, 2022b). The initial complaint explicitly mentions Epic Games’ use of dark patterns.
Evidence used: The complaint discusses the problematic designs in great detail, providing screenshots as referential exhibits, with additional facts detailing steps in a user’s interaction journey for in-game purchases. Other described evidence includes consumer complaints as well as inquiries to Epic Games staff or internal communications (involving responses from UX designers and product managers). The use of asymmetric designs was verified by both longitudinal evidence reported by Fortnite users and staff acknowledgement of how business goals were implicated by the specific designs under scrutiny. 
Methods: We also map this case to Qualitative content analysis and potentially User flow evaluation when used to identify asymmetries between customers’ and business goals.

D.C. v. Google LLC

Subject matter: A complaint filed by the D.C. Attorney General against Google directly mentions dark patterns within the context of deceptive practices for manipulating user settings choices regarding location data (D.C. v. Google LLC., 2022). 
Evidence used: While the publicly available complaint filing redacted some details prior to settlement, evidence gathered and presented to support the several allegations regarding location tracking include detailed descriptions of the implicated interfaces and how they were utilised. Here, evidence spanned several iterations of Google’s interfaces and various changes made to asymmetrically promote consent over alternative options. From what can be gathered from unredacted content, the complaint drew from consumer-accessible interfaces and documents like privacy policies or other disclosures, with some mention of internal correspondence. 
Methods: We map this case to Qualitative content analysis for assessing visual evidence.

Challenges, opportunities, and recommendations

Differences in evidentiary approaches between empirical research and enforcement investigations

Case-specific scope and resultant contextual depth

Cases inherently have narrow scope: typically fact-checking one organisation’s practices (interface or user journey element, and/or a specific subset of their digital service or product). Claims and evidence are primarily supported by multiple sources, like copies, policies, records, and screenshots of the user interfaces provided by or requested to the companies. Cases may also be supported by analysis of longitudinal samples of these evidence sources over time, including consumer complaints on publicly available sites as noted in Vonage (2022) and Epic Games (2022). Any gained evidence and insights derived from cases are bound to the case-specific granularity and depth that triangulates the complainant, the defendant, and the enforcer. This approach differs from academic work, which tends to analyse data to cover more breadth across organisations, or practices from a given organisation that has multiple users, particularly in a replicable manner and from a more impartial stance to draw broader conclusions. Thus, the scope of academic research is generally not confined to just one or a few aspects of a single organisation. Despite these tensions between empirical research and enforcement investigations, leveraging research for evidence-based enforcement allows us to trace how certain identification methods used in enforcement implicitly rely on similar identification logic as those developed in scholarly and industry contexts, even if not formally acknowledged. In this way, the comparison helps surface methodological overlaps, misalignments, and opportunities for cross-fertilisation between research and enforcement practice. Academic research could aggregate evidence from multiple cases for different insights, leveraging case-specific depth as well as scholarly breadth. This presents an opportunity for regulators to release case evidence as data for scholarly uses where or when possible. However, the inherent constraints of legal procedure often mandates the destruction of evidentiary data and related copies; this remains an open challenge in navigating what evidence might be acceptably retained and aggregated for scholarly use.

Access to private or internal evidence

Enforcers can access internal evidence that scholars typically cannot, like customer calls, chat records, letters exchanged between parties, training materials for customer representatives, and more. Regulators may request these as part of formal audits per-case or through other investigatory actions, and organisations must comply with such requests when necessary conditions are met. Other private evidence may also be given to authorities (though in such situations, companies’ voluntary provision is likely intended to demonstrate that dark patterns were not implicated). This evidence may include companies’ internal practices documents or data. Such evidence is neither available to consumers nor academic researchers under normal circumstances, echoing similar challenges in studying black-boxed systems or algorithms (Gryz & Rojszczak, 2021). Although business incentives and legal requirements may keep such materials protected from external view (except when formally audited by an agency), the glaring difference in access to this information between scholars and regulators highlights a fundamental knowledge and methodological gap between academics and enforcers. Internal evidence, especially staff correspondence or materials targeted at employees, are uniquely capable of demonstrating unjust benefit and to another extent, intent to deceive or conduct unfair practices. Intent has long been an interesting caveat foiling definitions of ‘darkness’ in academic research, with scholars disclaiming that the presence of dark patterns should not be interpreted as inherently demonstrative of malicious intent on behalf of the designer (Gray et al., 2018). However, as noted in cases like Epic Games (2022) and Vonage (2022), internal documentation sometimes reveals calculated strategy, explicit instructions from managers to designers, or business goals that are supported by the use of dark patterns.

Challenges for aligning investigatory methods

The two CNIL cases citing academic studies present some promise for the future inclusion of scholarly evidence within case-law, but different jurisdictions may have varied appreciation. The “if necessary and possible” restrictions (or use of only the “most highly qualified” scholarship) as “subsidiary” limitations on scholarship-as-evidence described earlier present obstacles for more closely aligned investigatory methods between both fields. Other differences between academia and enforcement present other obstacles for easy or direct transfer of knowledge and evidence between domains in both directions.

As inferred from case documents, case law and enforcement tends to rely on Expert evaluation techniques performed by enforcers who identify violations (and sometimes mention dark patterns) via Qualitative content analysis. However, these Expert evaluation techniques and their audit trails are not disclosed (or even named as such), making it difficult to assess methodological rigour using criteria common in academic research, which may be helpful for scholars seeking to design methods more aligned with enforcement practice. While findings from research studies are arguably likely to have probative value for regulatory decisions, these studies are scattered across various disciplines, such as computer science, social science, design, and law, and may not be immediately known to regulators or enforcers at the moment of decision-making. Quality criteria and leading scientific venues differ across domains, and those responsible for enforcement may not be keenly aware of these reputational variations. Peer reviews assure this quality purpose but are inherently tied to field-specific expertise. Thus, the effort to validate scholarship may be inefficient for regulators deciding upon a case, given the requisite domain expertise necessary for evaluation. Some methods used in scholarship or UXR come from fields outside of law and regulatory practices (e.g. heuristic analysis or web measurements). Translating such work into legally useful terms or necessary procedures may be difficult, and might not meet enforcement criteria for determining dark pattern practices. Similarly, to prove a case against a potentially non-compliant platform, regulators are bound to evaluate the evidence produced therein as a matter of procedure, even if scholarship may have identified the exact behaviour by other means.

Recommendations for improving the evidentiary impact of scholarship

Both fields naturally have differing primary objectives: scholars seek to create and share their research evidence up to scientific standards of rigour and to publish their work for broader societal or scholarly impact; enforcers analyse evidence to enforce the law and sanction rogue practices. These objectives warrant different approaches, but we argue overlap might be better utilized, and gaps could be partially bridged through collaboration or improved information sharing.

Align research methods with regulatory contexts

Some organisations already incorporate scholarship, like the Spanish DPA extending the Gray et al. ontological framework (2024) and utilising its hierarchy (AEPD, 2024, p. 12). Consumer protections and advocacy organisations like the Swedish consumer organisation also cite ontological pattern names (Sveriges Konsumenter, 2024, p. 16). These efforts help align terminology and concepts across academia and law, but opportunities remain in better leveraging methods and tools across disciplines.

One way to narrow the gap would be to adapt scholarly methods for the specific context and practices used by legal practitioners within a regulatory authority. This would entail dark patterns scholarship (e.g., on privacy or e-commerce) designed to inform specific regulators (data or consumer authorities, respectively), or legal domains (data protection law, consumer protection or competition law), with methods accordingly supporting the needs of that type of regulatory body. As noted previously, some work already seeks to inform regulators and enforcers, like work that articulates and categorises normative perspectives of dark pattern darkness (Mathur et al., 2021), or the expanse of dark patterns scholarship that specifically implicates GDPR compliance (Matte et al., 2020; Nouwens et al., 2020; Soe et al., 2020; Utz et al., 2019), or autonomy violations for DSA compliance (Santos et al., 2024). We encourage the continuation of these efforts, noting that such work increasingly targets specific regulations.

Perhaps the next step along this trend would involve deeper acquaintance with the regulator's evidentiary needs and specific investigatory methods, and design experimental protocols accordingly. One example is with the nature of case facts. In our cited cases, case facts often described interface navigation steps in succession, sometimes with screenshots of problematic designs provided in filing documents. In Epic Games, the FTC even details the button mechanics on a gaming controller as pertaining to Fortnite’s purchase interfaces (2022). As such, more descriptive, contextual evidence that captures the dynamic interaction flow of a business practice may better suit the often narrative structure of case facts. Heuristic analysis, which is more commonly used in industry UX research rather than scholarly research, may be an avenue for academics to explore further in support of regulatory efforts. In fact, more recent scholarship has begun to work towards developing such methods, analysing documents from an FTC complaint and subsequently building a “temporal analysis for dark patterns” methodology intended for scholarly and legal uses (Gray et al., 2025). Future work could continue this direct engagement and inspiration in both directions: from law to scholarship and vice versa.

Provide further clarification regarding cases’ investigatory methods

Enforcers’ documentation and analysis of case facts may not articulate sufficient or replicable context for scholars to easily interpret investigatory methods, particularly for the manner in which information was collected (e.g., from consumer complaint documents, from interaction flow walkthroughs, user interfaces, etc). Decisions and complaints also offer different reporting styles across jurisdictions, which makes analysis of facts, collected evidence (e.g. screenshots), and resultant legal reasoning potentially hard to systematically distinguish for other disciplines. Variance in case documentation is an inherent part of jurisdictional differences, and may be impossible to align fully across jurisdictions or for scholarly utility (for example, if methodological/investigatory confidentiality is required by each regulator’s internal policy). However, interdisciplinary researchers seeking to better support enforcement proceedings may benefit from even partial or implicit disclosure of enforcement’s investigatory methodologies. Increasing methodological disclosure within regulatory decisions, to some extent, has potential advantages: scholars may replicate or scale investigations in ways that courts or regulators may not have resources for (including, perhaps, evaluating industry trends and consensus to identify market outliers); for regulators, published or otherwise validated work using more closely aligned methods may improve the utility of that evidence for enforcement investigations.

Strengthen and leverage semi-formal avenues for collaboration

Finally, given inherent constraints per discipline, a potentially underutilised middle ground towards collaboration might be provided by semi-formal investigatory actions (though the extent to which such semi-formal actions may be used for formal enforcement versus general market analysis remains a broader debate not within the scope of this article). Such mechanisms already have some precedent, where civil society organisations (NOYB, 2021) have conducted website audits themselves that triggered enforcement procedures and informed official guidelines (EDPB, 2023). In the EU or in international collaboration, these actions might be performed through “sweeps,” and “audits,” while in the US these might be conducted through agency studies. Concerns identified during sweeps could not only result in follow-up work, such as outreach to organisations, but may also lead to the initiation of enforcement actions to address identified concerns. Table 4 portrays the methods used in known dark pattern sweeps and maps them to the methods used for identification of dark patterns. These investigations, insofar as their approaches are less constrained to one platform at a time, could utilise a variety of scholarly methods are a promising avenue for regulators to use scientific methods, and thus for methodology transfer between academia and law or even for direct collaboration between researchers and enforcement agencies. Formalising avenues for scholars to join sweeps, or to otherwise create processes for direct academic involvement, could be implemented to benefit regulatory action. That is, earlier inclusion could leverage academics’ methodological and technical expertise towards tailored, synchronised methodological development or investigations, e.g. for novel detection techniques or tool-building (automated pattern detection, usability studies, A/B testing), or user studies measuring the impact and effect of phenomena like dark patterns. Conversely, regulators may want to inform academic taxonomies or even guide study design to yield evidence or definitions in a format that may be immediately useful to ongoing investigations.

Conclusion

Our interdisciplinary team of researchers spanning human-computer interaction, design, computer science, and law presents dark patterns as a case study by which we reason through methodologies across disciplines. We begin by analysing groups of methods from scholarly literature and discuss their purpose, resultant evidence types, strengths, and limitations, then synthesise these methods against broader UX research methods and frame scholarly methods along different stages in product development or release. We then consider key, decided dark patterns enforcement cases and provide inferred mappings to scholarly methods from the cases provided. We also discuss other evidentiary sources available to enforcers.

Finally, we reflect upon our analyses and mappings considering the ongoing challenges of academic-enforcement collaboration for dark patterns. We propose suggestions for both scholars and enforcers to tighten feedback loops, and highlight informal investigation methods as an opportunity to strengthen collaboration.