The violence of the majority: Rethinking AI positionality in decision-making

The concept of consensus has long been celebrated as a cornerstone of democratic decision-making. However, when viewed through the lens of AI, consensus can reveal a darker side – what some call the "violence of the majority." AI systems, trained predominantly on majority perspectives, often marginalise the voices of underrepresented groups (Ricaurte, 2022; Shams et al., 2023). This exclusion is not necessarily the fault of engineers or designers but rather a structural issue rooted in insufficient or biased data. AI systems built on common population distributions inherently fail populations outside those datasets, leading to a vicious cycle where the underrepresented become further excluded (Buolamwini & Gebru, 2018).

Addressing these inequities begins with a fundamental question: Who builds the indicators and metrics that AI uses to guide decisions? Current systems, reliant on datasets that reflect societal biases, struggle to adapt without data representative of marginalised communities (Bose, 2024; Noriega Campero, 2019). This inability to "actively explore" or acquire new information exacerbates AI's limitations, particularly in contexts of distributional shifts, where the input data diverges significantly from the system's training data (Cherepanova et al., 2022). These challenges are particularly acute when AI interacts with underrepresented groups, risking reinforcement of existing disparities under the guise of objectivity.

The issue lies not merely in data imbalances but in the structural design of AI systems. For example, AI-based agricultural tools deployed in the Global South often fail to account for local languages, customs, or gender dynamics. In India, predictive tools designed for crop yield optimisation have struggled to address the unique challenges faced by smallholder farmers, particularly women, who often lack access to the technologies assumed to be universal (Rupavatharam et al., 2023). Including such global examples broadens the discussion and highlights the universal need for inclusive AI systems.

In fact, many AI models are optimised for accuracy across the largest dataset segments, inherently sidelining minority experiences (Buolamwini & Gebru, 2018). This approach reflects a utilitarian logic that prioritises efficiency over equity, creating a self-reinforcing cycle of exclusion (Ravanera & Kaplan ,2021). Consider facial recognition algorithms, which consistently perform worse for people with darker skin tones – a glaring example of how datasets drawn predominantly from lighter-skinned populations lead to biased outcomes (Buolamwini & Gebru, 2018). These biases are not incidental; they stem from the metrics and indicators chosen to guide AI decisions, metrics often defined without the input of marginalised groups (Ferrer et al., 2021; Hagendorff, 2022).

Moreover, AI’s limitations become even more pronounced in contexts of distributional shifts, where input data diverges from the training data. These shifts often occur in rapidly changing environments, such as public health crises, climate disasters, or sociopolitical upheavals (Rauba et al., 2024). In such cases, AI systems are prone to failure, as they cannot autonomously acquire or interpret new data reflective of the altered context. This inability to adapt is particularly detrimental for marginalised communities, whose experiences may deviate most significantly from the normative patterns encoded in AI models (Ofosu-Asare, 2024). For example, during the COVID-19 pandemic, predictive models for resource allocation often failed to account for the unique vulnerabilities of minority populations. To rectify this, AI systems must be designed not only to "learn" from historical data but to actively seek out and incorporate new information that captures diverse and shifting realities.

So what? The need to rethink the positionality of AI

The root of the problem lies in the positionality of those building and deploying AI systems. Positionality refers to the social and cultural context that influences an individual's identity, perspectives, and assumptions, which in turn affect how they engage with knowledge production, decision-making, and interactions with others. In academic research and critical theory, positionality emphasises the acknowledgment of one's own background, privileges, biases, and power dynamics in shaping their perspective. Positionality involves recognising how factors such as race, gender, socioeconomic status, and cultural background shape an individual's worldview. This concept is critical in fields like anthropology, sociology, and AI ethics, as it highlights the importance of reflexivity and the recognition of potential biases in knowledge creation and system design (O’Neill, 2024; Rose, 1997). In the context of AI, positionality is increasingly discussed as a factor influencing the design, development, and deployment of AI systems. For example, the creators’ positionality can lead to blind spots in system design, reinforcing existing biases and excluding marginalised voices (Aguilar et al., 2023).

In the context of AI, metrics and indicators, the lifeblood of AI decision-making, are rarely neutral. They are shaped by the perspectives, priorities, and blind spots of their creators. If the creators lack diverse representation, the resulting systems will reflect those gaps (Roberts, 2022). For instance, in predictive policing, data about past arrests often reflects systemic racism in law enforcement. When such data is fed into AI, the system perpetuates the same discriminatory patterns under the guise of scientific objectivity (Rossbach, 2023).

Addressing these inequities requires a fundamental shift in who builds and validates the systems. This involves not just diversifying the teams that create AI but also questioning the values and assumptions embedded in the technology (Cachat-Rosset & Klarsfeld, 2023). Positionality must be recognised as a critical factor in AI ethics, one that demands the inclusion of underrepresented perspectives at every stage – from data collection to algorithmic design and deployment.