Smart technologies

Introduction: the rise of smart ‘anywares’

A wide range of products and services is currently discussed under the heading of ‘smart’, loosely overlapping with the technologies of (cloud) robotics, Internet of Things (IoT) and more generally to hardwired applications of AI. 1 More concretely we can think of semi-autonomous vehicles (smart cars), cyberphysical infrastructures such as homes animated with sensor technologies that enable surreptitious adaptation of temperature, light, and all kinds of services (smart homes), or energy networks that afford real-time demand-side energy supply systems (smart grids) and myriad online architectures that require responsive and adaptive interaction, such as collaborative platforms (smart office) and e-learning (smart education). Most of these systems will be hybrid between online and offline, as exemplified by crime mapping (smart policing), social security fraud detection (smart forensics), the sharing economy (smart services), remote healthcare (smart healthcare) and initiatives in the realm of computational law that employ the alluring notion of smart law to sell their services. We seem to have arrived in the era of smart everyware (Greenfield, 2006), confronted with myriad smart anywares.

In this article I first ground the concept of smartness in the history of artificial intelligence and cybernetics (sections 1 and 2), highlighting the different strands in the research domains that enabled the development of smart technologies. Though this will bring out the smartness of those selling the idea of computational systems as cognitive engines, there is more to smart technologies than some social constructivists may wish to believe. 2 Grounded in postphenemenological philosophy and embodied enaction I will develop a layered approach to the kind of machine agency involved in smart technologies (section 3) that allows me to simultaneously reject confounding human and machine agency while nevertheless acknowledging the fact that their computational inferences are used to anticipate our behaviours, thus introducing agential affordances to our environments. Coupled with nudge theory such affordances have resulted in attempts to manipulate consumers (behavioural advertising, credit rating) and further manipulation in the political realm that has been reconfigured as a marketplace under the influence of neoliberal economics (section 4). In the final part I address the need to confront the affordances of smart environments, taking their machine agency seriously without buying into either marketing speak or social constructivist relativism (section 5).

1. Smart-arse technologies: avoiding the AI debate

Why would one call a technology smart? The Cambridge English Dictionary (2020) tells us that smart is about ‘having a clean, tidy, and stylish appearance (mainly in the UK)’, and/or refers to someone who is ‘intelligent, or able to think quickly or intelligently in difficult situations (mainly in the US)’. The geographical difference raises some interesting questions about cultural inclinations, but that may be merely a smart-arse move on my side, taking note that a smart-arse is defined in the same dictionary as ‘someone who is always trying to seem more clever than other people in a way that is annoying’. The answer must be situated in the computer-related adjective, which is defined as ‘[a] smart machine, weapon, etc. uses computers to make it work so that it is able to act in an independent way: Until the advent of smart weapons, repeated attacks were needed to ensure the destruction of targets’ (Cambridge Dictionary 2020). This must be what my article here aims to describe: technologies with a mindless mind of their own, ‘getting on with things’ such that we have time to do more important things. Let’s not forget, however, that the term normally refers to something aesthetic (perhaps to sleek interfaces that intuitively guide our usage based on a clean, tidy and stylish interface) and that it otherwise refers to a clever type of acuity, or intelligence. Acting in an independent way comes close, though one wonders in what sense smart weapons act independently, considering the fact that they are designed by humans with specific computational skills in mind and are also used by humans to destroy very precise targets. The independence seems to only refer to how these smart technologies achieve the goals set for them, even if one could say that they may be able to develop subgoals if they ‘think’ it will help to achieve our goals (Boulanin & Verbruggen, 2017). One could ask who the smart-arse is here: the human or the machine.

McCarthy, the AI pioneer who coined the term artificial intelligence (AI) in the famous 1956 Dartmouth conference (Leslie, 2019), certainly was smart. One of the other founding fathers of what we now refer to as AI, Herbert Simon, preferred to stick to the term ‘complex information processing’ (Leslie, 2019 at 32), because it supposedly better described the operations they were conceptualising. McCarthy, however, had the luminous insight that nobody would invest much in something so dry as ‘complex information processing’, whereas the mysterious idea of ‘artificial intelligence’ might trigger some real interest (Leslie, 2019 at 32) – let’s acknowledge McCarthy’s skills in the realm of public relations (PR). One of the reasons why I used the term smart technologies in one of my books, was to avoid the endless debates about whether machines can be intelligent, what is meant with intelligence and whether and, if so, how machine intelligence differs from both animal and human intelligence. Perhaps the more intriguing question is why it was not simply called machine or computational intelligence. I would actually claim that human intelligence itself is deeply artificial (Hildebrandt, 2020b; Plessner & Bernstein 2019). Speaking of machine or computational intelligence would probably highlight the difference that McCarthy wished to ignore, as becomes clear in the (cl)aim of those early pioneers who professed that (Moor, 2006):

[t]he study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.

Against such overreach, I dare say that the advantage of the term smart technologies is first and foremost that it avoids metaphysical debates on (1) whether machines ‘think’, (2) whether it suffices to answer that question in terms of observable behaviour only, and perhaps also (3) the debate on whether intelligence requires ‘thinking’ in the first place, considering that viruses and reptiles do very well in terms of independent navigation – even though we may think that they don’t think. 3 Speaking of smart instead of AI technologies allows us to be more pragmatic about their added value, steering free from the pseudo-religious (and thereby pseudo-scientific) claims around ‘artificial intelligence’ (Natale & Ballatore, 2020).

This may, however, result in those advocating smart technologies getting away with even bolder claims about their added value, precisely because the threat of a competing intelligence is sidetracked. Adopting the term may therefore be a smart move to downplay the extent to which smart technologies are capable of reconfiguring our choice architecture. This reconfiguration is often meant to manipulate our intent, as the entanglement of nudge theory and machine learning demonstrates in the realm of behavioural targeting (for advertising, recruiting, insurance, admission to college or parole decisions). Such entanglement hides the choices made at the backend of computing systems by those who stand to gain from their employment, or those intent on serving our best interests (though preferably behind our backs, see e.g. Yeung, 2017).

This confronts us with the question of ‘smart for whom?’, which is a political question. Ignoring it will not make such backend choices neutral. Perhaps we should coin these technologies, smart-arse technologies, highlighting their ability to manipulate our environment and our selves, without, however, attributing thoughts or intent to technologies that can only follow their masters’ script. Though their masters may actually be at a loss, due to the interacting network effects and emergent behaviours they generate (known as the Goodhart effect) (Strathern, 1997).

2. Smart technologies: embracing the cybernetic approach

Being at a loss is deeply human. Machines are never at a loss, because they lack the mind for it. But machines or technologies can help humans to gain control, if they somehow get smart about how to achieve their goals (I am glad to leave ‘they’ undefined here). The aim of increased control, often achieved by developing remote control, is key to another branch of what became AI, namely cybernetics (Wiener, 1948) or operations research (Pickering, 2002). Cybernetics is focused on developing the right kind of feedback loops, such that a technology is capable of adapting its behaviour to changing circumstances, based on its perception. This fits well with the idea that perception is triggered by the need to anticipate how one’s actions will affect one’s environment, and how this will in turn affect one’s action-potential, familiar in phenomenological research and affordance theory (Kopljar, 2016; Gibson, 2014). A narrower, machinic understanding often describes cybernetics as a specific type of regulation, entailing standard-setting, behaviour monitoring and behaviour modification. This obviously accords with computational systems, which require formalisation and thrive on disambiguated instructions. One could say that these systems require formalised regulation in order to regulate their environment, even if their approach is data-driven and based on machine learning. Even in that case input must be formalised (digitised, labelled or clustered) and the patterns mined are based on mathematical hypotheses (Blockeel, 2010). Cyber is Greek for steering (Hayles, 1999 at 8), and though steering can be done in many ways, computational systems necessarily reduce the art of steering to computational ‘algorithmics’. One could surmise that the notion of smart technologies is connected with the idea of remote control over an environment, for instance in the case of hardware failure detection combined with automated interventions once failure is detected.

In point of fact, SMART technologies originally referred to the acronym of self-monitoring, analysis and reporting technology (SMART), mainly used for failure prediction in hardware (Gaber et al., 2017). This is interesting because such SMART-ness remains focused on monitoring and reporting, without necessarily including an automated or even autonomous response in order to repair. In that sense the current use of the term smart is different as it refers to systems capable of responding to changes in their environment based on input data, and it is precisely the notion of control and the ability to steer that are central to what we now call smart technologies, both within the context of online and offline systems. Smart technologies are digital technologies of control, and the most important question will be whose control they facilitate. We will return to this question in section 4.

3. Levels of being smart: the diversity of non-human agents

The term smart is not a technical term within either computer science, law, the life sciences or social science. It may have been initiated as a marketing term, hoping to lure investors and potential users, offering an intuitive way to speak about computational systems that offer real world interventions supposedly achieving efficiency, convenience, or novel products and services. Though ‘stuff’ may be sold under the heading of smart even if it has no adaptive qualities, the core idea is that smart technologies do have some level of agency, in the practical sense of being able to respond to input data with behaviours that are conducive to our goals (who “our” refers to is the key question, see the final section).

Smartness, however, is not a categorical attribute of a technology, as there are different levels of being smart. Much depends on the extent to which its adaptive behaviours have been premeditated by the developer, which corresponds with the extent to which a technology can be said to express its own agency. Simultaneously, much depends on the complexity of the environment, taking note of the fact that environments are often stabilised or even closed, to ensure safe functioning of a smart technology. In robotics the environment of the robot, called its ‘envelop’, is usually developed in tandem with the robot, aligning its ‘degrees of freedom’ with the physical properties of the envelop (Floridi, 2014a). This way developers can more easily foresee the scenarios they must code for, preventing undefined behaviours of the device. Many roboticists foresee that self-driving cars will only hit the road if the road infrastructure is locked off from e.g. pedestrians, making sure the car runs within a relatively predictable environment, meeting only other self-driving cars on connected roads (Brooks, 2018).

If we acknowledge the fact that a technology may exhibit a specific type of agency, we can move away from discussions of whether they ‘have’ humanesque agency or not. My take is that they don’t, because they are embodied otherwise and that has consequences (Pfeifer & Bongard, 2007). Also, why invest in human-like agents if we already have wonderful ways to reproduce? Aficionados such as Kurzweil (2005) will claim that artificial agents will, however, soon outperform us and reach a singularity where humanity becomes a historical footnote to an intelligence too advanced for us to even conceive (though he did, nevertheless). I would say that here we enter the realm of pseudo-religious devotion if not hubris, observing that most believers in such singularity come from outside the domain of computer science, uninformed by the limits imposed by physics, complexity and much more (Walsh, 2017). It makes more sense to acknowledge that plants, animals and smart technologies all have different but nevertheless pivotal types of agency, and thereby differ from stones, screwdrivers and volcanoes. This will allow us to focus on figuring out what kind of agency is typical for smart technologies and raise the crucial question of how it may affect human agency. In this section I will discuss three levels of smartness, in the next I will probe into the implications for human agency.

Based on a series of seminal definitions of agency (Floridi & Sanders, 2004; Pfeifer & Bongard, 2007; Russell et al., 2010; Steels, 1995; Varela et al., 1991) I have compiled a definition that depicts agency as (Hildebrandt, 2015 at 22):

the capability:

• to act autonomously,
• to perceive and respond to changes in the environment,
• to endure as a unity of action over a prolonged period of time and
• to adapt behaviours to cope with new circumstances.

Let’s note that the term autonomous here does not refer to moral autonomy and does not assume human mindness, it simply refers to an entity capable of goal-oriented behaviour without intervention of its creator, designer or engineer. Let’s also note that the most crucial aspect of agency is the combination of perception and action; a simple thermostat perceives temperature and if a threshold is met it acts by triggering the heating system. Though nobody would imply that a thermostat can think, is autonomous in the sense of deciding which temperature we prefer or is in need of human rights to protect their dignity, a thermostat differs substantially from a water tap that only provides water after we open it. The thermostat has a minimal type of agency. Interestingly, we would not call such a thermostat smart, because it is not learning from our behaviour, it does not adapt to the way we set its temperature.

A smart thermostat, however, responds to our behaviour by adapting its own behaviour in line with how it profiles us. This may suit us, or suit those who profit from the data it provides about our behaviour. The smart thermostat may not be designed to endure as a unity of action over a prolonged period of time, which would imply self assessment and self repair. Similarly, it may not be designed to reconfigure its architecture to survive in a changing environment. And this brings me back to the first aspect of agency, being autonomy.

Here we need to distinguish between automation and autonomy. A simple and a smart thermostat both display automation, but their level of autonomy differs. Though one could claim that both are capable of ‘goal-oriented behaviour without intervention of its creator, designer or engineer’, the smart thermostat has a broader palette of goal-oriented behaviour due to its learning capacity. The difference between automation (which does not imply agency) and autonomy (which does) is usually sought in the ability of a system to respond to its environment in a way that sustains its own endurance and prevents or repairs failure modes, notably in the face of uncertainty (Endsley, 2017). This means that autonomy here refers to a technology capable of behaving as an adaptive unity of action over the course of time, highlighting the overlap and the interaction with the different aspects of machine agency or smartness.

This has led me to distinguish three levels of machine agency (Hildebrandt, 2015, chapter 2):

1. Agency of logic-based systems that are by definition deterministic and explainable, though in real life their interaction with unforeseen events may cause unforeseeable behaviour and the more complex their decision-tree and the more complex their environment, the more difficult it may be to reconstruct a correct explanation. A simple thermostat is a fine example of such a system, but so is a fraud detection system based entirely on a complex but preconceived decision tree. This is the type of agency closely related to good old fashioned artificial intelligence (GOFAI), grounded in knowledge representation and symbol manipulation (Haugeland, 1997, at 16).
2. Agency of machine learning systems that are capable of perceiving their environment (in terms of data), and of improving their performance of a specified task based on such perception. Such systems thrive on iterant feedback loops that are validated in terms of pattern recognition, which is ultimately grounded in statistical correlations expressed in mathematical functions. Examples of these types of systems abound: from e-learning to fintech and from load balancing in smart energy networks to facial recognition technology, machine translation as well as personal digital assistants such as Siri or Alexa. This type of agency is closely related to cybernetics and to artificial intelligence the modern approach (AIMA), grounded in inductive inferences and predictive analytics (Russell, Norvig, & Davis, 2010).
3. Agency of multi-agent systems (MASs) where a large number of agents interact, each based on their own goal or scripts, which results in emergent behaviours of the overall system due to network effects and complex dynamics (Vázquez-Salceda, Dignum, & Dignum, 2005). The agents can be logic-based or based on machine learning and the environment that a MAS navigates and regulates can be closed or open, predictable or unpredictable. Interestingly, a MAS will often be a distributed system that in itself constitutes an environment for humans and other types of agents, thus blurring the borders between an agent, an agent system and an environment. Whereas a smart fridge may be an identifiable agent in my home environment, my smart home that consists of interacting agents working together to better serve my needs (or those of the provider) is an environment rather than an agent. This is precisely where things become complicated, opaque and potentially turbulent. Some have termed this situation, where the environment itself takes on agency, an onlife world (Floridi, 2014b). Others have highlighted how the continuous and reflexive updating of one’s environment may reshape our own agency, though some do and others don’t make a difference between the nature of expectations between human agents and those between a human and a smart environment (Dittrich, 2002; Esposito, 2017).

Obviously, this tripartite categorisation is not the ultimate way to frame different levels of smartness, while noting that these levels interact and morph where smart technologies meet. This is where the difference between smart technologies and smart infrastructures becomes pivotal, especially since so many smart infrastructures are data-driven and surreptitiously reconfigure our choice architecture and our life world on the basis of our behavioural data. As indicated, this has been framed under the headings of Ambient Intelligence, the Internet of Things, cloud robotics, smart cities, connected cars or smart energy grids. All these headings refer to what computer science now refers to as cyberphysical infrastructures (Suh et al., 2014). As such infrastructure increasingly configures our environments, it becomes critical infrastructure. This will in turn augment our dependence on networked smart technologies and the energy requirements they involve, while also raising a number of issues of distributive justice, both with regard to access and with regard to potential unfair distribution of risks.

Although there seems to be an urge to discuss these societal implications in terms of ethics, I prefer to end this article with a discussion of the political economy of smart technologies, including some pointers to the importance of democracy and the rule of law. The main reason for this preference is the need to counter attempts to engage in ‘ethics-washing’ and ‘ethics shopping’ (Wagner, 2018) by teasing out the power relationships that are at stake. This is not to suggest that ethics is not relevant here (Hildebrandt, 2020a, chapter 11).

4. A political economy of smart technologies

For many decades economic analysis has been in the thralls of neo-classical economics, originating from the Chicago school of e.g. Milton Friedman, Gary Becker and Richard Posner, influenced by the Austrian-British philosopher and economist Hayek (2007), whose 1940 The Road to Serfdom celebrated an unconstrained market libertarianism. Becker’s rational choice theory inspired the Chicago school of political economy, applying the atomistic type of methodological individualism that grounds rational choice theory when used in for instance crime control and antitrust policy. Those unimpressed by the somewhat metaphysical and unsubstantiated claims of libertarian market fundamentalism usually address the conglomerate of neo-classical economic policies as ‘neoliberalism’ (Blakely, 2020). Under that heading I would also include the application of Kahneman and Tversky’s (1979) models of cognitive science to economics, for which they won the Nobel Prize. Their particular strand of neoliberalism identifies as behavioural economics, popularised as nudge theory by Thaler and Sunstein (2008). Nudge theory, in turn, fits very well with machine learning applications that aim to manipulate ‘users’ of smart technologies into specific behaviours, whether purchasing, voting or eating. Both nudge theory and machine learning thrive on an atomistic variant of methodological individualism that is closely aligned with utilitarianism (Hildebrandt, 2020a, chapter 11). Nudge theory’s alliance with behavioural economics has resulted in paternalistic libertarianism (Sunstein, 2016), though not everyone buys into the sneaky tyranny it professes (Yeung, 2017). Meanwhile most of our free services feed on behavioural advertising, which is firmly grounded on the pseudo-science of behavioural nudging by way of real-time-bidding systems, ultimately built on the quicksand of the same contested branch of cognitive science mentioned above (see Gigerenzer (2018) and Lepenies and Małecka (2019) on the pitfalls of nudge theory). The unwieldy marriage between machine learning and nudge theory has nevertheless been a major success for the rather deep pockets of big tech companies (Frederik & Martijn, 2019; Lomas, 2019a, 2019b).

To properly understand the political economy of smart technologies it may be more interesting to study the work of Karl Polanyi (2012), whose seminal The Great Transformation is inspiring a new generation of scholars. For instance, Benkler (2018), Britton-Purdy et al. (2020) and Cohen (2019) are intent on a better understanding of how law and the rule of law are implicated in the rise of economic superpowers such as big tech. They demonstrate the need to reinvent the countervailing powers of Montesquieu’s checks and balances, or calling for new versions of Polanyi’s counter-movements to put a halt to predatory capitalism. It is interesting that some of these scholars have been studying the advent of smart technologies for decades, notably investigating how the smart aspects of big tech products and services invalidate previous forms of legal protection, playing into the hands of those who control the backend systems.

In my own Smart Technologies and the End(s) of Law (Hildebrandt, 2015), I argue that we must learn to interact with smart systems, instead of perpetuating the illusion that we are using them whereas these systems are often using us (as data engines for their data-driven nudging). Moving from usage to interaction will require keen attention to effective and actionable transparency (Schraefel et al., 2020) and what some have called ‘throttling mechanisms’ that slow down high-frequency profiling by big tech platforms (Ohm, 2020). It seems clear that the kind of transparency and the human timescale that is needed to reorient cybernetic control back to individual human beings will require legislation, supervisory oversight and dedicated case law to rebalance current power relationships (Hildebrandt, 2018). Though developers of smart technologies often speak of human centred design, the current economic incentive structure reduces such humane approaches to PR about ‘the human in the loop’. Instead, we need an approach that puts the machine back in its proper place: smart technologies should be ‘in the loop’, while human beings navigate an animated technological landscape that serves them instead of spying on them (Zuboff, 2019).

5. Conclusion: human acuity and smart machines

Speaking of smart technologies could have the advantage of steering free from the snake oil narratives of AI, while still hinting at the cybernetics that inspires their design. But as it is not a technical term it can mean anything to anybody, and this becomes a drawback when it is used to sell systems based on unsubstantiated claims, as often happens under the heading of AI or machine learning (Giles, 2018; Lipton & Steinhardt, 2018).

Users and those forced to live with these systems (in smart cities, under surveillance of smart policing, at the mercy of smart recruiting) should ask themselves “smart for whom?”, “smart in what sense?” and “smart compared to what?”. Though many so-called smart applications are sold as ‘outperforming humans’ there is much to say about this (Cabitza et al., 2017), as such bombastic claims are based on performance metrics that refer to accuracy in the context of a data set, which may say little about the real-world performance (e.g. Caruana et al., 2015). Humans do not live in a data set, they navigate the material and institutional fabric of a real world, and they flourish based on an acuity that is not within the remit of smart machines.

Some will say the latter is an open question, others are certain that artificial general intelligence is around the corner. Still others focus their attention on how smart technologies affect human agency, taking a more pragmatic and simultaneously concerned approach. Referring to technologies as smart hopefully avoids metaphysical beliefs in technologies that cannot but follow a script, even if that script allows them to reorganise their internal structure to perform better on a given task. What matters is that human beings must be able to anticipate how the machinic agency of clever engineering feats profiles them, noting the difference between the fragile beauty of human acuity and the precocious foresight of machines that result from human ingenuity.