’Governance by Things’ as a challenge to regulation by law
AbstractIn this paper the authors examine how the rise of the Internet of Things will challenge regulatory structures. Coming from the idea of “code as law” the shift from technology governing online spaces to physical spaces is described as a new phenomenon. They call it ‘Governance by Things’. Some key observations of this structural shift are characterised in this article, for instance regarding its self-executing character and the imperfection of technology. Finally, the authors draw the conclusion that the ‘Governance by Things’ calls for a second-order regulation.
Licence: Creative Commons Attribution 3.0 Germany
Competing interests: The author has declared that no competing interests exist that have influenced the text.
Keywords: Governance by Things, Internet of things, Regulation, Private ordering
Citation: Schulz, W. & Dankert, K. (2016). ’Governance by Things’ as a challenge to regulation by law. Internet Policy Review, 5(2). DOI: 10.14763/2016.2.409
Recent debates in the media (Curtis, 2015; Hardy, 2015; Peterson, 2015) and academic discourse about the Internet of Things (hereinafter ‘IoT’) are stimulating public interest in this topic. Although the discussion around autonomous cars is one of the most prominent at this point, the phenomenon will likely not be restricted to the anticipated changes in traffic, but will further have an impact on all spheres of human life. Debates in the media and also in the legal academic discourse revolve around specific questions on the existing legal framework, especially rules of liability. In this article we propose taking a step back to examine the implications of the IoT for regulation and law from a broader perspective. Therefore, in this paper we highlight some of the important aspects surrounding how these developments will affect our ideas of governance and regulation by law. We believe that these issues indicate fundamental challenges for governance concepts.
There is no need to jump to the conclusion that the end of law has come (Hildebrandt, 2015). There have been statements to that effect in the 1960s that proclaimed the end of politics and law triggered by progress of technology and science (Schelsky, 1965: pp. 453 et seq.). Those statements proved to be wrong. The recent developments call, however, for new legal concepts.
I. The rise of the internet of things
One is hard pressed to find a convincing, universally accepted definition for the IoT. On the one hand, the subsequent discussion shows the ever-evolving character of the IoT (Santucci, 2008); on the other hand, different disciplines tend to have different perspectives. 1 Technically it means combining technologies – especially sensors, actuators, data processing and communication – into a bundle with new usability (Mattern & Flörkemeier, 2010). If scholars from different disciplines were asked to define the phenomenon, there would be numerous different approaches: an information scientist might emphasise the opportunities to automate processes. A sociologist might define the IoT as an ambivalent social development with possible beneficiaries and losers within society (Davies, 2014). An economist might see this process as a chance to increase efficiency and welfare by replacing manual human labour with the work of intelligent machines (Fleisch, Christ, & Dierkes, 2005).
Our perspective derives from legal sciences and is significantly influenced by ideas of regulation in the sense of the normative influence of law. Besides this traditional idea of the functioning of law, we consider that with the IoT the “code is law” paradigm (Lessig, 2006) might enter the physical world with all its consequences. We call this the ‘Governance by Things’.
Even though it is still uncertain in which direction the IoT will develop, we are convinced that it will have a considerable impact on the application and the requirements for modern forms of law.
In the following, we develop some basic ideas about the role of law in the future and how law could react to such far-reaching developments in our technological and social environment. This discussion ties in with old (Aultman, 1972) and recently refueled, more differentiated debates about “normative technologies” (e.g. Hildebrandt, 2015; Koops, 2007).
II. Governance by things - code becoming physical
Research on the normative factors influencing and determining human behaviour on the internet has led to at least four important circumscribable governance factors. These are social norms, law, contracts and code.2 Social norms, law and - as a surrogate of law - contracts, tell people what they should do. If people act against the rules, they will be sanctioned socially (e.g. social isolation), by law (penalties) or by contracts (contractual penalties). The fourth factor, code, which essentially describes the circumstances shaped by hardware and software, works differently: it sets the framework for behaviour in virtual spaces by defining the options and limits of interaction. Additionally, the code can nudge (Thaler & Sunstein, 2009) people with inscribed affordances, increasing the likelihood of a desired behaviour (Oermann, Lose, Schmidt & Johnsen, 2014: pp. 10-11).
(Oermann et al., 2014: p. 18.)
Like the above model shows, all of these factors are interweaved and potentially influence each other. It requires deep empirical insights to decipher these connections, like the influence of social norms on legislative procedures or the changing of social norms by the introduction of a new law. For code the knowledge about interdepencies with the other factors is in its infancy.3 Mostly literature has warned - from a theoretical point of view - about the determination by code in virtual spaces (Lessig, 2006; Reidenberg, 1997). Due to the emergence of smart things, we want to scrutinise what will happen if this paradigm becomes tangible and especially highlight the relevance of the Governance by Things in relation to traditional regulation by law.
1. The regulation by law
Black’s Law Dictionary used to define law as “that which is laid down [...]” (Black, 1910: 700). Nowadays this refers to the textual quality of law. In order to discuss substitutes for legal rules we first have to go back to the function the legal system fulfills in society. There are various approaches indicating how legal intuitions fit into the working of the overall social structure. Parsons has been instructive to our discussion and quite influential in the realms of sociology of law. According to him the major function of law is an integrative one (cf. Schur, 1968: 80 et seq.): “It serves to mitigate potential elements of conflict and to oil the machinery of social intercourse. It is hindered, only by adherence to a system of rules that systems of social interaction can function without breaking down into overt or chronic covert conflict.” (Parsons, 1962: 57 et seq.) It is noteworthy that according to Parsons the legal system has to address three problems to fulfill that function and those are legitimation, interpretation (establishing rights and obligations by determining the application of rules) and enforcement (including jurisdiction and sanctions).
Luhmann serves as another important reference point especially in the German discourse. He stated that, in an exceedingly complex and contingent world, social systems exist in order to reduce social complexity and give some stability to social expectations (Luhmann, 1993: 131-133). The legal system fulfills this function because it enables one to select between diverse choices on the basis of a binary code, “lawful” or “unlawful” (Recht-Unrecht), that cuts in “self-referentially” (Luhmann 1993: 165-173).
Llewellyn identified five "law jobs", which are relevant in our context. Law in any community serves to prevent disruptive conflicts within the community and helps maintain a peaceful, orderly society, and contribute to this stability by providing a means of resolving disputes (1940). Thus giving stability to social expectations.
Furthermore, conflict solving can serve as a common denominator for the functions of law among society (Röhl 1987: 576). Building on those functions, politics can make use of law to govern society.
The application of rules is the way in which the legal system fulfills this function. Textual law is a cultural artefact that enables people to get a glimpse of normative contents, although it is also a specific system with its own interpretation methods. This means there must be a differentiation between social systems and the methodological toolbox of law. In written law we have meta rules that are supposed to guarantee that the legal texts are drafted in a way that the normative content can be deduced. If rules were enacted orally and face-to-face, the addressee could simply ask about the potential meaning of the rule. With the invention of written words and rules, mass application of law was possible and there was no need for ruler and addressee to be in the same place at the same time (Hildebrandt, 2008). This, in turn, resulted in a need for legal certainty, which legal science tries to reach through the systematic interpretation of law.
2. Governance by code
As mentioned above, the advent of the IoT might give the discussion about code a new twist. To provide a better understanding of this debate, let us briefly introduce our understanding of code and its role in governance. This will lead to the concept of Governance by Things.
As far as IT-related developments go, the idea of governance by code is quite old. Lawrence Lessig developed this intriguing idea of implicit influence on user behaviour by hardware and software back in the 1990s. The most important aspects of the ongoing discussion on code were and still are the following:
Firstly, code is – besides other factors like social norms, law, and contracts – one of the factors regulating human behaviour by setting the rules for the usage of digital products. Unlike law and social norms, however, code is self-executing. It defines the environment for user behaviour instead of explicitly setting the rules by stipulating what one should do and defining legal sanctions for misbehaviour. Code therefore implicates restrictions, enables behaviour or nudges users in certain directions and therefore at least partly takes over functions of law as described above.
Secondly, code governed by private companies can result in a power shift: rules implemented in code can have a huge impact on what is allowed for a mass of people without sufficiently reflecting existing law or the will of society.4 An example might help illustrate the impact of regulation by code in contrast to traditional law: the political struggle for a federal minimum wage in Germany was a long and arduous one, before it was finally introduced in 2015. The online platform Upwork (formerly oDesk), which helps businesses and freelancers to connect, included global minimum wages simply by tweaking the source code of the platform. Following this change, some users were barred from entering a wage of under four US dollars.5 While the effectiveness of this simple alteration to the code is quite impressive, it pushed some low qualified workers out of the market completely, because they could no longer offer their work at conditions viable to them. Marginally higher qualified workers from other countries were hired instead. Ensuring a minimum payment standard takes on a peculiar flavour since the whole business model relies on percentage commissions based on the wages.6 Put bluntly, one could state that code is essentially a resource through which the ones designing the code can pursue their interests.
Thirdly, and connected to the second aspect, code is – as virtually all internet-driven technologies are – hard to address for national governments. Taken together, these three aspects dominate the debate on code.
3. Governance by things as a new paradigm?
In the literature reflecting on code, starting with Lawrence Lessig, the authors always took it for granted that code was a factor influencing human behaviour in virtual spaces. In contrast to this, the restraints by physical environments were called architecture (Lessig, 1998: p. 663 et seq.; 1999A: p. 506 et seq.), which can be used for regulation: for instance, if a rural community wants to prevent heavy goods vehicle traffic from passing through, they could forbid that kind of vehicles via law by putting a sign up (“unsuitable for heavy goods vehicles”). Another way would be to build up funnel shaped physical road embankments to make the road narrower, thus enabling only cars to pass through this architecture (Dankert, 2015: p. 52).
This shows that physical circumstances always were and still are of importance. Today the awareness throughout research disciplines, product design and planning of cities is rising. Urinals include games to nudge men to “aim” at the right spot (Sommer, 2009). This includes the idea of gamification. So-called “defensive architecture” keeps skaters from skating in certain places (Mersom, 2015) or spikes keep homeless people from sleeping in certain places (Quinn, 2014). These examples lead to two follow-up questions. Firstly, why was the idea of code limited to virtual spaces? Secondly, what do the developments connected to the IoT change concerning the differentiation between architecture and code?
The first question can be answered with respect to the concern Lessig wanted to formulate with the equation “code is law”. It was his concern to warn against an excessive technical determination; essentially this means a determination not by technology itself, but by a few decision-makers in private companies (Lessig, 2006: p. XV). In contrast to physical architecture, the example of Upwork and the minimum wage shows the differences strikingly: digital markets tend to stimulate the formation of monopolies supported by direct and indirect network effects (OECD, 2013: p. 170; Van Gorp & Batura, 2015: 22). If there are changes in the code, millions of users can be affected by them. Therefore, some companies without a binding to democratic control can set rules by editing their code. Additionally, this code opens up subtle options for the companies to nudge people unknowingly, analyse the output and optimise the code, which is not possible to this extent in physical spaces. So this is why code was, so far, only used in terms of virtual spaces.
This leads to the second question: Why does the IoT change this limitation, which leads to an idea of Governance by Things? One might say that the IoT has the capability to reduce the gap between reality and virtual datasets (Fleisch et al., 2005). In technological terms this means that “things” – including not only technical devices, but all kinds of things in a broader sense – can have sensors gathering data about their environment and communicating with each other. Whereas computers have been bound to a certain place and are mostly dependent on data input by humans, these processes are increasingly being replaced by automatic sensors and even the reaction can be automated by actuators (Hildebrandt, 2008). Therefore, datasets become more accurate in reproducing a picture of reality.
A glimpse of the utopia (or dystopia) of “ambient law” (Hildebrandt, 2008) is currently only real in demo smart cities, as can be seen for example in Saudi Arabia (Ouroussoff, 2010) or South Korea (O’Connell, 2005). But with the advent of the IoT it becomes obvious that legislators have to think carefully about the societal changes and implications for law that it will bring.7 The EU has acknowledged this need and put out an action plan to react to this development.8 Against this backdrop, it is specifically the first of the 14-point action plan we address, which states the goal of “defining a set of principles underlying the governance of IoT”. In this article, we want to step back even further and describe the developments initiated by the Governance by Things and their implications for the regulation of human behaviour by written norms.
We see structural challenges in at least four key aspects of the code paradigm entering the physical world that we want to highlight: firstly, the need to explicitly “regulate” situations which so far have not been regulated, secondly, the hermeneutical connection between the application of a norm and the construction of the norm, the impact on private ordering and finally the imperfection of technology.
a) Necessitas eget legem – Necessity needs law
A common thought experiment in ethics is the trolley problem (Ghanayim, 2006). In this thought experiment there is a runaway trolley racing down railway tracks that would kill five people if there were no intervention. The person in question could pull a lever, which would set the trolley on a different track where it would kill just one person. This has applications for other rules, such as the example set out by the character Mr. Spock in Star Trek that “the needs of the many outweigh the needs of the few”.9 This can indeed be an acceptable ethical norm for adherents of the utilitarian doctrine (Bentham, 2000).
Many legal systems address these kinds of exceptional cases with correctional norms on a secondary review level, such as the Choice-of-Evils in American criminal law or a comparable construction in German criminal law, which resolves this conflict on the level of unlawfulness when two obligations are in conflict (Lackner & Kühl, 2014: § 34 margin number 15). These exceptional cases – like the trolley problem – are systematically solved by moving them from the level of rules regulating behaviour to principles of justification. Thus, law does not have to provide rules on how to act when confronted with these kinds of decisions and still the underlying moral values, the rules in criminal law, can be upheld. However, at the same time, the law recognises that it would not be fair to punish someone who decides one way or the other. This follows Immanuel Kant’s discussion of the similar carneades-problem10 and supports the saying “necessitas non habet legem” – necessity has no law.
For autonomous cars there are at least two basic situations discussed (Bonnefon, Shariff, & Rahwan, 2015: p.3), which connect closely to the thought experiments above: 1) Autonomous cars must be programmed to choose between unavoidably harming either one person or several people; 2) The car is harming one person, but the chance to save this person will unavoidably harm the driver. This would be the case if e.g. one person surprisingly appears on the road and the only chance to swerve would be to drive into a wall, off a cliff, etc. Generally speaking when technical systems are built that anticipate critical decisions, necessity has to have a law or - in other words inspired by a recent post of an MIT blog - “self-driving cars must be programmed to kill” (Bonnefon et al., 2015). The rules that technology needs in order to function are also normative rules. Anticipating critical situations can be an opportunity to implement reasoned determinations in situations that would otherwise lead to arbitrary results, e.g. because reacting appropriately would be impossible due to time constraints.
Recently a survey in the field of experimental ethics showed that there might be a preference for “utilitarian cars”, which are programmed in a way to always minimise the death toll - even if this decision meant harming the ‘driver’s’ life (Bonnefon et al., 2015: p. 7). But likewise the survey showed that the acceptance of the programming of a car to sacrifice the driver potentially decreases, when this affects one’s own car (Bonnefon et al., 2015: p.7). Interestingly enough, the survey asked for the willingness to accept legal enforcement of self-sacrifice of the passenger of such a car. In this case the acceptance of a legal enforcement towards autonomous cars was higher than if it applied to the behaviour of humans (Bonnefon et al., 2015: p. 7 - 8). Without a doubt, a legal obligation to sacrifice oneself as a driver would not only face ethical challenges, but would at least in Germany also most likely not be compatible with constitutional values, such as human dignity or the right to live (BVerfGE 115, 118, p. 159).11 Nevertheless the intuition of the participants to regulate the implicit rules of autonomous cars seems to be right. When there is a rule-making process for a car, which results in a Governance by Things possibly anticipating a multiplicity of critical cases in a certain way, then law is an appropriate tool to accompany this process and leave room for a differentiated public discussion, whether Mr. Spock’s utilitarian approach was right or not.
If society wants to profit from the advantages of a highly efficient and consistent Governance by Things,12 the inscribed rules need to anticipative normative guidelines on how to solve critical decisions - in the figurative sense: “necessity needs law”.
b) The (self-)execution of normative rules
If we put a normative complexion on code, there are some parallels discussed by Lawrence Lessig and others between code and law. There is, however, at least one thing that sets code apart: its special self-executing character (Reidenberg, 1997: p. 569). Written law only provides psychic compulsion. This means the inscribed normative contents of law can motivate the addressee directly or indirectly to behave in a certain way, but even the enforcement of a specific norm by the organ in charge does not make the sanctioned person follow the rule of conduct which is enshrined in the legal rule. At best it can have an effect for future actions. Kelsen describes this paradox as follows: The “[...] sanction to be executed by the organ is provided for only in those concrete cases where the conduct which the legal order tries to bring about has not been ‘enforced’ and, thus, has proved not to be ‘enforcible’” (Kelsen, 2006: p.23). Thus normative rules provide an idea of what should be “normal” (meaning potentially benefiting a society), but they only come into actual effect whenever a discrepancy of the actual and desired behaviour occurs. The enforcement regards ex post penalties, but there is no other than psychic compulsion before someone decides not to conform to the law.
Governance by Things as described is also self-executing in the purest sense, in that the rules can be directly implemented in algorithms13 that control things, where the enforcement is anticipated and - in contrast to the psychic compulsion of law - can directly restrict or substitute human behaviour. As a result, regularly behaviour automatically conforms to the range of possibilities that the Governance by Things allows. Consider this famous example: the law can stipulate that one must fasten one’s seat belt while driving and the police can enforce this regulation. Smart cars, however, could simply refuse to start the engine if the sensors indicate that the driver has not buckled up properly. Hildebrandt states – building on Searle – that with normative technology regulative rules can become constitutive rules (Hildebrandt, 2008: pp. 169-183). While regulative rules leave the options open to either follow the rules or ignore them, constitutive rules only permit the action to be taken if the criteria the rules define are fulfilled.
Even in cases in which an algorithm does not fully prevent some kind of behaviour but just nudges someone in a certain direction, there is a significant difference when compared to law. The process of application lacks the complex interaction between abstract norms and the specific case at hand that makes each application of the law in itself a construction of the law. One consequence of the specific hermeneutics of law is a certain flexibility that is at least not inherent to algorithms. This leads to the question: what makes human rule-based decision-making so special? For human decision-making processes, psychologists differentiate between explicit and tacit knowledge. They say that human decision-making often is based on tacit knowledge (also ‘procedural knowledge’) (Polanyi, 1965: p. 16 et seq.). This type of knowledge is hard to verbalise. It is a fact that “we can know more than we can tell.” (Polanyi, 1965: p. 14). Theoretical models can be used to develop an abstract idea about explicit and implicit knowledge, their interaction and their influence on gathering knowledge (Nonaka & Takeuchi, 1995: pp. 61 et seq.).
In contrast, algorithms always use explicit knowledge to reach a solution (Carr, 2014: p. 8 et. seq.). The fact that no-one can describe the human decision-making process in every detail, because it is for the greater part based on tacit knowledge, led to the idea that complex interactions between humans and their environment (like driving a car) cannot be overtaken by algorithms (Levy & Murnane, 2004: p. 20). Although autonomous cars today prove the contrary, the automation of technical processes still is result-orientated. Algorithms with inherent social values conserve certain decisions according to predictable input-output-patterns, without a systematic self-conception. For instance, algorithmic rule-making reaches its’ limits, when rules have to be reasonably ignored or refined. Test-drives with autonomous cars show that they are able to stick to traffic rules slavishly, but still cannot decide when to override rules reasonably (Richtel & Dougherty: 2015).
At the same time, we also apply and (re-)construct tacit norms when we make decisions (cf. Kratochwil 1989: pp. 54-56). Similar to tacit knowledge, tacit norms are norms we follow without reflecting on them. They have become part of our scripts for decision-making without us even realising that we are applying a norm. These norms can emerge in interactions between people or they might be internalisations of explicit norms.
At this state, it might be possible for a car to drive autonomously, following traffic-rules in certain decision patterns. But this is just a translation of complex interactions between physical objects. Abstract judicial or ethical concepts like fairness or good faith are unwieldy to handle technically, because they are dependent on the implementation of social perception, which is based on tacit knowledge and tacit norms. To refer to the example from the beginning: legal research on the clarity of normative rules and the constitutional requirement of certainty showed that this principle paradoxically is loosely structured and dependent on smart decisions by judges (Towfigh, 2008: p. 15). As long as we cannot decipher what these abstract concepts mean, we are not able to translate them into explicit rules. At least this is likely to fail, like the seemingly elaborate robot laws in Isaac Asimov’s I, Robot, which led to chaos, because of the missing interpretation and legal discretion of the norms by robots (Söbbing, 2015: p.46).
In other words, in judicial processes even under a civil law system which is mainly governed by written laws, those written norms can be ‘updated’ since, each time, the legal text’s meaning is construed in view of the specific facts of the case at hand (Vesting, 2015: p. 140). This means that the specific inscribed normative content of a norm is constantly being redefined in step with social developments. In this regard, the uncertainty of law and the inclusion of human communication in this process are advantages.
As said before, the differences between law and the Governance by Things in terms of code can also be studied in the internet realm (Oermann et al.: p. 3 et seq; Kesan & Rajiv, 2005; Wagner, 2004).
c) Private ordering of things
The discussion about code and law has already triggered a debate about private ordering (Elkin-Koren, 2008: p. 5). Private ordering describes the process of setting up social norms and/or specific (self-)regulatory determinations and sanctions by private parties (Schwarcz, 2002: p. 319; Elkin-Koren, 2005). Besides the general attempt to regulating markets through private actors, private ordering can use regulatory measures, made by publicly empowered private authorities (Schwarcz, 2002). In the digital age, private ordering is seen as an efficient alternative way to regulate the information environment (Benkler, 2000: p. 2063), because its flexible structures complement the dynamic market structures in the technology sector. It has lengthy historical precedent that private actors – mainly companies and industry associations – at least partly take over regulatory tasks (Hofmann, Katzenbach, & Gollatz, 2014: pp. 12-13.; Feick & Werle 2010: p. 525). Nevertheless, this has expanded in scope over the past years (Schwarcz, 2002: p. 5 et seq.) and will probably increase further with the advent of the Governance by Things.
There are various forms of private ordering, a core element of which are contracts. One strategy of governments to reach certain regulatory goals can be to delegate responsibility to the industry. There is, however, also the possibility that the industry itself sets de facto standards. Governance structures on internet platforms go beyond simple contracts and also include the code created by the industry that can set standards, but which is not the result of a democratic rule-making process.14
If we push this idea further, we can consider how this argument might be transferred to other areas such as ‘smart cars’ or ‘smart cities’. As the gap between materiality and digital spaces narrows further (the architecture and the code), this discussion will become a core aspect of the regulation of the Governance by Things since public and commercial interests can collide. The ‘domestic authority’ line of argument returning from digital spaces back to physicality may sound reasonable to a certain degree, because consumers are able to ‘choose their authority’ by buying certain products or not. Rules set by companies with commercial interests might be appropriate and capable of substituting governmental decisions with Private Ordering in some cases.
On the other hand, it is known that digital markets tend to concentrate due to certain effects (e.g. power law effects or network effects), which seems to foster paradoxical situations in which the services with the highest market shares for a certain market may score low in customer satisfaction.15 This tendency for market concentration might spill over on IoT-markets, which are close to digital markets. This might make the domestic authority argument problematic in areas where an IoT product prevailed in the market and can set the implicit rules of Governance by Things while not taking consumer interests into account. If private companies driven by economic self-interest autonomously implement their ideas of ‘rightful’ behaviour in algorithms, there is no guarantee that they are complementary with social consent. Whether economic competition can guarantee that, depends on the market structure (e.g. the contestability of markets). At this point, the anticipated decisions embedded in algorithms can produce accomplished facts in large-scale scenarios, determining the behaviour of many members of society.
Among the important questions to be discussed at this crossroads is at what point the software behind the Governance by Things becomes a public affair, especially when it can have a direct physical impact. This often has far reaching consequences on the regulation by law: when a decision appears to be a public affair, questions of legitimacy come into play; the decision-maker is bound by human rights and has to adhere to the rule of law. Accompanying the latter is an expectation of transparency. Surely one cannot take these legal principles and impose them on Governance by Things. In contrast to law, which is open to be read by everyone from the broad public to lawyers, to the press, etc., algorithmic regulation in many cases relies on a lack of transparency to keep its intrinsic value, especially if it is connected to private companies’ economic interests. If a company like Google revealed how its search algorithm worked, this information would be used to manipulate search results. By the same token, if a producer of an algorithm for autonomous cars revealed its source code, it could be misused or competing companies could use it for their own products. But even in the unlikely scenario that there were participative structures and the source code was ‘open’, there would only be a small minority of people who could comprehend the logic of complex coding and really participate in the process of shaping algorithmic regulation (Oermann & Töllner, 2014: p. 8; Lessig, 1999B: p. 1418). We need to explore, which concepts of transparency can be helpful in this context.
Therefore, lawmakers have to learn about the interplay between governance factors and the impact of the Governance by Things to be able to analyse the need for an intervention by the state. If Public Ordering is the right mode of intervention, this leads to questions of how to regulate software, especially concerning the question of what the regulatory link is: is it the software itself, the standards behind it, the algorithms or even the maxims of coding?
d) The imperfection of technology
Research on the impact of the IoT and its normative implications tends to hold the theoretical view that technology is without failure and inevitable. The chief motivation for the establishment of automated cars is to reduce and eventually eliminate the possibility of road accidents. Likewise, ‘smart guns’ are supposed to help in achieving the aim of drastically decreasing the illegal use of weapons (Borrup, Heneghan, Hernández-Arranz, Luna, & Lapidus, 2014: p. 20 et seq.).
Although the Governance by Things can certainly be helpful to optimise traffic or other complex processes, in the end, every technology is a product of human work, so an infallible technology will never be invented.16 Therefore, not only will the possibility of misusing technology remain, but new types of failures and new opportunities for abuse will emerge as these become more ubiquitous. For instance, just recently technical security gaps in self-driving cars were revealed, which could be exploited to control certain functions of the cars remotely over the internet (Greenberg, 2015; Zetter, 2015). The option of tampering with imperfect technology is not new, but coupled with the self-executing and physical character of the Governance by Things, this phenomenon becomes a challenge. Every new technology involves new possibilities of abuse, but Governance by Things enables the possible manipulation of our physical surroundings once the IoT is ubiquitous.
Additionally, ‘bugs’ get a promotion with the Governance by Things. We already saw that algorithms can deliver odd outcomes, e.g. when automated credit ranking systems denied the former head of the US Federal Reserve Bank a restructuring of his mortgage. Even though “it would take fewer than three speeches for him to pay off his entire house” (Moore & Kasperkevic, 2014), some of the criteria led to a red flag on his refinancing inquiry. Questions about how to deal with such ‘bugs’ legally become even more pressing when algorithmic decisions can have an actual physical impact.
In our view, the imperfection of technology has two main implications for regulation: firstly, ‘smart’ things will replace human decision making processes and behaviour in an increasing number of situations, e.g. the autonomous car will substitute the decisions of drivers. This leads to several liability questions with potentially significant impacts on markets and innovation. Just imagine: who or possibly even what would be liable in the case of an accident involving or even caused by an autonomous car? It cannot be the car itself,17 although it seemingly ‘decided’ – maybe even wrongly – in a critical situation. But the anticipation of such algorithmic decisions can always be traced back to a manufacturer. At this stage and without changes in German law, there would possibly be a shift in liability towards the manufacturers of these cars, resulting in costly litigations against them (Jänich, Schrader, & Reck, 2015: p. 318). This scenario can come about, although it does not change the idea of owner liability: liability lies with the one who has increased the risk and enjoyed the advantages of something – which is still the owner of a car and not its manufacturer (Lutz, 2015: pp. 119 et seq.). Examples of shifting liability are not restricted to autonomous cars, but include a wide variety of products that could potentially harm people. In these cases, the pressure on producers to take precautions increases. Additionally, autonomous vehicles tend to “complicate the already complicated entanglements between insurance providers, plaintiffs, drivers/owners named as defendants, and manufacturers.” (Villasenor 2014: p.13). These problems have already been discussed and will likely result in new insurance concepts (Jain et al., 2015).
Furthermore, there will be a need for law to regulate situations where ‘software bugs’ do not merely affect software, but have a real-world physical impact in terms of security mechanisms preventing abuse or misuse of a technology. An obvious example are ‘smart guns’, which recognise their owners (Borrup et al., 2014) and - maybe in the future - are only able to be fired in situations of self-defence. Even if such a device has built-in security measures against non-conforming usage, there will be ways to bypass these measures and misuse the weapon for criminal purposes. The same goes for autonomous cars: it cannot be ruled out that such a car is hacked to make it drive markedly faster than allowed.
This implicit weakness of technology results in a significant gap in the Governance by Things and leaves the requirement for regulation by law and its educational character to prevent people from misusing technology and to shape social standards (Rüthers, Fischer, & Birk, 2013: p. 54).
IV. Conclusion: a call for second-order regulation of Governance by Things
According to a study, sometime after 2050 every vehicle will be driving autonomously (IHS, 2014). At that point, the implicit rules of the Governance by Things will have a huge impact on people’s lives, not only limited to autonomous cars. Currently, there are only a few examples of actual Governance by Things problems. Nevertheless, it seems evident that the development of new IoT technologies will give rise to a number of challenges in the not too distant future, which will be closely connected to the developments observed in internet governance.
We explained that on the one hand, there have to be concrete normative decisions in situations, which – so far – were not explicitly regulated (see section II. 2. a). On the other hand, we saw that these implicit normative decisions cannot substitute law, because technology lacks a systematic self-conception: Although law can seem antiquated in terms of efficiency, especially in contrast to constitutive normative rules of Governance by Things, we think that this will neither be the end of politics nor law. Lawmaking and court decisions are processes of communicative constructions, which include a variety of social perceptions. These are mostly based on tacit knowledge and tacit social norms. The uncertainty of these processes might seem inefficient but still has an advantage in keeping law up-to-date with social developments (see section II. 2. b)). Consequently, Governance by Things cannot replace law. But we think that it is time for lawmakers to accept and make use of the normative qualities of this “physical” code.
Therefore, lawmakers might be well advised to ponder a second-order regulation for the Governance by Things in some areas, which have to be analysed, since the influence of the Governance by Things will certainly increase, either not following specific democratic values or necessarily representing social perception. These self-executing rules with their regulative and constitutive elements include a subtle resource of governing people. We have learned that, in contrast to the Governance by Things, regulation by law is based on psychic compulsion, which means the enforcement of law is not identical to the intended normative content of a legal norm. A second-order regulation will not change this fact, but it would recognise the regulative and even constitutive elements of the Governance by Things and use them knowingly as a possibly strong resource of regulation.
These second-order rules can be perceived as legal rules defining explicit boundaries and guidelines for the producers of IoT-products, not really “taming” the code but rather socialising it. One striking example of how this could work are the “no fly zones“ programmed in drones, which can include areas like airports to prevent drones from harming flights. Regulation requiring to upload publicly curated maps for the operation to be legal could even include the opt-in or opt-out of private property owners who might want delivery drones to have access but exclude all others. At this point some drone manufacturers already include such measures voluntarily.18 Yet, it is conceivable to have a regulation by law defining these areas and the way it is included in the code.
However, it is not our intention to say that all kinds of product design should be in the hands of lawmakers – second-order regulation should be limited to certain areas, which must be carefully selected. The limits will be defined in a discourse on what is regarded as a “public affair” as outlined above. It is likely that the need of having an influence on regulation on implicit technological rules must be determined by the weight of the legal interests at stake. Especially when it comes to vital interests, like in health care or autonomous driving, software programming will not solely remain in the hands of market forces and their private ordering. Now governance experts have the chance of preparing for the coming challenge by creating second-order mechanisms that oversee the Governance by Things. One obvious first step they can take is to create interfaces between software engineers and experts in legal matters, ethics and governance. This will create the knowledge base for a meaningful debate on regulation in the era of the Governance by Things. Even nowadays some studies point in this direction (Bonnefon, Shariff, & Rahwan, 2015). Such work can contribute to defining the areas in which software programming can be seen as a public affair, and help in understanding what the points of reference and convenient methods of regulation are.
Based on the above discussion, there could be certain principles that technology must obey, conflict-solving mechanisms, and evaluation requirements; the whole toolbox of context regulation can be unpacked and filled with new instruments. It is a new El Dorado for governance research.
Aultman, M. (1972, January 1). Technology and the End of Law. American Journal of Jurisprudence, 17(1), 46-72.
Barbry, E. (2012). The Internet of Things, Legal Aspects: What Will Change (Everything)…, Communications and Strategies (87), 88 – 89. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2304137
Benkler, Y. (2000). An Unhurried View of Private Ordering in Information Transactions. Vanderbilt Law Review. Retrieved from http://www.benkler.org/UnhurriedView.pdf
Bentham, J. (2000). An Introduction to the Principles of Morals and Legislation. Kitchener: Batoche Books.
Black, H. C. (1910). Black’s Law Dictionary (2nd ed). St. Paul: West Publishing Co.
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2015). Autonomous Vehicles Need Experimental Ethics: Are We Ready for Utilitarian Cars? Retrieved from http://arxiv.org/abs/1510.03346
Borrup, K., Heneghan, K. A., Hernández-Arranz, G., Luna, R., & Lapidus, G. (2014). Accelerating the adoption of smart gun technology: a policy analysis. Quinnipiac Health Law Review Online, Annual Symposium, 1(1), p. 14-27. Retrieved from http://www.quinnipiac.edu/prebuilt/pdf/SchoolLaw/HealthLawJournalLibrary...
Brownsword, R. (2006). Neither East Nor West, is Mid-West Best? SCRIPT-ed, 3(1), 16-33. Retrieved from http://ssrn.com/abstract=1127125.
Carr, N. (2014). The Glass Cage: Automation and Us. New York, NY: Norton & Company.
Curtis, S. (2015, Jan 30). Who will pay for the Internet of Things? The Telegraph. Retrieved from http://www.telegraph.co.uk/technology/internet/11377083/Who-will-pay-for-the-Internet-of-Things.html
Dankert, K. (2015). Normative Technologie in sozialen Netzwerkdiensten - Neue Machtstrukturen als Anreiz für einen Paradigmenwechsel der Kommunikationsregulierung? KritV 2015, 50-72.
Davies, H. C. (2015, Apr 29). The Internet of Things: some implications for sociology. Retrieved from https://thesocietypages.org/sociologylens/2014/05/01/the-internet-of-thi...
Elkin-Koren, N. (2005). What Contracts Can't Do: The Limits of Private Ordering in Facilitating a Creative Commons. Fordham Law Review. Retrieved from http://ssrn.com/abstract=760906
Elkin-Koren, N. (2008). Governing Access to Users-Generated-Content: The Changing Nature of Private Ordering in Digital Networks. In Brousseau, E., Marzouki,, M. & Meadel, C. (Eds.), GOVERNANCE, REGULATIONS AND POWERS ON THE INTERNET, Cambridge University Press. Retrieved from http://ssrn.com/abstract=1321164
Feick, J., & Werle, R. (2010). Regulation of Cyberspace. In R. Baldwin, M. Cave & M. Lodge (Eds.), The Oxford Handbook of Regulation, 523-547. Oxford: Oxford University Press.
Fleisch, E., Christ, O., & Dierkes M. (2005). Die betriebswirtschaftliche Vision des Internets der Dinge. In E. Fleisch & F. Mattern (Eds.), Das Internet der Dinge: Ubiquitous Computing und RFID in der Praxis: Visionen, Technologien, Anwendungen, Handlungsanleitungen (pp. 8-9). Berlin, Heidelberg: Springer.
Ghanayim, K. (2006). Excused Necessity in Western Legal Philosophy. Canadian Journal of Law and Jurisprudence, 19(01), 31-66. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=995343
Green, P. (2014). Putting Magic in the Mundane. The New York Times. Retrieved from http://www.nytimes.com/2014/07/17/garden/putting-magic-in-the-mundane.html
Greenberg, A. (2015, Juli 21), Hackers Remotely Kill a Jeep on the Highway – With Me in It. Retrieved from http://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/
Hardy, Q. (2015, Jun 24). The Internet of Things Has Vast Economic Potential, McKinsey Report Says. Bits Blog, The New York Times. Retrieved from http://bits.blogs.nytimes.com/2015/06/24/the-internet-of-things-has-vast...
Hildebrandt, M. (2008). Vision of Ambient Law. In R. Brownsword & K. Yeung (Eds.), Regulating Technologies (175–191). Oxford. Retrieved from http://works.bepress.com/mireille_hildebrandt/4
Hildebrandt, M. (2008). Legal and technological normativity: more (and less) than twin sisters. Techné: Research in Philosophy and Technology, 12(3), 169-183. Retrieved from http://works.bepress.com/mireille_hildebrandt/13/
Hildebrandt, M. (2009). Technology and the End of Law. In E. Claes, W. Devroe & B. Keirsbilck (Eds.), Facing the limits of the law (443-464). Berlin, Heidelberg: Springer.
Hildebrandt, M. (2015). Smart Technologies and the End(s) of Law. Cheltenham: Elgar Publishing.
Hofmann, J., Katzenbach, C., & Gollatz, K. (2014). Between Coordination and Regulation: Conceptualizing Governance in Internet Governance. HIIG Discussion Paper Series No. 2014-4. Retrieved from http://dx.doi.org/10.2139/ssrn.2484463
IHS (2014). Self-Driving Cars Moving into the Industry’s Driver’s Seat. Retrieved from http://press.ihs.com/press-release/automotive/self-driving-cars-moving-i...
Jain, N., O’Reilly, J., & Silk, N. (2015, Jun 16). Driverless cars: Insurers cannot be asleep at the wheel. Retrieved from https://bankunderground.co.uk/2015/06/19/driverless-cars-insurers-cannot...
Jänich, V. M., Schrader, P. T., & Reck, V. (2015). Rechtsprobleme des autonomen Fahrens. NZV 2015, 313-319.
Kelsen, H. (2006). General Theory of Law & State. New Jersey: Transaction Publishers.
Kesan, J. P. & Rajiv C. S. (2005). Shaping Code. Harvard Journal of Law & Technology, 18(2), pp. 318-399. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=653143
Koops, B.-J. (2007). Criteria for Normative Technology: An Essay on the Acceptability of 'Code as Law' in Light of Democratic and Constitutional Values. In Brownsword, R. & Yeung, K. (Eds.), REGULATING TECHNOLOGIES (157-174). Oxford, Hart Publishing (TILT Law & Technology Working Paper Series No. 005/2007). Retrieved from http://dx.doi.org/10.2139/ssrn.1071745
Kratochwil, F. V. (1989). Rules, Norms, Decisions. Cambridge University Press.
Lackner, K., & Kühl, K. (2014). Strafgesetzbuch Kommentar (28th ed.). Munich.
Lessig, L. (1999A). The law of the horse: What cyberlaw might teach. Harvard Law Review, 113(2), 501-549. Retrieved from https://cyber.law.harvard.edu/works/lessig/finalhls.pdf
Lessig, L. (1999B). Open Code and Open Societies: Values of Internet Governance. Chicago -Kent. Law Review, 74(3), 1405-1420. Retrieved from http://scholarship.kentlaw.iit.edu/cklawreview/vol74/iss3/17
Lessig, L. (2006). Code, version 2.0. New York, NY: Basic Books. Retrieved from http://codev2.cc/download+remix/Lessig-Codev2.pdf
Levy, F. & Murnane, R. J. (2005). The New Division of Labor: How Computers Are Creating the Next Job Market. Princeton, NJ: Princeton University Press.
Llewellyn K. N. (1940). The Normative, the Legal, and the Law-Jobs: The Problem of Juristic Method. The Yale Law Journal. 49(8), 1355-1400.
Luhmann, N. (1995). Das Recht der Gesellschaft. Frankfurt am Main: Suhrkamp.
Luhmann, N. (2004). Law as a social system. K. A. Ziegert (Transl.), F. Kastner, R. Nobles, D. Schiff, & R. Ziegert (Eds.). Oxford: Oxford University Press.
Lutz, L. S. (2015). Autonome Fahrzeuge als rechtliche Herausforderung. NJW 2015, 119-124.
Mattern, F., & Flörkemeier, C. (2010). Vom Internet der Computer zum Internet der Dinge. Informatik-Spektrum, 33(2), 107-121. Retrieved from http://www.vs.inf.ethz.ch/publ/papers/Internet-der-Dinge.pdf
Maunz, T. & Dürig, G. (2015). Grundgesetz Loseblatt-Kommentar, 74. München: Beck Verlag.
Mersom, D. (2015, Oct 7). Bristol skateboarders take on ‘skatestopper’ defensive architecture. The Guardian. Retrieved from http://www.theguardian.com/cities/2015/oct/07/bristol-skateboarders-skat...
Moore, H & Kasperkevic, J. (2015, Oct 7). Ben Bernanke ran the Federal Reserve and can’t get a new mortgage. Can you? The Guardian. Retrieved from http://www.theguardian.com/money/us-money-blog/2014/oct/06/if-ben-bernanke-cant-refinance-his-mortgage-what-hope-for-the-rest-of-us
Nonaka, I. & Takeuchi, H. (1995). The Knowledge-Creating Company. Oxford, New York: Oxford University Press.
OECD (2013). Supporting Investment in Knowledge Capital, Growth and Innovation, OECD Publishing. Retrieved from http://dx.doi.org/10.1787/9789264193307-en
Oermann, M., Lose, M., Schmidt, & J.-H. & Johnsen, K. (2014). Approaching Social Media Governance. HIIG Discussion Paper Series No. 2014-05. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2498552
Oermann, M. & Töllner, N. (2014). The Evolution of Governance Structure in Cryptocurrencies and the Emergence of Code-Based Arbitration in Bitcoin. Retrieved from http://www.hans-bredow-institut.de/webfm_send/1070
O’Connell, P. L. (2005, Oct 5). Korea's High-Tech Utopia, Where Everything Is Observed. The New York Times. Retrieved from http://www.nytimes.com/2005/10/05/technology/techspecial/koreas-hightech-utopia-where-everything-is-observed.html
Ouroussoff, N. (2010, Dec 12). Saudi Urban Projects Are a Window to Modernity. The New York Times. Retrieved from http://www.nytimes.com/2010/12/13/arts/design/13desert.html?pagewanted=a...
Parsons, T. (1962). The Law and Social Control. In Evan, W. M. (ed.), Law and Sociology, Exploratory Essays. New York: Free Press of Glencoe.
Peterson, A. (2015, Aug 3). Connected medical devices: The Internet of things-that-could-kill-you. The Washington Post. Retrieved from https://www.washingtonpost.com/news/the-switch/wp/2015/08/03/connected-m...
Polanyi, M. (1985). Implizites Wissen. Frankfurt am Main: Suhrkamp.
Quinn, B. (2014). Anti-homeless spikes are part of a wider phenomenon of ‘hostile architecture’. The Guardian. Retrieved from http://www.theguardian.com/artanddesign/2014/jun/13/anti-homeless-spikes...
Reidenberg, J. R. (1997). Lex Informatica: The Formulation of Information Policy Rules through Technology. Texas Law Review, 76(3), 553-593. Retrieved from http://ir.lawnet.fordham.edu/faculty_scholarship/42/
Richtel, M. & Dougherty, C. (2015). Google’s Driverless Cars Run Into Problem: Cars With Drivers. The New York Times. Retrieved from http://www.nytimes.com/2015/09/02/technology/personaltech/google-says-it...
Röhl, K. F. (1987). Rechtssoziologie. Cologne: Heymanns.
Rosenthal et al. (2015). Europe’s Expanding ‘Right to Be Forgotten’. The New York Times. Retrieved from: http://www.nytimes.com/2015/02/04/opinion/europes-expanding-right-to-be-...
Rüthers, B., Fischer, C., & Birk, A. (2013). Rechtstheorie (7th ed.). Munich: Beck Verlag.
Santucci, G. (2008). Paper for the International Conference on Future Trends of the Internet. Retrieved from http://cordis.europa.eu/fp7/ict/enet/documents/publications/iot-between-...
Schelsky, H. (1965). Der Mensch in der wissenschaftlichen Zivilisation. In Schelsky, H. (ed.), Auf der Suche nach der Wirklichkeit (pp. 439-480). Düsseldorf-Cologne, Eugen Diederichs Verlag.
Schur, E. M. (1968). Law and Society. A Sociological View. New York: Random House.
Schwarcz, S. L. (2002), Private Ordering. In Northwestern University Law Review. Retrieved from http://dx.doi.org/10.2139/ssrn.298409
Sommer, J. (2009, Feb 7). When Humans Need a Nudge Toward Rationality. The New York Times. Retrieved from http://www.nytimes.com/2009/02/08/business/08nudge.html?_r=0
Söbbing, T. (2013). Rechtsfragen der Robotik. InTeR 2015, 43-46.
Thaler, R. & Sunstein, C. (2009). Nudge: Improving Decisions About Health, Wealth and Happiness. New York: Penguin Books.
Towfigh, E. V. (2008). Komplexität und Normenklarheit - oder: Gesetze sind für Juristen gemacht. Preprints of the Max Planck Institute for Research on Collective Goods. Retrieved from http://www.mpp-rdg.mpg.de/pdf_dat/2008_22online.pdf
Vap Gorp, N. & Batura, O. (2015). Challenges for Competition Policy in a Digitalised Economy, study for the European Parliament, IP/A/ECON/2014-12. Retrieved from http://www.europarl.europa.eu/RegData/etudes/STUD/2015/542235/IPOL_STU%2...
Vesting, Th. (2015). Rechtstheorie. 2nd. Ed. Munich: C. H. Beck.
Villasenor, J. (2014). Products Liability and Driverless Cars: Issues and Guiding Principles for Legislation. Brookings Institution. Retrieved from http://www.brookings.edu/~/media/research/files/papers/2014/04/products-...
Wagner, R. P. (2004). On Software Regulation. University of Penn. Law School, Public Law Working Paper 57; University of Penn, Institute for Law & Econ Research Paper 04-17. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=582422
Zarsky, T. (2015). Social Justice, Social Norms and the Governance of Social Media. Pace Law Review, 35(1), 138-172. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2576774
Ziewitz, M. (2016). Governing Algorithms: Myth, Mess, and Methods. Science, Technology & Human Values 2016, Vol. 41 (1), 3 - 16.
Zetter, K. (2015, Aug 6). Researchers hacked a Model S, but Tesla’s already released a patch. Retrieved from http://www.wired.com/2015/08/researchers-hacked-model-s-teslas-already/
1. For an overview on definitions of the IoT, see http://postscapes.com/internet-of-things-definition .
2. There are other approaches describing markets as a fourth factor instead of contracts, see e.g. Lessig, 1999A, The Law of the Horse, p. 506 et seq, in: 113 Harv. L. Rev. 501; we consider the governance factor of markets to be on another ontological level, see Oermann, Lose, Schmidt, & Johnsen (2014: pp. 8 et seq.).
3. For a case study on Social Media Governance see Oermann et al. (2014).
4. This point is quite tentative, for a differentiated examination of how code may correspond with law and the will of society, see Zarsky (2015: pp. 138-172, esp. pp. 141 et seq.).
5. Aside from these test-scenarios mentioned, Upwork introduces a global minimum wage of three US dollars for all users, see https://support.upwork.com/hc/en-us/articles/211062988-Minimum-Hourly-Rates .
6. Thanks to Vili Lehdonvirta (Oxford Internet Institute), who gave us interesting insights in online labour markets at the Hans-Bredow Institute, for details see: http://www.hans-bredow-institut.de/en/eventworkshop/online-labour-markets .
7. The EU expected a period of time of around 5 – 15 years for drastic changes in 2009, COM (2009) 278, see: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2009:0278:FIN:... .
9. Star Trek II, The Wrath of Khan (Paramount Pictures, 1982).
10. In this thought experiment two shipwrecked sailors only find one plank in the water. Both know that the plank can only save one from drowning. Sailor A, who was there first, is pushed away by sailor B and drowns. After B is rescued the question arises, whether he can be accused of murder or if he acted in self-defence.
11. English translation: http://www.bverfg.de/e/rs20060215_1bvr035705en.html, see para. 153 .
12. Overriding autonomous decisionmaking processes or implementing processes that generate random results may be considered in order to face critical situations or emergencies. Beiker, S. A. (2012). Legal Aspects of Autonomous Driving. In Santa Clara Law Review, 52 (4), 1154.
13. The discussion about algorithms is flourishing especially in social sciences at the moment with numerous definitions; a discussion we cannot descramble in this article. For a recent overview on research questions see Ziewitz (2016).
14. With the restriction that the legal frameworks for contracts and the limits with regard to content can be set by the legislator.
15. ACSI measured Facebook customer satisfaction to be comparably low over the last years, see http://www.theacsi.org/?option=com_content&view=article&id=149&catid=14&.... Moreover, the Pew Internet and American Life Project found that the majority of Facebook users has at least once voluntarily taken a break from the platform. The most frequently mentioned reasons are lack of time and lack of interest in the platform and its content http://www.pewinternet.org/2013/02/05/coming-and-going-on-facebook .
16. For a critical opinion on Google’s autonomous cars by Walker Smith, Driving at perfection, see http://cyberlaw.stanford.edu/blog/2012/03/driving-perfection .
17. Although “[…] there will certainly come a time when a thing will be seen as a ‘legal actor’”, Brabry (2012: pp. 88-89).