Driverless Cars and the Trolley Problem

The “Trolley Problem” is a long pondered ethical thought experiment; it is an intellectual exercise devised to highlight the moral conflicts that can arise in the making of decisions involving inescapable loss of life. Here is how Wikipedia presents it:

“A runaway trolley is barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person on the side track. You have two options:

  1. Do nothing, and the trolley kills the five people on the main track.
  2. Pull the lever, diverting the trolley onto the side track where it will kill one person (in some versions, a friend or family member).

Which is the most ethical choice?”

This thought experiment created by moral philosophers, now features frequently as a real problem in discussions about driverless cars. In its new form the trolley becomes a driverless car and the role of the man at the switch is assigned to the programmer of the algorithm that governs the car.

This modern version is presented in an MIT Technology Review article on driverless cars and reads like this:

“How should the [driverless] car be programmed to act in the event of an unavoidable accident? Should it minimize the loss of life, even if it means sacrificing the occupants, or should it protect the occupants at all costs? Should it choose between these extremes at random?”

In “The social dilemma of autonomous vehicles” Bonnefon et al subject these questions to questionnaire analysis. “Distributing harm” they explain, “is a decision that is universally considered to fall within the moral domain. Accordingly, the algorithms that control AVs will need to embed moral principles guiding their decisions in situations of unavoidable harm.” These guiding principles, in a democracy, should reflect societal values – otherwise known as public opinion. To find these values they conducted six questionnaire surveys. Here is what they found:

“Autonomous Vehicles (AVs) should reduce traffic accidents, but they will sometimes have to choose between two evils – for example, running over pedestrians or sacrificing itself and its passenger to save them. Defining the algorithms that will help AVs make these moral decisions is a formidable challenge. We found that participants to six studies approved of utilitarian AVs (that sacrifice their passengers for the greater good), and would like others to buy them, but they would, themselves, prefer to ride in AVs that protect their passengers at all costs. They would disapprove of enforcing utilitarian AVs, and would be less willing to buy such a regulated AV. Accordingly, regulating for utilitarian algorithms may paradoxically increase casualties by postponing the adoption of a safer technology.”

Two problems with the Trolley Problem

  1. The interviewees in the Bonnefon study were offered an unrealistic choice. They were presented with the Trolley Problem as a real problem – one in which they, as car occupants, had to decide which road user should die. But as Andrew Chatham , a principal engineer on the Google driverless project observed: “The main thing to keep in mind is that we have yet to encounter one of these problems,” he said. “In all of our journeys, we have never been in a situation where you have to pick between the baby stroller or the grandmother. … It takes some of the intellectual intrigue out of the problem, but the answer is almost always ‘slam on the brakes … So it would need to be a pretty extreme situation before that becomes anything other than the correct answer.”
  2. But more importantly, the Bonefon study, and all other invocations of the Trolley Problem that I can find, reveal a profoundly biased view of the role that driverless cars might play in future urban transport systems.

In my last post  I looked at the influential role played by public opinion in determining who should have priority on the road. The book I was reviewing, Fighting Traffic, explored how “public” opinion on this issue was formed, and how the triumph of “Motordom” secured dominance for the motorist over vulnerable road users – pedestrians and cyclists – with whom they had previously shared the road. This battle, between cars and vulnerable road users, is about to be reignited by driverless cars – or maybe it’s been already lost.

The MIT review and the Bonnefon study referred to above are representative of everything I can find on the Internet about the problems that driverless cars might have in sharing the road with pedestrians and cyclists. All of the questions put to the survey groups in the Bonefort study invited them to assume they were answering the survey questions as drivers or car passengers. For example: “Participants did not think that AVs should sacrifice their passenger when only one pedestrian could be saved.” The views of the singular pedestrian, or cyclist were not solicited.

It was presumed that the societal values that should be programmed into the algorithms of driverless cars would be exclusively the values of the people in the cars. I can find no examples of the application of the Trolley Problem that acknowledge the existence of the concerns of vulnerable road users, or of policies and programmes being pursued to encourage more walking and cycling.

At present Google advertises the extreme deference with which its cars can respond to vulnerable road users. The most famous example is in this TED Talk video of a woman in an electric wheelchair trying to chase a duck off the road in Mountain View California; this can be seen in the video about 11 minutes in. All the impressive examples of deference to vulnerable road users shown in the video are displayed on roads with very few of them. How will the Google car address the problem of deferential paralysis  [1] in dense urban areas with large numbers of pedestrians and cyclists? This is a question yet to be answered.

[1] Driverless Cars and the Sacred Cow Problem, published in mangled version in City Metric, 5 September 2016.


Fighting Traffic: the next battle

Amazon Review

Fighting Traffic: The Dawn of the Motor Age in the American City,

by Peter D Norton

5 out of 5 stars     

9 May 2017

Format: Paperback Verified Purchase

Fighting traffic is an instructive account of the social reconstruction of American cities that led to their domination by motordom – the powerful collective of interests dedicated to clearing a path for the car. The most important period in the rise of motordom was the 1920s. Norton charts this transformation in terms of the insults that the competitors for road space traded with each other: motorists became “joy riders”, “road hogs” and “speed demons”, and their machines “juggernauts” and “death cars”, while pedestrians became “jaywalkers” and street cars became “traffic obstructions”. Norton explains how the road hogs won, how roads that were previously shared spaces were taken over by the car.
He attributes this victory to motordom’s awareness of the importance of shaping attitudes, the impressive resources that they had available to apply to this task, and their ultimate success in establishing that urban roads were, almost exclusively, for cars. By 1930 the battle had been won: “most street users agreed that most streets were chiefly motor thoroughfares.”
“Motordom”, Norton notes, “had effective rhetorical weapons, growing national organization, a favourable political climate, substantial wealth, and the sympathy of a growing minority of city motorists. By 1930, with these assets, motordom had redefined city streets.”
This is how he accounts for the dramatic change in attitudes, over a short space of time, about who should have the right of way on American streets: “From American ideals of political and economic freedom, motordom fashioned the rhetorical lever it needed. In these terms, motorists, though a minority, had rights that protected their choice of mode from intrusive restrictions. Their driving also constituted a demand for street space, which, like other demands in a free market, was not a matter for expert scrutiny.”
Norton’s account is not of mere historical interest. Today the five most valuable companies in the world – Apple, Alphabet (Google), Microsoft, Amazon and Facebook – plus Tesla and Uber and all the major traditional car manufacturers, are promoting driverless cars. And they promise to reopen the argument over who should have the right of way on city streets.
They boast that their cars will able to respond with extreme deference to all pedestrians, cyclists and children encountered in the street, thereby liberating them to enjoy their pre-motordom freedom to venture safely into the road. But they concede that if this freedom were widely exercised in dense urban areas motor traffic would grind to a halt. So, who will command the streets in dense urban areas? The promoters of driverless cars are also the world’s preeminent shapers of public opinion.
PS A sixth star for clear and persuasive writing.


The Pathway to Driverless Cars and the Sacred Cow Problem

The Pathway to Driverless Cars and the Sacred Cow Problem

Last Thursday (April 27, 2017) I was one of two speakers invited to lead the discussion at a National Infrastructure Commission roundtable on Connected and Autonomous Vehicles. The first speaker discussed the “Readiness of the road network for connected and autonomous vehicles”. My presentation was subtitled “Some behavioural challenges to think about.” The PowerPoint notes version of my presentation can be found here. What follows is a brief synopsis of the presentation.


In the Pathway to driverless cars the Government sets out its vision of what lies at the end of the pathway in rather deterministic terms:

     “Automated vehicle technology will profoundly change the way we travel … fully automated vehicles will transport people and goods to their destination without any need for a driver.”

The Government evinces no doubt that we will get there.   Further, the Government wants to get us there as soon as possible: “The Government wants to secure the UK’s position at the forefront of this change for the development, construction, and use of automated vehicle technologies.”

Two (possibly insurmountable) obstacles lie in the path of this vision: driver distraction and sacred cows. The Pathway to the driverless future envisioned by the Government depends on the development of Advance Driver Assistance Systems. These systems will take over more and more of the driving task until, finally, it takes over all of it and we can all be transported to our destinations without any need for a driver. The closer we get to this end state the greater the problem of distracted driving becomes. Why pay attention when the car is doing everything for you? The difficulty (impossibility?) of overcoming this problem is why Google removed the steering wheel.

This recent piece in the Guardian (27 April, 2017) illustrates the point: “In a little more than two years, a fleet of driverless cars will make its way from Oxford to London, with the entire journey, on urban streets and motorways, conducted automatically.” It then adds “The Government backed project, announced yesterday, expects to have a human in the driving seat, ready to take over if necessary.” Why should one stay alert for the whole of a journey from Oxford to London to respond to an incident that you have been persuaded is highly unlikely to happen?

The best answer that the Pathway can provide is Rule 150 of the Highway Code, which both acknowledges the problem of distracted driving and provides the solution. The acknowledgement: “There is a danger of driver distraction being caused by in-vehicle systems such as satellite navigation systems, congestion warning systems, PCs, multi-media, etc. And the solution: You MUST (sic) exercise proper control of your vehicle at all times. Do not rely on driver assistance systems such as cruise control or lane departure warnings.”

Most of the 307 rules in the Highway Code are merely advisory, but when a rule uses capitalized MUST it has the force of law. If you disobey the rule “you are committing a criminal offence.” The best solution to the distracted driving problem offered thus far is to declare that distracted driving is against the law.

Assuming we get past the distracted driving problem and arrive at the Pathway’s destination we encounter the sacred cow problem.

One can find innumerable demonstrations by the promoters of driverless cars of their ability to programme their cars to respond with extreme deference to any pedestrians or cyclists who might wander into their path – and numerous acknowledgements of the necessity of this programming if driverless cars are to be permitted on public highways. Their cars, they boast, will respond with extreme deference to all vulnerable road users. One can find numerous illustrations of the ability of real sacred cows to cause traffic paralysis in Indian cities. Why would sacred humans, aware that the algorithms of the driverless cars that they encounter have been programmed to defer to them, not take advantage of their sacred status in ways that would lead to deferential paralysis in cities with lots of pedestrians and cyclists, and perhaps newly liberated free-range children?

One looks in vain for an answer to this question. One can find a few acknowledgements that it a real problem:

“Driving in cities would be unacceptably slow if autonomously-operating cars were required to assume that every pedestrian might jump into traffic as fast as humanly possible. But if pedestrians came to learn that cars would always avoid them then they would likely act in much less controlled ways on streets and pavements.”

But solutions have yet to be found:

“Studies are underway using driving simulators to determine the optimal ways to design the human-machine interactions, but there are no clear answers today regarding design principles or standards.”

More commonly one finds a complete lack of awareness of the existence of such a problem. Here from Autoexpress:

“the Highway Code will need to change to get the most out of them. The tech will allow more accurate driving so, for example, cars could overtake cyclists more closely… “



I ended my presentation with a question. I noted that the Department of Transport, in addition to its driverless car initiative also had a well-funded project promoting walking and cycling. I asked the roundtable “Are these two initiatives talking to each other?” The answer was an implicit “NO”. Although I had been invited as one of two Round Table discussion leaders to talk about behavioural challenges confronting the driverless car project, neither of my two behavioral challenges to the feasibility of a driverless future was discussed. Not surprisingly perhaps for a National Infrastructure Commission Round Table, the discussion focused entirely on the infrastructure problems that needed to be overcome to make the driverless future a reality. There was no serious challenge to the assumption that the nation was on the Pathway to a driverless future. The guiding assumption of the discussion was that the nation was on The Pathway, and the job of those concerned with its infrastructure was to help “secure the UK’s position at the forefront of this change”.


The impending competition for road space between driverless cars and pedestrians and cyclists looks like being an unequal contest. In terms of money, political influence and friendly media coverage the driverless car project starts with an enormous advantage.


If I have wrongly characterized the discussion that took place at the Roundtable I hope those who took part will feel free to correct me in the comment section below.


PS   For a highly readable account of how the earlier battle for road space was won by the car in America see Fighting Traffic: the dawn of the motor age in the American city, by peter Norton.

And for a view from the financial sector of the problem of sacred cows click here – Sacred cows in the road




Driverless cars and the sacred cow problem

The promoters of driverless cars have demonstrated remarkable progress in their ability to program their vehicles to respond with extreme deference to pedestrians, cyclists, and cars with human drivers. Such programming confers sacred cow status on all road users not in self-driving vehicles. The developers of autonomous vehicles acknowledge the need for new road safety rules to accommodate these revolutionary vehicles on public highways. But would-be regulators have yet to propose a set of rules that would allow these sacred cows to move about freely in dense urban areas without creating a state of deferential paralysis for those in autonomous vehicles.

Full essay here (pdf)


Risk and Culture

Risk, most dictionaries agree, involves exposure to the possibility of loss or injury. Perceptions of this possibility are embedded in culture and vary enormously over space and time. One frequently encounters the contention that it is important to distinguish between “real”, “actual”, “objective” risks and those that are merely “perceived”. But all risk is perceived. Risk is a word that refers to the future, and the future exists only in the imagination. And the imagination is a product of culture.

Opening paragraph of Chapter 7 of  Routledge Handbook of Risk Studies – click here for the complete chapter



Environmental groups’ failure over HS2

Letter in Telegraph, 17 April2016

Environmental groups’ failure over HS2

SIR – It is now very clear indeed that the hugely expensive HS2 project is fundamentally flawed; yet it continues to make progress towards delivery in spite of compelling evidence justifying its cancellation.

Its passage has been assisted by two important factors that are as problematic as the project itself. The first is the failure of both governmental and non-governmental supporters to change direction on the basis of evidence. The second is the dramatic transformation of so-called environmental groups.

The Campaign for Better Transport, Friends of the Earth, the Campaign to Protect Rural England and Greenpeace have assisted this extremely environmentally damaging project at every stage.

These groups have betrayed their members as the project will, without question, add to greenhouse gas emissions, seriously damage the countryside, destroy woodland and generate levels of noise greater than those set in World Health Organisation community noise standards.

This marks a serious decline in the legitimacy of these environmental groups. It can be seen as a huge loss in a democracy constantly struggling with the excesses of government policies that emphasise the importance of the environment but in practice contribute to its degradation.

The environmental movement has embraced the old maxim, “if you can’t beat them, join them” – and we are all the losers.

John Whitelegg

[former Board member of Transport 2000 – now the Campaign for Better Transport]

John Adams

Emeritus Professor, University College London [and member of the original board of directors of FoE]

Mayer Hillman

Senior Fellow Emeritus, Policy Studies Institute

Stephen Plowden

[independent transport planner]


Cycling and Safety: change must take root in people’s minds


Last March I took part in a conference devoted to the promotion of cycling in Madrid. My presentation, in essay form, has now been published by World Transport: Policy and Practice. Herewith the abstract –

This essay is a response to an invitation to provide an overview of the current state of cycling in Britain, and more specifically London, for a conference in Madrid – a city, like London, striving to promote more cycling. The essay focuses on the importance of both the volume of motorised traffic and perceptions of safety as determinants, over time, of the volume of cycling. It notes the dramatic decline (over 95%) since 1950 in the road accident fatality rate in Britain as cyclists, pedestrians and motorists competed for the right to the use of limited road space and how, in selected areas of London, cyclists are in the process of regaining their right to the road.

And here is Figure 7 from the essay:

graph for blog

From 1950 to 1973 (the year of the energy crisis) the number of kilometres cycled in Britain plummeted – by about 80%. Over the same period the fatal risk of cycling, per kilometre, increased dramatically. The enormous increase in motoring was, physically, driving cyclists off the road. This displacement was officially sanctioned by what became known as the “predict and provide” policy underpinning transport planning. Forecasters were employed to predict future levels of car ownership and car use, and official policy was to provide sufficient road space to accommodate the forecasts. At public inquiries into road-building plans the problems of cyclists and pedestrians did not feature.

Their problems are only now beginning to be acknowledged as issues deserving of consideration alongside those of motorists stuck in traffic jams. Change does appear to be taking root in people’s minds.

The published paper can be found here (starting on page 10) –


The Driverless Car Revolution – Amazon Review

Driverless Car Revolution: buy mobility not metal

by Rutt Bridges

Review for **** 25 June, 2015

Highly recommended, but …

Mobility Not Metal is an impressively clear and comprehensive account of the potential of the driverless car revolution – with a significant omission that we will come to shortly. It provides an intelligible description of both the technology on which it depends, and the extraordinary pace of its development. The essence of the mobility vision it presents is a world transported by Uber without the drivers. It also presents convincing evidence that it will be possible for this technology to operate safely for all road users, both in and outside cars.

Bridges is clear that driverless cars will be a massively disruptive technology, and identifies the potential winners and losers in the revolution that he forecasts. Per mile travelled it will be hugely cheaper, safer and more efficient for all travellers, but especially for those denied a driving licence because of age, disability or disqualification. The potential losers include the motor manufacturing industry (far fewer cars will be needed), motor mechanics (the new “Autos” will be much more reliable and less expensive to maintain) and an army of drivers (from those driving taxis and delivery vans to pizza delivery boys on bicycles). He is clear that the gains will greatly exceed the losses – driverless cars “will have a profoundly positive influence on all of us and our planet as well” – and that the winners will prevail – Chapter 1 is entitled “How your world will [not might] change.”

He argues his case with the help of a number of American vignettes: “For simplicity’s sake, we’ll focus just on mobility services in the U.S., provided to city and suburban markets by electric vehicles.” More specifically his demonstration of the economic advantages of driverless cars is based on their potential use in Denver: “The City and County of Denver will be the theoretical test market for the Mobility economic model.” And herein lies a problem.

Bridges’ vision of the future of driverless cars is global – he speaks of its impact on “our planet” – and he envisages its principal market being “densely populated metro areas” everywhere. But by global standards the city and county of Denver are not densely populated. 82% of its commuters currently travel by car, truck, or van, mostly from sprawling suburbs. The other favourite demonstration laboratory for driverless cars is Mountain View California also, by global urban standards an area of extremely low settlement density. How, one might ask, would the self-driving car cope in a truly densely populated metro area such as London with a population density more than 150 times that of Mountain View?

This question is not addressed. Truly densely populated urban areas have dense pedestrian traffic and, increasingly, dense cycling traffic. And, after many decades of neglect, it is now official policy in most such areas that these modes of travel should be encouraged. How might driverless cars manage in such areas?

Most of the video demonstrations of driverless cars that I have been able to find show them performing on roads from which cyclists and pedestrians have been excluded. Where they are shown interacting with pedestrians and cyclists the pedestrians and cyclists are very occasional impediments, and the driverless cars are shown responding very deferentially. They are, according to Bridges, “inherently polite” and programmed to be “the ultimate defensive drivers”. It is difficult to see why, at the busy times of day when driverless cars would be most in demand, they would not suffer deferential paralysis in areas with high volumes of pedestrian and cycle traffic.

Bridges observes “it could be the regulatory challenges and not technology that pose the greatest risk for bringing the benefits of driverless cars to the people of the planet.” He fears that America might not rise to these challenges and asks: “Will America let politics and bureaucracy block real progress?” If not he fears that China might steal the lead in the race to bring his vision of progress to the world.

But he offers no clue as to how the challenges posed by high pedestrian and cyclist densities might be met. What regulatory changes would be needed to allow Autos free movement in such areas? His book does not acknowledge the existence of the problem. A challenge readers might find interesting: Google “China (or Netherlands) bicycles” and click on images, and then devise a set of regulatory changes that would allow Bridges’ Autos to share the streets with the traffic in the pictures displayed.

Buy this book. **** is a compromise. This is essential reading for those seeking to understand the evangelical enthusiasm of the advocates of driverless cars and the extraordinary wealth and political muscle of their promoters, and fearful of the consequences for pedestrians and cyclists who threaten to impede their progress.


PS (1 January 2016) A perceptive contribution to this theme can be found on the website of People’s Cycling Front of South Gloucestershire.


Self driving cars and the child-ball problem

“…if a ball were to roll onto a road, a human might expect that a child could follow. Artificial intelligence cannot yet provide that level of inferential thinking.”

This quotation from 2012 has already been overtaken by the extraordinary progress in the development of self-driving cars. But programming a self-driving car to anticipate a child following a ball is the easy part of the problem.  The tricky bit is programming the car’s response. read more


Risk: mathematical and otherwise

Draft of essay commissioned for a special issue (June 2015) of the Mathematics Enthusiast entitled “Risk: mathematical or otherwise”. Still time to make changes so critical advice welcomed, especially from mathematicians.


What role might mathematicians have to play in the management of risk? The idea of turning a risk, a possibility of loss or injury, into a “calculated” risk, a quantified probability of loss or injury, is one that has obvious appeal not just to statisticians and mathematicians – but to large numbers of others who would like to know the probability of failure before pursuing some intended course of action.

Conclusion: even when risks can be calculated with great precision, they can only be used to inform judgment, but not substitute for it. And it matters who is making the judgment. Read more …


Older posts «