Trolley Problem: Autonomous cars must not make moral decisions

October 28, 2016

Last week, Mercedes backtracked on comments by driver assistance systems and active safety manager Christoph von Hugo, which suggested their cars would prioritise the safety of the driver over that of pedestrians. Instead, Mercedes parent Daimler has said that: ‘For Daimler it is clear that neither programmers nor automated systems are entitled to weigh the value of human lives. 

The company added: ‘There is no instance in which we’ve made a decision in favour of vehicle occupants. To make a decision in favour of one person and thus against another is not legally permissible in Germany. There are similar laws in other countries as well.’ 

They also point to the need for a wider societal debate about what kind of mechanical future the world wants, saying: ‘To clarify these issues of law and ethics in the long term will require broad international discourse. This is the only way to build a comprehensive consensus and promote acceptance for the results.’ 

The issue has provoked considerable debate on LinkedIn this week. 

The ethical problem they allude to is the Trolley Problem. In the past, accidents on the motorway were largely the result of human error. However, with the introduction of autonomous cars, since computers do not make errors, programmers have to decide how a car will respond to situations on the road, which could expose the car to a situation where it has no choice but to kill. 

The Trolley Problem in its general form considers a runaway trolley rolling at speed down some railway tracks. Ahead on the tracks, five people are tied up on the tracks and unable to move. The trolley is heading straight for them, unable to stop in time. However, there is a side track to the trolley’s right which has only one person tied up on the tracks. There are two options: do nothing and kill five people, or divert on to the side track and kill one person. 

Now imagine the trolley is a car, and the five people are the occupants of the car, who will die if the car does nothing and crashes into an incoming object, but there is an option for the car to change course and save the occupants of the car, but kill a pedestrian in the process. What should the car be programmed to do, prioritise its occupants or pedestrians? 

There are great legal consequences to the decision the car makes. If the car is programmed to kill the pedestrian, the pedestrian’s family could sue the carmaker. In the large car insurance market, who pays if an autonomous car is programmed to crash into another autonomous car? The decision can no longer be left to being an ‘act of God’, caused by human error. 

Daimler has concluded that autonomous cars must not be programmed to make moral decisions. Instead, the car will simply aim to avoid problems as much as possible – leaving who it saves to the roll of the dice. 

Autonomous cars continue to face major hurdles across technological, regulatory and consumer spheres, and as such are a long way from coming to our roads. 

Daimler said: ‘Our development work focuses on completely avoiding a dilemma situation by, for example, implementing a risk-avoiding operating strategy in our vehicles. We continue to adhere to the principle of providing the highest possible level of safety for all road users.’ 

Hugo said: ‘There are situations that today’s driver can’t handle, that—from the physical standpoint—we can’t prevent today and automated vehicles can’t prevent, either. [The self-driving car] will just be far better than the average [human] driver.’

Also in News & Insights

UPDATE: News & Insights have moved to the Autovista Group website

March 31, 2017

Our regular automotive industry News & Insights are no longer being published on the Autovista Group Market Reports website.

Instead, you can now find the latest updates at our central Autovista Group website, home to our pan-European brands including Autovista, Eurotax, Glass's and Schwacke.

To stay up to date with rapidly changing market trends, we recommend signing up to our free Autovista Group Daily Brief which delivers our daily news stories directly to your inbox.

You can still find our in depth market reports here on the Autovista Group Market Reports site. Keep checking back as we have an exciting new report due to be launched shortly!

PSA to boost UK presence in the event of a ‘hard Brexit'

March 08, 2017

UK sales flat despite upcoming road tax hike, diesel demand plummets

March 08, 2017

UK new car registrations fell annually by 0.3% in February to 83,115 units according to the SMMT, driven down by weaker demand from individuals and companies. More noticeable, however, was the 9.2% drop in demand for diesels compared to February 2016, a steeper drop than the decline in Germany...