Article |

Who would you blame if an autonomous car killed you?

EM
Eduardo Martos
CTO & Software Architect
Este artículo también está disponible en español.

Imagen generada con ChatGPT

Image generated with ChatGPT.
There is an image that has been on my mind for a while. You are walking peacefully down the street, with your usual problems: work, kids, bills, the school WhatsApp group. You hear a loud noise, turn your head, see a car losing control… and a few seconds later, everything goes dark.

You have been killed by an autonomous car.

But who would you really blame?

The car manufacturer? The programmer who wrote the braking algorithm? The team that trained the AI model with millions of examples? The company that put it on the street halfway because “it will improve with user data”? The regulator who authorized it knowing it wasn’t perfect?

And beneath all this is an even more uncomfortable question: What happens when a machine “decides” that you are the acceptable collateral damage?

🕒 Summary for busy people

Estimated reading time for the full article: 12 minutes.

An autonomous car hits you. There is no driver. There is no intent. Just a chain of technical, commercial, and moral decisions that someone made years ago, which, in that moment, turns you into a statistic.

Who would you blame? The manufacturer, the programmer, the model, the regulator… or a little bit of everyone?

The uncomfortable truth is that responsibility dilutes as artificial intelligence decides for us. Each layer—the algorithm, the data, the company, the state—shares a bit of the blame, and no one signs the final decision.

Autonomous cars are the most visible example of a much larger dilemma: we are delegating moral judgment to systems that have no consciousness or fear of making mistakes. Technology is not neutral: it reflects our priorities and our biases. And when those priorities are hidden under the word progress, the damage becomes invisible.

Perhaps the problem is not that machines make decisions, but that we stop assuming the derived responsibility.

The comforting lie of neutral technology

We love to think that technology is neutral. That algorithms only apply logic and mathematics. That if something goes wrong, it was an accident and not a decision.

But there is nothing more political, more moral, and more biased than a complex system that makes automatic decisions.

An autonomous car does not “see” the world like you and I do. It interprets pixels, distances, probabilities, trajectories… And based on that interpretation, someone (a team, a company, a committee) has defined what is an acceptable risk, what is safe behavior, and what is prioritized when not everyone can be saved.

That is where the myth of neutrality ends and uncomfortable decisions begin.

Because when you program such a system, you are answering questions that sound like a philosophy exam, not a technical specification:

  • Is the life of a passenger worth the same as that of a pedestrian?
  • Should the car always protect whoever is inside, no matter the cost?
  • What if to save five people, it has to hit one?
  • What if that person is a child and the five are adults?
  • Are there cases where the car should sacrifice its own occupant?

This is no longer a theoretical problem. It translates into code, parameters, priorities, and relative weights. And you do not decide on any of them.

The train dilemma turned into a business requirement

The famous train dilemma is known by almost everyone: A train is heading without brakes directly towards five people. You can pull a lever to divert the train to another track where there is only one person. If you do nothing, five will die. If you act, one will die. What do you do?

Until recently, this was left to bar debates, ethics classes, or Twitter threads. Now it is literally a functional specification:

“Define damage minimization strategy in scenarios of inevitable collision.”

In other words, the train dilemma has become a business requirement.

Someone has to make that decision. Not as an intellectual exercise, but as part of a product that will later be sold, certified, and driven in your neighborhood.

And be careful, we are not only talking about the extreme case where the car chooses who to kill. The same bias appears at many more subtle levels: When does it brake too soon? When does it decide that the risk is acceptable and proceeds? What types of errors are tolerated and which are unacceptable?

Each of those decisions pushes the statistics in one direction: more accidents avoided here, more risk accepted there.

The inevitable bias of programming

When we think of bias, we often imagine something crude: explicit racism in data, models that discriminate by gender… that kind of thing. And yes, that is real and serious. But in the case of autonomous cars, there is an even more inevitable form of bias: the need to choose a priority.

You cannot build a complex system without deciding what it optimizes, what it minimizes, and what it sacrifices when things get tough.

If the goal is to minimize total damage, the car may have to make decisions that, from your individual perspective, seem horrible:

“The car swerved towards me because, statistically, killing me reduced overall damage.”

No one needs to write it that way in the commercial documentation, but deep down, that’s what it is.

And the more sophisticated the system, the more the feeling that “someone” made that decision fades. The developer adjusts parameters. The product manager defines requirements. The ethics committee discusses cases. The regulator demands guarantees. The model is trained with millions of examples. Each one does a small part.

In the end, however, the consequence is very concrete in certain scenarios:

One person alive. Another dead.

The real cases we don’t want to look at

All this would be theoretical if it hadn’t already happened. And not just once, but several times.

1) Uber, 2018 — The first documented fatal case

An autonomous Uber car hit and killed Elaine Herzberg in Tempe (Arizona). The system detected her in advance but failed to classify her correctly. Uber had disabled automatic braking to avoid false positives. The driver was distracted. The regulation did not require explainability1.

Everything failed a little. Nothing failed completely. Elaine Herzberg died.

And the only one accused was the driver, not the company.

2) Tesla — When marketing weighs more than reality

In 2016, a Tesla Model 3 with Autopilot activated fatally crashed into a white truck that was overturned on a highway in Florida. The system mistook the trailer for the bright sky and did not identify it as an obstacle. The radar was also set to ignore large, static objects that could cause unnecessary braking. The car did not brake. The driver, confident, neither did.

Tesla insisted that Autopilot was just a driving aid, but presented it as if it were almost autonomous.

Everything failed a little. Nothing failed completely. Joshua Brown died.

And the responsibility floated between marketing, technology, and misplaced expectations.

3) Cruise, 2023 — When autonomy fails in unforeseen situations

In 2023, a Cruise autonomous car hit a woman who had just been struck by another vehicle in San Francisco. The system did not understand the scene: it detected an obstacle but did not interpret that it was an injured person on the ground. When trying to move the car to safety, it dragged her several meters without understanding the context.

Authorities immediately suspended Cruise’s license. The company withdrew its entire fleet from circulation.

Everything failed a little. Nothing failed completely.

And this time there was no driver to blame: just a system that did not know what to do when reality stopped resembling the simulation.

If we observe these cases together, the conclusion is inevitable: the future is already here, but we have not yet decided who should be held accountable when it makes a mistake.

Where we really are: technology and regulation in 2025

Autonomous driving is no longer an experiment. Waymo operates robotaxis in several U.S. cities, Tesla continues to expand its Full Self-Driving system, and other companies are testing driverless vehicles in real situations. The technology is impressive, but it remains fragile. It works brilliantly in normal scenarios and stumbles when the unexpected occurs: construction, unpredictable pedestrians, emergency vehicles, unusual lighting conditions. Just the moments where a bad decision costs a life.

Regulators are trying to catch up. In Europe, the AI Act classifies these systems as “high risk,” requiring more documentation, oversight, and traceability. And the regulation UNECE R157 defines how level 3 systems should behave2: when they can take control and when they must return it to the driver. In the United States, the approach is more practical and decentralized. States like California allow robotaxi testing but do not hesitate to suspend operations when there is an evident risk, as happened in the Cruise case.

The result is an uncomfortable middle ground: the technology is good enough for us to delegate to it, but not yet enough to do so without reservations. We are learning to coexist with systems capable of making critical decisions without having fully decided how we want them to do it or who is responsible when they fail.

Distributed responsibility, diluted blame

In classic systems, the chain of responsibility was clearer. If a brake failed, you looked to the manufacturer of the part, the workshop, the maintenance protocol. There were errors, but the map was more or less understandable.

With an autonomous car, the chain is much more diffuse because more actors, innovative technologies, and a factor that introduces undeniable imbalance come into play: a machine that decides for you.

And in addition, a myriad of overlapping layers that make it very difficult to isolate an error (or a decision): the software manufacturer, the hardware creator, the data labeler, the model trainer, the safety policy definer, the market release approver, the regulator, and finally, the driver who activates autonomous mode and delegates, from that moment on, all decisions to an autonomous system.

No one controls the entire system from end to end. Each actor can argue that they did their part. And that is true. But the result of that sum of parts is a system capable of killing without there being a single identifiable culprit.

It is the perfect storm: many partial responsibilities, one very specific victim, and a technical context complex enough that no one fully understands it.

And as if that weren’t enough, we add marketing:

“Our system reduces traffic accidents by 90%.”

Is it true? It might be. But when you are the remaining 10%, statistics provide little comfort.

The false comfort of continuous improvement

Another classic argument is:

“Yes, there will be failures at first, but the system will learn. The more cars there are on the road, the better the model will become.”

From a technical standpoint, this makes sense. From a human standpoint, it is unsettling. Because what we are basically saying is:

“We accept that there will be victims during the learning phase.”

The difference with other products is that, in this case, the learning phase does not take place in a controlled environment, but on the street, among real people who have not signed special consent to be guinea pigs.

Are we really afraid of the machine?

I have the feeling that, many times, what worries us is not so much the possibility of dying, but the idea of dying because of something we consider inhuman.

It is easier for us to accept that another driver gets distracted, that someone drinks and drives, that a truck veers into the lane due to human error. It angers us, it outrages us, but we understand it: we are fragile and we make mistakes.

What we struggle to accept is that a set of sensors connected to a machine learning model, fine-tuned according to business criteria and supervised by overwhelmed regulators, decided that, in a split second, our life was the sacrificial piece on the board.

A breath of fresh air: what we are doing right

Nevertheless, not everything is so dark. No potentially dangerous system has reached the market without undergoing a sometimes long and painful process of social and technical adaptation. This happened with aviation, nuclear energy, medications, the internet, and practically any innovation that now seems normal to us.

The difference, in this case, is not so much the technology itself as the type of decisions we delegate to it. We used to discuss infrastructure; now we discuss criteria, judgment, what it means to “do the right thing” when there is no obvious answer.

Still, something is changing for the better. Regulators are beginning to demand traceability so that we can reconstruct which model made which decision and with what data. Companies are increasingly aware that they can no longer hide bias under the rug, because public opinion does not tolerate it and technological reputation has become fragile. Society is also finally starting to ask uncomfortable questions that just a few years ago were not even in the conversation. Meanwhile, research is moving towards more auditable, more explainable models, and in some cases, deliberately more conservative ones.

We are not going to eliminate risk, but we can reduce it, understand it, and make it visible. Transparency does not prevent mistakes, but it prevents us from normalizing them. And perhaps, with enough social pressure, we will end up building systems that are not only safer but also fairer.

A better future is not guaranteed, but it is not ruled out either. It largely depends on what we demand today.

So, who would you blame?

If an autonomous car kills you, you could try to blame many, from the brand for selling it to the regulators for authorizing it, going through the entire manufacturing and development chain. Even society for assuming that “AI is the future” without asking too many questions.

Legally, the easiest thing will be to point to the company that puts its logo on the hood and signs the warranty. It has lawyers, insurance policies, and a communications department prepared for these crises.

Morally, it is more complicated. Because if you scratch a little, you discover that the car has applied exactly the logic that someone decided was correct, the model has responded according to the data that someone chose, and the priority of protecting the passenger over the pedestrian, or vice versa, was made in a meeting years ago, in a room where no one was thinking of you yet.

The inevitable extrapolation

We talk about autonomous cars because it is a concrete, visible, and easy-to-imagine example. But the real problem, beyond the car, is the pattern. Artificial intelligence is starting to take on more tasks, more roles, and more decisions that we previously took for granted. And every time we delegate one of them without understanding it, without supervising it, or without demanding clear accountability, we cede a small fragment of our judgment to something that has neither consciousness nor context, and that certainly does not put its own skin on the line.

Today it is the steering wheel. Tomorrow it will be medical diagnoses, judicial processes, candidate selection, public resource allocation, or school performance evaluation. If we continue to advance without asking who is responsible when an AI makes a mistake, we will end up building systems where everyone delegates and no one takes responsibility. And perhaps what is truly dangerous is not that machines make decisions, but that we stop making them.

An uncomfortable suspicion

I have a suspicion that I cannot shake off. We are fascinated by artificial intelligence, in part, because it helps us distribute blame. If a human makes a mistake, we can point to them. If a machine makes a mistake, there is always room to say that “it’s a system failure.” It allows us to keep moving forward, keep innovating, keep selling. It allows us to sleep more soundly.

But deep down we know that when an AI “decides” poorly, it is not the machine that fails. It is us delegating moral decisions to processes that we prefer not to look too closely at.

I return to the beginning: who would you blame if an autonomous car killed you? Perhaps the most honest question is this one:

Who is willing to assume, in writing, that they programmed a system where your death could be a legitimate option?

As long as no one wants to sign that, we will continue to hide the problem behind nice words: innovation, progress, safety, damage minimization, assisted driving, artificial intelligence.

And perhaps the real challenge is not to build machines that make fewer mistakes than we do, but to learn to face the decisions we hide within their code.

References

1
The ability of a model or system to explain why it made a decision in a way that humans can understand.
2
Autonomous driving systems are classified according to the SAE (Society of Automotive Engineers) into six levels, from 0 to 5:

• Level 0 — No automation: The car does not make decisions by itself. All driving depends on the human.

• Level 1 — Driver assistance: The system can perform one specific task (e.g., maintain speed or stay in lane), but the driver controls everything else.

• Level 2 — Partial automation (what Tesla uses today): The car can accelerate, brake, and steer simultaneously, but the driver must supervise and is responsible at all times.

• Level 3 — Conditional automation: The system drives by itself in specific situations (e.g., well-marked highways). The car is responsible for driving while the system is active, but it can ask the human to take control.

• Level 4 — High automation: The car can drive without human intervention in very specific areas or conditions (certain cities, certain weather). If something goes wrong, the car must be able to “resolve it” without human help.

• Level 5 — Full automation: Drives itself in any environment, without a steering wheel or pedals. It does not yet exist commercially.