“Who would buy a car programmed to sacrifice the owner?”

"One of the underlying assumptions in the discussing the ethics of the self-driving car is that there is a knowable outcome and, working backwards, a right answer...

Equally at fault for the failure of making such a car an attractive product for consumers is our inability to objectively asses risk. The probability of dying in a terrorist attack are infinitesimal compared to a lightning strike or in a plane compared to a car crash [7], [8]. It comes down to how we perceive risk vs. the actual likelihood of the risk [9]. Even though autonomous cars would dramatically reduce overall accidents, injuries, and car-related fatalities, because the risks are beyond our control, are in the future, and unknown, they loom large."

https://medium.com/@scweiss1/who-would-buy-a-car-programmed-to-sacrifice-the-owner-28b93db822a3

I really enjoyed how Stephen lays out all the issues here.

It makes me think about the difference between our idea of machines/computers and our idea of tools - cars were originally tools, that we operated in order to move me quickly. More and more, they are machines that we input information into and expect certain outputs from. And while, right now, that relationship is pretty clear (I apply a certain amount of pressure to the brakes, turn the wheel a certain way, turn the knob for the volume, choose a certain gear...) we are adding more and more black boxes (with GPS, we trust it to find the best route with the least traffic, ask it to find the cheapest nearby gas or nearest parking lot; with proximity sensors, we ask the car to beep before we back into our neighbor's bushes and so we've stopped looking out ourselves).

On the one hand, there seems to be a desire for a world where we don't have so much decision fatigue and can avoid uninteresting and frustrating quotidian tasks like figuring out how directions to unknown locations or remembering when our next dental appointment is. (or maybe this is what the tech world wants, and where capitalists see potential for economic growth). It's the Jetsons, a 1950s ideal of the future that still shapes our imagination.

But... Sometimes these tasks can be part of how we feel connection to our lives and our days, and when we cede them to machines built by strangers we lose a personal touch to our own decisions
There is a danger of mindlessness if we can't replace those moments of connection, if we are convinced that these machines let us avoid focusing on the current moment.

My solution to this has been to connect to my machines as though they are tools, whose black boxes I can wield if I try to maintain self awareness and think about what I NEED instead of what some unknown designers and builders imagine that I "could do".

I think about the conveyor belt that pulled the Jetsons through their morning, and I feel no joy or life in that. But I love that my needs are met, and my capabilities augmented, by tools.