Re: exactly the sort of thing we want machines to do.
"automation of repetitive manual labour has always been a net good for society. Past performance is not a guarantee of future performance."
It is a damned good predictor and acting without historical reasoning really does require some serious reason. Repetitive manual labour is also boring and physical. Two things a lot of people like to reduce doing naturally which is why automation tends to improve things.
"But the problem is that no-one at the time knew any of that was going to happen."
Nobody knows what people desire and how desires change. It is the experimentation through cheaper resources and increased time that allows for exploration of desires.
"The likes of Amazon and Uber don't want to replace all their employees with robots because it will lead to a utopian future, they want to do it to cut costs, and screw what happens to the little people."
Which ultimately helps the little people as money is information and reducing costs is improved productivity. One of the greatest successes is money. It has reduced hardship and improved lives globally where all other wishful thinking and well meaning has failed.
"But we also should take the chance to think about what those people could now do instead and how we actually want society to develop, rather than just blindly rush ahead assuming everything will somehow turn out OK."
That sounds like central planning aka the wishful thinking and well meaning that has caused so much misery in the world. We shouldnt be thinking of what people can go do instead of standing in the field, they can go do something else. We dont need to think about what people will do instead of climbing up a chimney or walking miles to collect water or crawling under dangerous machines, they can go do something else. Instead of trying to think up jobs for back breaking labourers we can let them go do what they want to do.