"Our comfort comes at the price of someone else's discomfort, it's as simple as that."
Those workers would rather be in the factory than back on the farm, and you would deny them that choice?
The developing world is climbing a ladder, each rung better than the last. It will be a while before they reach the conditions of the developed countries, but that's no reason to kick the ladder away.