The algorithm is only as good as its training data. Improve the data, maybe tweak the algorithm and its completely neutral.
The problem with facial recognition isn't that it 'doesn't work'. It does. Rather well, in fact. Its like ANPR for people. (...and nobody complained when ANPR was rolled out so that your vehicle could be tracked anywhere, anytime. You really have to pay attention to these things.....now everyone's making a lot of noise about facial recognition but its really too late to do much about it, at least at the government/law enforcement level)