"“We’re always judging people in all sorts of ways, but without data we do it with a selection bias,” he said. “We base it on stuff we know about people, but that usually means favoring people who are most like ourselves.” Familiarity is a crude form of risk management, since we know what to expect. But that doesn’t make it fair...
Algorithms are written by human beings. Even if the facts aren’t biased, design can be, and we could end up with a flawed belief that math is always truth.
Upstart’s Mr. Gu, who said he had perfect SAT scores but dropped out of Yale, wouldn’t have qualified for an Upstart loan using his own initial algorithms. He has since changed the design, and he said he is aware of the responsibility of the work ahead.
“Every time we find a signal, we have to ask ourselves, ‘Would we feel comfortable telling someone this was why they were rejected?’ ” he said."
http://bits.blogs.nytimes.com/2015/07/26/using-algorithms-to-determine-character/?_r=0
I think this is a good direction as long as we have the awareness that it is imperfect; better than the nepotism we currently have <how nepotistic are we>, but super vulnerable to serious flaws.
No comments:
Post a Comment