"Psychology’s Favorite Tool for Measuring Racism Isn’t Up to the Job"

"A pile of scholarly work, some of it published in top psychology journals and most of it ignored by the media, suggests that the IAT falls far short of the quality-control standards normally expected of psychological instruments. The IAT, this research suggests, is a noisy, unreliable measure that correlates far too weakly with any real-world outcomes to be used to predict individuals’ behavior — even the test’s creators have now admitted as such... 

The most important benchmarks pertain to a test’s reliability — that is, the extent to which the test has a reasonably low amount of measurement error (every test has some) — and to its validity, or the extent to which it is measuring what it claims to be measuring... 

The IAT’s architects have reported that overall, when you lump together the IAT’s many different varieties, from race to disability to gender, it has a test-retest reliability of about r = .55... 

Surprisingly, there’s a serious dearth of published information on test-retest reliability of the race IAT specifically...

And when you use meta-analyses to examine the question of whether IAT scores predict discriminatory behavior accurately enough for the test to be useful in real-world settings, the answer is: No. Race IAT scores are weak predictors of discriminatory behavior...

there have always been alternate potential explanations for what the IAT really measures. From early on, skeptics of Greenwald and Banaji’s claims have highlighted the possibility that the test doesn’t really, or doesn’t only, capture implicit bias; in 2004, for example, Hal Arkes and Tetlock published a paper entitled “Would Jesse Jackson ‘Fail’ the Implicit Association Test?” in which they argued that it could be the case that people who are more familiar with certain stereotypes score higher on the IAT, whether or not they unconsciously endorse those stereotypes in any meaningful way. Along those same lines, some researchers have suggested that it could be the case that those who empathize with out-group members, and are therefore well aware of the negative treatment and stereotypes they are victimized by, have an easier time forming the quick negative associations with minority groups that the IAT interprets as implicit bias against those groups."

http://nymag.com/scienceofus/2017/01/psychologys-racism-measuring-tool-isnt-up-to-the-job.html

There is a TON of detail in this very, very thoroughly researched article, I skimmed heavily. (I would actually advise starting at the last section, starting with "It’s hard not to see Blanton’s point", and the reading up for more details as desired) 

And I'm sad, I liked the IAT. 

Although I suppose that this is much more interesting.

FB: :/ "The IAT, it turns out, has serious issues on both the reliability and validity fronts, which is surprising given its popularity and the very exciting claims that have been made about its potential to address racism. That’s what the research says, at least, and it raises serious questions about how the IAT became such a social-science darling in the first place."