The Goldberg Paradigm and Elegant Gender Research

[CW: discussion of gender research and reactions therein. More research analysis than social justice.]

Recently, I’ve been having a number of conversations about methodology in psychology. I’ve been mentioning that psychology of gender research has tended to have excellent methodology, relative to the baseline. In response, I’ve gotten shocked laughter, nervous giggles, and utter confusion. I expect that some of this comes from researcher fears about one’s research being misinterpreted for great big arguments on the internet, defenses of sexism, etc.

For instance, imagine this entirely fictional scenario in which you take a careful look at Norwegian households and chores. (pdf, all in Norwegian until p. 223) You notice that the households with more parity in housework sharing are also households that break up more frequently. You write two hundred and twenty nine pages,* carefully thinking through causes for the results, including statements like

“Untraditional couples, where he does the most of the housework, may hold a less traditional or more modern view about marriage, whereby marital dissatisfaction more easily leads to marital break-up. If so, the division of housework is no “cause” of later divorce”

and after all of that work, you get deep media analysis like:

In what appears to be a slap in the face for gender equality, the report found the divorce rate among couples who shared housework equally was around 50 per cent higher than among those where the woman did most of the work.” [Telegraph]

Ladies, you may want to think twice before asking your husband to help out around the house. [thanks, HuffPo]

And then there was the pushback: who is this guy? There could be so many other mediating variables! (The pushback was impressively thoughtful, but I still regularly hear that research says women should do housework/chore-sharing will ruin your relationship). In short, people might not have strong and divisive opinions about the truth of research investigating how people pee in public restrooms. Said research is unlikely to be a question they’ve wrestled with, a formative part of their identity, or purporting to answer a question of how one should handle a highly-valued relationship, in the way gender relations is.

But, write a study showing that women are slightly better at a female-associate skill, that men are better at a male-associated skill, that there’s no difference in gender performance on a specific-gender-associated skill, that traditionally structured relationships succeed in specific ways, that non-traditionally structured relationships succeed in specific ways….do any of those things, and people will have Opinions and other people will have Disagreements, all of them asserting that they’re experts. I think this knowledge: that your research will be endlessly misinterpreted and picked apart in the public sphere, keeps psych of gender researchers constructing careful, airtight, designs and cautious discussion sections.

So, let’s talk about some really fantastic gender research: the Goldberg Paradigm.

The Goldberg paradigm is elegantly, beautifully, simple. It’s the sort of experimental design that throws me in to raptured lectures and hand-waving.

Picture this: you create one written profile of a potential job candidate. However, you assign it two different names: one male, one female. Then, armed with you two different job candidates, you hand the profiles to a number of department chairs, and see what happens. In fact what happens (and has prompted over a hundred replications since) is that the female-named resumes get less response. They’re less likely to be called, less likely to be viewed as qualified, less likely to be offered mentorship. This effect appears without a difference between men and women potential-employers.

The research done by Goldberg originally took place more than thirty years ago, but a more recent test of the paradigm showed that it continues to replicate. Variations are also used to test religious bias and prejudice against ‘non-American’ names.

So what makes the Goldberg Paradigm so cleanly beautiful? In short, it handles lots of tricky confounds. In many aspects of workplace treatment, disentangling other factors of communication style and halo effect  makes for an impossible game. Is it that employers favor men or that other, simple, psychology serves people who push hard for raises, and men are more likely to push for raises? The Goldberg-based resumes create two identical career histories, remove any appearance or behavioral variables, and see what happens.

After taking a minute to think about variables other than implicit gender bias that could result in this sort of robust effect, here’s the only ones I came up with:

Names: there’s some name research that perhaps certain names are associated with certain skills. If the male name on the Goldberg resume is more congruent with the job in question than the female name, eh….maybe? I’m skeptical of this research already, and have a hard time imagining that this effect, if it even exists in the wild, could be contributing to every single replication that found the effect.

Job-Gender congruency: For the most part, these were for male dominated fields: lab manager, researcher. If you were to do this with a nurses job, I would expect the results to reverse or disappear. (However, I want to point out more gender research, somewhat old, suggesting that perceptions of men doing ‘women’s’ jobs and women doing ‘men’s’ jobs are treated differently, and are far more likely to be rewarded for ‘token’ status.)

That’s….not so bad, as far as confounds in psychology go. And sure, some of that has to do with the subject at hand. “Does gender impact the interpretation of concrete details and qualifications on resumes” is a bit of an easier game than “How does gender impact subtle social status signaling in conversations in the workplace?” It’s just slightly harder to get the latter.  The former can give us some sense of the landscape, without building so much on tenuous connections and oceans of confounds.

And since I started by talking about poor interpretations of psych of gender research, let’s talk about what happens here, what to make of the results. They’re sometimes used to argue that ‘being a man is equivalent to X years of experience”. Which…is more rhetorical than purely correct, because of course it’s not that easy to quantify. What I think the cost that’s being indicated by the resume test is that of attributional ambiguity.

Imagine this: you’re half the Goldberg paradigm come to life. You, a woman, submit your application materials to a lab where you’d like to work as a research assistant. Several days later, you hear that a candidate has been selected–a man. This is the fifth time you’ve submitted materials, and you feel quite as qualified as the other applicants. On one hand, it might be that you’re unlucky. Or there’s something about your resume that’s off putting–a typo, or just failing to properly advertise yourself. It might have been that each of those times you applied, you were competing against an extraordinary candidate, who was hired each time.

Or, it might be that you’re a woman, and accidentally on the receiving end of some unconscious bias. You have no idea, and what’s more, you have no way to find out. You might be able to change some aspect of the application, or get more experience before applying to that sort of job again, but you also don’t know if that’s what works. You might be wasting extra work on nothing. And that, perhaps, is the most damning thing the Goldberg paradigm points at.


*This also puts me in mind of the Rind, et al controversy.

Goldberg, P. (1968). Are women prejudiced against women? Transaction, 5, 316-322.
Fidell, L.S. (1975). Empirical verification of sex discrimination in hiring practices in psychology. In R.K. Inger & F.L. Denmark (Eds.), Woman: Dependent or independent variable? (pp. 774-782). New York: Psychological Dimensions.
Rudman, L.A. and Glick, P. (2010). The social psychology: How power and intimacy shape gender relations. New York: The Guilford Press.
Advertisements

3 thoughts on “The Goldberg Paradigm and Elegant Gender Research

  1. There probably needs to be a meta-analysis to tease out the effect that arises if 50% of all of a certain category of resumes with names like DeMarcus on them are fake or fraudulent submissions by other university researchers, while 99% of resumes with names like John are not fraudulent.

    Academics complain about this in general all the time (see someone like Andrew Gelman), and also fraudulent-but-not-for-research contacts from people in India, China, Nigeria. Most people doing hiring in most fields are probably unaware of the extent of this type of research but it could be specifically creating its own problems.

    1. Oh, HUNH. My guess, if I understand what you’re referring to, is that the effect of people doing this research is small enough to not be changing how people react to not-traditionally-named resumes, but yeah, I’d want to confirm that.

  2. Oh, thanks for responding and sorry for my own delays getting back quicker after the 4th.

    20 years ago, there might have been a dozen researchers with a few studies. There is no doubt now that the numbers have increased massively in the intervening years. In little time one is able to find a lot of examples; all of the below have distinctly very large sample sizes.

    This is a classic game theoretical failing, besides the competition between researchers for publications and presige, increasing the raw numbers that even one researcher sends out is easy with computer technology and makes for nice and shiny statistics to publish.

    http://oreopoulos.faculty.economics.utoronto.ca/wp-content/uploads/2014/03/Why-Do-Skilled-Immigrants-Struggle-in-the-Labor-Market.pdf
    http://faculty.chicagobooth.edu/matthew.notowidigdo/research/Kroft_Lange_Noto_Resume_Study.pdf
    http://johnnunley.org/under_unemp_april_2014.pdf

    The number of actual people, for instance actual college graduates, in categories of underrepresented groups (certain minorities, women, etc) might hardly be larger than the number of false hits being sent out. The effect we’d worry about ethically is plausibly even because representing many false persons from one single institution could have disproportionate harm to that institution and the people therein.This is still excluding actual fraud perpetuated by real people but in the wild – foreigners spamming fraudulent information and so on which again academics seem explicitly aware of.

    There is at present no centralized organization for approval/ethics and no pre-registration of these studies, both of which would help. It seems like researchers in many fields don’t even cite all the papers that other fields put out, say economists are mutually unaware of psychologists and sociologists and both of other field specific audit studies, so getting a sense of the extent right now is hard.

    At the least it seems like there are some tens of thousands of false resumes being sent out each year, and it could actually be much much more. If this doesn’t matter for majority groups where the number of real people is much larger – people named John and not Aaliyah, or merely studies that look at young workers vs old workers, I’m not so sure about the ethics for all subgroups.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s