ARI SHAPIRO, HOST:
The results of an IQ test can depend on the gender of the person who is conducting the test. Likewise, studies of pain medication can be completely thrown off by the gender of the experimenter. NPR's Richard Harris reports that this underappreciated problem is one reason that some scientific findings don't stand the test of time.
RICHARD HARRIS, BYLINE: Colin Chapman found out about this problem the hard way. He had traveled to Sweden on a Fulbright scholarship to launch his career in neuroscience. And he decided to study whether a nasal spray containing a hormone called oxytocin would help control obesity.
COLIN CHAPMAN: I was really excited about this project. From what I understood about how the brain works, I thought it was kind of a slam dunk.
HARRIS: Chapman set up the experiment, and then left for a few years to attend Harvard Law School. When he returned, the findings were not at all what he expected.
CHAPMAN: And I was really disappointed because this was kind of my baby. It was, like, my big project going into neuroscience.
HARRIS: But Chapman, who is now a graduate student at the University of Uppsala, says his idea turned out to be right after all.
CHAPMAN: There was another research group that, around the same time, came up with the same idea. And they ran basically the same project, and they got exactly the results that I was expecting to get.
HARRIS: That led him to wonder what had gone wrong with his experiment. One possibility was that the hormone he was using - oxytocin - can waft through the air and affect social interactions, particularly between the sexes.
So he started to worry that the natural hormone from the experimenters could have been messing up his results. One man and two women had been conducting the actual experiments.
CHAPMAN: I was asking around. I said, do you know which trial was run by who? And nobody had kept track of that because it's not something that's commonly kept track of in science, just in general.
HARRIS: Writing in the online journal Science Advances, Chapman and two colleagues now argue that's a huge mistake. He dug back through the history of science and found many, many examples of studies that are influenced by whether the experimenter and the subject are the same gender.
CHAPMAN: Even something that's supposedly as stable as IQ can be affected by the gender of the experimenter. If you have a female experimenter with a male student, for instance, you're going to see higher IQ scores.
HARRIS: It's also a big problem in pain research. A heterosexual man participating in a pain experiment will report more pain to a male tester than to a female.
Chapman suspects this is partly because the man is, subconsciously or otherwise, trying to impress the woman, and partly because the biochemistry of sexual attraction is at work.
CHAPMAN: If you're testing out a new drug for pain and you're getting these kinds of great results, you might want to look at who's running the experiment and who's participating in the experiment, because that could explain it more than the drug itself.
HARRIS: And the subjects of these experiments don't even have to be humans. In 2014, researchers discovered that the sex of a laboratory worker could completely screw up the results of pain experiments in rats and mice. Jeffrey Mogil, a neuroscience professor at McGill University, headed that study and says it's due to chemicals in the scent of males.
JEFFREY MOGIL: And if you present the male chemical in front of rats or mice, they are stressed. And that stress ends up killing pain.
HARRIS: Once you published this, what happened?
MOGIL: You know, scientists change their practices very, very slowly. And I think that it would be fair to say very little has happened.
HARRIS: He and Chapman agree that a simple first step would be for scientists to report the gender of the people who run these kinds of experiments. Richard Harris, NPR News.