The British Journal of Psychiatry should be embarrassed.

PZ Myers shows us the fiasco of a putative meta-analysis of mental health risks of abortion, published by the British Journal of Psychiatry and torn apart by Jim Coyne at Psychology Today. The problems with the analysis are briefly summarized as follows:

1. The author has a conflict of interest on the subject, as she is an anti-abortion advocate, and failed to disclose this in her submission to the journal.

2. The analysis used 22 studies, half of which were conducted by the author herself. She did not disclose which studies were excluded and why.

3. Her own studies used in the analysis range from unreliable to meaningless.

Since when did scientific rigor ever get in the way of a good scare tactic? Coyne helpfully quotes National Right to Life News as summarizing conclusions such as:

“Women who aborted have a 55 percent higher risk of mental health problems compared to women with an ‘unplanned’ pregnancy who gave birth.

Yeah, I just love the scare quotes around “unplanned.”

NRtLN’s summary conflates the comparison between women who have aborted vs. not aborted, with those who have aborted vs. given birth. It confuses an outcome for a given pregnancy with lifetime experience. IOW: it is possible for a woman to have at least one abortion AND have at least one live birth. The majority of women having a first abortion are already mothers, and many others have children later.

If what they mean by “aborted vs. given birth” is the comparison of women who’ve had at least one abortion with those who’ve had at least one live birth and no abortions, then they should freaking well say so, and furthermore, they need to limit the comparison to women who became pregnant when they didn’t want to. Since this is an organization that uses scare quotes around unplanned pregnancy, such respect for confounding factors is probably too much to ask.

Anyway, that’s just one anti-choice website’s trumpeting the study’s conclusions. What else do we need to know about Priscilla Coleman’s analysis?

What is the issue with Coleman’s choice of comparisons/control groups? She relied heavily on comparisons between women with received an abortion for unwanted pregnancy and women who had completed a pregnancy that was planned or wanted. Of course, this strategy cannot get at the effects of abortion because presumably the circumstances of having an unwanted or unplanned pregnancy are different from the circumstances of completing a wanted pregnancy. It is not surprising that women having a wanted pregnancy have better mental health than women who did not want to be pregnant.

Perhaps Dr. Coleman is operating from the same state of mind as National Right to Life News, with the scare quotes for unplanned pregnancies. If Dr. Coleman believes there is no meaningful difference between an intended pregnancy and one conceived by carelessness or coercion, then we cannot expect her to control for the difference in her studies. However, the British Journal of Psychiatry should have noted this lack of control before they published the analysis.

Another criticism of Coleman’s review is that she often did not control for prior mental health. So, if a woman smoked marijuana or consumed alcohol after an abortion, she was compared to women who dealing with a newborn child, and any differences were attributed to the first group of women having had an abortion, even in situations where ascertainment of drug or alcohol use occurred before the abortion.

There are two layers of crappy science here. One is that Coleman’s analysis expects us to believe that mental health differences after an abortion or live birth are not attributable to differences before the reproductive decision. In order to reach that conclusion, you need to control for differences before the decision. The other issue is that these are very shoddy criteria for assessing mental health. Are we to assume that mentally healthy women don’t drink or get stoned, ever? Are we also expected to abstain from caffeinated beverages?

If you examine Figures 1 and 2 in Coleman’s review, you can see that she counts each of her own studies multiple times in her calculation of the effects attributable to abortion. This practice was also roundly criticized in the E-letter responses to her article because each study should only be entered once, if the conditions are met for integrating results of studies in a meta-analysis and providing a test of the statistical significance of the resulting effect size. This may sound like a technical point, but it is something quite basic and taught in any Meta-Analysis 101.

Coleman’s calculation of overall effect sizes for the negative mental health effects of abortion involve integrating multiple effects obtained from the same flawed studies into  a single effect size that cannot accurately characterize any of the individual effects – anxiety, depressionsubstance abuse, and suicide – that went into it. Again we are encountering a nonsensical statistic. In meta-analysis workshops and basic courses, it has become standard practice to include a slide allegedly of a road sign announcing that you are entering New Cuyama, California on which  a population of 562, an elevation of 2150 feet, and a date of establishment of 1951 are summed up to a total of a total of 4663. This is meant to be a spoof of the meaninglessness of combining diverse outcomes into a single summary effect size.

“Integrating multiple effects obtained from the same flawed studies into a single effect size.”

That’s why she had to count her own studies multiple times each: she had to count up the incidence of anxiety AND depression AND substance abuse AND suicide, which meant she had to multiply the population size by all those issues!

Meanwhile, wasn’t it helpful of her to supply all of that data from her own research, and wasn’t it fortunate that she was there to figure out how many times it needed to be counted!

(*wipes sarcasm off keyboard*)

I suppose Dr. Coleman needs to do all this work herself, since there’s a vast liberal conspiracy to suppress the truth about the mental health risks of abortion. No, really, just ask her yourself.

The E-letter from Ben Goldacre is particularly noteworthy. A British psychiatrist, he was recently appointed by BJP editor as Co-Editor for Debate, and is best known as the author of a Guardian column and book, both with the same title, Bad Science.

Goldacre, true to form, laid bare the pseudoscience of the review. He tracked down a PowerPoint presentation of Priscilla Coleman, author of the review and raised the issue of a serious conflict of interest that she did not disclose. Goldacre noted that the PowerPoint presentation explicitly declared:

We need to develop organized research communities to continue the research, apply for grants, recruit young academics, critique data produced by pro-choice researchers, challenge politically biased professional organizations, train experts to testify, and disseminate cohesive summaries of evidence.

Yet, take a closer look at Coleman’s PowerPoint presentation that even more boldly reveals her  perspective:

The rapidly accumulating literature on the negative effects of abortion is rarely made available to practitioners and to women considering abortion as professional organizations, including the APA and the AMA, along with the liberal press expend  incredible amounts of energy to hide the now scientifically verified truths.

Remember AAPLOG? They’re the ones who argue that abortion should be restricted so that obstetricians can get paid for more live births. Dr. Coleman made a presentation to them in which she stated that they need to develop research communities to “critique data produced by pro-choice researchers” because there’s a far-reach liberal conspiracy including the APA and AMA dedicated to hiding the truth that having abortions makes women crazy. This is a person who has already demonstrated that she has a dog in the fight, and this should have been disclosed to BJP.
In realistic terms, though, it shouldn’t be a surprise that she didn’t tell BJP that she’s a conspiracy-minded ideologue who’s dedicated to eliminating reproductive freedom. She’s against abortion rights, and she wants her research to provide fuel to the anti-choice movement, so she has no incentive to disclose a conflict of interest. If she honestly thinks the liberal press is conspiring to hide the truth, then it’s only natural for her to conclude that it’s acceptable—nay, necessary—for her to do whatever it takes to get her “scientifically verified truths” into a prestigious journal.
Perhaps the editors of BJP aren’t responsible for digging up the dirt on contributors to find out conflicts of interest they may not have disclosed, but in this case, it was more than just a matter of not knowing. They probably should have been a bit more familiar with David Reardon, the Moses of the anti-abortion movement:
For the purpose of passing restrictive laws to protect women from unwanted and/or dangerous abortions, it does not matter if people have a pro-life view. The ambivalent majority of people who are willing to tolerate abortion in “some cases” are very likely to support informed consent legislation and abortion clinic regulations, for example, because these proposals are consistent with their desire to protect women. In some cases, it is not even necessary to convince people of abortion’s dangers. It is sufficient to simply raise enough doubts about abortion that they will refuse to actively oppose the proposed anti-abortion initiative. In other words, if we can convince many of those who do not see abortion to be a “serious moral evil” that they should support anti-abortion policies that protect women and reduce abortion rates, that is a sufficiently good end to justify NRS efforts. Converting these people to a pro-life view, where they respect life rather than simply fear abortion, is a second step. The latter is another good goal, but it is not necessary to the accomplishment of other good goals, such as the passage of laws that protect women from dangerous abortions and thereby dramatically reduce abortion rates.
This guy co-authored ten of the studies that Dr. Coleman performed and used in the meta-analysis. He has described, in so many words, his intention to sow just enough confusion around the supposed dangers of abortion to diffuse opposition to anti-choice laws. It doesn’t stop there, though.
One disturbing possibility is by way of the influence of an Associate Editor of the BJP, Irish psychiatrist Patricia Casey. She, unbelievably, has an even stronger track record than Coleman of letting ideology intrude into evaluations supposedly based on evidence. Not only does she have ties to the American anti-abortion movement, she is on record as claiming this scientific evidence backs that having parents divorce is more traumatic for young children than having one of the parents die.

I guess Dr. Casey is one of the brave scientists fighting the vast liberal conspiracy to hide the truth about abortion, and as far as she’s concerned, it’s not a conflict of interest, it’s a crushing blow to the pro-choice enemy.

Peer review is part of the self-correcting mechanism which separates science from ideology. The responses to the Coleman analysis, including Dr. Goldacre’s rebuttal, can be found here. I will quote a snippet from epidemiologist Chelsea Polis’s letter:

Coleman continues to ignore the scientific importance of accounting for pregnancy intention in this body of literature. If women who abort (many of which are unintended pregnancies) are compared against women who deliver (many of which are intended pregnancies), effects of unintended pregnancy are difficult to disentangle from effects of abortion. Circumstances surrounding an intentional versus an unintentional conception or pregnancy may be related to mental health outcomes. Most aborted pregnancies in the United States were unintended.[7] Coleman wrongly assumes that since nearly half of pregnancies in the US are unintended, most births are too, failing to acknowledge that almost half of unintended pregnancies end in abortion.[8] Thus, her assertion that “the majority of women in the control groups in studies comparing abortion with term pregnancy actually delivered unintended pregnancies even if the variable was not directly assessed” has no empirical grounding. Similarly, her assertion that a “no pregnancy” group may be a “cleaner” comparison group ignores the fact that the “no pregnancy” group would not have experienced unintended pregnancy.

It takes some very fuzzy math indeed to conclude that most live births are from unintended pregnancies when approximately half of total pregnancies are unintended. In all seriousness, though, I appreciate Dr. Polis’s distinction between the mental health effects of unintended pregnancy vs. effects of abortion, especially w/r/t circumstances of conception. I couldn’t have said it better myself.