Mother Jones Meltdown

facepalmWhen you read that “researchers have found a statistically significant correlation between [thing x and thing y]” what do you imagine the percent of correlation to be (after, of course, you realize that since it’s just correlation so all they’ve found is that two things sometimes happen at the same time, without any idea of what causes them, and with the distinct possibility that they are completely unrelated.) Still, what would your expectation be if a source that it supposed to be reporting the news said that there was a “statistically significant correlation?” Would you expect that correlation to be 80%?  50%?  Would you expect it to be .02 percent?

That’s exactly what Mother Jones did on their Facebook post today.  Their teaser for an article they linked to by Pacific Standard called  “Grand Obese Party” (see what they did there – so clever) read “Researchers have found a statistically significant correlation between support for Mitt Romney and a pudgy populace.” The story this linked to reported a study that had found that “a one percent increase in county-level support for Romney corresponds to a 0.02 percent increase in age-adjusted obesity rates.” I guess “Having Cured All Diseases and Ended Poverty and Hunger, Researchers are Spending Money to See if Republicans are Fatter than Democrats” was just too long for a headline.

[Edit for quick discussion of statistical significance:  The reason I ask about expectation is that the paper is either counting on the readers to not understand statistical significance or they don’t understand it themselves. Either way, just really poor journalism. Statistical significance is about the probability that an effect doesn’t occur by happenstance.  One of the first things I learned in my stats and research methods classes (right after correlation never ever implies causation) is that statistical significance is not an indicator of meaningful results.  Unfortunately reporters who don’t understand research often think that “statistically significant” sounds important and all “science-y” and this kind of “news story” is the result – where they report the results as if they are important because they were “statistically significant”  and don’t tell you that, for example, the sample was a small group of middle aged white men which severely limits how they can extrapolate results.

Even more ridiculous:

The researchers argue this reflects poorly on the Republican party’s emphasis on “personal responsibility” for reducing obesity risk. Successful fat-fighting strategies “will necessarily involve government intervention,” they argue, “because they involve workplace, school, marketing and agricultural policies.”

Bigger government or bigger waistlines: The choice is yours.

First of all, nobody has a “successful” intervention for obesity – we have no idea how to make fat people not fat.  What we do have is a political climate where any idea that someone suggests will “reduce obesity” is likely to be implemented with absolutely no evidence required.  That’s how we ended up with programs in schools that didn’t do anything to make fat students smaller, but did increase eating disorders.

So let’s review:  Mother Jones contributed to a climate of bullying, stigma, and shaming of fat people for a .02% correlation, some cheap alliteration, and the suggestion that people should vote for the party that will best eradicate a group of people based on how they look.   Pretty sure this is the definition of being a hack.

This member of the “pudgy populace” is not impressed.


Let’s give some feedback:

Pacific Standard (who ran the piece):

Mother Jones:

Comment on MJ’s Facebook thread about the piece:

31 thoughts on “Mother Jones Meltdown

  1. I’m getting a headache from rolling my eyes… I guess that’s better than getting a headache from beating my head against the wall. I’m no scientist, but .02% does not sound all that significant. Maybe I’m missing something here.

    1. The term “statistical significance” simply means that the researchers agree that there is a probability that the correlation is not merely the product of chance. Nothing more. It has nothing to do with the dictionary definition of “significance.”

      1. Thanks Helena… I got my lone B in Statistics (I’m a straight A type of gal), and I think it was a sympathy B. At the time I was deploying heavily to war zones and I think the teacher just said, eh, her other scores rock and she did try to understand the subject while emailing in her work from a place with wonky internet connections.

  2. 0.02% really? Must not understand the term significant.

    But then, she wasn’t looking for real science. She was looking for a reason to reinforce bigotry and “make her point”. Science is just what she hides behind in an attempt to create legitimacy.

    Not impressed either.

    1. Well, there’s a difference between “statistical significance” and “actual significance”. Something might be statistically significant but be irrelevant in the real world. The mistake is when journalists see “significant!” and run wild with it.

    2. I hate to talk about diets here, but I’m going to for just a sec, because it’s an example that well demonstrates the difference between statistical and practical significance (it’s also an example that showed up a lot in my stats class.)

      As Ragen has often repeated, and I once read the study myself, Weight Watchers studies showed that people lost an average of 5 lbs in a year of WW, which is statistically significant (i.e. repeatable in experiments), but anyone wanting to diet probably wants to lose more than 5 lbs, so it’s not practically significant.

    1. “I bet you’re fat, too!” –Insult hurled at me by a Democrat on a (somewhat) Left-tolerant board. (Let the record state: I’m a Green. The DP and the GOP are both at all times invited to kiss my fat ass.)

  3. I enjoy your blog, but I have to defend “statistically significant.” Although it does sound small, statistically significant is not an arbitrary term. There is an actual mathematical formula used to determine when something is statistically significant. Of cause, correlation does not mean causation. I’m sure we could come up with tons of random things (people who like ballons, people who dislike bananas, etc.) and compare them to their presidential candidate choice and find lots of statistically significant correlations. It doesn’t make it interesting or worthy of writing an article.

    1. Hi Kate,

      Thank you for your comment and your kind words about my work. Since you brought it up, I’ll talk about it here. Statistical significance is about the probability that an effect doesn’t occur by happenstance. [edited to remove a sentence that, though I didn’t intend it to, sounded super rude when I re-read it. Sorry!] Statistical significance is used in various contexts and is based on a number of factors some of which (like P values) may be chosen by the researcher. One of the first things I learned in the first of those classes (right after correlation never ever implies causation) is that statistical significance is not an indicator of meaningful results.

      I was actually going to break this down in the original post with a discussion of statistical significance but I decided that the fact that they are reporting a 1 percent to .02 percent correlation really had already done the work for me. Their choice to focus on the “statistical significance” suggests, to me, their lack of understanding of the way that research actually works. It’s a term that sounds “science-y” and that is widely misunderstood and I see it used this way all the time. When I asked people what they expected when they heard that results were “statistically significant” I was asking about what news outlets would think was newsworthy, not what could actually be defined as statistically significant based on the researchers’ choices.

      Hope that makes sense, thanks again for your comment!


  4. .02??? for serious??? when did MJ become the Star? They used to stand for real news, and the type of reporting other magazines other publications wouldn’t touch…seems they have become something my college professors would have laughed at in private, and suggested “needed further study” and “showed insufficient proof” on the grade….

  5. The term “statistical significance” usually designates a formal concept in statistics, usually measured in units of standard deviations from the mean, or “sigmas”, assuming a normal distribution. I didn’t read the original piece, but in media coverage of scienctific studies, this is often a problem: journalists read “statistically significant” in the abstract, introduction, or conclusion (aka the parts that don’t have lots of math and technical jargon aimed at peer professionals), and repeat the term. The audience then reads the term and understands it in ordinary language, where “statistically significant” means something like “important math stuff” or “its a big deal!” without reference to the technical concept.

    There are a lot of legitimate criticisms of the concept of statistical significance made by people who debate stuff like problems in classical statistics. One of those issues is that, in virtue of being stipulative (rather than, say, axiomatic — a mathematical measure we made up like how long is a foot or an inch, rather than a mathematical axiom we discovered like the Pythagorean theorem or Pi) statistical significance doesn’t track features of mathematical reality (the way that the Pythagoriean theorem or Pi does) but rather just detects specific connections we have designed it to detect (sort of like a programmed formula in an Excel spreadsheet or something). Statistical significance requirements vary based on the discipline and kinds of data sets.

    So, the thing is Ragen, the correlation detected here — between body size and party affiliation — may actually be statistically significant by the measures typically used in research for political demographics(and those measures may vary from those used in, say, clinical trials), since that is determine by a formula set up to look for significant correlations in various populations. (The same formulas might be used to try and discover, hypothetically, if Republicans are typically more religious than Democrats, or if Democrats are typically more educated than Republicans, or if there Democrats tend to live in cities or suburbs, or whatever). I don’t know — since I clicked around a bit and can’t seem to find the link in Mother Jones to the original study, and so haven’t reviewed their statistical methodology.

    However, the main component — drawing conclusions about “personal responsibility” from a statistically significant correlation — really is just shit science reporting. If I were to report on children in the US who graduate from foster care, I would report that there statistically significant correlations between graduating from foster care and committing suicide (or homelessness or drug addiction). However, the correlation is not self-explaining, causally — it needs to be supplemented with additional data (for example, from developmental psychology) that explains why this correlation exists. Similarly, you can’t say “This correlation (assuming the researchers used statistics properly and it is statistically significant!) means that most Republicans aren’t personally responsible!” without looking at any other factor. And it certainly doesn’t justify a moral judgment about Republicans or an argument in favor the Democratic party — for all we know, there is a statistically significant correlation between being Democrat and being bulimic/anorexic that would explain the difference, for instance. In the absence of any other data or explanatory connection, all this statistic tells us that people who self-identify as Republicans have larger body size.

    1. In the absence of any other data or explanatory connection, all this statistic tells us that people who self-identify as Republicans have larger body size.

      Actually, it doesn’t even tell you that. It tells you that people who *live in Republican counties* are very slightly more likely to have a larger body size.

      Not criticizing your overall statement, just that the correlation is even fuzzier.

      1. And the irony is that given the large concentration of conservatives in the South, these counties are probably largely filled with fat people who are also POC (specifically Black people and Latin@s)… who in most cases would in no way vote for Mitt Romney (if their rights to vote were protected, of course, which they aren’t… thanks Supreme Court!).

  6. Well, at least they’re honest. They’re not giving us a “so conceerrrrned” spiel; they want us gone and they want government intervention to make it happen. The sad/scary thing is, people will read this plea for the government to step up and actively work to eradicate a group of people with nothing in common but a single cosmetic characteristic and think it sounds perfectly reasonable because the group in question is us. Feeling really bummed right now. Nothing cute to say.

    1. Plus, I feel like the implied statement is “don’t vote for Republicans, you’ll get FAT.”

      As a Fat Republican, I’m pretty sure that the box I check on a piece of paper has precisely zero impact on my weight.

      1. As a fat Democratic Socialist living in GOP Central, if I haven’t lost weight by now from running for my life, I’m never going to. 😉

  7. Reblogged this on Sly Fawkes and commented:
    Hey, this is fun! Let me try one!
    I am fat.
    I have an underactive thyroid gland.
    Being fat causes your thyroid gland to be underactive!
    Now I just need to write the article stating that if everyone would just become a Socially Acceptable Size (TM) their thyroid problems would magically disappear!
    So, what I’m getting from this Mother Jones article is that being fat causes you to vote for Republicans.
    Well, if only I’d known that! But I didn’t know, so I voted for the Other Guy.

  8. Sweet flaming Jeebus. You can correlate anything to anything, really. I saw an unusual number of blue cars on my driving route today, and it was 65F. So I can conclude that on days that are exactly 65F, people who own blue cars are more likely to drive them on my route.


  9. Would statistical significance indicate that something might benefit from further study while if it’s below statistical significance you would look in another direction?

  10. “Bigger government or bigger waistlines: The choice is yours.”

    Yes! Be so scared of the ZOMG DEATHFAT that you’re willing to abandon your beliefs! Obviously, you’re so stupid that you need the government to tell you how to live, or you wouldn’t be fat! DO YOU NOT REALIZE THAT AN EXTRA .02% OF PEOPLE WHO LIVE NEAR A FEW EXTRA REPUBLICANS ARE FAAAAAT?!?!?!?!

    It should be funny. Except it’s not, because being fat is so wrong to these people that you should put avoiding fatness before anything else that means anything to you in your life. You know, just in case.

    Was today really so completely devoid of news that this seemed a good idea to put forth as journalism?

  11. I was pretty upset when I read the post headline on my FB Feed as well, and let them know on their FB page – not that they will see it or care.

    I can only conclude that Mother Jones Magazine has some self-image issues and perhaps their editors feel the online mag is just not legit enough to resist using outrageous click bait. Sad =(

  12. Someone at work last week was trying to tell me that green tea ‘burns weight’ because they drank green tea for 2 weeks and lost two pounds. I reponded by sarcastically commenting how that sounded to be statistically significant. Their response? ‘Well yeah it is actually because I’m a qualified nutritionist’. Headdesk.

    1. You know, I could probably drink nothing but Red Bull for two weeks and achieve similar results. Of course, I’m willing to try, with the proper funding.

      For science, of course.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.