junk science

Junk Science: Vitamin Mania

vitamins

538, the new platform for stats-oriented analysis from Nate Silver at ESPN, has this good story by Emily Oster about the junky science of vitamins: “Don’t Take Your Vitamins.”

Many medical studies show positive health effects from higher vitamin levels. The only problem? These studies often can’t tease out the effect of the vitamins from the effect of other factors, such as generally healthy living. Studies that attempt to do this typically show no impact from vitamin use — or only a very tiny one on a small subset of people. The truth is that for most people, vitamin supplementation is simply a waste of time.

To get a little more concrete — and to understand how we got to that endless row of vitamins at CVS — it’s useful to look at a couple of examples: vitamin D and vitamin E. These are among the most popular vitamin supplements: In the 2009-2010 NHANES, 34 percent of adults reported taking vitamin D supplements and 30 percent reported taking vitamin E.

One can find plenty of support for this supplementation behavior in the medical literature. A recent review identified 290 observational studies on vitamin D. For the most part, these studies measure the amount of 25-hydroxy vitamin D — the marker of vitamin D concentration — in participants’ blood and analyze the relationship between that concentration and various measures of health.

Using this approach, researchers have found that higher concentrations of vitamin D are linked to less cardiovascular disease, lower overall mortality, less weight gain, less diabetes, less likelihood of getting infectious diseases, less multiple sclerosis, fewer mood disorders, better cognitive function — basically, every outcome under the sun. Based on these studies, vitamin D is pretty much the philosopher’s stone.

A bit less magical, vitamin E has also been credited (again, in observational studies) with everything from better pregnancy outcomes to lower mortality. In the most striking result, a large study published in the early 1990s found a 40 percent reduction in mortality risk from taking vitamin E supplements for two years. This effect is enormous.

But as striking as these results on both vitamin D and vitamin E are, they fall short of the standard for causality. These studies were not randomized controlled trials, which means other factors could have influenced their outcomes. The authors did try to adjust for some variables — age and whether the subjects smoke, for example — but these may not be sufficient. Yet people believe the results: 25 percent of adults reported taking vitamin E in 1989, and the share rose to almost 40 percent by 2003.

As is often the case, striking observational results like these were followed by large randomized controlled trials — many of them. A study run through the National Institutes of Health called the Women’s Health Initiative analyzed the impact of vitamin D and calcium supplementation in 36,000 post-menopausal women. Another large trial out of Harvard — the Physician’s Health Study — looked into vitamin E supplementation among 14,000 male physicians.

In these trials, participants were randomly assigned to take supplements. Because the assignment was random — and the trials were big — the demographic and health characteristics of the supplement group and the non-supplement group were similar before the study started. When researchers looked at participants’ health over the long term, they could therefore be confident that any differences they saw across groups were due to the supplements, and not some other factor.

When the results of these studies came out, they largely refuted the idea that these supplements offered benefits. Vitamin E appears to have no impact on cancer or heart disease. Results from the Women’s Health Study, released in 2005, showed no relationship between vitamin E supplementation and overall mortality. Later results from the men in the Physicians’ Health Study showed the same: no relationship.

For vitamin D, the randomized trials (nicely summarized here) refuted virtually all of the purported benefits to diabetes, weight loss and cancer. For elderly women, there is some evidence of a small reduction in mortality with supplementation, but well below what was seen in observational data and only marginally statistically significant.

Randomized controlled trials are not actually required to draw some conclusions in some cases; the problem is that it is easy and cheap to study correlations, as in those studies that show correlations between blood levels of vitamins and some health benefit. Taking vitamins is part of a constellation of habits of organized, health-conscious people, so naturally people who take vitamins tend to have many other healthy habits and so their vitamin levels often correlate with good outcomes. Researchers do the easy studies first, then get funding for the much more expensive studies to look for causation; in this case, very little causation is turned up. So don’t feel bad about taking vitamins — you can make a case for the multivitamin as insurance against deficiencies you may not be aware of. There is little downside to moderate doses of vitamins. But a good diet with diverse foods generally provides all of the vitamins most people need.

Vitamin D levels in blood correlate with low rates of dementia, for example, but that may well be because people who eat oily coldwater fish regularly are being protected by the fish oils and not the vitamin D they contain.

The “junk science” here is not the correlation studies, but the conclusion that they prove anything that should be acted on.

More “50 Shades of Grey” Pseudoscience Reporting

“Fifty Shades of Grey” cover

Perhaps it’s not really “pseudoscience” — which connotes the promotion of definitely false beliefs — but “junk science,” where research studies are done by academics who cloak themselves in the authority of Science but actually commit logical fallacies in promoting their work to appear to confirm the beliefs they already hold about the world.

My earlier post, “Reading “50 Shades of Grey” Gives You Anorexia and an Abusive Partner!”, reporting on the feminist junk science study trying to associate 50 Shades of Grey with anorexia and abusive relationships has company in two much-more-detailed critiques in Psychology Today blogs. Both are worth reading in full, but I’ll quote some points. First, we have this piece by Robert James King, Ph.D.:

Let’s start with the so-called eating disorders. What was actually measured? Two things.

Q1 “Have you ever fasted for a day (or more)?”

and

Q2 “Have you ever used diet aids?”

That’s the lot. No. Calling the use of diet aids an eating disorder is just scare-mongering. These people didn’t have eating disorders—at least not that we know of. I could have got the same results by dividing the group into “gym members” and “non-gym members”

The binge-drinking measure (“Having 5 or more drinks on 6 or more days in the last month”) was technically correct. For doctors, five or more drinks is a binge. For most of us—it’s a quiet night in—but let’s pass over that one to the juicy stuff, “risky sexual practices”.

The criteria the authors use for prevalence of risky sexual practices were two:

Q1: “Have you had five or more sexual partners?”

Q2: “Have you ever had anal sex?”

Really? Boy, you young people! These are the criteria for risky sexuality? ….

Running a bunch of correlations without having any controls is a thing that we scientists call (and stop me if I’m getting too technical here) “Not doing science”.

Let’s say I have a hunch that there is a nefarious plot that links people who have been killed by falling out of bed and the number of lawyers in Puerto Rico. It correlates .96 over twelve years—I bet you didn’t know that. Well, how could you? It’s a plot by Puerto-Rican Lawyers…

Well, I can search around for some correlations—but all this really tells the world is that I am obsessed with Puerto Rican lawyers (for some reason). (3) That’s why scientist control for certain factors. Here are some possible ones that don’t seem to have occurred to the authors:

Did their sample read other books? Other erotic books? Do they even have sexual partners? Have they ever had sex at all?

Here’s the thing—if you build in your assumptions at the start (that kink is bad) then maybe you can find a correlation there. Without controls all the authors have done is import their moralising and attached some numbers to it.

The next piece is by Jen Kim (who has read the books!):

“The study did not distinguish whether women experienced the health behaviors before or after reading the books.”

Lead researcher Amy Bonomi says it’s a potential problem either way, but I have to disagree.

The distinction is quite important, because one interpretation suggests that girls who read the book already have a proclivity for certain behaviors, while the other suggests that the book creates such behaviors.

As a reader of two (!) out of the three books by E.L. James, I have a difficult time seeing the latter case.

Ana Steele, the dimwitted ingénue of the story, chooses to participate in a consensual S&M relationship with handsome stalker, Christian Grey. As far as I can recall, she enjoys (or is at least, open) to this arrangement. Furthermore, Christian has more of a predilection for physical spanking than verbal abuse, right?
For some reason, Ana forgets to eat. It is certainly never implied that she is purposely trying to lose weight or suffers from an eating disorder. The girl just doesn’t like to eat.

I haven’t read the third book, but I believe Ana gets drunk once in the first book (to the point of getting sick) and then gets tipsy a few more times. How does this behavior differ from any other 21 year old’s?
Ana’s only sexual partner was and is Christian, which hardly makes her as promiscuous as this study claims its readers to be….

A lot of this criticism is reminiscent of the blowback against violent video games and films like The Matrix in the wake of the Columbine shooting. Keanu’s kung-fu skills were to blame, not the shooters’ mental health or upbringing.

But, a recent study from the University of Oxford, The University of Rochester and the company Immersyve, has found that playing so-called violent video games (e.g., Call of Duty or Grand Theft Auto) do not give rise to real-world aggression.

Previous post on topic: Reading “50 Shades of Grey” Gives You Anorexia and an Abusive Partner!

For more on pop culture:

“Game of Thrones” and the Problem of Power The Lessons of Walter White
“Blue Valentine”
“Mad Men”
The Morality of Glamour
“Mockingjay” Propaganda Posters
“Big Bang Theory” — Aspergers and Emotional/Social Intelligence
Real-Life “Hunger Games”: Soft Oppression Destroys the Poor
Reading “50 Shades of Grey” Gives You Anorexia and an Abusive Partner!
YA Dystopias vs Heinlein et al: Social Justice Warriors Strike Again
“Raising Arizona” — Dream of a Family

“Parallel Science Propaganda Machine”

Science! – Breaking Bad

Noticing the ways in which “Science!” is being used as an authority signal by all sorts of propagandists now — including Vani Hari “Food Babe” and Dr. Oz — it alerts us to a much broader problem: lack of rigorous scientific method has spread from social sciences where researchers typically misunderstand statistics and fail to use proper controls, then have their press release end up interpreted as “proving” some more broadly-stated claim by popularizing media reports. This tendency has infected all the other sciences.

Aside from simple distortion, there are a variety of other methods to mislead by borrowing the authority science brings to any debate position. Activist organizations (and business lobbying groups as well!) now find a way to get studies done that support their politics, then proclaim the studies as proving their point of view correct. Reason Hit and Run’s Ronald Bailey brought my attention to a good story on this phenomenon by Marcel Kuntz of the Genetic Literacy Project — the large number of marginal journals and “institutes” that will cooperate in promoting junk science:

Political ecologists–commentators in the media and among NGO advocacy groups–like science…when it confirms their views. When it contradicts them, rather than changing their minds, they often prefer to change the science to fit their ideology. They have thus created a “parallel science.” Which should not be confused with pseudo-sciences (e.g. astrology, false medicine, the paranormal, ufology, etc.).

Pseudo-sciences may harm naive believers, parallel “science” is harming democracy. It is a component of a predetermined political project to the exclusive benefit of the ideological views of a minority. “Parallel science” seemingly resembles science, but it differs from science since its conclusions precede experimentation.

Parallel “science” has been created to replace scientists, especially in risk assessment, by “experts” (often self-proclaimed) supportive of a political project. This parallel “science” is hidden behind positive-sounding terms, such as “citizen science” or “independent” or “whistleblower”, while mainstream scientists are accused of having “conflicts of interest” or having ties with “industry”. In order to further propagate distrust in current risk assessment, parallel “science” will invoke unrelated past health problems or environmental damages, but never to the way science has solved problems. …

Why is parallel “science” not discredited and why is it represented so uncritically by the media? The answer partly lies in the current dominance of a relativist ideology. The danger of such a postmodern approach to science is that it considers all points of views to be equally valid and thus raises the value of “independent” (in fact ideological) views to the same level as scientific ones.

Ronald Bailey adds an incident illustrating the problem of politically-biased sources cited as “Science!”:

Let me give an example of how “parallel science” manufactures propaganda for activist groups with which they can mislead the credulous. In March, 2014, Doug Gurian-Sherman, senior scientist in Food and Environment Program at the Union of Concerned Scientists, stated in no less a place than MIT’s Technology Review:

It’s also worth noting that there’s no real consensus on GMO crop safety.

As evidence that there’s “no real consensus,” to what website did Gurian-Sherman link? A declaration issued by the European Network of Scientists for Social and Environmental Responsibility. ENSSER is a collection of long-time foes of agricultural biotechnology. The disingenuous statement has so far been signed by fewer than 300 scientists, including such anti-biotech luminaries as Charles Benbrook, Vandana Shiva, Gilles-Eric Seralini, and Gurian-Sherman himself. Referring to this declaration as evidence against biotech crop safety is akin to citing a statement from tobacco company scientists asserting that cigarette smoking isn’t a risk factor for lung cancer.

Vani Hari’s mistake — what makes her so easy to discredit — is that she incorporated as a profit-making LLC, when sophisticated propagandists know to start a nonprofit NGO and simply pay themselves a hefty salary out of nonprofit funds. The effect is the same — corrupt dollars from sponsors and special interests go in, propaganda and a comfortable lifestyle for the propagandists go out. The difference is it’s harder to attack the nonprofit.

Death by HR: How Affirmative Action Cripples Organizations

[From Death by HR: How Affirmative Action Cripples Organizations, available now in Kindle and trade paperback.]

The first review is in: by Elmer T. Jones, author of The Employment Game. Here’s the condensed version; view the entire review here.

Corporate HR Scrambles to Halt Publication of “Death by HR”

Nobody gets a job through HR. The purpose of HR is to protect their parent organization against lawsuits for running afoul of the government’s diversity extortion bureaus. HR kills companies by blanketing industry with onerous gender and race labor compliance rules and forcing companies to hire useless HR staff to process the associated paperwork… a tour de force… carefully explains to CEOs how HR poisons their companies and what steps they may take to marginalize this threat… It is time to turn the tide against this madness, and Death by HR is an important research tool… All CEOs should read this book. If you are a mere worker drone but care about your company, you should forward an anonymous copy to him.

Jeb Kinnison

Writing About Life and Love

junk science

Junk Science: Vitamin Mania

Like this:

“Parallel Science Propaganda Machine”

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: