Wednesday, July 1, 2009

Pseudoscientific Research -- In Nutrition & Education

The May-June issue of the Skeptical Inquirer prints an important article on the science of nutrition by Dr. Reynold Spector, who is currently a clinical professor at the Robert Wood Johnson Medical School in New Jersy. (He formerly taught at Iowa , Stanford, and Harvard-MIT and was formerly executive VP at Merck in charge of drug development.)

Spector begins by pointing out that “in recent years” progress in research on nutrition has “lagged behind” many other medical fields.

Some of this lag is the result of pseudoscientific beliefs and practices that funders and some researchers, but certainly many science journalists and much of the public erroneously believe is based on scientific methods.

This, my readers from the world of educational policy research will recognize, is all too reminiscent of the mantra in our field: “Studies show….” A phrase that -- in the field of education -- should always cause the waving of a warning flag.

Spector notes that critics of the scientific field of nutrition quite understandably say that “much nutritional research and practice” is, to paraphrase Thomas Hardy, “science’s laughingstock.”

Spector contends that the field of nutrition and the people who rely on its findings are harmed by the absence of “rigorous scientific principles and methods.” He uses data to show that much conventional wisdom about dietary pyramids, food supplements, megavitamins, and weight-loss diets is “unproven, erroneous, or even harmful and is often based on pseudoscience” resting on shoddy research methods.

In nutrition research, there is too much reliance on epidemiology/observation studies for causal conclusions about nutritional facts and practices. Instead, Spector says, such conclusions should be based on sound, rigorous scientific studies (randomized, single variable, hypothesis-driven, with validated instruments and proper statistical analyses) .

[Reading this in Spector’s article immediately reminded me of the fact that when we were both in the U.S. Department of Education, IES Director Russ Whitehurst and I were delighted by Gary Taubes's pro-rigorous-research article on health (“Do We Really Know What Makes Us Healthy?”) in the Sept. 16, 2007 New York Times Sunday Magazine. Taubes had earlier authored the path-breaking 2002 article “ What if It's All Been a Big Fat Lie? ]

Here (from Spector’s article) is a chart showing how the public has been led astray by reliance on false-positive results from epidemiology/observation studies:



Spector points out that the appropriate methodology for finding the truth in nutrition research “has been known for decades.” Nonetheless, Spector says, it is often “either not followed or scientific data are resisted.”

Spector suggests that to understand why this happens, we take a sociological approach. He says, “A useful part of such an analysis is the question: who benefits from a particular event or behavior?” He proposes that the following groups benefit from shoddy nutrition research:

• Journals and their editors (a constant flow of content);
• Academics (“It’s easy to publish almost anything; certain types of studies (e.g., case…studies) require much less effort and resources than controlled trials to yield a publication.”);
• Business interests (who market products and services based on the shoddy research); and
• The news media (controversy and novelty sell newspapers and bring in TV viewers).

Many might say that like beneficiaries are present in the education industry as well. Some might add as potential beneficiaries in the education industry: the education Establishment and the politicians allied with them. Spector also notes that “the unwillingness of investigators who perform pseudoscientific studies” to acknowledge error “cannot be underestimated.”

Spector insists that “unless proper studies are done,” the research literature is “doomed to potential, often-unknown bias and confounding.” He adds that although “it is difficult and expensive” to do “long-term adequate…studies,” they can be done and have been done.

Who, Spector asks, is harmed by shoddy nutritional research? Spector provides a list of those harmed. If we modified his list to apply instead to the education industry, it would read as follows:

• Students and their parents;
• Effective teachers and other efficacy-minded instructional specialists (textbook writers, in-service trainers of teachers, etc.);
• Scientifically-minded education researchers and ed-school professors;
• Resources (which are wasted);
• The news media (“Reporting advice that ultimately requires revision or repudiation makes them look foolish."); and
• Professional standards (undermined by lack of rigor in pursuit of the truth).

Spector concludes by making recommendations for readers of future research in the field of nutrition:

1. "Readers of medical reports and journals should focus on studies that employ methods that test a hypothesis definitively.
2. "Readers should be skeptical of the results of [epidemiology/observation studies] that test a contributory causal hypothesis and draw causal conclusions…. Such studies must be considered at best hypothesis-generating. Moreover, unless such studies have a clear “upfront” hypothesis and prespecified data analysis plan and are not the result of “data-dredging,” they merit even less credence....
3. "Readers should encourage journal editors, academicians, and funding agencies to support quality studies (e.g., randomized controlled studies) rather than those unlikely to answer questions definitively (e.g., [epidemiology/observation studies ], case-control studies, or cohort studies)."

17 comments:

  1. Hey Bill,

    I would like to get the following piece published at Hoover. Is that a possibility?

    see link for piece:

    A Plan to Increase Achievement:
    Explicit Instruction Not National Standards


    Thanks,

    Dan

    ReplyDelete
  2. Hi,
    I've few education websites and blogs. I'm very much interested sharing my website information with your website. If you wish then please get back to me so that we can discuss further. Waiting your reply. Wishing you a nice day.
    Cheers,
    Christine.
    christine09.smith@gmail.com

    ReplyDelete
  3. It is good to see that information, really like to share this, this database really helps, Thanks for the information.
    Term papers

    ReplyDelete
  4. I've learned from experience to be a bit more skeptical whenever someone starts a sentence with "studies show.." Thanks for posting this.

    ReplyDelete
  5. I am hoping the same best work from you in the future as well.

    ReplyDelete
  6. This is very good information on Dr Reynold Spector. This is also a very good part of history about the Robert Wood Johnson medicial school. This I will share the class. Good job.

    ReplyDelete
  7. That article is so interesting and makes a very nice image in my mind. Thank you for sharing this info with us!!!!

    ReplyDelete
  8. Very interesting! Thanks a lot for post.

    ReplyDelete
  9. Hello my friend! I want to say that this article is awesome, nice written and include almost all vital info. I would like to see more posts like this.

    ReplyDelete
  10. great) liked everything very much) keep it up and dont stop)

    ReplyDelete
  11. Great post, I enjoyed ready reading it, Keep posting good stuff like this.

    ReplyDelete
  12. but certainly many science journalists and much of the public erroneously believe is based on scientific methods.

    ReplyDelete
  13. If you're intent to buy online, there are several alternative routes that you can access, all of which involve you hunting around on the internet at various different web sites in order to locate your dream Nike Air Max.Heartily welcome all of you to our Nike Dunks Outlet.It has enjoyed a good reputation for superior service for many years.

    atlanta roof repair

    ReplyDelete
  14. Aim should be there to hit goals in life. Without this no one can survive.This is the reason for selecting college paper writing service in students life to achieve the goal. They may have aim but if they are not good in academic assignment activities they will fail to meet their goal.

    ReplyDelete