Is Evidence Based Medicine a Movement in Crisis?

April 1, 2016

In my early twenties, I tore a carotid artery out of my neck in a motorcycle accident. The emergency procedure that followed undoubtedly saved my life and from that point forward I had no time to listen to people complaining about the limitations of the nationalised health care that exists in the UK. In my thirties, the same system of evidence-based medicine let me down badly, and in the end, it was a chiropractor who describes himself as “the least evidence-driven person he knows” that turned my life around.

Now I’m slightly confused. I am aware that the evidence-based system has a lot to offer, but I don’t feel I can use it effectively without the assistance of someone like Dr. Tommy Wood. Why?

The system is in crisis.

The system is in crisis in part because of the sheer volume of evidence being generated, much of which is unreproducible or misappropriated by vested interests. Building a meta-analysis atop of this shoddy foundation does little to clarify the situation.

Animal and cell models.

Are used as supporting evidence but frequently what happens outside of the body or in another animal has little or nothing to do with what happens in humans in a real-life setting. The models are becoming cheap enough for scientists to make multiple attempts at finding the “correct” answer, and much of the data are never published.

Lies, damned lies, and statistics.

You cannot prove something to be true using statistical methods.

Science advances one funeral at a time.

Handing out a grant to the young and inexperienced is a risky thing to do, and so inevitably the money follows experience. But imagine if your entire life’s work was based on ideas that were later shown to be wrong. Would you come clean?

In this candid interview with Dr. Tommy Wood.

Tommy talks about the limitations of the evidence-based system, but his intention is not to throw science under the bus. Rather, he wants us all to understand the limitations and proceed with caution. As a scientist that does peer review for a living, Tommy’s goal is to leave science in better shape than he found it.

Since recording this interview, we have adopted the Agile Scrum sprint system of health coaching.

And it’s going great! We’re excited about the new system, and the first health sprints are underway. I’m still very keen to know what you think about the idea, so please complete the one question (three clicks total) survey I mention in the show.

Survey: The scrum and sprint Agile methods are applicable to health coaching.

Here’s the outline of this interview with Dr. Tommy Wood

0:00:15    The main paper from Tommy’s PhD thesis: Treatment temperature and insult severity influence the neuroprotective effects of therapeutic hypothermia.

0:01:28    Newborn babies with brain injury.

0:02:20    The cooling is only 2 or 3 degrees.

0:02:33    What's the best temperature?

0:03:28    Figure 1: Hemispheric area loss after moderate hypoxia-ischaemia.

0:04:01    Richard D. Feinman: “Nobody loses an average amount of weight.”

0:04:49    Tommy may sound anti-science, but this is what he does for a living and he hopes to leave science in a better state than when he left it.

0:06:04    People are selective about the studies they choose to dismantle.

0:06:39    The Cochrane Collaboration.

0:07:04    Cochrane do systematic reviews and meta-analysis.

0:07:37    The meta-analysis is only as good as the studies on which it's based.

0:08:13    Publication bias is a problem.

0:08:58    Anyone can learn how to do a Cochrane review.

0:09:32    Questionable discussion: Effects of low-carbohydrate diets v. low-fat diets on body weight and cardiovascular risk factors: a meta-analysis of randomised controlled trials.

0:09:55    Tommy’s letter: The cardiovascular risk reduction benefits of a low-carbohydrate diet outweigh the potential increase in LDL-cholesterol.

0:10:47    Self-appointed expert that cannot be wrong.

0:12:11    The are a number of different funding sources.

0:13:20    It's less risky to give money to the more experienced.

0:13:35    Max Planck: “Science advances one funeral at a time.”

0:14:29    We don't know what people are doing but not publishing.

0:15:04    Cell and animal models enable you to make multiple attempts without publishing.

0:15:31    Statistics never prove anything to be true.

0:15:51    The emphasis is on new things that sound sexy.

0:16:30    One editor questioned Tommy’s animal model when a human trial was underway.

0:17:30    Papers are retracted from big journals.

0:18:27    In finance this is called backfitting.

0:19:05    How can we expect the animal model to translate to humans?

0:19:34    Most animal models don't reflect what happens in humans at all.

0:20:03    1,026 experimental treatments in acute stroke.

0:20:58    Tommy's study was a model of something that we know works in humans and other animals.

0:22:17    Because he started with the basics, Tommy is confident this work will extrapolate to humans.

0:22:59    Should consumers of science be interested in cell and animal models?

0:23:06    Tommy finds it very difficult to get excited about a mouse study.

0:23:13    There are some genetic mutations that cause Alzheimer's very young.

0:23:29    These do not translate well into humans.

0:23:48    Genomic responses in mouse models poorly mimic human inflammatory diseases.

0:24:54    The immune system is particularly complex and difficult.

0:25:41    Don't take supplements based on studies done in rats!

0:26:01    Spirulina for detox.

0:26:48    This is not even remotely close to what's going on inside a living human.

0:27:30    It's easier than ever before to get access to the science.

0:29:06    As a computer scientist, I've never done science.

0:29:43    Spend time on the basics.

0:30:28    Abraham Lincoln? “Give me six hours to chop down a tree, and I will spend four hours sharpening the axe."

0:31:17    Dr. Dan Kalish describes himself as not being evidence-driven but he still gets fantastic results.

0:31:47    Do we need to be 100% evidence-based?

0:32:04    Why Most Published Research Findings Are False.

0:32:33    Evidence based medicine: a movement in crisis?

0:32:48    Evidence-based as a starting point, not a wall of protection.

0:33:54    Anecdote is the lowest form of evidence.

0:35:52    There's no protocol for debugging a computer program but there is a system.

0:37:06    Our plans are evidence-based, but some of the newer supplements lack data.

0:37:57    My swipe file is getting out of control.

0:38:59    Gary Ralston Consulting.

0:39:07    Agile project management.

0:39:28    Could the same system work for health coaching?

0:39:53    The traditional Waterfall model.

0:41:23    Please let me know if this is a good idea.

0:41:57    I've been using Zoom.

0:42:18    Our kanban board has four columns, Todo, Doing, Review, and Done.

0:47:11    Podcast on organic acids.

0:47:49    The year one cost of our programme costs around $8,000. For the performance and longevity orientated, the cost per year thereafter is about $5,000.

0:51:18    The stool tests are not perfect, sometimes the Doctor’s Data test comes up trumps, sometimes BioHealth is best.

0:51:29    The GI-MAP™ stool test.

0:51:44    Our programme is about the cost of a mountain bike.

0:52:09    But the testing will make you faster!

0:52:49    If you're not completely happy in 8 months I will refund all of the money you spent on my time.

0:55:16    Initial round of testing and then we retest.

0:55:38    This adds accountability.

0:56:24    Agile also adds accountability.

0:57:58    My last round of testing found a blasto infection, yeast and clostridial overgrowth, b6 and carnitine deficiency.

0:58:36    How long does insulin resistance takes to happen?

0:58:55    Insulin resistance can start in your 20s and 30s.

0:59:38    Total maintenance costs.

1:00:10    $1,500 a year on supplements.

1:01:02    The people spending the most money are also the most engaged.

1:01:36    Trying too many things at once can be a problem.

1:02:19    We concentrate on teaching.

1:03:27    Podcast on the FDN training course.

1:03:57    Chet Morjaria: Coach and Director of Editorial at Breaking Muscle.

1:04:03    The best coach should be trying to work herself out of a job.

1:05:20    Getting people to pay for all of my time can be challenging.

1:05:44    The tests are more expensive on directlabs.com.

1:06:21    You can't make a living selling supplements.

1:06:23    More recently I've been charging $250 an hour for my time, but I don't bill for as much time as I spend.

1:07:00    We do lots of work on the backend but I only bill for 30 minutes of my time.

1:08:04    Dripping tests onto people.

1:08:29    If you'd like to do just one test, please order it soon!

1:10:23    We haven't made any money but we are having fun.