I sometimes wonder, what concept of scientific evidence is acceptable - and if some members of this forum are aware of the differences in standards.
Now let’s be clear: people can have different standards for risk taking depending on their individual circumstances. If you have a specific and severe medical condition or are of quite advanced age you might be willing to try sometimes unproven substances. That is understandable and is not part of my elaboration.
But there should be no dissent on what constitutes good scientific practice and evidence. Therefore sometimes I find it difficult to understand the excitement of some (not all) about a broad array of supplements.
In this thread I want to focus on supplements and interventions that promise an increase in life-span – your rapamycin & Co. Please correct me if I’m wrong, but in order of validity this is the main type evidence you can encounter:
Place number 1. Well done placebo-controlled studies in humans.
The gold standard. Not fool-proof – dosing can be sub-clinical, it can be underpowered etc. – but it’s the best there is. After a couple of years of follow-up there should be markedly reduction in mortality numbers.
2. (longitudinal) Epidemiological studies in humans.
Just behind RCTs. Ideally prospective cohort studies of sufficient time-span and magnitude. If controlled for a lot of relevant confounding factors it can at least give a good indication that there might be an effect.
Problems: can easily be under-powered (ideally you have 100.000s participants) , ridden with statistical artifacts (monitoring and censoring effects in the intervention group etc.), lack of information about dosing/adherence, not accounting for important confounders
3. Lifespan studies in rodents or monkeys.
Mice (and monkeys) are not Men. But evolutionary they are fairly close – compared to c.elegans. If you have an intervention, that prolongs life in mice – or even better monkeys – it’s at least indicative that it might work in other mammals.
Problems: bad animal husbandry can distort the effect of interventions - unusually short lived or metabolically deranged controls are a sign for that. The mice model might be a heavily in-bread or genetically modified disease-model (which makes it hard to understand the relevance for “normal” humans). The study might be just under-powered or lack a control group.
These 3 – in descending order – are what I would consider necessary evidence to at least consider an intervention as a possible strategy. You still have to spend a lot of thought on something, if the only data supporting improved mortality is coming from mice (is the dosing really translatable to humans? Etc.)
The following evidence is more problematic. It can serve as support for data uncovered in 1.-3. or help to explain the mechanisms of action. But in and by itself it is in my eyes not a justification to do an intervention. Or vice-versa it cannot counter-proof positive results from 1.-3. :
4. case-control studies in humans
5. case-studies in humans and anecdotal evidence
6. studies in primitive organisms (flies, c.elegans, yeast cells)
7. ex-vivo and tissue studies
8. in-vitro studies
Especially 8. - in-vitro studies – are often over-interpreted.
Generally their only use in my opinion is for analyzing the mechanisms of action for certain interventions to better understand their application. You might find an interesting bio-marker or intra-cellular process that can help to judge the dosing of a substance found in 1.-3. But it is unsettling that frequently authors are giving a call-for-action entirely based on an in-vitro result (despite a dosing which literally is 1000 times the human dosing and no knowledge of metabolic interactions in living animals). They are easy and cheap to do – so you sometimes find dozends of in-vitro study for every in-vivo study on pubmed. But they are really the weakest evidence you can look for to judge an intervention for its results.
I’d like to hear if you follow this reasoning so far about the quality and validity of scientific evidence.
Edited by Guest, 19 October 2020 - 02:20 PM.