RCTs
Science quality assurance, Science fraud incentives
Question: how important are RCT? Upon learning that there is or isn't a RCT supporting an assertion, how should that impact my beliefs?
A Randomized Controlled Trial is taken as a gold standard for science. Unfortunately they can be expensive to do, especially when taking people as subjects, so we have to content ourselves with epidemiological studies in some cases.
In all too many cases, the demand for ‘confirmatory’ statistical evidence is a red herring. Consider the tactics employed for decades by the pro-smoking lobby in successfully blocking anti-smoking legislation. By underscoring the lack of statistical tests of significance based on the results of an RCT, the gold standard of proof for many 20th c, hard-core empiricists such as R. A. Fisher, anti-smoking legislation was derailed until more commonsense standards prevailed. Similar tactics are in use today by the anti-global warming lobby.
The point is that many of the most important discoveries in the history of science have not relied on either RCTs or tests of significance. For instance, astronomy is a foundational scientific endeavor whose discoveries, by definition, are not and cannot be based on tests of significance. It is impossible to conduct an RCT with the cosmos! (At least, to date. It may be the case that some brilliant astrophysicist will yet figure out how to conduct such an experiment).
Next, consider John Snow’s map of mid-19th c London’s cholera epidemics (www1.udel.edu/johnmack/frec682/cholera/). Nowhere does it contain a test of significance. Regardless, he conclusively demonstrated the loci of contagion, resulting in the elimination of cholera as a threat.
Then, too, much of Louis Pasteur’s work in bacteriology and hygiene had nothing to do with significance tests (e.g., Bruno Latour’s book, The Pasteurization of France).
Other examples abound but these few suffice.
Demanding RCT may also be an excuse to not listen. Especially where a RCT is difficult to do, you're facetious if you don't get off your horse and discuss the evidence we do have. Astronomers and historians are able to do work without RCTs, see History is not a science.