Video: https://www.tiktok.com/@distilledscience/video/7510083269375823134
Transcript
The Maha Report cites studies that don't exist, but I thought my role is to run an agency that is committed to gold standard science. So what happened? I blame AI, but more on that soon. Let's play good science, bad science. The Make America Healthy Again Commission, chaired by RFK Jr., just put out a 70-page report that examined the root causes of deteriorating child health to establish a clear evidence-based foundation for the policy interventions. And it concluded that the primary causes were food, chemical exposure, technology use, and over-medication. Which is of course everything that our FK Jr. was saying when he commissioned the report. And these are problems, but they're broad. In order to fix the problems, we need a targeted approach. Exactly which foods are problematic and at what amounts. Everything is a chemical. Most are harmless. Many that we are exposed to do pose some health risks, but which ones should we devote our time and money towards eliminating or reducing? Technology can enrich or destroy. Medications can save lives or ruin them. government should be intervening in all of these guided by good science. This was not good science. It contained over 500 citations, but an investigation by notice found that seven of these were completely made up, and many others that did exist were used as evidence for things that they didn't talk about. The report listed study authors, titles, and DOI numbers. When Notice reached out to some of the listed authors, they got responses like, were involved with. In other cases, when the report made claims like antidepressant prescription rates in teens increased by 1,400% between 1987 and 2014, even though a systematic overview shows that psychotherapy is just as effective as drugs in the short term and potentially more effective in the long term. It cited this study, which exists, but only compared the effects of different drugs. It did not include psychotherapy as a comparison. To me, this is a clear, obvious example of AI gone wrong. I spend most of my days reading through scientific research, and I try to do that. all of the AI tools as they come out. It used to be that if you asked ChachyPT to compile some sort of scientific report for you, it would cite tons of studies that sound legit with titles and authors and DOI numbers, but they were completely made up. Oftentimes it would list authors who were real people who had published on that same sort of related topic, but not those precise papers. These days, many of the AI features like deep research are getting way better, with most of their citations being pretty accurate. But I never use any statement from any of these where I haven't checked the original paper and read it myself before citing it. It looked like what happened here is that RFK Jr. asked his team for a report that supported his existing views, which normally is just asking for cherry picking. But they were under a lot of pressure and time crunch, and so used AI, which drummed up all sorts of sources to back his statements. But they were bad at science and bad at using AI, which I trained people to do, by the way. Now this doesn't mean that all of the report's conclusions were wrong, just that we can't trust it either way. Final verdict? Science.
Additional notes
Is this the real life, or is it just fantasy? 🤦♂️ What do things like this make you think?
References
- Make America Healthy Again Commission report discussed in transcript; direct URL not listed in workbook.
- Notice investigation of fabricated/misused citations mentioned in transcript; direct URL not listed in workbook.
- Systematic overview and drug-comparison study mentioned in transcript; titles/DOI/PMID not listed in workbook.