Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

Science

Showing Original Post only (View all)

SorellaLaBefana

(395 posts)
Sat Apr 26, 2025, 03:59 PM Apr 26

Another, well-done, failure of replication study discussed in Nature--this one focused on basic methods [View all]

Here's what Nature had to say (paywalled)
Huge reproducibility project fails to validate dozens of biomedical studies

...The teams were able to replicate the results of less than half of the tested experiments. That rate is in keeping with that found by other large-scale attempts to reproduce scientific findings. But the latest work is unique in focusing on papers that use specific methods and in examining the research output of a specific country, according to the research teams...

They ended up selecting three of these methods: an assay of cell metabolism, a technique for amplifying genetic material and a type of maze test for rodents. Then the authors randomly selected biomedical papers that relied on those methods and were published from 1998 to 2017 by research teams in which at least half the contributors had a Brazilian affiliation....

The authors judged a paper’s replicability by five criteria, including whether at least half of the replication attempts had statistically significant results in the same direction as the original paper. Only 21% of the experiments were replicable using at least half of the applicable criteria...

Marcelo Mori, a biologist at the State University of Campinas in Brazil, agrees. “Exact replication of an experiment is particularly challenging in the life sciences,” he says. “Living organisms, such as rodents and cell cultures, can respond differently when exposed to environmental variations like temperature, diet, microbiota, or culture medium composition.” Reagents, such as antibodies, can exhibit batch-to-batch variation, he adds...

https://www.nature.com/articles/d41586-025-01266-x?utm_source=Live+Audience&utm_campaign=88bbdfbc5f-nature-briefing-daily-20250425&utm_medium=email&utm_term=0_b27a691814-88bbdfbc5f-49942924

Very much doubt that it reflects anything unique about the country where the study was done—which the Nature article takes pains to explicitly state. Further it mentions "Publish or Perish" as a problem in all countries. Original study (still a preprint) is Amaral, O. B. et al. Preprint at bioRxiv (2025)
https://doi.org/10.1101/2025.04.02.645026

Yet another reminder that, even with an apparently well done study, in clinical medicine it is, in my long held opinion, foolish and potentially unsafe to make practice changes based on only ONE study. Nonetheless, it has been my observation that this is quite often what happens.
3 replies = new reply since forum marked as read
Highlight: NoneDon't highlight anything 5 newestHighlight 5 most recent replies
Latest Discussions»Culture Forums»Science»Another, well-done, failu...»Reply #0