By New Scientist
ER Productions Limited/Getty Images
With researchers, journals, politicians, journalists and social media influencers all capable of espousing misleading or unverified scientific findings, it pays to be able to recognise the telltale signs of a study that might be poor. Here are seven potential warning flags:
Study is published on a blog, preprint server or social media
Such science can still be valid, but exercise caution because it is unlikely to have been reviewed by independent experts or undergone much vetting.
Study has only one author
This can sometimes be a warning sign that a paper or report is an early exploration of an idea or a tentative new hypothesis that shouldn’t necessarily be taken too seriously.
Advertisement
The researchers are from a surprising field of study
All fields of research require specialist training and knowledge, but that doesn’t stop some people applying themselves to subjects they know little about.
The analysis is very fast
When studies are published within days or weeks of an event, or report results from a clinical trial that hasn’t finished yet, these first impressions should be treated with caution until researchers have had more time to validate their work.
The study is very small
Medical studies that look at small numbers of people are less likely to have results that will stand up in further trials. There are no strict rules, but anything under 50 participants is highly tentative, and studies involving at least hundreds of patients or volunteers are preferable.
The trial has no placebo group
It isn’t always possible to design experiments to include a placebo group, but without one, it can be difficult to know for sure if the observed effects are meaningful.
The study reports a correlation or association
Many factors can be linked without one causing the other. In observational studies, factors such as age, wealth or sex may also have an effect on the issue in question. Even when studies say they have accounted for such confounding factors, it is possible that their effects haven’t been completely removed from the analysis.
More on these topics:
