How do you know if research evidence is trustworthy?27 October 2020
We’ve all seen ‘scientific’ claims made in adverts and in the news - and sometimes it can be hard to tell which ones can be trusted.
It can feel even more difficult to decide what the facts are when the claims made relate to what may or may not be beneficial for our health.
At Versus Arthritis, we will never stop asking questions and seeking robust evidence-based answers in our research.
We hope this information will help to demystify the types of evidence used in research and help you to decide what is trustworthy.
What are the different types of evidence?
Evidence can be broadly split into five different kinds, depending on the source and reliability.
1. Using 'sciency' words and little ‘concrete’ evidence
This type of evidence often includes scientific terms, such as ‘antioxidant’ or ‘detox’, which can make people believe it is based on real research, but this is often not the case and no real science evidence is given.
You might have seen a few cosmetic advertisements which use this type of language.
2. Sharing one person’s experience
It’s useful to share personal experiences and stories as they can feel more relatable, especially if we have had similar challenges or positive highlights in our own lives.
Personal experiences are often used in news stories, and they’re usually presented as case studies of individual experiences, which may not include any facts or figures.
For this reason, they should not be used in the place of scientific evidence. A personal experience about using CBD oil for pain is an individual’s experience.
It’s important to remember, what works for one person may not work for another.
3. Using just one scientific study
A well-designed and detailed experimental research study can provide strong evidence.
We should still ask questions to find out more about the quality and to reassure ourselves that the claims are based on strong foundations.
- How large was the study?
- Did they compare the new treatment to an existing treatment or a placebo (a treatment that has no known effects)?
- Who produced the research? Has it been published in a reputable science journal? For example, Cloudy with a Chance of Pain was published in a peer-reviewed journal.
4. One scientific study is reported in different ways
Sometimes when a piece of research makes the headlines its results can be misunderstood or interpreted differently by different reporters.
The actual findings can even be overlooked or exaggerated. For example, the health benefits of eating chocolate or drinking red wine.
To check if the headlines reflect the research, and if the research is good quality, we should look for a link or reference to the original research paper, and ask the same questions mentioned above.
5. Assessing similar scientific studies
The strongest kinds of evidence are systematic reviews.
This is a type of literature review that critically looks at all available research on a topic and combines and assesses previous relevant studies to develop a single conclusion. For example, assessing which types of exercise are best for osteoarthritis, using data from 103 different clinical trials.
These methods filter and analyse results from only the highest quality studies in order to scientifically test a claim.
Other evidence to consider
Here’s a few other areas to look out for to help you understand the ‘accuracy’ of evidence.
Who has a vested interest?
It’s important to find out who funded what you’re reading and ask whether this might have had an influence on how the results are presented.
In scientific journals, academics should always disclose their ‘interests’. For example, if they have connections to a private company whose business may be affected by the results in a journal.
Equally, celebrity endorsements and testimonies should be questioned if it appears that they are promoting a particular product. Ask yourself if the results align with their interests.
Has the research been peer reviewed?
This is when the scientific community gives feedback on the quality of new research before it can be published.
It’s important as it’s the academic publishing standard for knowing if scientific results are valid, significant and original. It another check that the researchers’ opinions are supported by the results of the study.
Many of the claims we see online may not have come from research published in a peer reviewed journal, so always ask: has this been peer reviewed? If not, why not?
What is the ‘illusionary truth effect’?
The illusionary truth effect is a psychological phenomenon which is simply, the more times we hear something, the more accurate we think it is.
The human brain finds it easier to process phrases and ideas that are familiar, meaning that we trust claims more readily if they are repeated over time or across several sources.
This is something we’ve all probably experienced, especially when you’ve heard a news story multiple times or a topic is trending on social media.
Find out more about evidence
Learn more about Sense about Science, they’re passionate about making evidence-based science available to the public.
How to get involved in our research
We will shortly be launching new and exciting opportunities to get involved in our research. For more information, or to find out how you can become involved, please email: email@example.com
Get the support you need
You might also be interested in...
How can our research help to improve the diagnosis of osteoporosis fractures?
We wanted to celebrate one of our amazing research projects aiming to improve the lives of people living with osteoporosis.
Meet the researchers: Dr Daniel Whibley is exploring how better sleep and exercise can help to manage persistent pain.
Dr Daniel Whibley shares what inspired his research and why the links between sleep, exercise and chronic pain are important for developing treatments for arthritis.