The G-cat

Telling the real from the fake

The phrase ‘fake news’ seems to get thrown around ad nauseum these days, but there’s a reason for it (besides the original somewhat famous coining of the phrase). Inadvertently bad, or sometimes downright malicious, reporting of various apparent ‘trends’ or ‘patterns’ are rife throughout nearly all forms of media. Particularly, many entirely subjective or blatantly falsified presentations or reports of ‘fact’ cloud real scientific inquiry and its distillation into the broader community. In fact, a recent study has shown that falsified science spreads through social media at orders of magnitude faster than real science: so why is this? And how do we spot the real from the fake?

It’s imperative that we understand what real science entails to be able to separate it from the pseudoscience. Of course, scientific rigour and method are always of utmost importance, but these can be hard to detect (or can be effectively lied through colourful language choices). When reading a scientific article, whether it’s direct from the source (a journal, such as Nature or Science) or secondarily through a media outlet such as the news or online sources, there’s a few things that you should always look for that will help discern between the two categories.

Peer-review and adequate referencing

Firstly, is the science presented in an objective, logical manner? Does it systematically demonstrate the study system and question, with the relevant reference to peer-reviewed literature? Good science builds upon the wealth of previously done good science to contribute to a broader field of knowledge; in this way, critical observations and alternative ideas can be compared and contrasted to steer the broader field. Even entirely novel science, which go against the common consensus, will reference and build upon prior literature and justify the necessity and design of the study. Having written more than one literature review in my life, I can safely assure you that there is no shortage of relevant scientific studies that need to be read, understood and built upon in any future scientific study.

 

Methods, statistics and sampling

Secondly, is there a solid methodological basis for the science? In almost all cases this will include some kind of statistical measure for the validity (and accuracy) of the results. How does the sample size of the study measure up to what the target group? Remember, a study size of 500 people is definitely too small to infer the medical conditions of all humans, but rarely do we get sample sizes that big in evolutionary genetics studies (especially in non-model species). The sampling regime is extremely important for interpreting the results: particularly, keep in mind if there is an inherent bias in the way the sampling has been done. Are some groups more represented than others? Where do the samples come from? What other factors might be influencing the results, based on the origin of the samples?

Presentation and language of findings

Thirdly, how does the source present the results? Does it make claims that seem beyond a feasible conclusion based on the study itself? Even if the underlying study is scientific, many secondary sources have a tendency to ‘sensationalise’ the results in order to make them both more appealing and more digestible to the general public. This is only exacerbated by the lack of information of the scientific method of the original paper, actual statistics, or the accurate summation of those statistics. Furthermore, a real scientific study will try to (in most cases) avoid evocative words such as ‘prove’, as a fundamental aspect of science is that no study is 100% ‘proven’ (see falsifiability below). Proofs are a relevant mathematical concept though, but these fall under a different category altogether.

Here’s an example: recently, an Australian mainstream media outlet (among many) shared a story about a ‘recent’ (six months old) study that found that second-born children are more likely to be criminals and first-born children have higher IQ. As you might expect, the original study does not imply that being born second will make you a sudden murderer nor will being the first born make you a prodigy. Instead, the authors suggest that there may be a link between differential parental investment/attention (between different age order children) as a potential mechanism. They ruled out, based on a wealth of statistics, the influence of alternative factors such as health or education (both in quality and quantity). Thus, there is a correlative (read: not causative) effect of age on these characteristics. If you directly interpreted the newscast (or read some of the misguided comments), you might think otherwise.

Falsifiability 

Fourthly, are the hypotheses in the study falsifiable? One of the foundations of the modern scientific method includes the requirement of any real scientific hypothesis to be falsifiable; that is, there must be a way to show evidence against that hypothesis. This can be difficult to evaluate, but is why some broad philosophical questions are considered ‘unscientific’. A classic example is the phrase “all swans are white”, which was apparently historically believed in Europe (where there are no black swans). This statement is technically falsifiable, since if one found a non-white swan it would ‘disprove’ the hypothesis. Lo and behold, Europeans arrive in Australia and find that, actually, some swans are black. The original statement was thus falsified.

The role of the peer: including you!

Peer-review is a critical aspect of scientific process, and despite some conspiracy-theory-esque remarks about the secret Big Science Society, it generally works. While independent people inevitably have their own personal biases and are naturally subjective to some degree (no matter how hard we may try to be objective), a larger number of well-informed, critical thinkers help to broaden the focus and perspective surrounding any scientific subject. Remember, nothing is more critical of science than science itself.

While peer-review is technically aimed at other scientists as a way to steer and inform research, the input of outsider, non-specialist readers can still be informative. By closely looking at science, and better understanding both how it is done and what it is showing, can help us evaluate how valuable science is to broader society and shift scientific information into useful, everyday applications. Furthermore, by educating ourselves on what is real science, and what is disruptive drivel, we can aid the development of science and reduce the slowing impact of misinformation and deceit.