In high school, statistics may well have been one of the most tedious and hated forms of mathematics. However, statistics play an important role in the science, particularly in the medical sciences. Interestingly though, statistics were involved almost every day for us all during the covid pandemic. Some were misleading, which is why this edition of “what is science?” will be focusing on statistics and how it is used in science. Hopefully, this article may assist readers in critically thinking about scientific articles using statistics.
Lets start with the basics. Statistics is concerned with the collection, analysis and interpretation of data. In the natural sciences, this may be something like interpreting the effects of a drug on the heart rate of individuals. In essence, statistics deals with samples taken from a population, because it is virtually impossible to measure the heart rate of 7 billion people. As such, a sample is taken and the results are used to make assumptions about the population. But how is this done? This is where the maths comes in
Firstly, we’ll deal with the mean, standard deviation, and more. The mean is a fancy way of saying average. So if I measure the heart rates of 3 individuals to be 60, 65 and 70, the mean is 65. The standard deviation describes the spread of the data from the mean. So the standard deviation of those 3 values I mentioned would be calculated as 5. However, if I had different values with the same mean, say 60, 60 and 75, that standard deviation would be 7.1. What does that mean? Well, the 2nd set of values are more spread out from the mean than the first value. Why is this important? Well imagine you’re choosing between two jobs with the same average income, the standard deviation would tell you whether the job’s income varies between $1000 or $10,000.
To kick it up a notch, we’ll move onto the normal, or Gaussian, distribution. Many types of data, such as the height of people or blood pressure, will form a normal distribution in which a bell shape arises where the mean is located in the middle (figure 1). Additionally, 68% of the data will be located within 1 standard deviation from the mean, 95% will be located within 2 standard deviations and 99.7% of the data will be located within 3 standard deviations from the mean. If I were to take a random sample of a population and the dataset follows a normal distribution, it is likely the population will as well. Due to these rules, this allows us to use statistical tests that assume a normal distribution to determine population-level attributes based of a random sample.
Figure 1. Example of a normal distribution. Shown is the mean (µ) in the center and the standard deviations (σ) away from the mean. 68% of the data is found 1 standard deviation away from the mean, 95% is found 2 standard deviations away and 99.7% is found 3 standard deviations away from the mean.
When I take a random sample and work out the mean, how can I be certain that the true population mean is the same or is at least between a certain range. A confidence interval can be used for this and it will develop a range in which, a certain percentage of the time, will estimate where the population mean may fall in when multiple samples are taken. Say for example, I want to calculate the confidence intervals of mean blood pressure of 65 and, using a 95% confidence interval, I get the values 77 and 53. I can therefore be certain that 95% of the means of random samples from the population will fall between this range and there’s only a 5% chance the true population mean doesn’t fall within this range. Confidence intervals are sensitive to variability in data and as such, if we were comparing two groups of blood pressure, it allows us to determine the spread, or standard deviation, of each group. One important thing when interpreting confidence intervals is that the smaller the sample size, the larger the interval. As such, if a study claimed that 90% of people developed cancer due to red meat consumption, but the sample size is small, this can mean that the true amount of people who develop cancer by consuming red meat may be much lower than 90%. In contrast, a large sample size will often contain a much lower confidence interval and you can be more certain the population mean is close to the sample mean.
Another statistical tool used is the p-value. If we cast our minds back to the scientific method, when developing our hypothesis of a phenomenon, such as whether caffeine causes an increased heart rate, a null hypothesis will state that there is no relationship between 2 variables. The p-value is the likelihood that the results of an experiment can occur if we were to assume the null hypothesis to be true. As such, p-values are generally very small, with the most common p-value being below 0.05. If we were to find that caffeine increases heart rate and a p-value is below 0.05, that means only 5% of the time you would detect these results if you assumed the null hypothesis to be true, that is that caffeine does NOT increase heart rate. However, for those who aren’t happy with a 5% chance this can be observed assuming the null hypothesis was correct, they can use 0.01 for a 1% chance or even 0.001 for a 0.1% chance. The value decided upon is known as the alpha level and the aim is to generate a p-value below the alpha. Several statistical tests utilize the p-value, one of them being the t-test, which is used to compare the means and standard deviations of 2 groups, typically the control and the treated group. A p-value below the alpha level states that there is a significant difference between the control and treated groups. However the t-test will only compare 2 groups and if you wished to compare more than that then a one-way ANOVA may be used, which is essentially the same as the t-test except it can compare multiple groups.
Much like with confidence intervals, tests using a p-value can be misused. One way is if the sample data does not follow a normal distribution. This is an issue because statistical tests like the t-test and one-way ANOVA assume a normal distribution. Additionally, these problems arise when the sample size is too small as it is unlikely that such a sample size will follow a normal distribution. Outliers in a dataset may also likely affect the outcomes of these tests as they may significant alter the mean or standard deviation, which are utilized in these tests. Lastly, these tests can be misused if the datasets are not truly independent. For example, if I were to measure the resting heart rate of an individual and then measure their heart rate after giving them caffeine, those datasets are not independent. To gain independent data I need to measure the heart rate of one individual and then measure the heart rate of another individual given caffeine. However, just because a dataset doesn’t follow normal distribution, or are not independent, it doesn’t mean that there aren’t any other tests that can be used. However, using the wrong test for the wrong type of dataset may mean that the authors are trying to mislead the readers into believing that they have recorded significant results when, in reality, the results are not significant.
Statistics are a useful way of making assumptions about a population from a random sample. However, as we have seen it is definitely possible to use statistics to mislead readers about the significance of an experiment. Hopefully, this article will give you all the basic tools to critically think about a scientific journal article and decide for yourselves what the results truly mean.
