Scenario 1: Independent samples with 2 known variances

For the first scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances \(\sigma^2 = 1\) in both populations and that we would like to test whether the two population means are different.

So we have:

• 5 observations in each sample: \(n_1 = n_2 = 5\)
• mean of sample 1: \(\bar{x}_1 = 0.02\)
• mean of sample 2: \(\bar{x}_2 = 0.06\)
• variances of both populations: \(\sigma^2_1 = \sigma^2_2 = 1\)

Following the 4 steps of hypothesis testing we have:

1. \(H_0: \mu_1 = \mu_2\) and \(H_1: \mu_1 - \mu_2 \ne 0\). (\(\ne\) because we want to test whether the two means are different, we do not impose a direction in the test.)
2. Test statistic: \[z_{obs} = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}}\] \[= \frac{0.02-0.06-0}{0.632} = -0.063\]
3. Critical value: \(\pm z_{\alpha / 2} = \pm z_{0.025} = \pm 1.96\) (see a guide on how to read statistical tables if you struggle to find the critical value)
4. Conclusion: The rejection regions are thus from \(-\infty\) to -1.96 and from 1.96 to \(+\infty\). The test statistic is outside the rejection regions so we do not reject the null hypothesis \(H_0\). In terms of the initial question: At the 5% significance level, we do not reject the hypothesis that the two population means are the same, or there is no sufficient evidence in the data to conclude that the two populations considered are different.

Scenario 2: Independent samples with 2 equal but unknown variances

For the second scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances in both populations are unknown but equal (\(\sigma^2_1 = \sigma^2_1\)) and that we would like to test whether the mean of population 1 is larger than the mean of population 2.

So we have:

• 6 observations in sample 1: \(n_1 = 6\)
• 5 observations in sample 2: \(n_2 = 5\)
• mean of sample 1: \(\bar{x}_1 = 1.247\)
• mean of sample 2: \(\bar{x}_2 = 0.1\)
• variance of sample 1: \(s^2_1 = 0.303\)
• variance of sample 2: \(s^2_1 = 0.315\)

Following the 4 steps of hypothesis testing we have:

1. \(H_0: \mu_1 = \mu_2\) and \(H_1: \mu_1 - \mu_2 > 0\). (> because we want to test if the mean of the first population is larger than the mean of the second population.)
2. Test statistic: \[t_{obs} = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{s_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}\] where \[s_p = \sqrt{\frac{(n_1-1)s^2_1+ (n_2 - 1)s^2_2}{n_1 + n_2 - 2}} = 0.555\] so \[t_{obs} = \frac{1.247-0.1-0}{0.555 * 0.606} = 3.411\] (Note that as it is assumed the variances of the two populations are equal, a pooled (common) variance, denoted \(s_p\), is computed.)
3. Critical value: \(t_{\alpha, n_1 + n_2 - 2} = t_{0.05, 9} = 1.833\)
4. Conclusion: The rejection region is thus from 1.833 to \(+\infty\) (there is only one rejection region because it is a one-sided test). The test statistic lies within the rejection region so we reject the null hypothesis \(H_0\). In terms of the initial question: At the 5% significance level, we conclude that the mean of population 1 is larger than the mean of population 2.

Scenario 3: Independent samples with 2 unequal and unknown variances

For the third scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances in both populations are unknown and unequal (\(\sigma^2_1 \ne \sigma^2_1\)) and that we would like to test whether the mean of population 1 is smaller than the mean of population 2.

So we have:

• 5 observations in sample 1: \(n_1 = 5\)
• 6 observations in sample 2: \(n_2 = 6\)
• mean of sample 1: \(\bar{x}_1 = 0.42\)
• mean of sample 2: \(\bar{x}_2 = 1.247\)
• variance of sample 1: \(s^2_1 = 0.107\)
• variance of sample 2: \(s^2_1 = 0.303\)

Following the 4 steps of hypothesis testing we have:

1. \(H_0: \mu_1 = \mu_2\) and \(H_1: \mu_1 - \mu_2 < 0\). (< because we want to test if the mean of the first population is smaller than the mean of the second population.)
2. Test statistic: \[t_{obs} = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{\frac{s^2_1}{n_1} + \frac{s^2_2}{n_2}}}\] \[= \frac{0.42-1.247-0}{0.268} = -3.084\]
3. Critical value: \(-t_{\alpha, \upsilon}\) where \[\upsilon = \frac{\bigg(\frac{s^2_1}{n_1} + \frac{s^2_2}{n_2} \bigg)^2}{\frac{\bigg(\frac{s^2_1}{n_1}\bigg)^2}{n_1 - 1} + \frac{\bigg(\frac{s^2_2}{n_2}\bigg)^2}{n_2 - 1}} = 8.28\] so \[-t_{0.05, 8.28} = -1.851\] Note: The degrees of freedom 8.28 does not exist in the standard Student distribution table, so simply take 8, or compute it in R with qt(p = 0.05, df = 8.28). For simplicity, this number of degrees of freedom is sometimes approximated as \(df = min(n_1 - 1, n_2 - 1)\), so in this case it would be \(df = 4\).
4. Conclusion: The rejection region is thus from \(-\infty\) to -1.851. The test statistic lies within the rejection region so we reject the null hypothesis \(H_0\). In terms of the initial question: At the 5% significance level, we conclude that the mean of population 1 is smaller than the mean of population 2.

Scenario 4: Paired samples where the variance of the differences is known

Student’s t-test with paired samples are a bit different than with independent samples, they are actually more similar to one sample Student’s t-test. Here is how it works. We actually compute the difference between the two samples for each pair of observations, and then we work on these differences as if we were doing a one sample Student’s t-test by computing the test statistic on these differences.

In case it is not clear, here is the fourth scenario as an illustration. Suppose the data below. Moreover, suppose that the two samples are dependent (matched), that the variance of the differences in the population is known and equal to 1 (\(\sigma^2_D = 1\)) and that we would like to test whether the mean difference between the two populations is different than 0.

The first thing to do is to compute the differences for all pairs of observations:

So we have:

• number of pairs: \(n = 5\)
• mean of the difference: \(\bar{D} = 0.04\)
• variance of the difference in the population: \(\sigma^2_D = 1\)
• standard deviation of the difference in the population: \(\sigma_D = 1\)

Following the 4 steps of hypothesis testing we have:

1. \(H_0: \mu_D = 0\) and \(H_1: \mu_D \ne 0\)
2. Test statistic: \[z_{obs} = \frac{\bar{D} - \mu_0}{\frac{\sigma_D}{\sqrt{n}}} = \frac{0.04-0}{0.447} = 0.089\] (This formula is exactly the same than for one sample Student’s t-test with a known variance, except that we work on the mean of the differences.)
3. Critical value: \(\pm z_{\alpha/2} = \pm z_{0.025} = \pm 1.96\)
4. Conclusion: The rejection regions are thus from \(-\infty\) to -1.96 and from 1.96 to \(+\infty\). The test statistic is outside the rejection regions so we do not reject the null hypothesis \(H_0\). In terms of the initial question: At the 5% significance level, we do not reject the hypothesis that the mean difference between the two populations is equal to 0.

Scenario 5: Paired samples where the variance of the differences is unknown

For the fifth and final scenario, suppose the data below. Moreover, suppose that the two samples are dependent (matched), that the variance of the differences in the population is unknown and that we would like to test whether a treatment is effective in increasing running capabilities (the higher the value, the better in terms of running capabilities).

The first thing to do is to compute the differences for all pairs of observations:

So we have:

• number of pairs: \(n = 5\)
• mean of the difference: \(\bar{D} = 8\)
• variance of the difference in the sample: \(s^2_D = 16\)
• standard deviation of the difference in the sample: \(s_D = 4\)

Following the 4 steps of hypothesis testing we have:

1. \(H_0: \mu_D = 0\) and \(H_1: \mu_D > 0\) (> because we would like to test whether the treatment is effective, so whether the treatment has a positive impact on the running capabilities.)
2. Test statistic: \[t_{obs} = \frac{\bar{D} - \mu_0}{\frac{s_D}{\sqrt{n}}} = \frac{8-0}{1.789} = 4.472\] (This formula is exactly the same than for one sample Student’s t-test with an unknown variance, except that we work on the mean of the differences.)
3. Critical value: \(t_{\alpha, n-1} = t_{0.05, 4} = 2.132\) (n is the number of pairs, not the number of observations!)
4. Conclusion: The rejection regions are thus from 2.132 to \(+\infty\). The test statistic lies within the rejection region so we reject the null hypothesis \(H_0\). In terms of the initial question: At the 5% significance level, we conclude that the treatment has a positive impact on the running capabilities (because the mean of the differences is greater than 0)

This concludes how to perform the different versions of the Student’s t-test for two samples by hand. In the next sections, we detail how to perform the exact same tests in R.

Scenario 1: Independent samples with 2 known variances

For the first scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances \(\sigma^2 = 1\) in both populations and that we would like to test whether the two population means are different.

dat1 <- data.frame(
  sample1 = c(0.9, -0.8, 0.1, -0.3, 0.2),
  sample2 = c(0.8, -0.9, -0.1, 0.4, 0.1)
)
dat1

##   sample1 sample2
## 1     0.9     0.8
## 2    -0.8    -0.9
## 3     0.1    -0.1
## 4    -0.3     0.4
## 5     0.2     0.1

dat_ggplot <- data.frame(
  value = c(0.9, -0.8, 0.1, -0.3, 0.2, 0.8, -0.9, -0.1, 0.4, 0.1),
  sample = c(rep("1", 5), rep("2", 5))
)

library(ggplot2)

ggplot(dat_ggplot) +
  aes(x = sample, y = value) +
  geom_boxplot() +
  theme_minimal()

Note that you can use the {esquisse} RStudio addin if you want to draw a boxplot with the package {ggplot2} without writing the code yourself. If you prefer the default graphics, use the boxplot() function:

boxplot(value ~ sample,
  data = dat_ggplot
)

The two boxes seem to overlap which illustrate that the two samples are quite similar, so we tend to believe that we will not be able to reject the null hypothesis that the two population means are similar. However, only a formal statistical test will confirm this belief.

Below a function to perform a t-test with known variances, with arguments accepting:

• the two samples (x and y),
• the two variances of the populations (V1 and V2),
• the difference in means under the null hypothesis (m0, default is 0),
• the significance level (alpha, default is 0.05)
• and the alternative (alternative, one of "two.sided" (default), "less" or "greater"):

t.test_knownvar <- function(x, y, V1, V2, m0 = 0, alpha = 0.05, alternative = "two.sided") {
  M1 <- mean(x)
  M2 <- mean(y)
  n1 <- length(x)
  n2 <- length(y)
  sigma1 <- sqrt(V1)
  sigma2 <- sqrt(V2)
  S <- sqrt((V1 / n1) + (V2 / n2))
  statistic <- (M1 - M2 - m0) / S
  p <- if (alternative == "two.sided") {
    2 * pnorm(abs(statistic), lower.tail = FALSE)
  } else if (alternative == "less") {
    pnorm(statistic, lower.tail = TRUE)
  } else {
    pnorm(statistic, lower.tail = FALSE)
  }
  LCL <- (M1 - M2 - S * qnorm(1 - alpha / 2))
  UCL <- (M1 - M2 + S * qnorm(1 - alpha / 2))
  value <- list(mean1 = M1, mean2 = M2, m0 = m0, sigma1 = sigma1, sigma2 = sigma2, S = S, statistic = statistic, p.value = p, LCL = LCL, UCL = UCL, alternative = alternative)
  # print(sprintf("P-value = %g",p))
  # print(sprintf("Lower %.2f%% Confidence Limit = %g",
  #               alpha, LCL))
  # print(sprintf("Upper %.2f%% Confidence Limit = %g",
  #               alpha, UCL))
  return(value)
}

test <- t.test_knownvar(dat1$sample1, dat1$sample2,
  V1 = 1, V2 = 1
)
test

## $mean1
## [1] 0.02
## 
## $mean2
## [1] 0.06
## 
## $m0
## [1] 0
## 
## $sigma1
## [1] 1
## 
## $sigma2
## [1] 1
## 
## $S
## [1] 0.6324555
## 
## $statistic
## [1] -0.06324555
## 
## $p.value
## [1] 0.949571
## 
## $LCL
## [1] -1.27959
## 
## $UCL
## [1] 1.19959
## 
## $alternative
## [1] "two.sided"

The output above recaps all the information needed to perform the test: the test statistic, the p-value, the alternative used, the two sample means and the two variances of the populations (compare these results found in R with the results found by hand).

The p-value can be extracted as usual:

test$p.value

## [1] 0.949571

The p-value is 0.95 so at the 5% significance level we do not reject the null hypothesis of equal means. There is no sufficient evidence in the data to reject the hypothesis that the two means in the populations are similar. This result confirms what we found by hand.

Note that a similar function exists in the {BSDA} package:⁴

library(BSDA)

z.test(dat1$sample1,
  dat1$sample2,
  alternative = "two.sided",
  mu = 0,
  sigma.x = 1,
  sigma.y = 1,
  conf.level = 0.95
)

## 
##  Two-sample z-Test
## 
## data:  dat1$sample1 and dat1$sample2
## z = -0.063246, p-value = 0.9496
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.27959  1.19959
## sample estimates:
## mean of x mean of y 
##      0.02      0.06

Scenario 2: Independent samples with 2 equal but unknown variances

For the second scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances in both populations are unknown but equal (\(\sigma^2_1 = \sigma^2_1\)) and that we would like to test whether the mean of population 1 is larger than the mean of population 2.

dat2 <- data.frame(
  sample1 = c(1.78, 1.5, 0.9, 0.6, 0.8, 1.9),
  sample2 = c(0.8, -0.7, -0.1, 0.4, 0.1, NA)
)
dat2

##   sample1 sample2
## 1    1.78     0.8
## 2    1.50    -0.7
## 3    0.90    -0.1
## 4    0.60     0.4
## 5    0.80     0.1
## 6    1.90      NA

dat_ggplot <- data.frame(
  value = c(1.78, 1.5, 0.9, 0.6, 0.8, 1.9, 0.8, -0.7, -0.1, 0.4, 0.1),
  sample = c(rep("1", 6), rep("2", 5))
)

ggplot(dat_ggplot) +
  aes(x = sample, y = value) +
  geom_boxplot() +
  theme_minimal()

Unlike the previous scenario, the two boxes do not overlap which illustrates that the two samples are different from each other. From this boxplot, we can expect the test to reject the null hypothesis of equal means in the populations. Nonetheless, only a formal statistical test will confirm this expectation.

There is a function in R, and it is simply the t.test() function. This version of the test is actually the “standard” Student’s t-test for two samples. Note that it is assumed that the variances of the two populations are equal so we need to specify it in the function with the argument var.equal = TRUE (the default is FALSE) and the alternative hypothesis is \(H_1: \mu_1 - \mu_2 > 0\) so we need to add the argument alternative = "greater" as well:

test <- t.test(dat2$sample1, dat2$sample2,
  var.equal = TRUE, alternative = "greater"
)
test

## 
##  Two Sample t-test
## 
## data:  dat2$sample1 and dat2$sample2
## t = 3.4113, df = 9, p-value = 0.003867
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
##  0.5304908       Inf
## sample estimates:
## mean of x mean of y 
##  1.246667  0.100000

The output above recaps all the information needed to perform the test: the name of the test, the test statistic, the degrees of freedom, the p-value, the alternative used and the two sample means (compare these results found in R with the results found by hand).

The p-value can be extracted as usual:

test$p.value

## [1] 0.003866756

The p-value is 0.004 so at the 5% significance level we reject the null hypothesis of equal means. This result confirms what we found by hand.

Unlike the first scenario, the p-value in this scenario is below 5% so we reject the null hypothesis. At the 5% significance level, we can conclude that the mean of population 1 is larger than the mean of population 2.

A nice and easy way to report results of a Student’s t-test in R is with the report() function from the {report} package:

# install.packages("remotes")
# remotes::install_github("easystats/report") # You only need to do that once
library("report") # Load the package every time you start R

report(test)

## Effect sizes were labelled following Cohen's (1988) recommendations.
## 
## The Two Sample t-test testing the difference between dat2$sample1 and
## dat2$sample2 (mean of x = 1.25, mean of y = 0.10) suggests that the effect is
## positive, statistically significant, and large (difference = 1.15, 95% CI
## [0.53, Inf], t(9) = 3.41, p = 0.004; Cohen's d = 2.07, 95% CI [0.75, Inf])

As you can see, the function interprets the test (together with the p-value) for you.

Note that the report() function can be used for other analyses. See more tips and tricks in R if you find this one useful.

If your data is formatted in the long format (which is even better), simply use the tilde (~). For instance, imagine the exact same data presented like this:

dat2bis <- data.frame(
  value = c(1.78, 1.5, 0.9, 0.6, 0.8, 1.9, 0.8, -0.7, -0.1, 0.4, 0.1),
  sample = c(rep("1", 6), rep("2", 5))
)
dat2bis

##    value sample
## 1   1.78      1
## 2   1.50      1
## 3   0.90      1
## 4   0.60      1
## 5   0.80      1
## 6   1.90      1
## 7   0.80      2
## 8  -0.70      2
## 9  -0.10      2
## 10  0.40      2
## 11  0.10      2

Here is how to perform the Student’s t-test in R with data in the long format:

test <- t.test(value ~ sample,
  data = dat2bis,
  var.equal = TRUE,
  alternative = "greater"
)
test

## 
##  Two Sample t-test
## 
## data:  value by sample
## t = 3.4113, df = 9, p-value = 0.003867
## alternative hypothesis: true difference in means between group 1 and group 2 is greater than 0
## 95 percent confidence interval:
##  0.5304908       Inf
## sample estimates:
## mean in group 1 mean in group 2 
##        1.246667        0.100000

test$p.value

## [1] 0.003866756

The results are exactly the same.

Scenario 3: Independent samples with 2 unequal and unknown variances

For the third scenario, suppose the data below. Moreover, suppose that the two samples are independent, that the variances in both populations are unknown and unequal (\(\sigma^2_1 \ne \sigma^2_1\)) and that we would like to test whether the mean of population 1 is smaller than the mean of population 2.

dat3 <- data.frame(
  value = c(0.8, 0.7, 0.1, 0.4, 0.1, 1.78, 1.5, 0.9, 0.6, 0.8, 1.9),
  sample = c(rep("1", 5), rep("2", 6))
)
dat3

##    value sample
## 1   0.80      1
## 2   0.70      1
## 3   0.10      1
## 4   0.40      1
## 5   0.10      1
## 6   1.78      2
## 7   1.50      2
## 8   0.90      2
## 9   0.60      2
## 10  0.80      2
## 11  1.90      2

ggplot(dat3) +
  aes(x = sample, y = value) +
  geom_boxplot() +
  theme_minimal()

There is a function in R for this version of the test as well, and it is simply the t.test() function with the var.equal = FALSE argument. FALSE is the default option for the var.equal argument so you actually do not need to specify it. This version of the test is actually the Welch Student’s test, used when the variances of the populations are unknown and unequal. To test if two population variances are equal, you can use the Levene’s test (leveneTest(dat3$value, dat3$sample) from the {car} package, or simply by comparing the dispersion of the two samples via a dotplot or a boxplot). Note that the alternative hypothesis is \(H_1: \mu_1 - \mu_2 < 0\) so we need to add the argument alternative = "less" as well:

test <- t.test(value ~ sample,
  data = dat3,
  var.equal = FALSE,
  alternative = "less"
)
test

## 
##  Welch Two Sample t-test
## 
## data:  value by sample
## t = -3.0841, df = 8.2796, p-value = 0.007206
## alternative hypothesis: true difference in means between group 1 and group 2 is less than 0
## 95 percent confidence interval:
##        -Inf -0.3304098
## sample estimates:
## mean in group 1 mean in group 2 
##        0.420000        1.246667

The output above recaps all the information needed to perform the test (compare these results found in R with the results found by hand).

The p-value can be extracted as usual:

test$p.value

## [1] 0.00720603

The p-value is 0.007 so at the 5% significance level we reject the null hypothesis of equal means, meaning that we can conclude that the mean of population 1 is smaller than the mean of population 2. This result confirms what we found by hand.

Scenario 4: Paired samples where the variance of the differences is known

For the fourth scenario, suppose the data below. Moreover, suppose that the two samples are dependent (matched), that the variance of the differences in the population is known and equal to 1 (\(\sigma^2_D = 1\)) and that we would like to test whether the mean difference between the two populations is different than 0.

dat4 <- data.frame(
  before = c(0.9, -0.8, 0.1, -0.3, 0.2),
  after = c(0.8, -0.9, -0.1, 0.4, 0.1)
)
dat4

##   before after
## 1    0.9   0.8
## 2   -0.8  -0.9
## 3    0.1  -0.1
## 4   -0.3   0.4
## 5    0.2   0.1

dat4$difference <- dat4$after - dat4$before

ggplot(dat4) +
  aes(y = difference) +
  geom_boxplot() +
  theme_minimal()

Since there is no function in R to perform a t-test with paired samples where the variance of the differences is known, here is one with arguments accepting:

• the differences between the two samples (x),
• the variance of the differences in the population (V),
• the mean of the differences under the null hypothesis (m0, default is 0),
• the significance level (alpha, default is 0.05)
• and the alternative (alternative, one of "two.sided" (default), "less" or "greater"):

t.test_pairedknownvar <- function(x, V, m0 = 0, alpha = 0.05, alternative = "two.sided") {
  M <- mean(x)
  n <- length(x)
  sigma <- sqrt(V)
  S <- sqrt(V / n)
  statistic <- (M - m0) / S
  p <- if (alternative == "two.sided") {
    2 * pnorm(abs(statistic), lower.tail = FALSE)
  } else if (alternative == "less") {
    pnorm(statistic, lower.tail = TRUE)
  } else {
    pnorm(statistic, lower.tail = FALSE)
  }
  LCL <- (M - S * qnorm(1 - alpha / 2))
  UCL <- (M + S * qnorm(1 - alpha / 2))
  value <- list(mean = M, m0 = m0, sigma = sigma, statistic = statistic, p.value = p, LCL = LCL, UCL = UCL, alternative = alternative)
  # print(sprintf("P-value = %g",p))
  # print(sprintf("Lower %.2f%% Confidence Limit = %g",
  #               alpha, LCL))
  # print(sprintf("Upper %.2f%% Confidence Limit = %g",
  #               alpha, UCL))
  return(value)
}

test <- t.test_pairedknownvar(dat4$after - dat4$before,
  V = 1
)
test

## $mean
## [1] 0.04
## 
## $m0
## [1] 0
## 
## $sigma
## [1] 1
## 
## $statistic
## [1] 0.08944272
## 
## $p.value
## [1] 0.9287301
## 
## $LCL
## [1] -0.8365225
## 
## $UCL
## [1] 0.9165225
## 
## $alternative
## [1] "two.sided"

The output above recaps all the information needed to perform the test (compare these results found in R with the results found by hand).

The p-value can be extracted as usual:

test$p.value

## [1] 0.9287301

The p-value is 0.929 so at the 5% significance level we do not reject the null hypothesis of the mean of the differences being equal to 0. There is no sufficient evidence in the data to reject the hypothesis that the mean difference between the two populations is equal to 0. This result confirms what we found by hand.

Scenario 5: Paired samples where the variance of the differences is unknown

For the fifth and final scenario, suppose the data below. Moreover, suppose that the two samples are dependent (matched), that the variance of the differences in the population is unknown and that we would like to test whether a treatment is effective in increasing running capabilities (the higher the value, the better in terms of running capabilities).

dat5 <- data.frame(
  before = c(9, 8, 1, 3, 2),
  after = c(16, 11, 15, 12, 9)
)
dat5

##   before after
## 1      9    16
## 2      8    11
## 3      1    15
## 4      3    12
## 5      2     9

dat5$difference <- dat5$after - dat5$before

ggplot(dat5) +
  aes(y = difference) +
  geom_boxplot() +
  theme_minimal()

There is a function in R for this version of the test, and it is simply the t.test() function with the paired = TRUE argument. This version of the test is actually the standard version of the Student’s t-test with paired samples. Note that the alternative hypothesis is \(H_1: \mu_D > 0\) so we need to add the argument alternative = "greater" as well:

test <- t.test(dat5$after, dat5$before,
  alternative = "greater",
  paired = TRUE
)
test

## 
##  Paired t-test
## 
## data:  dat5$after and dat5$before
## t = 4.4721, df = 4, p-value = 0.005528
## alternative hypothesis: true mean difference is greater than 0
## 95 percent confidence interval:
##  4.186437      Inf
## sample estimates:
## mean difference 
##               8

Note that we wrote after and then before in this order. If you write before and then after, make sure to change the alternative to alternative = "less".

If your data is in the long format, use the tilde ~:

dat5 <- data.frame(
  value = c(9, 8, 1, 3, 2, 16, 11, 15, 12, 9),
  time = c(rep("before", 5), rep("after", 5))
)
dat5

##    value   time
## 1      9 before
## 2      8 before
## 3      1 before
## 4      3 before
## 5      2 before
## 6     16  after
## 7     11  after
## 8     15  after
## 9     12  after
## 10     9  after

test <- t.test(value ~ time,
  data = dat5,
  alternative = "greater",
  paired = TRUE
)
test

## 
##  Paired t-test
## 
## data:  value by time
## t = 4.4721, df = 4, p-value = 0.005528
## alternative hypothesis: true mean difference is greater than 0
## 95 percent confidence interval:
##  4.186437      Inf
## sample estimates:
## mean difference 
##               8

The output above recaps all the information needed to perform the test (compare these results found in R with the results found by hand).

The p-value can be extracted as usual:

test$p.value

## [1] 0.005528247

The p-value is 0.006 so at the 5% significance level we reject the null hypothesis of the mean of the differences being equal to 0, meaning that we can conclude that the treatment is effective in increasing the running capabilities (because the mean of the differences is greater than 0). This result confirms what we found by hand.

Combination of plot and statistical test

After having written this article, I discovered the {ggstatsplot} package which I believe is worth mentioning here, in particular the ggbetweenstats() and ggwithinstats() functions for independent and paired samples, respectively.

These two functions combine a boxplot—representing the distribution for each group—and the results of the statistical test displayed in the subtitle of the plot.

See examples below for scenarios 2, 3 and 5. Unfortunately, the package does not allow to run tests for scenarios 1 and 4.

# load package
library(ggstatsplot)
library(ggplot2)

# plot with statistical results
ggbetweenstats(
  data = dat2bis,
  x = sample,
  y = value,
  plot.type = "box", # for boxplot
  type = "parametric", # for student's t-test
  var.equal = TRUE, # equal variances
  centrality.plotting = FALSE # remove mean
) +
  labs(caption = NULL) # remove caption

# plot with statistical results
ggbetweenstats(
  data = dat3,
  x = sample,
  y = value,
  plot.type = "box", # for boxplot
  type = "parametric", # for student's t-test
  var.equal = FALSE, # unequal variances
  centrality.plotting = FALSE # remove mean
) +
  labs(caption = NULL) # remove caption

ggwithinstats(
  data = dat5,
  x = time,
  y = value,
  type = "parametric", # for student's t-test
  centrality.plotting = FALSE # remove mean
) +
  labs(caption = NULL) # remove caption