R on Stats and R

McNemar's test in R

Wed, 19 Aug 2026 00:00:00 +0000

Introduction

In a previous article, we showed how to perform the Chi-square test of independence in R in order to test whether two qualitative variables are related. As mentioned in that article (and in the one showing how to do the Chi-square test of independence by hand), this test requires that observations are independent. When observations are dependent, that is, when the two measurements are collected on the same subjects (paired samples), the McNemar’s or Cochran’s Q tests should be used instead.

This article is dedicated to the first one: the McNemar’s test. It is used to compare two related (paired) proportions measured on a qualitative variable with only two possible levels. In practice, it is mostly used when the same subjects are measured twice (typically before and after an intervention), or when two raters or two conditions are applied to the same subjects.

In a way, the McNemar’s test is to two paired proportions what the paired Student’s t-test is to two paired means: in both cases we take advantage of the fact that the two measurements belong to the same individuals, the difference being that here the variable of interest is binary instead of quantitative.

Note also that the McNemar’s test is limited to exactly two related measurements. If you have more than two (for example, the same question asked at three different time points), the appropriate extension is the Cochran’s Q test, of which the McNemar’s test is the special case for two measurements. If you are unsure about which test is appropriate for your own data, see this overview of the most common statistical tests.

In the remaining of the article, we present the data used for the illustration, the aim, hypotheses and assumptions of the test, and finally how to perform it in R and how to interpret its results.

Data

A dataset with a paired binary structure is not so easy to find among the datasets shipped with R, so we simulate our own data for this article.

Suppose that we ask 200 randomly selected citizens whether they are in favor of a new policy in their city (answer “Yes” or “No”), that we then have them watch a public debate on this policy, and that we ask them exactly the same question again right after the debate:

# number of respondents
n <- 200

# opinion before the debate
before <- sample(c("Yes", "No"),
  size = n,
  replace = TRUE,
  prob = c(0.4, 0.6)
)

# opinion after the debate (respondents who were in favor
# tend to keep their opinion, while those who were against
# are more likely to change their mind)
after <- ifelse(before == "Yes",
  sample(c("Yes", "No"), size = n, replace = TRUE, prob = c(0.9, 0.1)),
  sample(c("Yes", "No"), size = n, replace = TRUE, prob = c(0.4, 0.6))
)

# dataset
dat <- data.frame(
  respondent = 1:n,
  before = factor(before, levels = c("Yes", "No")),
  after = factor(after, levels = c("Yes", "No"))
)

head(dat)

##   respondent before after
## 1          1    Yes   Yes
## 2          2    Yes   Yes
## 3          3     No   Yes
## 4          4    Yes   Yes
## 5          5    Yes   Yes
## 6          6     No    No

(Note that a seed has been set in the background with set.seed(42), so the simulated data and all results below are reproducible.)

Each row corresponds to one respondent and contains two measurements of the same binary variable: the opinion before and the opinion after the debate. The two samples are thus paired, since the two answers on a given row belong to the same person.

As always, it is a good practice to start with some descriptive statistics. Here, the proportion of respondents in favor of the policy at each of the two time points:

# install.packages("dplyr")
library(dplyr)

dat %>%
  summarise(
    prop_before = mean(before == "Yes"),
    prop_after = mean(after == "Yes")
  )

##   prop_before prop_after
## 1        0.46       0.61

In our sample, the proportion of respondents in favor of the policy went from 46% before the debate to 61% after the debate.

These two proportions are computed on the same people, so comparing them as if they came from two independent groups would ignore the pairing. What matters for the McNemar’s test is the way each respondent moved (or did not move) from one answer to the other, and this information is contained in the 2 $\times$ 2 contingency table of the paired answers:

tab <- table(dat$before, dat$after,
  dnn = c("Before", "After")
)

tab

##       After
## Before Yes No
##    Yes  81 11
##    No   41 67

This table must be read pair by pair, and not cell by cell as we usually do:

the two cells on the diagonal are the concordant pairs: 81 respondents answered “Yes” twice and 67 answered “No” twice, so these 148 respondents did not change their mind,
the two cells outside the diagonal are the discordant pairs: 11 respondents were in favor before the debate but against after, while 41 were against before but in favor after.

Only the discordant pairs carry information about a change of opinion (a respondent who gave twice the same answer tells us nothing about the effect of the debate), and this is precisely what the McNemar’s test is built on.

The same information can be visualized with a simple barplot of the paired counts:

# install.packages("ggplot2")
library(ggplot2)

ggplot(dat) +
  aes(x = before, fill = after) +
  geom_bar(position = "dodge") +
  labs(
    x = "Opinion before the debate",
    y = "Number of respondents",
    fill = "Opinion after the debate"
  )

From the table and the plot, we see that the changes of opinion do not balance out: many more respondents switched from “No” to “Yes” than the opposite. The question is now whether this imbalance is large enough to be declared significant, or whether it could reasonably be explained by chance alone (that is, by sampling fluctuations).

McNemar’s test

Aim and hypotheses

The McNemar’s test is used to compare two related proportions, so it allows to determine whether the proportion of subjects belonging to a given category changed between two dependent measurements.

The null and alternative hypotheses of the McNemar’s test are:

$H_0$: the two related proportions are equal (marginal homogeneity, that is, there is no systematic change between the two measurements)
$H_1$: the two related proportions are different (there is a significant change between the two measurements)

Since concordant pairs bring no information about a change, the test is based only on the two discordant cells. Denoting by $b$ the number of subjects who answered “Yes” then “No”, and by $c$ the number of subjects who answered “No” then “Yes”, the hypotheses can equivalently be written as:

$H_0: p_b = p_c$
$H_1: p_b \ne p_c$

where $p_b$ and $p_c$ are the probabilities of the two possible types of change. Under the null hypothesis, a change in one direction is as likely as a change in the other direction, so the test statistic

\[\chi^2 = \frac{(b - c)^2}{b + c}\]

follows a Chi-square distribution with 1 degree of freedom. By default, R applies a continuity correction (see more on this below), which replaces the numerator by $(|b - c| - 1)^2$.

In the context of our example, the McNemar’s test helps us to answer the following question: “Did the public debate significantly change the proportion of citizens in favor of the new policy?”.

Rejecting $H_0$ would mean that the proportion of citizens in favor of the policy is significantly different before and after the debate, so that the changes of opinion observed in our sample are unlikely to be due to chance only. On the contrary, not rejecting $H_0$ would mean that we do not have enough evidence to conclude that opinions changed: the switches observed in the two directions would then be compatible with random fluctuations.

Note that, as for many tests, the McNemar’s test does not indicate the direction of the change. The direction must be read from the contingency table or from the marginal proportions computed in the previous section.

Assumptions

For the results of the McNemar’s test to be valid, the following assumptions must be met:

Paired measurements on a binary variable. The two measurements must be collected on the same subjects, or on matched pairs (twins, or patients matched on age and sex for instance), and the variable of interest must be qualitative with exactly two levels (“Yes”/“No”, success/failure, present/absent, etc.). If the two samples are independent instead of paired, use the Chi-square test of independence.
Data organized in a 2 $\times$ 2 contingency table of the paired outcomes. Each subject contributes to one and only one cell of the table, so the sum of the four cells equals the number of subjects (200 in our case), and not twice this number.
Pairs are independent of each other. Within a pair, the two measurements are of course dependent, and this is precisely the reason why we use this test. Between pairs, however, independence is required: one subject’s answers must not influence another subject’s answers. As for many statistical tests, this assumption is usually verified based on the design of the experiment rather than via a formal test. A random and representative sample of the population of interest is generally sufficient. In our example, respondents have been selected at random and answered the question individually, so we consider this assumption as met.
Enough discordant pairs. The $p$-value returned by mcnemar.test() is based on a Chi-square approximation, which is reliable only if the number of discordant pairs is large enough. A common rule of thumb is that $b + c$ should be at least 25. In our sample, $b + c$ = 52, so the approximation can be used safely.

When the number of discordant pairs is small, it is preferable to use the exact version of the test, which is based on a binomial distribution instead of the Chi-square approximation. It boils down to testing whether, among the discordant pairs, changes in one direction are as frequent as changes in the other direction, so it can be performed in base R with the binom.test() function:

# exact version of the McNemar's test
binom.test(tab[1, 2], tab[1, 2] + tab[2, 1], p = 0.5)

## 
## 	Exact binomial test
## 
## data:  tab[1, 2] and tab[1, 2] + tab[2, 1]
## number of successes = 11, number of trials = 52, p-value = 3.589e-05
## alternative hypothesis: true probability of success is not equal to 0.5
## 95 percent confidence interval:
##  0.1106115 0.3470376
## sample estimates:
## probability of success 
##              0.2115385

Note that the {exact2x2} package also provides a dedicated mcnemar.exact() function, which returns the same $p$-value together with a confidence interval for the odds ratio.

In R

The McNemar’s test can be performed in R with the mcnemar.test() function, applied on the contingency table of the paired outcomes:

mcnemar.test(tab)

## 
## 	McNemar's Chi-squared test with continuity correction
## 
## data:  tab
## McNemar's chi-squared = 16.173, df = 1, p-value = 5.781e-05

The test can also be applied directly on the two variables, without building the contingency table first (results are of course identical):

mcnemar.test(dat$before, dat$after)

## 
## 	McNemar's Chi-squared test with continuity correction
## 
## data:  dat$before and dat$after
## McNemar's chi-squared = 16.173, df = 1, p-value = 5.781e-05

The output shows:

the title of the test, together with the mention that a continuity correction has been applied,
the data which have been used,
the test statistic (McNemar's chi-squared),
the degrees of freedom (always equal to 1 for a 2 $\times$ 2 table) and
the $p$-value.

As mentioned above, R applies a continuity correction by default. This correction makes the test slightly more conservative (that is, it gives a larger $p$-value), and it can be removed thanks to the correct = FALSE argument:

mcnemar.test(tab, correct = FALSE)

## 
## 	McNemar's Chi-squared test
## 
## data:  tab
## McNemar's chi-squared = 17.308, df = 1, p-value = 3.179e-05

With 52 discordant pairs, both versions lead to the same conclusion. The correction really matters only when the number of discordant pairs is small, and in that case the exact version presented in the previous section is a better option anyway.

It is the $p$-value which is of interest to conclude the test. If you are not familiar with $p$-values, I invite you to read this section.

Interpretations

Based on the McNemar’s test, we reject the null hypothesis and we conclude that the proportion of citizens in favor of the new policy is significantly different before and after the debate ($p$-value < 0.001).

$\Rightarrow$ In our context, rejecting the null hypothesis means that the debate is associated with a significant change of opinion. Looking at the direction of this change, the proportion of citizens in favor of the policy increased from 46% before the debate to 61% after the debate.

(For the sake of illustration, if the $p$-value had been larger than the significance level $\alpha = 0.05$: we could not have rejected the null hypothesis, so we could not have concluded that the proportion of citizens in favor of the policy was different before and after the debate.)

Contrary to the tests comparing three groups or more, no post-hoc test is required after a significant McNemar’s test: only two related measurements are compared, so a significant result already tells us which two proportions differ. Post-hoc comparisons become relevant again with more than two related measurements, in which case you should turn to the Cochran’s Q test.

Summary

In this article, we reviewed the aim and the hypotheses of the McNemar’s test, which is used to compare two related proportions measured on the same subjects, together with its underlying assumptions (paired measurements on a binary variable, independence between pairs and a sufficient number of discordant pairs). We then showed how to perform it in R with the mcnemar.test() function, applied either on the 2 $\times$ 2 contingency table of the paired answers or directly on the two variables, and how to interpret its results by comparing the $p$-value with the significance level $\alpha$. Remember that it is the special case of the Cochran’s Q test for exactly two related measurements, and that with independent samples the Chi-square test of independence should be preferred.

Thanks for reading.

I hope this article helped you to understand the McNemar’s test and how to perform it in R.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Bayesian Neural Networks in {tidymodels} with {kindling}

Fri, 20 Mar 2026 00:00:00 +0000

What Are Bayesian Neural Networks?

Standard neural networks learn fixed weights during training and produce a single point estimate for each input — with no sense of how confident the model is. Bayesian Neural Networks (BNNs) replace those fixed weights with probability distributions (Neal 2012). Instead of learning a single value per weight, the network learns a mean and a variance, and samples from that distribution at every forward pass.

Formally, a BNN places a prior $p(\mathbf{w})$ over the weights and updates it with observed data $\mathcal{D}$ via Bayes’ theorem:

\[p(\mathbf{w} \mid \mathcal{D}) = \frac{p(\mathcal{D} \mid \mathbf{w})\, p(\mathbf{w})}{p(\mathcal{D})}\]

For large and deep networks, the posterior $p(\mathbf{w} \mid \mathcal{D})$ is intractable. BNNs typically use variational inference to approximate it with a tractable distribution $q_\phi(\mathbf{w})$ by minimising the KL divergence between the two (Blundell et al. 2015).

From point estimates to distributions

A standard linear layer computes $\mathbf{y} = \mathbf{W}\mathbf{x} + \mathbf{b}$, where $\mathbf{W}$ and $\mathbf{b}$ are fixed. In a Bayesian linear layer, they are random variables. We place a Gaussian prior over the weights:

\[\mathbf{W} \sim \mathcal{N}(\mu_{\text{prior}},\ \sigma_{\text{prior}}^2)\]

and during training we learn a variational posterior:

\[q(\mathbf{W}) = \mathcal{N}(\mu_W,\ \sigma_W^2)\]

Each weight therefore has two learnable scalars: a mean $\mu_W$ and a log-standard-deviation $\log \sigma_W$.

The reparameterisation trick

Sampling directly from $q(\mathbf{W})$ is not differentiable with respect to $\mu_W$ and $\sigma_W$. We resolve this with the reparameterisation trick (Kingma and Welling 2013):

\[\mathbf{W} = \mu_W + \sigma_W \odot \varepsilon, \quad \varepsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I})\]

We store $\log \sigma_W$ (not $\sigma_W$ directly) to ensure positivity during unconstrained optimisation:

\[\sigma_W = \exp(\texttt{weight}_{\log\sigma})\]

This yields the sampled weight used in the forward pass:

\[\mathbf{W}_{\text{sample}} = \mu_W + \exp(\texttt{weight}_{\log\sigma}) \odot \varepsilon_W\]

and similarly for the bias:

\[\mathbf{b}_{\text{sample}} = \mu_b + \exp(\texttt{bias}_{\log\sigma}) \odot \varepsilon_b\]

Is it possible?

At the moment, {tidymodels} offers limited APIs for neural network architectures, mostly around standard MLPs. A few packages make {torch} easier to use at a higher level:

{brulee} bridges {torch} with {tidymodels}. It is ergonomic, but currently focused on MLPs for tabular data.
{cito} does not integrate with {tidymodels} and leans more toward statistical applications.
The {torch} team maintains {luz}.
{tabnet} integrates with {tidymodels}, but is dedicated to the TabNet architecture.

{kindling} does not replace these packages. However, it helps close some of these gaps by supporting custom architectures with flexible depth. As a result, custom neural network architectures, including BNNs, can be used within {tidymodels} workflows. For a broader overview of what {kindling} enables in R, see this earlier post.

Setup

On March 3, 2026, {kindling} v0.3.0 was released on CRAN. Remember that utils::install.packages() in R always installs the latest version.

install.packages("kindling")

If you want the development version, you can retrieve the package by installing it from GitHub:

pak::pak("joshuamarie/kindling")

I recommend using {pak}: it is fast and very good at resolving package dependencies across environments, including system libraries and {renv}-managed projects.

Before using {kindling}, install LibTorch — the C++ backend shared by PyTorch and the {torch} R package (Falbel and Luraschi 2023):

torch::install_torch()

Loading namespace

When teaching R, I recommend using box::use() and qualifying the names you import. Create a bnn folder in your project root, then clone this repository to use BNNs in R (this module is adapted from torchbnn (Kim 2020)): https://github.com/joshuamarie/RTorchBNN.

You can run a specific command:

git clone --branch main https://github.com/joshuamarie/RTorchBNN bnn

Then, load the following:

box::use(
  bnn = . / bnn,
  kindling[train_nnsnip, nn_arch, act_funs],
  parsnip[fit, augment],
  yardstick[metrics]
)

Then register the models in {parsnip} with:

loadNamespace("kindling")

The `train_nnsnip()` Interface

In {kindling} v0.3.0, train_nnsnip() was introduced as a {parsnip}-compatible model specification that bridges the generalized neural network trainer with the {tidymodels} ecosystem. Unlike mlp_kindling(), which is specific to feedforward networks, train_nnsnip() is architecture-agnostic: you describe the layer topology via nn_arch(), allowing BNN-style or other custom layer types to slot in without changing the training logic.

Classification: Iris

We train a Bayesian Neural Network model to predict species in the iris dataset via train_nnsnip(), by configuring arch with nn_arch(). Inside nn_arch(), set nn_layer to bnn$BayesLinear instead of the default linear layer, then define the arguments for each layer with layer_arg_fn.

nn_model <-
  train_nnsnip(
    mode = "classification",
    arch = nn_arch(
      nn_layer = bnn$BayesLinear,
      out_nn_layer = torch::nn_linear,
      layer_arg_fn = ~ if (.is_output) {
        list(.in, .out)
      } else {
        list(
          in_features = .in,
          out_features = .out,
          prior_mu = 0,
          prior_sigma = 0.1
        )
      }
    ),
    hidden_neurons = c(64, 32),
    activations = act_funs(relu, elu),
    loss = "cross_entropy",
    epochs = 50,
    verbose = TRUE,
    learn_rate = 1e-3,
    optimizer_args = list(weight_decay = 0.01)
  ) |>
  fit(Species ~ ., data = iris)

nn_model |>
  augment(new_data = iris) |>
  metrics(truth = Species, estimate = .pred_class)

## # A tibble: 2 × 3
##   .metric  .estimator .estimate
##   <chr>    <chr>          <dbl>
## 1 accuracy multiclass      0.68
## 2 kap      multiclass      0.52

Averaging predictions over multiple forward passes gives both a point estimate and a measure of predictive spread — all within a standard {tidymodels} pipeline.

The drawbacks

For now, there is no built-in option to update the loss dynamically during training. To use a custom objective such as the Evidence Lower Bound (ELBO), you currently need to define it manually and retrain with that loss.

Updating the model

That said, you can still apply a custom loss by setting the loss parameter to a torch function, for example torch::nnf_mse_loss(). If you want to train the model with an ELBO-style loss, do the following:

make_elbo_loss <- function(bnn_model, n_obs, kl_weight = 1.0) {
  box::use(
    torch[nnf_cross_entropy, torch_tensor]
  )

  function(input, target) {
    ce <- nnf_cross_entropy(input, target)
    device <- input$device
    kl_val <- torch_tensor(0.0, device = device, requires_grad = FALSE)
    for (m in bnn_model$modules) {
      if (inherits(m, "BayesLinear") && !is.null(m$kl)) {
        kl_val <- kl_val + m$kl
      }
    }

    ce + kl_weight * kl_val / n_obs
  }
}

elbo_loss <- make_elbo_loss(nn_model$fit$model, n_obs = nrow(iris), kl_weight = 1.0)

nn_model2 <-
  train_nnsnip(
    mode = "classification",
    arch = nn_arch(
      nn_layer = bnn$BayesLinear,
      out_nn_layer = torch::nn_linear,
      layer_arg_fn = ~ if (.is_output) {
        list(.in, .out)
      } else {
        list(
          in_features = .in,
          out_features = .out,
          prior_mu = 0,
          prior_sigma = 0.1
        )
      }
    ),
    hidden_neurons = c(64, 32),
    activations = act_funs(relu, elu),
    loss = elbo_loss,
    epochs = 50,
    verbose = TRUE,
    learn_rate = 1e-3,
    optimizer_args = list(weight_decay = 0.01)
  ) |>
  fit(Species ~ ., data = iris)

nn_model2 |>
  augment(new_data = iris) |>
  metrics(truth = Species, estimate = .pred_class)

## # A tibble: 2 × 3
##   .metric  .estimator .estimate
##   <chr>    <chr>          <dbl>
## 1 accuracy multiclass      0.72
## 2 kap      multiclass      0.58

As always, if you have any question related to the topic covered in this post, please add it as a comment so other readers can benefit from the discussion.

References

Blundell, Charles, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. “Weight Uncertainty in Neural Network.” International Conference on Machine Learning, 1613–22.

Falbel, Daniel, and Javier Luraschi. 2023. Torch: Tensors and Neural Networks with GPU Acceleration. https://github.com/mlverse/torch.

Kim, Harry. 2020. Bayesian Neural Network for PyTorch [Torchbnn]. https://github.com/Harry24k/bayesian-neural-network-pytorch.

Kingma, Diederik P, and Max Welling. 2013. “Auto-Encoding Variational Bayes.” arXiv Preprint arXiv:1312.6114.

Neal, Radford M. 2012. Bayesian Learning for Neural Networks. Vol. 118. Springer Science & Business Media.

You can do more for neural networks in R with {kindling}

Thu, 19 Feb 2026 00:00:00 +0000

Why this post matters

Neural networks in R are no longer niche. Today, we can choose among:

{nnet} for classic, small-scale neural nets,
{neuralnet} another classic neural nets package besides {nnet},
{keras} / {keras3} for the Keras API (typically with Python backends such as TensorFlow/JAX/Torch),
{torch} for native-R deep learning with explicit model and training control.

So why discuss another package?

In our experience, many day-to-day projects sit in the middle: we want more flexibility than a classical model, but less boilerplate than writing a full {torch} training loop. {kindling} is designed for that middle ground.

In this article, we focus on what {kindling} does well, where it helps, and where it is still limited.

What problem `{kindling}` solves (and what it does not)

{kindling} is a higher-level interface built on top of {torch}. In practice, it reduces repetitive code for:

defining common network architectures,
training feedforward and recurrent models,
integrating with {tidymodels} ({parsnip}, {recipes}, {workflows}, {tune}).

It does not replace low-level {torch} for highly customized research code, and it does not make hardware setup disappear. You still need a working LibTorch installation and an environment that can use CPU/GPU properly.

At the time of writing (CRAN version 0.2.0), core built-in architectures include FFNN/MLP and recurrent variants (RNN/LSTM/GRU).

Setup

Install this package in two different ways:

Solely install {kindling} (as it installs the package dependencies internally)

install.packages("kindling")

You can install the package dependencies separately:

install.packages(c(
  "kindling",
  # dependencies
  "torch", "dplyr", "rsample", "recipes", "yardstick",
  "workflows", "parsnip", "tibble", "ggplot2", "mlbench", "vip"
))

Then load the packages in two ways:

Using library() traditionally:

library(kindling)
library(torch)
library(dplyr)
library(rsample)
library(recipes)
library(yardstick)
library(workflows)
library(parsnip)
library(tibble)
library(ggplot2)
library(mlbench)
library(vip)

In this article, we use library() for clarity and copy-paste reproducibility.

If you prefer a module-based and more explicit import style, use box::use() (read Box: Placing module system into R for more details).

Here’s the equivalent code as above:

box::use(
kindling[...],
dplyr[...],
rsample[...],
recipes[...],
yardstick[...],
workflows[...],
parsnip[...],
tibble[...],
ggplot2[...],
mlbench[...],
vip[...]
)

{kindling} uses {torch} as backend, so LibTorch must be installed once:

if (!torch::torch_is_installed()) {
  torch::install_torch()
}

Three levels of interaction

{kindling} is useful because you can work at different abstraction levels. Here are three ways to interact with the package, ordered from lowest to highest level of abstraction:

1) Generate model code (`*_generator()`)

If you want to inspect architecture code before training:

ffnn_generator(
  nn_name = "MyNet",
  hd_neurons = c(64, 32, 16),
  no_x = 10,
  no_y = 1,
  activations = act_funs(
    relu,
    "softplus(beta = 0.5)",
    selu
  )
)

## torch::nn_module("MyNet", initialize = function () 
## {
##     self$fc1 = torch::nn_linear(10, 64, bias = TRUE)
##     self$fc2 = torch::nn_linear(64, 32, bias = TRUE)
##     self$fc3 = torch::nn_linear(32, 16, bias = TRUE)
##     self$out = torch::nn_linear(16, 1, bias = TRUE)
## }, forward = function (x) 
## {
##     x = self$fc1(x)
##     x = torch::nnf_relu(x)
##     x = self$fc2(x)
##     x = torch::nnf_softplus(x, beta = 0.5)
##     x = self$fc3(x)
##     x = torch::nnf_selu(x)
##     x = self$out(x)
##     x
## })

Technical note: To specify a parametric activation functions like Softplus under act_funs() function, set softplus = args(beta = 0.5) or softplus[beta = 0.5] (available in v0.3.x and later), not just in a stringly typed expression, e.g. "softplus(beta = 0.5)".

This returns an unevaluated torch::nn_module expression you can inspect or modify.

2) Direct Training

The functions available (for now) are ffnn() and rnn(). For many applied tasks, this is the fastest way to fit a network from a formula.

mini_direct <- ffnn(
  mpg ~ .,
  data = mtcars,
  hidden_neurons = 8,
  activations = act_funs(relu),
  epochs = 20,
  verbose = FALSE
)

predict(mini_direct, newdata = head(mtcars, 3))

## [1] -0.01706841 -0.04320815 -0.71570069

3) ML Framework Integration: `{tidymodels}` with `mlp_kindling()` / `rnn_kindling()`

{kindling} functions to directly train the said models also (currently) integrates with {tidymodels}. This level is ideal when we want recipes, workflows, resampling, and tuning.

mini_spec <- mlp_kindling(
  mode = "regression",
  hidden_neurons = 8,
  activations = act_funs(relu), # or just "relu"
  epochs = 20,
  verbose = FALSE
)

mini_wf <- workflow() |>
  add_formula(mpg ~ .) |>
  add_model(mini_spec)

mini_fit <- fit(mini_wf, data = mtcars)

predict(mini_fit, new_data = head(mtcars, 3))

## # A tibble: 3 × 1
##   .pred
##   <dbl>
## 1  9.28
## 2  9.31
## 3  5.63

Now that we’ve covered the three levels of interaction, let’s see {kindling} in action with two “realistic” examples. We’ll demonstrate how to structure reproducible workflows, handle feature preprocessing, and evaluate results properly. Along the way, you’ll see which abstraction level works best for different scenarios.

Example 1: Iris classification with reproducible good practice

This first example uses direct training with ffnn(), but still follows practical steps:

train/test split,
feature scaling,
validation split during training,
regularization,
out-of-sample evaluation.

Data Preprocessing

Train/test split is performed to prevent evaluating only on training data.

data(iris)

iris_split <- initial_split(iris, prop = 0.8, strata = Species)
iris_train <- training(iris_split)
iris_test <- testing(iris_split)

iris_recipe <- recipe(Species ~ ., data = iris_train) |>
  step_normalize(all_predictors())
iris_prep <- prep(iris_recipe, training = iris_train)
iris_train_processed <- bake(iris_prep, new_data = NULL)
iris_test_processed <- bake(iris_prep, new_data = iris_test)

MLP with 2 Hidden Layers

set.seed(20260218)

iris_mlp <- ffnn(
  Species ~ .,
  data = iris_train_processed,
  hidden_neurons = c(32, 16),
  activations = act_funs(relu, "softplus(beta = 0.5)"),
  loss = "cross_entropy",
  optimizer = "adam",
  learn_rate = 0.01,
  penalty = 1e-4,
  mixture = 0,
  batch_size = 16,
  epochs = 200,
  validation_split = 0.2,
  verbose = FALSE
)

iris_mlp

## 
## ======================= Feedforward Neural Networks (MLP) ======================
## 
## 
## -- FFNN Model Summary ----------------------------------------------------------
## 
## 
## -------------------------------------------------------------------------------------
##   NN Model Type           :             FFNN    n_predictors :                    4
##   Number of Epochs        :              200    n_response   :                    3
##   Hidden Layer Units      :           32, 16    reg.         :   [λ = 1e-04, α = 0]
##   Number of Hidden Layers :                2    Device       :                  mps
##   Pred. Type              :   classification                 :                     
## -------------------------------------------------------------------------------------
## 
## 
## 
## -- Activation function ---------------------------------------------------------
## 
## 
##                -------------------------------------------------
##                  1st Layer {32}    :                      relu
##                  2nd Layer {16}    :      softplus(beta = 0.5)
##                  Output Activation :   No act function applied
##                -------------------------------------------------

iris_pred <- predict(iris_mlp, newdata = iris_test_processed, type = "response")

iris_eval <- tibble(
  truth = iris_test_processed$Species,
  .pred_class = iris_pred
)

metrics(iris_eval, truth = truth, estimate = .pred_class)

## # A tibble: 2 × 3
##   .metric  .estimator .estimate
##   <chr>    <chr>          <dbl>
## 1 accuracy multiclass     0.933
## 2 kap      multiclass     0.9

conf_mat(iris_eval, truth = truth, estimate = .pred_class)

##             Truth
## Prediction   setosa versicolor virginica
##   setosa         10          0         0
##   versicolor      0          9         1
##   virginica       0          1         9

Why these choices?

Here’s the brief explanation on how we set up the models from {kindling}:

Architecture (c(32, 16)): The architecture has 2 layers with 32 units for the 1st layer and 16 for the 2nd layer. It is enough capacity for Iris, but still simple.
Activations: The architecture has 2 hidden layers, each layer is specified with different activation functions.
1. relu: Also called Rectified Linear Unit (ReLU); the most stable default for hidden layers.
2. softplus: Similar to ReLU but much smoother in approximation; beta is adjusted to 0.5.
Scaling: crucial because neural networks are sensitive to predictor scale.
Regularization (penalty and mixture): helps reduce overfitting even on small data. mixture is set to 0, which means the optimization is performed with L2 regularization.
Validation split: monitors generalization during training.

Example 2: A more realistic tabular benchmark (Sonar)

Iris is useful for teaching, but sometimes considered (too) easy. The Sonar dataset (60 numeric predictors, binary outcome) is a better stress test for tabular classification.

data("Sonar", package = "mlbench")

sonar <- Sonar |>
  mutate(Class = factor(Class))

sonar_split <- initial_split(sonar, prop = 0.8, strata = Class)
sonar_train <- training(sonar_split)
sonar_test <- testing(sonar_split)

Now fit through {tidymodels} so preprocessing and modeling stay in one pipeline:

sonar_rec <- recipe(Class ~ ., data = sonar_train) |>
  step_normalize(all_predictors())

sonar_spec <- mlp_kindling(
  mode = "classification",
  hidden_neurons = c(64, 32),
  activations = act_funs(relu, relu),
  optimizer = "adam",
  learn_rate = 0.001,
  penalty = 1e-4,
  mixture = 0,
  batch_size = 16,
  epochs = 250,
  validation_split = 0.2,
  verbose = FALSE
)

sonar_wf <- workflow() |>
  add_recipe(sonar_rec) |>
  add_model(sonar_spec)

set.seed(20260218)
sonar_fit <- fit(sonar_wf, data = sonar_train)

sonar_pred <- augment(sonar_fit, new_data = sonar_test)

metrics(sonar_pred, truth = Class, estimate = .pred_class)

## # A tibble: 2 × 3
##   .metric  .estimator .estimate
##   <chr>    <chr>          <dbl>
## 1 accuracy binary         0.860
## 2 kap      binary         0.720

In practice, this setup is often a strong baseline for tabular binary classification before trying larger architectures.

About early stopping and callbacks

If you come from Keras/TensorFlow, you may be used to callback objects (early stopping, learning-rate schedules, etc.).

With {kindling} 0.2.0, practical overfitting control is usually handled through:

validation_split,
regularization (penalty, mixture),
tuning model size and training length (epochs),
proper resampling with {tidymodels}.

In other words, we can approximate early-stopping behavior by selecting epochs via validation/resampling, even without a callback-heavy workflow.

Supported architectures (current scope)

Architecture	Main function(s)	Typical use
Feedforward (MLP/FFNN)	ffnn(), mlp_kindling()	Tabular regression/classification
RNN	rnn_kindling(rnn_type = "rnn")	Sequential patterns
LSTM	rnn_kindling(rnn_type = "lstm")	Longer-range sequence dependencies
GRU	rnn_kindling(rnn_type = "gru")	Sequence modeling with fewer parameters

Variable importance (FFNN)

Interpretability for neural networks is imperfect, but {kindling} integrates established approaches for FFNN models.

# For FFNN fits:
garson(iris_mlp, bar_plot = FALSE)

##        x_names y_names  rel_imp
## 1 Petal.Length       y 34.31493
## 2  Sepal.Width       y 24.68379
## 3  Petal.Width       y 22.43818
## 4 Sepal.Length       y 18.56311

olden(iris_mlp, bar_plot = FALSE)

##        x_names y_names    rel_imp
## 1 Petal.Length       y -5.1100721
## 2  Petal.Width       y -2.9793542
## 3  Sepal.Width       y  0.8302629
## 4 Sepal.Length       y -0.1882139

# Via vip (Olden/Garson methods supported by kindling S3 methods)
vi(iris_mlp, type = "olden") |>
  vip()

We recommend using these as directional diagnostics, not as causal evidence.

When `{kindling}` is a good fit

{kindling} is a practical choice when our projects:

work mostly in R and want to stay inside {tidymodels},
need neural nets for tabular or moderate sequence tasks,
want less boilerplate than raw {torch} but still meaningful control.

It may be less ideal when our projects need:

custom research architectures and training loops,
mature callback ecosystems similar to high-level Keras workflows,
highly optimized distributed production pipelines.

Limitations to keep in mind

Hardware setup still matters: GPU usage depends on a correct {torch}/LibTorch installation and supported hardware.
Ecosystem maturity: this is a young package; interfaces can evolve.
Debugging depth: for deeply custom debugging, low-level {torch} remains the reference path.
Model class assumptions: recurrent models are for sequence-structured data; using them on plain tabular data is usually not appropriate.

Takeaways

{kindling} is not about replacing {torch} or {keras3}. It is about reducing friction for common deep-learning workflows in R.

For many applied projects, a robust pattern is:

preprocess with {recipes},
start with a modest MLP architecture,
use validation split + regularization,
evaluate on held-out test data,
tune only after you have a strong baseline.

If this matches your workflow, {kindling} is worth trying.

As always, if you have any question related to the topic covered in this post, please add it as a comment so other readers can benefit from the discussion.

nycOpenData: A unified R interface to NYC Open Data APIs

Tue, 27 Jan 2026 00:00:00 +0000

Guest post by Christian Martinez, developer of the nycOpenData package in R.

nycOpenData: A unified R interface to NYC Open Data APIs

I am pleased to announce the release of nycOpenData, an R package providing convenient, tidy access to dozens of datasets from the New York City Open Data platform.

The package is designed as part of an open-science and reproducible-research effort, with the goal of lowering the friction between public data and statistical analysis—especially for teaching, exploratory research, and applied civic work.

Why nycOpenData?

NYC Open Data hosts hundreds of datasets covering topics such as public safety, housing, transportation, education, health, and city services. While these datasets are publicly accessible through the Socrata API, working with them directly often requires:

knowing dataset identifiers,
manually constructing API queries,
handling pagination, timeouts, and rate limits,
and performing repetitive data-cleaning steps.

These barriers can slow down exploratory analysis and make public data less accessible to students, researchers, and practitioners who primarily work in R.

nycOpenData was built to remove these obstacles by providing a consistent, user-friendly interface that returns clean tibbles ready for analysis—without requiring users to interact directly with the API.

What does the package do?

The package provides a growing collection of wrapper functions, each corresponding to a specific NYC Open Data dataset or dataset family. All functions follow a shared design pattern and support:

row limits,
optional filtering via named lists,
sorting,
and graceful handling of API errors and timeouts.

Examples of currently supported domains include:

311 service requests
Transportation and for-hire vehicles
Motor vehicle collisions
Department of Buildings permits and complaints
Education and school reporting
Juvenile justice and public safety
Street trees and environmental data
Permitted events (historical)

A typical call looks like this:

library(nycOpenData)

nyc_311(
  limit = 1000,
  filters = list(borough = "BROOKLYN")
)

## # A tibble: 1,000 × 40
##    unique_key created_date          agency agency_name complaint_type descriptor
##    <chr>      <chr>                 <chr>  <chr>       <chr>          <chr>     
##  1 67613985   2026-01-26T02:06:05.… NYPD   New York C… Noise - Resid… Banging/P…
##  2 67609553   2026-01-26T02:02:09.… NYPD   New York C… Noise - Resid… Banging/P…
##  3 67610990   2026-01-26T01:58:58.… NYPD   New York C… Illegal Parki… Blocked H…
##  4 67615428   2026-01-26T01:56:49.… NYPD   New York C… Noise - Resid… Banging/P…
##  5 67609568   2026-01-26T01:48:16.… NYPD   New York C… Noise - Resid… Loud Musi…
##  6 67612476   2026-01-26T01:47:10.… NYPD   New York C… Noise - Resid… Loud Musi…
##  7 67614152   2026-01-26T01:46:26.… DSNY   Department… Snow or Ice    Snow Trac…
##  8 67614054   2026-01-26T01:44:50.… DSNY   Department… Dirty Conditi… Trash     
##  9 67606570   2026-01-26T01:41:32.… NYPD   New York C… Noise - Resid… Banging/P…
## 10 67610091   2026-01-26T01:35:51.… NYPD   New York C… Noise - Vehic… Car/Truck…
## # ℹ 990 more rows
## # ℹ 34 more variables: location_type <chr>, incident_zip <chr>,
## #   incident_address <chr>, street_name <chr>, cross_street_1 <chr>,
## #   cross_street_2 <chr>, intersection_street_1 <chr>,
## #   intersection_street_2 <chr>, address_type <chr>, city <chr>,
## #   landmark <chr>, status <chr>, community_board <chr>,
## #   council_district <chr>, police_precinct <chr>, bbl <chr>, borough <chr>, …

The result is returned as a tidy tibble of the 1,000 most recent NYC 311 requests, making it immediately compatible with the tidyverse ecosystem for visualization, modeling, and reporting.

Mini analysis

One of the strongest qualities this function has is its ability to filter based on multiple columns. Let’s put everything together and get a dataset of the last 1,000 311 requests from the New York Police Department in Brooklyn.

# Creating the dataset
brooklyn_nypd <- nyc_311(limit = 1000, filters = list(agency = "NYPD", borough = "BROOKLYN"))

# Calling head of our new dataset
head(brooklyn_nypd)

## # A tibble: 6 × 39
##   unique_key created_date           agency agency_name complaint_type descriptor
##   <chr>      <chr>                  <chr>  <chr>       <chr>          <chr>     
## 1 67613985   2026-01-26T02:06:05.0… NYPD   New York C… Noise - Resid… Banging/P…
## 2 67609553   2026-01-26T02:02:09.0… NYPD   New York C… Noise - Resid… Banging/P…
## 3 67610990   2026-01-26T01:58:58.0… NYPD   New York C… Illegal Parki… Blocked H…
## 4 67615428   2026-01-26T01:56:49.0… NYPD   New York C… Noise - Resid… Banging/P…
## 5 67609568   2026-01-26T01:48:16.0… NYPD   New York C… Noise - Resid… Loud Musi…
## 6 67612476   2026-01-26T01:47:10.0… NYPD   New York C… Noise - Resid… Loud Musi…
## # ℹ 33 more variables: location_type <chr>, incident_zip <chr>,
## #   incident_address <chr>, street_name <chr>, cross_street_1 <chr>,
## #   cross_street_2 <chr>, intersection_street_1 <chr>,
## #   intersection_street_2 <chr>, address_type <chr>, city <chr>,
## #   landmark <chr>, status <chr>, community_board <chr>,
## #   council_district <chr>, police_precinct <chr>, bbl <chr>, borough <chr>,
## #   x_coordinate_state_plane <chr>, y_coordinate_state_plane <chr>, …

# Quick check to make sure our filtering worked
nrow(brooklyn_nypd)

## [1] 1000

unique(brooklyn_nypd$agency)

## [1] "NYPD"

unique(brooklyn_nypd$borough)

## [1] "BROOKLYN"

We successfully created our dataset that contains the 1,000 most recent requests regarding the NYPD in the borough Brooklyn.

Now that we have successfully pulled the data and have it in R, let’s figure out what NYC residents in Brooklyn are complaining about to the NYPD.

To do this, we will create a bar graph of the complaint types.

# Visualizing the distribution, ordered by frequency
library(ggplot2)

ggplot(brooklyn_nypd, aes(y = reorder(complaint_type, complaint_type, length))) +
  geom_bar(fill = "steelblue") +
  theme_minimal() +
  labs(
    title = "Most Recent NYPD 311 Complaints (Brooklyn)",
    subtitle = "Top 1,000 service requests",
    x = "Number of Complaints",
    y = "Type of Complaint"
  )

Figure 1: Bar chart showing the frequency of NYPD-related 311 complaint types in Brooklyn from the 1,000 most recent service requests.

This graph shows us not only which complaints were made, but how many of each complaint were made.

Designed for reproducible workflows

A core design principle of nycOpenData is reproducibility. Rather than downloading static CSV files that can change over time or be accidentally modified, analyses can explicitly document:

which dataset was used,
how many rows were requested,
which filters were applied,
and when the data were accessed.

This makes the package particularly useful for:

reproducible research projects,
classroom assignments,
data journalism,
and exploratory civic analysis.

The package is also designed to be API-polite, with configurable timeouts and safeguards that help prevent common failure modes when querying large public datasets.

Who is it for?

nycOpenData is intended for a broad audience, including:

students learning statistics or data science using real-world data,
instructors teaching reproducible research or applied data analysis,
researchers conducting exploratory or descriptive analyses,
data journalists and civic technologists,
and anyone interested in working with NYC public data in R.

The goal is not to abstract away the data itself, but to make access predictable, transparent, and easy to integrate into standard R workflows.

Availability

The package is available on CRAN and can be installed using:

install.packages("nycOpenData")

Development continues on GitHub, where new datasets and improvements are added regularly.

Acknowledgements

This package was developed alongside teaching and applied research projects in reproducible data science, with inspiration from open-source contributors across the R community and the NYC Open Data program.

Useful links

CRAN package: https://CRAN.R-project.org/package=nycOpenData
pkgdown site: https://martinezc1.github.io/nycOpenData/
GitHub repository: https://github.com/martinezc1/nycOpenData
NYC Open Data portal: https://opendata.cityofnewyork.us/

As always, feedback, bug reports, and dataset requests are very welcome.

Thanks for reading!

AssociationExplorer: A user-friendly shiny application for exploring associations and visual patterns

Tue, 16 Dec 2025 00:00:00 +0000

I am pleased to announce the publication of our paper “AssociationExplorer: A user-friendly Shiny application for exploring associations and visual patterns” in the journal SoftwareX, together with the official release of the AssociationExplorer2 R package on CRAN.

Both the paper and the software are part of an open-science effort aimed at making exploratory data analysis more accessible to non-technical users.

Why AssociationExplorer?

Exploring multivariate datasets is now central in social sciences, data journalism, and education. However, identifying and interpreting associations between variables often requires programming skills and a solid background in statistics, which can represent a substantial barrier for many users.

AssociationExplorer was designed to lower this barrier by providing an interactive, visual, and statistically grounded tool for exploring associations between quantitative and qualitative variables, without requiring users to write any code.

The application is primarily intended for:

journalists and data journalism practitioners,
teachers and students,
researchers in the exploratory phase of an analysis,
engaged citizens interested in understanding public or survey data.

What does the app do?

AssociationExplorer follows a simple and guided workflow:

Import data (CSV or Excel files)
Interactively select variables of interest
Automatically compute association measures adapted to variable types:
- Pearson’s $r$ correlation for numeric–numeric pairs,
- Cramer’s V for categorical–categorical pairs,
- the correlation ratio $\eta$ for mixed numeric–categorical pairs
Filter associations using user-defined thresholds
Visualize results through:
- an interactive correlation network,
- contextual bivariate visualizations (scatter plots, mean plots, and colored contingency tables)

This workflow is designed to support transparent, reactive, and interpretable exploratory data analysis.

A paper published in SoftwareX

The SoftwareX paper provides a detailed description of:

the motivation and intended audience of the tool,
the software architecture,
the methodological choices underlying the association measures,
an illustrative case study based on the European Social Survey,
and perspectives for future development.

Link to the paper: https://doi.org/10.1016/j.softx.2025.102483

In line with the journal’s standards, the code, documentation, and example data are fully open and reproducible.

The R package is also on CRAN

Alongside the paper, an R package is also available on CRAN, making installation and use straightforward:

install.packages("AssociationExplorer2")
library(AssociationExplorer2)
run_associationexplorer()

The CRAN release ensures:

standardized installation,
better integration with existing R workflows,
clearer versioning and dependency management.

Who is it for, and how can it be used?

AssociationExplorer is particularly useful for:

exploratory analysis prior to formal modeling,
teaching concepts related to association and dependence,
data storytelling and journalistic exploration,
the analysis of survey data and public datasets.

The goal is not to replace confirmatory statistical analyses, but to provide a robust tool for understanding the structure of the data before modeling.

Acknowledgements

This work was carried out in collaboration with Cédric Heuchenne, Arnaud Claes, and Antonin Descampe, whom I warmly thank.

The project is supported by the Walloon Region and SPW Recherche within the ODALON research project.

Useful links

📄 SoftwareX paper: https://doi.org/10.1016/j.softx.2025.102483
📦 CRAN package: https://CRAN.R-project.org/package=AssociationExplorer2
💻 GitHub repository:
- of the paper
- of the R package

As always, feedback, bug reports, and suggestions are very welcome.

Thanks for reading!

References

Soetewey, A., Heuchenne, C., Claes, A. and Descampe, A. (2026). AssociationExplorer: A user-friendly shiny application for exploring associations and visual patterns. SoftwareX, 33(102483). https://doi.org/10.1016/j.softx.2025.102483

Soetewey, A., Heuchenne, C., Claes, A. and Descampe, A. (2025). AssociationExplorer2: A User-Friendly ‘shiny’ Application for Exploring Associations and Visual Patterns. R package version 0.1.4, https://github.com/AntoineSoetewey/AssociationExplorer2

Nonparametric serial interval estimation

Mon, 18 Aug 2025 00:00:00 +0000

Motivation

Epidemiological delays inform about the time between two well-defined events related to a disease. The serial interval (SI) of an infectious disease is defined as the time between symptom onset in a primary case (infector) and symptom onset in a secondary case (infectee). It is a widely used epidemiological delay quantity and plays a central role in mathematical/statistical models of disease transmission. There exists a tight link between the reproduction number (average number of secondary infections generated by an infected individual) and the serial interval. Therefore, getting accurate knowledge about the SI distribution is key to gain a clear understanding of transmission dynamics during outbreaks. Timings of symptom onset for infector-infectee pairs can be obtained from line list data and observations usually consist of calendar dates. From a mathematical perspective, it is more convenient to work with numbers than with calendar dates and the latter are typically transformed to integers for the sake of statistical analysis.

The main challenge when working with SI data is censoring in the sense that exact symptom onset times are usually unobserved and only known to have occurred between two time points. If the time resolution of a reported timing of illness onset is a calendar day, for instance July 15, there is not enough information to determine the exact time of illness onset within that day. As such, symptom onset is assumed to have occurred between July 15 and July 16 and we say that serial interval data are interval-censored. The figure below illustrates the coarse structure of SI data that adds a layer of complexity to the estimation problem.

Source: Gressani O, Hens N. (2025). Nonparametric serial interval estimation with uniform mixtures. PLoS Comput Biol 21(8): e1013338.

A recent article by Gressani and Hens (2025) published in PLOS Computational Biology proposes a new estimator of the cumulative distribution function of the serial interval without making parametric assumptions regarding the underlying SI distribution. The estimator is based on mixtures of uniform distributions and only requires left and right bounds of serial interval windows of infector-infectee pairs as a main input ($s_{iL}$ and $s_{iR}$ in the above figure). Point estimates of different serial interval features are available in closed-form and the bootstrap is used to compute confidence intervals. The nonparametric methodology is relatively simple and computationally fast and stable. Moreover, a user-friendly routine is available in the EpiDelays package written in R. This post aims at giving users a simple first experience with this new nonparametric methodology for serial interval estimation. The package can be installed from GitHub (using devtools) as follows:

install.packages("devtools")
devtools::install_github("oswaldogressani/EpiDelays")

Simulated data

The estimSI() routine of the EpiDelays package can be used to compute nonparametric estimates (point estimates with standard errors and confidence intervals) of different serial interval features (e.g. the mean, median, standard deviation). The routine is simple to use and requires only two inputs:

x: A data frame with $n$ rows (corresponding to the number of transmission pairs for which illness onset data is available) and two columns containing the lower bound of the SI window $s_{iL}$ (first column) and the upper bound of the SI window $s_{iR}$ (second column).
nboot: An integer for the bootstrap sample size (default is 2000) used to construct ($90\%$ and $95\%$) confidence intervals (CIs).

We start by illustrating the use of estimSI() on simulated data. The simSI() routine can be used to simulate artificial serial interval data with SI windows having a width (coarseness) of at least two days. The underlying target SI distribution is assumed to have a Gaussian distribution with mean muS and standard deviation sdS that have to be specified by the user. The code below can be used to generate $n=15$ SI windows from a Gaussian distribution with a mean of $3$ days and standard deviation of $2$ days. More details regarding the data generating mechanism can be found in the article.

set.seed(2025)
simdata <- simSI(muS = 3, sdS = 2, n = 15)
gt::gt(round(simdata, 2))

s	sl	sr	sw
4.24	3	5	2
3.07	2	4	2
4.55	4	6	2
5.54	5	7	2
3.74	3	5	2
2.67	2	4	2
3.79	3	5	2
2.84	2	4	2
2.31	1	5	4
4.40	3	6	3
2.21	1	3	2
-0.51	-2	1	3
2.16	1	3	2
4.53	2	5	3
5.13	3	7	4

The first column of the simulated dataset contains the true (unobserved) serial interval value generated from the chosen Gaussian distribution. The second and third columns contain the left and right bound of the SI window (sl and sr). Finally, the last column contains the width of the observed SI window, i.e. sw=sr-sl. The underlying target SI distribution is specified to be Gaussian with a mean of $3$ days and standard deviation of $2$ days. The 5th, 25th, 75th and 95th quantiles of the latter distribution are:

round(qnorm(p = c(0.05, 0.25, 0.75, 0.95), mean = 3, sd = 2), 1)

## [1] -0.3  1.7  4.3  6.3

We now create a data frame containing sl and sr and use the latter as an input in the estimSI() routine. Nonparametric estimates of different SI features can be accessed with $npestim$ .

xdf <- data.frame(sl = simdata$sl, sr = simdata$sr)
SIfit <- estimSI(x = xdf, nboot = 2000)
gt::gt(round(SIfit$npestim, 1),
  rownames_to_stub = TRUE
)

	mean	sd	q0.05	q0.25	q0.5	q0.75	q0.95
point	3.4	1.7	0.3	2.5	3.5	4.6	6.0
se	0.4	0.3	1.1	0.4	0.3	0.4	0.5
ci90l	2.8	1.1	-1.3	1.8	2.9	3.9	5.0
ci90r	4.0	2.1	2.1	3.2	4.1	5.2	6.6
ci95l	2.7	1.1	-1.3	1.5	2.8	3.8	4.9
ci95r	4.2	2.2	2.1	3.3	4.2	5.4	6.6

The output shows point estimates (point), standard errors (se) and confidence intervals bounds (ci) for the serial interval mean, standard deviation (sd) and 5th, 25th, 50th, 75th and 95th quantiles denoted by q0.05, q0.25, etc. We can also plot the estimated cumulative distribution function (cdf) obtained with the nonparametric approach and compare it with the target Gaussian cdf. The quality of the fit will typically depend on the sample size $n$ and on the degree of coarseness present in the data. Note that the nonparametric methodology naturally deals with negative SI values.

sl <- simdata$sl
sr <- simdata$sr
Fhat <- function(s) (1 / SIfit$n) * sum((s - sl) / (sr - sl) * (s >= sl & s <= sr) + (s > sr))
sf <- seq(-3, 8, length = 100)
plot(sf, sapply(sf, Fhat), type = "l", lwd = 2, xlab = "Serial interval", ylab = "cdf")
grid()
lines(sf, pnorm(sf, mean = 3, sd = 2), col = "blue", lwd = 2)
legend("topleft", c("Estimated cdf of SI", "Target cdf of SI"), col = c("black", "blue"), lwd = c(2, 2), bty = "n")

Real data

Lessler et al. (2009) share a dataset containing serial interval windows obtained from $n=16$ infector-infectee pairs for Influenza A (2009 H1N1 influenza) at a New York City school. The SI windows are directly available from the supplementary appendix of the latter reference and are encoded in a data frame xNY. Nonparametric estimates of serial interval features are then obtained with estimSI().

xNY <- data.frame(sl = c(1, 1, 1, 0, 0, 4, 2, 3, 0, 3, 0, 3, 4, 1, 3, 3), sr = c(3, 3, 3, 2, 2, 6, 4, 5, 2, 5, 2, 5, 6, 3, 5, 5))
set.seed(123)
SIfitNY <- estimSI(xNY, nboot = 2000)
gt::gt(round(SIfitNY$npestim, 1),
  rownames_to_stub = TRUE
)

	mean	sd	q0.05	q0.25	q0.5	q0.75	q0.95
point	2.8	1.5	0.4	1.5	2.8	4.1	5.2
se	0.3	0.1	0.2	0.4	0.6	0.4	0.3
ci90l	2.2	1.3	0.2	1.1	1.8	3.2	4.7
ci90r	3.4	1.7	1.1	2.2	3.7	4.6	5.6
ci95l	2.1	1.2	0.2	1.0	1.8	3.0	4.6
ci95r	3.5	1.7	1.1	2.5	3.8	4.7	5.7

Interested readers can find more real data examples in Gressani and Hens (2025) and learn about the strengths and limitations of this new nonparametric methodology for serial interval estimation.

References

Gressani, O. and Hens, N. (2025). Nonparametric serial interval estimation with uniform mixtures. PLoS Computational Biology 21(8):e101338. https://doi.org/10.1371/journal.pcbi.1013338

Gressani; O. (2025). EpiDelays: A Software for Estimation of Epidemiological Delays (version 0.0.1). https://github.com/oswaldogressani/EpiDelays

Lessler, J., Reich, N. G., Cummings, D. A., and the New York City Department of Health and Mental Hygiene Swine Influenza Investigation Team. (2009). Outbreak of 2009 pandemic influenza A (H1N1) at a New York City school. New England Journal of Medicine 361(27), 2628-2636. https://www.nejm.org/doi/full/10.1056/NEJMoa0906089

EpiLPS for estimation of incubation times

Thu, 01 Aug 2024 00:00:00 +0000

Motivation

A group of researchers from the Data Science Institute (DSI) at Hasselt University developed a new statistical model to estimate the incubation period of a pathogenic organism based on coarse data. The incubation period of an infectious disease (defined as the time elapsed between infection and the manifestation of first symptoms) is of great importance as it permits to shed light on the epidemic potential of a disease and to optimize the length of quarantine periods to freeze transmission. The article (Gressani et al. 2024) was recently published in the American Journal of Epidemiology with practical implementation of the methodology accessible through the EpiLPS package (Gressani et al. 2022).

Coarse data

What makes estimation of incubation times so challenging in the first place? The devil lies in the data. True infection times are stealthy and rarely observed. In information-theoretic jargon this phenomenon is called “imperfect information’’ but statisticians prefer to call it censoring. To be more precise, infection times are interval censored, i.e. one part of the collected data contains exposure intervals $\mathcal{E}=[t^{E_L},t^{E_R}]$ reported by individuals that are part of the study, where $t^{E_L}$ and $t^{E_R}$ stand for the left and right bound, respectively, of the exposure window. The other part of the data contains symptom onset times $t^{\mathcal{S}}$. This is a more easily accessible piece of information -people tend to remember the day when first symptoms appeared- and so the timing of symptom onset is assumed to be exactly observed. Subtracting the exposure bounds from the symptom onset time, one obtains the incubation interval $\mathcal{I}=[t^{\mathcal{I}_L}, t^{\mathcal{I}_R}]$ with lower bound $t^{\mathcal{I}_L}=t^{\mathcal{S}}-t^{E_R}$ and upper bound $t^{\mathcal{I}_R}=t^{\mathcal{S}}-t^{E_L}$, characterizing the coarse data structure which will be the main model input.

Simulated example

In EpiLPS, the estimIncub() routine is designed to compute an estimate of the incubation density based on the methodology of Gressani et al. (2024). Giving a detailed account of the methodology would be out of scope for this blog and the reader is redirected to the article for technicalities. In a nutshell, it is a Bayesian approach making use of (penalized) B-splines, Laplace approximations and Markov chain Monte Carlo (MCMC) methods to derive a semi-parametric estimate of the incubation density. An attractive feature of the estimIncub() routine for the end-user is the minimalistic input it requires to work, namely:

x: A data frame containing the lower and upper bound of the incubation interval.
K: An integer specifying the number of B-splines to smooth the incubation density.
niter: The number of MCMC samples required.

In practice, only the data frame x is required as the remaining inputs are assigned default values. Computationally, the routine requires a small amount of resources as costly subroutines are coded in C++ and integrated in R via the Rcpp package. The structure of x is quite simple. It should be a data frame with two columns containing the left bound $t^{\mathcal{I}_L}$ of the incubation interval (in the first column) and the right bound $t^{\mathcal{I}_R}$.

Let’s start with a simple example where x is simulated based on a data generating mechanism assuming a known incubation distribution. This can be achieved with the incubsim() routine in EpiLPS. We choose x to be generated according to a Lognormal incubation distribution with a mean of 5.5 days and a standard deviation of 2.1 days following Ferretti et al. (2020). Simulation of $n=40$ observations with an average exposure window of 2 days is implemented as follows:

set.seed(2024)
simdat <- incubsim(incubdist = "LogNormal", n = 40, coarseness = 2)
gt(head(simdat$Dobsincub, 5))

tL	tR
3.724500	5.510304
4.381377	6.654224
4.588020	5.483614
3.847835	6.003948
3.522282	5.033487

By typing simdat$Dobsincub, the user has access to the generated incubation intervals (expressed in days), corresponding here to a data frame with two columns and $n=40$ rows. This is the data frame that is injected in the estimIncub() routine:

fit <- estimIncub(x = simdat$Dobsincub, verbose = TRUE)

## ----------------------------------------------------------------
## Time elapsed: 1.339 seconds.
## Fitted density is Log-Normal with meanlog=1.617 and sdlog=0.317.
## Mean incubation period (days): 5.298 with 95% CI: 5.086-5.615.
## 95th percentile (days): 8.484 with 95% CI: 8.121-9.089.
## ----------------------------------------------------------------

The output in the R console can easily be interpreted. It tells us that the model chooses a Lognormal density fit for the incubation period with a mean of 5.3 days (95% CI: 5.0-5.6 days). More detailed summary statistics are accessible by typing fit$stats, such as the posterior standard deviation and additional percentiles. What happens under the hood? Basically, the model computes a semi-parametric fit to the data and compares it with classic parametric fits (Lognormal, Weibull and Gamma) used for incubation modeling. The candidate with the lowest Bayesian information criterion (BIC) wins the game and is finally selected (here the Lognormal distribution). The incubation windows and the fitted incubation density can be obtained by typing:

grid.arrange(plot(fit, typ = "incubwin"), plot(fit, type = "pdf"), nrow = 1)

Real data example

The flexible Bayesian methodology is illustrated on SARS-CoV-2 symptom onset and exposure window data extracted from cases in Vietnam. The dataset was analyzed in Bui et al. (2020) and is publicly available on the GitHub repository provided in the article (https://github.com/longbui/Covid19IncubVN; last accessed July 17, 2024). The dataset contains information about $n=19$ cases identified from January 23, 2020 to April 13, 2020. After continuity corrections (required to change calendar dates into continuous time points), the left and right incubation bounds are given by:

# Left incubation bound
tL <- c(0.504, 2.983, 5.343, 6.969, 6.990, 2.570, 6.870, 1.263, 0.693, 1.869, 1.151, 2.748, 1.209, 1.161, 4.982, 4.017, 2.170, 9.805, 1.659)

# Right incubation bound
tR <- c(1.491, 7.762, 11.777, 12.821, 11.359, 8.551, 24.954, 4.144, 6.408, 7.083, 4.191, 10.326, 9.942, 7.254, 12.690, 10.825, 14.478, 16.592, 18.649)

# Data set
dataVietnam <- data.frame(tL = tL, tR = tR)

The estimIncub() routine provides a Weibull fit with a mean incubation period of 6.7 days (95% CI: 5.8-7.4 days) and the standard deviation (extracted by typing incubfit$stats) is 3.3 days (95% CI: 3.0-3.8 days). A figure of the incubation windows and the fitted Weibull density (with 95% credible interval) is also provided.

set.seed(2024)
incubfit <- EpiLPS::estimIncub(x = dataVietnam, verbose = TRUE, tmax = 25)

## ----------------------------------------------------------------
## Time elapsed: 1.13 seconds.
## Fitted density is Weibull with shape=2.142 and scale=7.521.
## Mean incubation period (days): 6.661 with 95% CI: 5.753-7.375.
## 95th percentile (days): 12.552 with 95% CI: 11.931-14.087.
## ----------------------------------------------------------------

grid.arrange(plot(incubfit, typ = "incubwin"), plot(incubfit, type = "pdf"), nrow = 1)

References

Gressani, O., Torneri, A., Hens, N. and Faes, C. (2024). Flexible Bayesian estimation of incubation times. American Journal of Epidemiology (Accepted manuscript). https://doi.org/10.1093/aje/kwae192

Gressani, O., Wallinga, J., Althaus, C. L., Hens, N. and Faes, C. (2022). EpiLPS: A fast and flexible Bayesian tool for estimation of the time-varying reproduction number. PLoS Comput Biol 18(10): e1010618. https://doi.org/10.1371/journal.pcbi.1010618

Eddelbuettel, D. and Francois, R. (2011). Rcpp: Seamless R and C++ Integration. Journal of Statistical Software, 40(8), 1–18. https://doi.org/10.18637/jss.v040.i08

Ferretti, L. et al. (2020). Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 368, eabb6936. https://doi.org/10.1126/science.abb6936

Bui LV, Nguyen HT, Levine H, Nguyen HN, Nguyen T-A, Nguyen TP, et al. (2020) Estimation of the incubation period of COVID-19 in Vietnam. PLoS ONE 15(12): e0243889. https://doi.org/10.1371/journal.pone.0243889

Binary logistic regression in R

Tue, 30 Jan 2024 00:00:00 +0000

Introduction

Regression is a common tool in statistics to test and quantify relationships between variables.

The two most common regressions are linear and logistic regressions. A linear regression is used when the dependent variable is quantitative, whereas a logistic regression is used when the dependent variable is qualitative.

Both linear and logistic regressions are divided into different types. Before detailing them, let’s first recap of which type a variable can be:

A quantitative variable measures a quantity, the values it can take are numbers. It is divided into:
- discrete: the values it can take are countable and have a finite number of possibilities (the values are often integers, for example the number of children), and
- continuous: the values it can take are not countable and have an infinite number of possibilities (the values are usually with decimals, or at least decimals are technically possible, for example the weight).
A qualitative variable (also known as categorical) is not numerical and its values fit into categories. It is also divided into two types:
- nominal: no ordering is possible or implied in the categories (for example the sex), and
- ordinal: an order is implied in the categories (for example the health status, such as poor/reasonable/good).

A binary variable, also known as dichotomous, is a special case of qualitative nominal variable when there are only two categories.

See more details and examples about variable types if needed.

Now that the types of a variable is clear, let’s summarize the different types of regression:

Linear regression:
- Simple linear regression is used when the goal is to estimate the relationship between a quantitative continuous dependent variable (also often called outcome or response variable) and only one independent variable (also often called explanatory variable, covariate or predictor) of any type.
- Multiple linear regression is used when the goal is to estimate the relationship between a quantitative continuous dependent variable and two or more independent variables (again, of any type).
Logistic regression:
- Binary logistic regression is used when the goal is to estimate the relationship between a binary dependent variable (= two outcomes), and one or more independent variables (of any type).
- Multinomial logistic regression is used when the goal is to estimate the relationship between a nominal dependent variable with three or more unordered outcomes, and one or more independent variables (of any type).
- Ordinal logistic regression is used when the goal is to estimate the relationship between an ordinal dependent variable with three or more ordered outcomes, and one or more independent variables (of any type).

Note that there exists another type of regression; the Poisson regression. This type of regression is used when the goal is to estimate the relationship between a dependent variable which is in the form of count data (number of occurrences of an event of interest over a given period of time or space, e.g., $0, 1, 2, \ldots$), and one or more independent variables.

Logistic regressions and poisson regressions are both part of a broader type of model called generalized linear models (abbreviated as GLM). The name “generalized linear models” comes from the fact that these models allow to “generalize” the classic linear model. Indeed, it can be used in many situations, for example when analyzing a dependent variable which is not necessarily quantitative continuous or when residuals are not normally distributed (which are prerequisites for a linear model).

Linear regression and its application in R have already been presented in this post. It is now time to present the logistic regression.

Binary logistic regression being the most common and the easiest one to interpret among the different types of logistic regression, this post will focus only on the binary logistic regression. Other types of regression (multinomial & ordinal logistic regressions, as well as Poisson regressions are left for future posts).

In this post, we will first explain when a logistic regression is more appropriate than a linear regression. We will then show how to perform a binary logistic regression in R, and how to interpret and report results. We will also present some plots in order to visualize results. Finally, we will cover the topics of model selection, quality of fit and underlying assumptions of a binary logistic regression. We will try to keep this tutorial as applied as possible by focusing on the applications in R and the interpretations. Mathematical details will be as concise as possible.

Linear versus logistic regression

We know that a linear regression is a convenient way to estimate the relationship between a quantitative continuous dependent variable, and one or more independent variables (of any type).

For instance, suppose we would like to estimate the relationship between two quantitative variables, $X$ and $Y$. Using the ordinary least squares method (the most common estimator used in linear regression), we obtain the following regression line:

Now suppose we are interested in estimating the impact of age on whether or not a patient has a certain disease. Age is considered as a quantitative continuous variable, while having the disease is binary (a patient is either ill or healthy).

Visually, we could have something like this:

If we fit a regression line (using the ordinary least square method) to the points, we obtain the following plot:

We see that the regression line goes below 0 and above 1 with respect to the $y$-axis. Since the dependent variable disease cannot take values below 0 (= healthy) nor above 1 (= ill), it is obvious that a linear regression is not appropriate for these data!

In addition to this limitation, the assumptions of normality and homoscedasticity, which are required in linear regression, are clearly not appropriate with these data since the dependent variable is binary and follows a Binomial distribution! R will not stop you from performing a linear regression on binary data, but this will produce a model of little interest.

This is where a logistic regression becomes handy as it takes into consideration these limitations.

Applied to our example, here is how the points are fitted using a binary logistic regression:

It is clear that this model is more appropriate.

The curve (known as a sigmoid) is obtained via a transformation of the predicted values. There are several possible choices for the link function, which aim is to constrain predicted values to be within the range of observed values.

The most widely used in practice is the logit function, which relates the probability of occurrence of an event (bounded between 0 and 1) to the linear combination of independent variables. The logit function also turns out to be the canonical link function for a Bernoulli or Binomial distribution. This transformation ensures that no matter in which range the $X$ values are located, $Y$ will only take numbers between 0 and 1.

One could say that the fitted values (= represented by the blue curve) taking values between 0 and 1 also does not seem to make sense since a patient can only be healthy or ill (and thus the dependent variable can only take the value 0 or 1, respectively). However, in a binary logistic regression it is not the outcome no disease/disease that is directly modeled, but the likelihood that a patient has the disease or not given his or her characteristics. This likelihood will be framed in terms of a probability to observe or not the disease in a patient, which is indeed included between 0 and 1, or 0% and 100%.

More generally, with a logistic regression we would like to model how the probability of success varies with the independent variables and determine whether or not these changes are statistically significant. We are actually going to model the logarithm of the odds, and the logistic regression model will be written as follows:

\[\begin{align} \log(odds(success)) &= logit(\pi) \\ &= \log\left(\frac{\pi}{1 - \pi}\right) \\ &= \beta_0 + \beta_1 X_1 + \cdots + \beta_p X_p \end{align}\]

where $\pi$ $(0 \le \pi \le 1)$ is the probability of an event happening (success) and denoted $\pi = P(success)$. We find the values of $\hat{\beta}_0$, $\hat{\beta}_1$, $\ldots$, $\hat{\beta}_p$, which are used as estimates for $\beta_0$, $\beta_1$, $\ldots$, $\beta_p$, using the maximum likelihood method. This method is one of several methods used in statistics to estimate parameters of a mathematical model. The goal of the estimator is to estimate the parameters $\beta_0$, $\beta_1$, $\ldots$, $\beta_p$ which maximize the log likelihood function. Different algorithms have been established over the years for this non-linear optimization, but this is beyond the scope of the post.

Logistic regressions are very common in the medical field, for example to:

estimate the risk factors associated with a disease or a harmful condition,
predict the risk of developing a disease based on a patient’s characteristics, or
determine the most important biological factors associated with a specific disease or condition.

However, logistic regressions are used in many other domains, for instance in:

banking sector: estimate a debtor’s creditworthiness based on his or her profile (income, assets, liabilities, etc.),
marketing: estimate a customer’s propensity to buy a product or service based on his or her profile (age, sex, salary, previous purchases, etc.),
sports: estimate the probability of a player winning against another player as a function of the characteristics of the two opponents,
politics: answer the question “Would a citizen vote for our political party at the next elections?”
etc.

Univariable versus multivariable logistic regression

Now that it is more clear when a binary logistic regression should be used, we show how to perform one in R. We start by presenting univariable binary logistic regressions, and then multivariable binary logistic regressions.

Remember that in both cases, the dependent variable must be a qualitative variable with two outcomes (hence the name binary logistic regression). The difference between a univariable and multivariable binary logistic regression lies in the fact that:

for a univariable binary logistic regression, there is only one independent variable, while
for a multivariable binary logistic regression, there are two ore more independent variables.

It is true that the term “univariable” may be confusing here because there are two variables in the model (i.e., one dependent variable and one independent variable). However, it is called univariable binary logistic regression to indicate that only one independent variable is considered in the model, as opposed to multivariable binary logistic regression where several independent variables are considered in the model.

To draw a parallel with linear regression:

a univariable binary logistic regression is the equivalent of a simple linear regression, whereas
a multivariable binary logistic regression is the equivalent of a multiple linear regression

when the dependent variable is binary instead of quantitative continuous. This is the reason a univariable binary logistic regression is sometimes called a simple binary logistic regression and a multivariable binary logistic regression sometimes called a multiple binary logistic regression.

The terms univariable/multivariable should not be confused with univariate/multivariate. The number of dependent variables characterizes the model as univariate or multivariate; univariate refers to a model with only one dependent variable, while multivariate refers to a model that simultaneously predicts more than one dependent variable. Usually, the intent is to differentiate models based on the number of independent variables. This distinction is made thanks to the terms univariable and multivariable. Multivariable refers to a model relating multiple predictor variables to a dependent variable, whereas univariable refers to a model relating one single independent variable to a dependent variable.

Data

For these illustrations, we use the “Heart Disease” dataset, available from the {kmed} R package. This data frame consists of 14 variables, of which only 5 of them are kept for this post:

age: age in years
sex: sex (FALSE = female, TRUE = male)
cp: chest pain type (1 = typical angina, 2 = atypical angina, 3 = non-anginal pain, 4 = asymptomatic)
thalach: maximum heart rate achieved
class: diagnosis of heart disease (divided into 4 classes)

# import and rename dataset
library(kmed)
dat <- heart

# select variables
library(dplyr)
dat <- dat |>
  select(
    age,
    sex,
    cp,
    thalach,
    class
  )

# print dataset's structure
str(dat)

## 'data.frame':	297 obs. of  5 variables:
##  $ age    : num  63 67 67 37 41 56 62 57 63 53 ...
##  $ sex    : logi  TRUE TRUE TRUE TRUE FALSE TRUE ...
##  $ cp     : Factor w/ 4 levels "1","2","3","4": 1 4 4 3 2 2 4 4 4 4 ...
##  $ thalach: num  150 108 129 187 172 178 160 163 147 155 ...
##  $ class  : int  0 2 1 0 0 0 3 0 2 1 ...
##  - attr(*, "na.action")= 'omit' Named int [1:6] 88 167 193 267 288 303
##   ..- attr(*, "names")= chr [1:6] "88" "167" "193" "267" ...

Note that the pipe operator |> and the {dplyr} package is used to select variables. See more data manipulation techniques using this package if you are interested.

For greater readability, we rename the variables cp, thalach and class with more informative names:

# rename variables
dat <- dat |>
  rename(
    chest_pain = cp,
    max_heartrate = thalach,
    heart_disease = class
  )

We transform the variables sex and chest_pain into factor and set the labels accordingly:

# recode sex
dat$sex <- factor(dat$sex,
  levels = c(FALSE, TRUE),
  labels = c("female", "male")
)

# recode chest_pain
dat$chest_pain <- factor(dat$chest_pain,
  levels = 1:4,
  labels = c("typical angina", "atypical angina", "non-anginal pain", "asymptomatic")
)

For a binary logistic regression in R, it is recommended that all the qualitative variables are transformed into factors.

In our case, heart_disease (our dependent variable) is currently encoded as integer with values ranging from 0 to 4. Therefore, we first classify it into 2 classes by setting 0 for 0 values and 1 for non-0 values, using the ifelse() function:

# recode heart_disease into 2 classes
dat$heart_disease <- ifelse(dat$heart_disease == 0,
  0,
  1
)

We then transform it into a factor and set the labels accordingly using the factor() function:

# set labels for heart_disease
dat$heart_disease <- factor(dat$heart_disease,
  levels = c(0, 1),
  labels = c("no disease", "disease")
)

Keep in mind the order of the levels for your dependent variable, as it will have an impact on the interpretations. In R, the first level given by levels() is always taken as the reference level.

In our case, the first level is the absence of the disease and the second level is the presence of the disease:

levels(dat$heart_disease)

## [1] "no disease" "disease"

This means that when we will build the models, we will estimate the impact of the independent variable(s) on the presence of the disease (and not the absence!).

This is the reason that, for dependent variables of the type no/yes, false/true, absence/presence of a condition, etc. it is recommended to set the level no, false, absence of the condition, etc. as the reference level. It is indeed usually easier to interpret the impact of an independent variable on the presence of a condition/disease than the opposite.

If you want to switch the reference level, this can be done with the relevel() function.

Here is a preview of the final data frame and some basic descriptive statistics:

# print first 6 observations
head(dat)

##   age    sex       chest_pain max_heartrate heart_disease
## 1  63   male   typical angina           150    no disease
## 2  67   male     asymptomatic           108       disease
## 3  67   male     asymptomatic           129       disease
## 4  37   male non-anginal pain           187    no disease
## 5  41 female  atypical angina           172    no disease
## 6  56   male  atypical angina           178    no disease

# basic descriptive statistics
summary(dat)

##       age            sex                 chest_pain  max_heartrate  
##  Min.   :29.00   female: 96   typical angina  : 23   Min.   : 71.0  
##  1st Qu.:48.00   male  :201   atypical angina : 49   1st Qu.:133.0  
##  Median :56.00                non-anginal pain: 83   Median :153.0  
##  Mean   :54.54                asymptomatic    :142   Mean   :149.6  
##  3rd Qu.:61.00                                       3rd Qu.:166.0  
##  Max.   :77.00                                       Max.   :202.0  
##     heart_disease
##  no disease:160  
##  disease   :137  
##                  
##                  
##                  
##

The data frame is now ready to be analyzed further through univariable and multivariable binary logistic regressions.

Binary logistic regression in R

Univariable binary logistic regression

As mentioned above, we start with a univariable binary logistic regression, that is, a binary logistic regression with only one independent variable.

In R, a binary logistic regression can be done with the glm() function and the family = "binomial" argument. Similar to linear regression, the formula used inside the function must be written as dependent variable ~ independent variable (in this order!).

While the dependent variable must be categorical with two levels, the independent variable can be of any type. However, interpretations differ depending on whether the independent variable is qualitative or quantitative.

For completeness, we illustrate this type of regression with both a quantitative and a qualitative independent variable, starting with a quantitative independent variable.

Quantitative independent variable

Suppose we want to estimate the impact of a patient’s age on the presence of heart disease. In this case, age is our independent variable and heart_disease is our dependent variable:

# save model
m1 <- glm(heart_disease ~ age,
  data = dat,
  family = "binomial"
)

Results of the model is saved under the object m1. Again, similar to linear regression, results can be accessed thanks to the summary() function:

# print results
summary(m1)

## 
## Call:
## glm(formula = heart_disease ~ age, family = "binomial", data = dat)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -3.05122    0.76862  -3.970  7.2e-05 ***
## age          0.05291    0.01382   3.829 0.000128 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 409.95  on 296  degrees of freedom
## Residual deviance: 394.25  on 295  degrees of freedom
## AIC: 398.25
## 
## Number of Fisher Scoring iterations: 4

The most important results in this output are displayed in the table after Coefficients. The bottom part of the output summarizes the distribution of the deviance residuals. In a nutshell, deviance residuals measure how well the observations fit the model. The closer the residual to 0, the better the fit of the observation.

Within the Coefficients table, we focus on the first and last columns (the other two columns correspond to the standard error and the test statistic, which are both used to compute the $p$-value):

the column Estimate corresponds to the coefficients $\hat{\beta}_0$ and $\hat{\beta}_1$, and
the column Pr(>|z|) corresponds to the $p$-values.

R performs a hypothesis test for each coefficient, that is, $H_0: \beta_j = 0$ versus $H_1: \beta_j \neq 0$ for $j = 0, 1$ via the Wald test, and print the $p$-values in the last column. We can thus compare these $p$-values to the chosen significance level (usually $\alpha = 0.05$) to conclude whether or not each of the coefficient is significantly different from 0. The lower the $p$-value, the more evidence that the coefficient is different from 0. This is similar to linear regression.

Coefficients are slightly harder to interpret in logistic regression than in linear regression because the relationship between dependent and independent variables is not linear.

Let’s first interpret the coefficient of age, $\hat{\beta}_1$, which is the most important coefficient of the two.

First, since the $p$-value of the test on the coefficient for age is < 0.05, we conclude that it is significantly different from 0, which means that age is significantly associated with the presence of heart disease (at the 5% significance level). Note that if the test was not significant (i.e., the $p$-value $\ge \alpha$), we would refrain from interpreting the coefficient since it means that, based on the data at hand, we are unable to conclude that age is associated with the presence of heart disease in the population.

Second, remember that:

when $\beta_1 = 0$, $X$ and $Y$ are independent,
when $\beta_1 > 0$, the probability that $Y = 1$ increases with $X$, and
when $\beta_1 < 0$, the probability that $Y = 1$ decreases with $X$.

In our context, we have:

when $\hat{\beta}_1 = 0$, the probability of developing a heart disease is independent of the age,
when $\hat{\beta}_1 > 0$, the probability of developing a heart disease increases with age, and
when $\hat{\beta}_1 < 0$, the probability of developing a heart disease decreases with age.

Here we have $\hat{\beta}_1 =$ 0.053 $> 0$, so we already know that the older the patient, the more likely he or she is to develop a heart disease. This makes sense.

Now that we know the direction of the relationship, we would like to quantify this relationship. This is easily done thanks to odds ratios (OR). OR are found by taking the exponential of the coefficients. In R, the exponential is done thanks to the exp() function.

OR can be interpreted as follows: the OR is the multiplicative change in the odds in favor of $Y = 1$ when $X$ increases by 1 unit.

Applied to our context, we compute the OR for the age by computing $\exp(\hat{\beta}_1) =$ exp(0.053).

Using R, this gives:

# OR for age
exp(coef(m1)["age"])

##      age 
## 1.054331

Based on this result, we can say that an extra year of age increases the odds (that is, the chance) of developing a heart disease by a factor of 1.054.

Therefore, the odds of developing a heart disease increases by (1.054 - 1) $\times$ 100 = 5.4% when a patient becomes one year older.

To sum up:

when the coefficient $\hat{\beta}_1 = 0 \Rightarrow$ OR $= \exp(\hat{\beta}_1) = 1 \Rightarrow P(Y = 1)$ is independent of $X \Rightarrow$ there is no relationship between $X$ and $Y$,
when the coefficient $\hat{\beta}_1 > 0 \Rightarrow$ OR $= \exp(\hat{\beta}_1) > 1 \Rightarrow P(Y = 1)$ increases with $X \Rightarrow$ there is a positive relationship between $X$ and $Y$, and
when the coefficient $\hat{\beta}_1 < 0 \Rightarrow$ OR $= \exp(\hat{\beta}_1) < 1 \Rightarrow P(Y = 1)$ decreases with $X \Rightarrow$ there is a negative relationship between $X$ and $Y$.

We now interpret the intercept $\hat{\beta}_0$.

First, we look at the $p$-value of the test on the intercept. This $p$-value being < 0.05, we conclude that the intercept is significantly different from 0 (at the 5% significance level).

Second, similar to linear regression, in order to obtain an interpretation of the intercept, we need to find a situation in which the other coefficient, $\beta_1$, vanishes.

In our case, it happens when a patient is 0 year old. We may or may not need to interpret results in a such a situation, and in many situations the interpretation does not make sense, so it is more a hypothetical interpretation. However, again for completeness we show how to interpret the intercept. Note that it is also possible to center the numeric variable so that the intercept has a more meaningful interpretation. This, however, goes beyond the scope of the post.

For a patient aged 0 year, the odds of developing a heart disease is $\exp(\hat{\beta}_0) =$ exp(-3.051) = 0.047. When interpreting an intercept, it often makes more sense to interpret it as the probability that $Y = 1$, which can be computed as follows:

\[\frac{\exp(\hat{\beta}_0)}{1 + \exp(\hat{\beta}_0)}.\]

In our case, it corresponds to the probability that a patient of age 0 develops a heart disease, which is equal to:

# prob(heart disease) for age = 0
exp(coef(m1)[1]) / (1 + exp(coef(m1)[1]))

## (Intercept) 
##  0.04516478

This means that, if we trust our model, a newborn is expected to develops a heart disease with a probability of 4.52%.

For your information, a confidence interval can be computed for any of the OR using the confint() function. For example, a 95% confidence interval for the OR for age:

# 95% CI for the OR for age
exp(confint(m1,
  parm = "age"
))

##    2.5 %   97.5 % 
## 1.026699 1.083987

Remember than when evaluating an OR, the null value is 1, not 0. An OR of 1 in this study would mean that there is no association between the age and the presence of heart disease. If the 95% confidence interval of the OR does not include 1, we conclude that there is a significant association between the age and the presence of heart disease. On the contrary, if it includes 1, we do not reject the hypothesis that there is no association between age and the presence of heart disease.

In our case, the 95% CI does not include 1, so we conclude, at the 5% significance level, that there is a significant association between age and the presence of heart disease.

You will notice that it is the same conclusion than with the $p$-value. This is normal, it will always be the case:

if the $p$-value < 0.05, 1 will not be included in the 95% CI, and
if the $p$-value $\ge$ 0.05, 1 will be included in the 95% CI.

This means that you can choose whether you draw your conclusion about the association between the two variables based on the 95% CI or the $p$-value.

Estimating the relationship between variables is the main reason for building models. Another goal is to predict the dependent variable based on newly observed values of the independent variable(s). This can be done with the predict() function.

Suppose we would like to predict the probability of developing a heart disease for a patient aged 30 years old:

# predict probability to develop heart disease
pred <- predict(m1,
  newdata = data.frame(age = c(30)),
  type = "response"
)

# print prediction
pred

##         1 
## 0.1878525

It is predicted that a 30-year-old patient has a 18.79% chance of developing a heart disease.

Note that if you would like to construct a confidence interval for this prediction, it can be done by adding the se = TRUE argument in the predict() function:

# predict probability to develop heart disease
pred <- predict(m1,
  newdata = data.frame(age = c(30)),
  type = "response",
  se = TRUE
)

# print prediction
pred$fit

##         1 
## 0.1878525

# 95% confidence interval for the prediction
lower <- pred$fit - (qnorm(0.975) * pred$se.fit)
upper <- pred$fit + (qnorm(0.975) * pred$se.fit)
c(lower, upper)

##          1          1 
## 0.07873357 0.29697138

If you are a frequent reader of the blog, you are probably know that I like visualizations. The plot_model() function available in the {sjPlot} R package does a good job of visualizing results of the model:

# load package
library(sjPlot)

# plot
plot_model(m1,
  type = "pred",
  terms = "age"
) +
  labs(y = "Prob(heart disease)")

For those of you who are familiar with the {ggplot2} package, you will have noticed that the function accepts layers from the {ggplot2} package. Note also that this function works with other types of model (such as linear models).

The above plot shows the probability of developing a heart disease in function of age, and confirms results found above:

We see that the probability of developing a heart disease increases with age (which was expected given that the OR for the coefficient of age is > 1),
and we also see that the probability of developing a heart disease for a 30-year-old patient is slightly below 20%.

Qualitative independent variable

Suppose now that we are interested in estimating the relationship between the probability of developing a heart disease and the sex (which is a qualitative variable).

Recall that when the independent variable was quantitative, $\exp(\hat{\beta}_1)$ was the multiplicative change in the odds in favor of $Y = 1$ as $X$ increases by 1 unit.

With $X$ being the sex, the only unit increase possible is from 0 to 1 (or from 1 to 2 if sex is encoded as a factor), so we can write an interpretation in terms of female/male:

$\exp(\hat{\beta}_1)$ is the multiplicative change of the odds in favor of $Y = 1$ as a female becomes a male.

Again, keep in mind what is the order of the level for the variable sex. In our case, the level female comes before the level male:

# levels for sex
levels(dat$sex)

## [1] "female" "male"

So it is indeed the multiplicative change of the odds in favor of $Y = 1$ as a female becomes a male. If the level male came before the level female in our dataset, it would have been the opposite.

You will concede that it is rather strange to interpret odds as a female becomes a male, or vice versa. Therefore, it is better to say:

$\exp(\hat{\beta}_1)$ is the multiplicative change of the odds in favor of $Y = 1$ for males versus females.

In our case, we obtain the following results:

# save model
m2 <- glm(heart_disease ~ sex,
  data = dat,
  family = "binomial"
)

# print results
summary(m2)

## 
## Call:
## glm(formula = heart_disease ~ sex, family = "binomial", data = dat)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  -1.0438     0.2326  -4.488 7.18e-06 ***
## sexmale       1.2737     0.2725   4.674 2.95e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 409.95  on 296  degrees of freedom
## Residual deviance: 386.12  on 295  degrees of freedom
## AIC: 390.12
## 
## Number of Fisher Scoring iterations: 4

# OR for sex
exp(coef(m2)["sexmale"])

##  sexmale 
## 3.573933

which can be interpreted as follows:

The $p$-value of the test on the coefficient for sex is < 0.05, so we conclude that the sex is significantly associated with the presence of heart disease (at the 5% significance level).
Moreover, when looking at the coefficient for sex, $\hat{\beta}_1 =$ 1.274, we can say that:
- For males, the odds of developing a heart disease is multiplied by a factor of exp(1.274) = 3.574 compared to females.
- In other words, the odds of developing a heart disease for males are 3.574 times the odds for females.
- This means that, the odds of developing a heart disease are (3.574 - 1) $\times$ 100 = 257.4% higher for males than for females.

These results again make sense, as it known that men are more likely to have a heart disease than women.

The interpretation of the intercept $\hat{\beta}_0 =$ -1.044 is similar than in the previous section in the sense that it gives the probability of developing a heart disease when the other coefficient, $\beta_1$, is equal to 0.

In our case, $\beta_1 = 0$ simply means that the patient is a female. Therefore, the probability of developing a heart disease for a woman is:

# prob(disease) for sex = female
exp(coef(m2)[1]) / (1 + exp(coef(m2)[1]))

## (Intercept) 
##   0.2604167

The avid reader will notice that a univariable binary logistic regression with a qualitative independent variable will lead to the same conclusion than a Chi-square test of independence:

chisq.test(table(dat$heart_disease, dat$sex))

## 
## 	Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  table(dat$heart_disease, dat$sex)
## X-squared = 21.852, df = 1, p-value = 2.946e-06

Based on this test, we reject the null hypothesis of independence between the two variables and we thus conclude that there is a significant association between the sex and the presence of heart disease ($p$-value < 0.001).

The advantage of a univariable binary logistic regression over a Chi-square test of independence is that it not only tests whether or not there is a significant association between the two variables, but it also estimates the direction and strength of this relationship.

As for a univariable logistic regression with a quantitative independent variable, predictions can also be made with the predict() function. Suppose we would like to predict the probability of developing a heart disease for a male:

# predict probability to develop heart disease
pred <- predict(m2,
  newdata = data.frame(sex = c("male")),
  type = "response"
)

# print prediction
pred

##         1 
## 0.5572139

Based on this model, it is predicted that a male patient has 55.72% chance of developing a heart disease.

We can also visualize these results thanks to the plot_model() function:

# plot
plot_model(m2,
  type = "pred",
  terms = "sex"
) +
  labs(y = "Prob(heart disease)")

The points correspond to the predicted probabilities, and the bars correspond to their confidence intervals.

The plot of the results in terms of probabilities confirms what was found above, that is:

women are less likely to develop a heart disease than men,
the probability that a woman develops a heart disease is expected to be slightly above 25%, and
the probability that a man develops a heart disease is expected to be around 55%.

Multivariable binary logistic regression

The interpretation of the coefficients in multivariable logistic regression is similar to the interpretation in univariable regression, except that this time it estimates the multiplicative change in the odds in favor of $Y = 1$ when $X$ increases by 1 unit, while the other independent variables remain unchanged.

This is similar to multiple linear regression, where a coefficient gives the expected change of $Y$ for an increase of 1 unit of $X$, while keeping all other variables constant.

The main advantages of using a multivariable logistic regression compared to a univariable logistic regression are to:

consider the simultaneous (rather than isolated) effect of independent variables,
take into account potential confounding and/or interaction effects, and
improve predictions.

For this illustration, suppose we would like to estimate the relationship between heart disease and all variables present in the data frame, that is, age, sex, chest pain type and maximum heart rate achieved:

# save model
m3 <- glm(heart_disease ~ .,
  data = dat,
  family = "binomial"
)

# print results
summary(m3)

## 
## Call:
## glm(formula = heart_disease ~ ., family = "binomial", data = dat)
## 
## Coefficients:
##                             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                -0.060150   1.962091  -0.031 0.975544    
## age                         0.042814   0.019009   2.252 0.024302 *  
## sexmale                     1.686330   0.349352   4.827 1.39e-06 ***
## chest_painatypical angina  -0.120481   0.641396  -0.188 0.851000    
## chest_painnon-anginal pain -0.124331   0.571093  -0.218 0.827658    
## chest_painasymptomatic      1.963723   0.548877   3.578 0.000347 ***
## max_heartrate              -0.030326   0.007975  -3.802 0.000143 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 409.95  on 296  degrees of freedom
## Residual deviance: 275.26  on 290  degrees of freedom
## AIC: 289.26
## 
## Number of Fisher Scoring iterations: 5

Note that the formula heart_disease ~ . is a shortcut to include all variables present in the data frame in the model as independent variables, except heart_disease.

Based on the $p$-values displayed in the last column of the coefficients table, we conclude that, at the 5% significance level, age, sex and maximum heart rate achieved are all significantly associated with heart disease ($p$-values < 0.05).

For the variables age, sex and max_heartrate, there is only one $p$-value (the result of the test on the nullity of the coefficient).

For the variable chest_pain, 3 $p$-values are displayed. This is normal: similar to linear regression when a categorical variable with more than two levels is included in the model, one test is performed for each comparison between the reference level and the other levels.

In our case, the reference level for the variable chest_pain is typical angina as it comes first:

levels(dat$chest_pain)

## [1] "typical angina"   "atypical angina"  "non-anginal pain" "asymptomatic"

Therefore, a test is performed for the comparison between:

typical angina and atypical angina,
typical angina and non-anginal pain, and
typical angina and asymptomatic.

But here we are not interested in comparing levels of the variable chest pain, we would like to test the overall effect of chest pain on heart disease. For this, we are going to compare two models via a likelihood ratio test (LRT):

a model which includes all the variables of interest and the variable chest_pain, and
the exact same model but which excludes the variable chest_pain.

The first one is referred as the full or complete model, whereas the second one is referred as the reduced model.

We compare these two models with the anova() function:

# save reduced model
m3_reduced <- glm(heart_disease ~ age + sex + max_heartrate,
  data = dat,
  family = "binomial"
)

# compare reduced with full model
anova(m3_reduced, m3,
  test = "LRT"
)

## Analysis of Deviance Table
## 
## Model 1: heart_disease ~ age + sex + max_heartrate
## Model 2: heart_disease ~ age + sex + chest_pain + max_heartrate
##   Resid. Df Resid. Dev Df Deviance  Pr(>Chi)    
## 1       293     325.12                          
## 2       290     275.26  3    49.86 8.558e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Notice that the reduced model must come before the complete model in the anova() function. The null hypothesis of this test is that the two models are equivalent.

At the 5% significance level (see the $p$-value at the right of the R output), we reject the null hypothesis and we conclude that the complete model is significantly better than the reduced model at explaining the presence of heart disease. This means that chest pain is significantly associated with heart disease (which was expected since the comparison between typical angina and asymptomatic was found to be significant).

A comparison of two models via the LRT will be shown again later, when discussing about interactions.

Now that we have shown that all four independent variables were significantly associated with the presence of heart disease, we can interpret the coefficients in order to know the direction of the relationships and most importantly, quantify the strength of these relationships.

Like univariable binary logistic regression, it is easier to interpret these relationships through OR. But this time, we also print the 95% CI of the OR in addition to the OR (rounded to 3 decimals) so that we can easily see which ones are significantly different from 1:

# OR and 95% CI
round(exp(cbind(OR = coef(m3), confint(m3))), 3)

##                               OR 2.5 % 97.5 %
## (Intercept)                0.942 0.020 44.353
## age                        1.044 1.006  1.084
## sexmale                    5.400 2.776 10.971
## chest_painatypical angina  0.886 0.252  3.191
## chest_painnon-anginal pain 0.883 0.293  2.814
## chest_painasymptomatic     7.126 2.509 22.030
## max_heartrate              0.970 0.955  0.985

From the OR and their 95% CI computed above, we conclude that:

Age: the odds of having a heart disease are multiplied by a factor of 1.04 for each one-unit increase in age, all else being equal.
Sex: the odds of developing a heart disease for males are 5.4 times the odds for females, all else being equal.
Chest pain: the odds of developing a heart disease for people suffering from chest pain of the type “asymptomatic” are 7.13 times the odds for people suffering from chest pain of the type “typical angina”, all else being equal. We refrain from interpreting the other comparisons as they are not significant at the 5% significance level (1 is included in their 95% CI).
Maximum heart rate achieved: the odds of having a heart disease are multiplied by a factor of 0.97 for each one-unit increase in maximum heart rate achieved, all else being equal.
Intercept: we also refrain from interpreting the intercept as it is not significantly different from 0 at the 5% significance level (1 is included in the 95% CI).

If you are interested in printing only the OR for which the $p$-value of the coefficient is < 0.05, here is the code:

exp(coef(m3))[coef(summary(m3))[, "Pr(>|z|)"] < 0.05]

##                    age                sexmale chest_painasymptomatic 
##              1.0437437              5.3996293              7.1258054 
##          max_heartrate 
##              0.9701289

Remember that we can always write the interpretations in terms of the percentage increase/decrease in odds with the formula $(OR - 1) \times 100$, where OR corresponds to the odds ratio.

For instance, for the maximum heart rate achieved, the OR = 0.97, so the interpretation becomes: the odds of developing a heart disease increases by (0.97 $- 1) \times 100 =$ -3% for each one-unit increase in maximum heart rate achieved, which is equivalent to say that the odds of developing a heart disease decreases by 3% for each one-unit increase in maximum heart rate achieved.

For illustrative purposes, suppose now that we would like to predict the probability that a new patient develops a heart disease. Suppose that this patient is a 32-year-old woman, suffering from chest pain of the type non-anginal and she achieved a maximum heart rate of 150. The probability that she develops a heart disease is:

# create data frame of new patient
new_patient <- data.frame(
  age = 32,
  sex = "female",
  chest_pain = "non-anginal pain",
  max_heartrate = 150
)

# predict probability to develop heart disease
pred <- predict(m3,
  newdata = new_patient,
  type = "response"
)

# print prediction
pred

##          1 
## 0.03345948

If we trust our model, the probability that this new patient will develop a heart disease is predicted to be 3.35%.

We can also visualize the results thanks to the plot_model() function, three effects at the same time:

effect of age, sex and chest pain type on the predicted probability of developing a heart disease, and
effect of maximum heart rate achieved, sex and chest pain type on the predicted probability of developing a heart disease.

# 1. age, sex and chest pain on prob of disease
plot_model(m3,
  type = "pred",
  terms = c("age", "chest_pain", "sex"),
  ci.lvl = NA # remove confidence bands
) +
  labs(y = "Prob(heart disease)")

# 2. max heart rate, chest pain and sex on prob of disease
plot_model(m3,
  type = "pred",
  terms = c("max_heartrate", "chest_pain", "sex"),
  ci.lvl = NA # remove confidence bands
) +
  labs(y = "Prob(heart disease)")

For more clarity in the plots, confidence bands are removed thanks to ci.lvl = NA.

These plots confirm results obtained above, that is:

there is a positive relationship between age and the presence of heart disease,
there is a negative relationship between maximum heart rate achieved and the presence of heart disease,
the odds of developing a heart disease is higher for patients suffering from chest pain of the type asymptomatic and similar for the 3 other types of chest pain, and
the odds of developing a heart disease is higher for males than for females.

Interaction

In the previous sections, potential interaction effects were omitted.

An interaction occurs when the relationship between an independent variable and the outcome variable depends on the value or the level taken by another independent variable. On the contrary, if the relationship between an independent variable and the dependent variable remains unchanged no matter the value taken by another independent variable, we cannot conclude that there is an interaction effect.

In our case, there would be an interaction if for example the relationship between age and heart disease depends on the sex. There would be an interaction, for instance, if the relationship between age and heart disease was positive for females, and negative for males, or vice versa. Or if the relationship between age and heart disease was much stronger or much weaker for females than for males.

Let’s see if there is an interaction between age and sex, and more importantly, whether or not this interaction is significant. For this, we need to build two models:

one model containing only the main effects, so without the interaction, and
one model containing the main effects and the interaction.

# save model without interaction
m4 <- glm(heart_disease ~ age + sex,
  data = dat,
  family = "binomial"
)

# save model with interaction
m4_inter <- glm(heart_disease ~ age * sex,
  data = dat,
  family = "binomial"
)

We first assess the interaction visually via the plot_model() function:

# plot
plot_model(m4_inter,
  type = "pred",
  terms = c("age", "sex"),
  ci.lvl = NA # remove confidence bands
) +
  labs(y = "Prob(heart disease)")

Since the two curves of the predicted probabilities are relatively similar and follow the same pattern, the relationship between age and the presence of heart disease does not seem to depend on the sex, indicating that there may indeed be no interaction. However, we would like to test it more formally via a statistical test.

For this, we can compare the two models (the one without compared to the one with interaction) with a likelihood ratio test (LRT), using the anova() function:

anova(m4, m4_inter,
  test = "LRT"
)

## Analysis of Deviance Table
## 
## Model 1: heart_disease ~ age + sex
## Model 2: heart_disease ~ age * sex
##   Resid. Df Resid. Dev Df Deviance Pr(>Chi)
## 1       294     364.43                     
## 2       293     364.23  1  0.20741   0.6488

Remember that it is always the reduced model as the first argument in the anova() function, and then the more complex model as the second argument.

The test confirms what we supposed based on the plot: at the 5% significance level, we do not reject the null hypothesis that the two models are equivalent. Since the only difference between the two models is that an interaction term is added in the complete model, we do not reject the hypothesis that there is no interaction between the age and the sex ($p$-value = 0.649).

This conclusion could have also been obtained more simply with the drop1() function:

drop1(m4_inter,
  test = "LRT"
)

## Single term deletions
## 
## Model:
## heart_disease ~ age * sex
##         Df Deviance    AIC     LRT Pr(>Chi)
## <none>       364.23 372.23                 
## age:sex  1   364.43 370.43 0.20741   0.6488

As you can see, this gives the same $p$-value of 0.649.

In practice, a non-significant interaction is removed from the model before interpreting its results. In our case, only the main effects of age and sex would remain in the model.

This leads us to model selection, or which variables should be included in our final model. This is discussed in the next section.

Model selection

In practice, we often have several models, corresponding to the different combinations of independent variables and their interactions. Finding the best model is not easy.

In general, the best practice is to obtain a final model that is as parsimonious as possible, that is, with as few parameters as possible. A parsimonious model is easier to interpret and generalize, and also more powerful from a statistical point of view. On the other hand, it should not be too simple so that it still captures the variations or patterns in the data. In general, while more variables are often better than one, too many is often worse than a few.

The two most common approaches to obtain a final model are the following:

Adjust the model by removing the main effects and their interactions which are not significant with respect to their $p$-values, obtained by testing the nullity of the corresponding coefficients using a statistical test such as the likelihood ratio test or the Wald test. If there are several main effects or interactions which are not significant, interactions must be removed before removing any main effect. Moreover, it is recommended to remove interactions and independent variables one by one (starting with the one with the highest $p$-value), as removing a variable or an interaction may make another variable or interaction that was initially non-significant significant.
Adjust the model by using AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion). These procedures allow to select the best model (according to AIC or BIC) by finding an equilibrium between simplicity and complexity. These selection processes usually lead to a final model with as few parameters as possible, but which captures as much information in the data as possible. Note that only models with the same dependent variable can be compared using AIC or BIC. Models with different dependent variables cannot be compared using these criteria.

The first method requires that:

the underlying assumptions are valid,
the sample size is sufficiently large, and
the models are nested (i.e., the complete model includes at least all the variables included in the reduced model).

Moreover, the second method can be used with widely used criteria when selecting variables, and more importantly, in a completely autonomous way in R.

For this reason, the second method is recommended and more often used in practice.

This second method, referred as the stepwise selection, is divided into 3 types:

backward selection: we start from the most complete model (containing all independent variables and usually also their interactions), and the interactions/main effects are deleted at each step until the model cannot be improved,
forward selection: we start from the most basic model containing only the intercept, and the independent variables/interactions are added at each step until the model cannot be improved, or
mixed selection: we apply both the backward and forward selection to determine the best model according to the desired criterion.

We show how to select the best model according to AIC using the mixed stepwise selection, illustrated with all variables present in the data frame as independent variables and all possible second order interactions:

# save initial model
m5 <- glm(heart_disease ~ (age + sex + chest_pain + max_heartrate)^2,
  data = dat,
  family = "binomial"
)

# select best model according to AIC using mixed selection
m5_final <- step(m5,
  direction = "both", # both = mixed selection
  trace = FALSE # do not display intermediate steps
)

# display results of final model
summary(m5_final)

## 
## Call:
## glm(formula = heart_disease ~ age + sex + chest_pain + max_heartrate + 
##     age:max_heartrate, family = "binomial", data = dat)
## 
## Coefficients:
##                              Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                19.4386591  8.1904201   2.373 0.017628 *  
## age                        -0.3050017  0.1414986  -2.156 0.031122 *  
## sexmale                     1.7055353  0.3507149   4.863 1.16e-06 ***
## chest_painatypical angina  -0.0086463  0.6547573  -0.013 0.989464    
## chest_painnon-anginal pain -0.0590333  0.5844000  -0.101 0.919538    
## chest_painasymptomatic      1.9724490  0.5649516   3.491 0.000481 ***
## max_heartrate              -0.1605658  0.0540933  -2.968 0.002994 ** 
## age:max_heartrate           0.0023314  0.0009433   2.471 0.013457 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 409.95  on 296  degrees of freedom
## Residual deviance: 268.60  on 289  degrees of freedom
## AIC: 284.6
## 
## Number of Fisher Scoring iterations: 5

According to the AIC (which is the default criterion when using the step() function), the best model is the one including:

age,
sex,
chest_pain,
max_heartrate, and
the interaction between age and max_heartrate.

For your information, you can also easily compare models manually using AIC or the pseudo-$R^2$ with the tab_model() function, also available in the {sjPlot} R package:

tab_model(m3, m4, m5_final,
  show.ci = FALSE, # remove CI
  show.aic = TRUE, # display AIC
  p.style = "numeric_stars" # display p-values and stars
)

	heart disease	heart disease	heart disease
Predictors	Odds Ratios	p	Odds Ratios	p	Odds Ratios	p
(Intercept)	0.94	0.976	0.01 ^***	<0.001	276759408.51 ^*	0.018
age	1.04 ^*	0.024	1.07 ^***	<0.001	0.74 ^*	0.031
sex [male]	5.40 ^***	<0.001	4.47 ^***	<0.001	5.50 ^***	<0.001
chest pain [atypical angina]	0.89	0.851			0.99	0.989
chest pain [non-anginal pain]	0.88	0.828			0.94	0.920
chest pain [asymptomatic]	7.13 ^***	<0.001			7.19 ^***	<0.001
max heartrate	0.97 ^***	<0.001			0.85 ^**	0.003
age × max heartrate					1.00 ^*	0.013
Observations	297	297	297
R² Tjur	0.393	0.142	0.409
AIC	289.263	370.435	284.599
* p<0.05 p<0.01 * p<0.001

Pseudo-$R^2$ is a generalization of the coefficient of determination $R^2$ often used in linear regression to judge the quality of a model. Like the $R^2$ in linear regression, the pseudo-$R^2$ varies from 0 to 1, and can be interpreted as the percentage of the null deviance explained by the independent variable(s). The higher the pseudo-$R^2$ and the lower the AIC, the better the model.

Note that there are several pseudo-$R^2$, such as:

Likelihood ratio $R^2_{L}$,
Cox and Snell $R^2_{CS}$,
Nagelkerke $R^2_{N}$,
McFadden $R^2_{McF}$, and
Tjur $R^2_{T}$.

The tab_model() function gives the Tjur $R^2_{T}$ by default.

Based on the AIC and the Tjur $R^2_{T}$, the last model is considered as the best one among the 3 considered.

Note that, even though a model is deemed the best one among the ones you have considered (based on one or several criteria), it does not necessarily mean that it fits the data well. There are several methods to check the quality of a model and to check if it is appropriate for the data at hand. This is the topic of the next section.

Quality of a model

Usually, the goal of building a model is to be able to predict, as precisely as possible, the response variable for new data.

In the next sections, we present some measures to judge the quality of a model, starting with the easiest and most intuitive one, followed by two widely used in the medical domain, and finally two other metrics common in the field of machine learning.

Validity of the predictions

Accuracy

A good way to judge the accuracy of a model is to monitor its performance on new data and count how often it predicts the correct outcome.

Unfortunately, when we have access to new data, we often do not know the real outcome and we cannot therefore check if the model does a good job in predicting the outcome. The trick is to:

train the model on the initial data frame,
test the model on the exact same data (just like if it was a complete different data frame for which we do not know the outcome), and then
compare the predictions made by the model to the real outcomes.

To illustrate this process, we take the model built in the previous section and test it on the initial data frame.

Moreover, suppose that if the probability for the patient to develop a heart disease is below 50%, we consider that the predicted outcome is the absence of the disease, otherwise the predicted outcome is the presence of the disease.

# create a vector of predicted probabilities
preds <- predict(m5_final,
  newdata = select(dat, -heart_disease), # remove real outcomes
  type = "response"
)

# if probability < threshold, patient is considered not to have the disease
preds_outcome <- ifelse(preds < 0.5,
  0,
  1
)

# transform predictions into factor and set labels
preds_outcome <- factor(preds_outcome,
  levels = c(0, 1),
  labels = c("no disease", "disease")
)

# compare observed vs. predicted outcome
tab <- table(dat$heart_disease, preds_outcome,
  dnn = c("observed", "predicted")
)

# print results
tab

##             predicted
## observed     no disease disease
##   no disease        132      28
##   disease            33     104

From the contingency table of the predicted and observed outcomes, we see that the model:

correctly predicted the absence of the disease for 132 patients,
incorrectly predicted the presence of the disease for 28 patients,
incorrectly predicted the absence of the disease for 33 patients, and
correctly predicted the presence of the disease for 104 patients.

The percentage of correct predictions, referred as the accuracy, is the sum of the correct predictions divided by the total number of predictions:

accuracy <- sum(diag(tab)) / sum(tab)
accuracy

## [1] 0.7946128

This model has an accuracy of 79.5%.

Although accuracy is the most intuitive and easiest way to measure a model’s predictive performance, it has some drawbacks, notably because we have to choose an arbitrary threshold beyond which we classify a new observation as 1 or 0. A more detailed discussion about this can be found on Frank Harrell’s blog.

In this illustration, we chose 50% as the threshold beyond which a patient was considered as having the disease. Nonetheless, we could have chosen another threshold and the results would have been different!

Sensitivity and specificity

If you work in the medical field, or if your research is related to medical sciences, you have probably already heard about sensitivity and specificity.

The sensitivity of a classifier, also referred as the recall, measures the ability of a classifier to detect the condition when the condition is present. In our case, it is the percentage of diseased people who are correctly identified as having the disease. Formally, we have:

\[ Sensitivity = \frac{\text{True positives}}{\text{True positives} + \text{False negatives}},\]

where true positives are people correctly diagnosed as ill and false negatives are people incorrectly diagnosed as healthy.

The specificity of a classifier measures the ability of a classifier to correctly exclude the condition when the condition is absent. In our case, it is the percentage of healthy people who are correctly identified as not having the disease. Formally, we have:

\[Specificity = \frac{\text{True negatives}}{\text{True negatives} + \text{False positives}},\]

where true negatives are people correctly diagnosed as healthy and false positives are people incorrectly diagnosed as ill.

In R, sensitivity and specificity can be computed as follows:

# sensitivity
sensitivity <- tab[2, 2] / (tab[2, 2] + tab[2, 1])
sensitivity

## [1] 0.7591241

# specificity
specificity <- tab[1, 1] / (tab[1, 1] + tab[1, 2])
specificity

## [1] 0.825

With our model, we obtain:

sensitivity = 75.9%, and
specificity = 82.5%.

The closer the sensitivity and the specificity are to 100%, the better the model.

AUC and ROC curve

We have already seen that the better the quality of the model, the better the predictions.

Another common and less arbitrary way to judge the quality of a model is by computing the AUC (Area Under the Curve) and plotting the ROC (Receiver Operating Characteristic) curve.

This can be achieved easily thanks to the {pROC} package:

# load package
library(pROC)

# save roc object
res <- roc(heart_disease ~ fitted(m5_final),
  data = dat
)

# plot ROC curve
ggroc(res, legacy.axes = TRUE)

# print AUC
res$auc

## Area under the curve: 0.87

As the ggroc() function works with layers from the {ggplot2} package, we can print the AUC directly in the title of the plot of the ROC curve:

# plot ROC curve with AUC in title
ggroc(res, legacy.axes = TRUE) +
  labs(title = paste0("AUC = ", round(res$auc, 2)))

These two quality metrics can be interpreted as follows:

in the plot, the closer the ROC curve is to the upper left-hand corner, the better the model, and
the closer the AUC is to 1, the better the model.

Based on the ROC curve and the AUC, we can say that this model is good to very good. This means that the model is appropriate for these data, and that it can be useful to predict whether or not a patient will develop a heart disease!

Reporting results

As we have seen before, odds ratios are useful when reporting results of binary logistic regressions.

Computing these odds ratios together with the confidence intervals is not particularly difficult. However, presenting them in a table for a publication or a report can quickly become time consuming, in particular if you have many models and many independent variables.

Luckily, there are two packages which saved me a lot of time and which I use almost every time I need to report results of a logistic regression.

The first package, called {gtsummary} is useful to report results of one regression at a time. The second one is the {finalfit} package.¹ This packages is more appropriate if you need to report results of several regressions at a time.

{gtsummary} package

Here is an example with one of the models built previously:

# load package
library(gtsummary)

# print table of results
tbl_regression(m5_final, exponentiate = TRUE)

Characteristic	OR¹	95% CI¹	p-value
age	0.74	0.55, 0.96	0.031
sex
female	—	—
male	5.50	2.82, 11.2	<0.001
chest_pain
typical angina	—	—
atypical angina	0.99	0.27, 3.65	>0.9
non-anginal pain	0.94	0.30, 3.07	>0.9
asymptomatic	7.19	2.44, 22.8	<0.001
max_heartrate	0.85	0.76, 0.94	0.003
age * max_heartrate	1.00	1.00, 1.00	0.013
¹ OR = Odds Ratio, CI = Confidence Interval

What I like with this package is its ease of use, and the fact that all results are nicely formatted in a table. This is a very good starting point when I need to create a table for a publication or a report, for one regression at a time.

The second package becomes interesting when you need to report results for several models at once.

{finalfit} package

The {finalfit} package allows to report odds ratios, their confidence intervals and the $p$-values in a very efficient way. Moreover, it is quite easy to do so for many regressions at the same time.

Let me present the package by reporting results of:

all univariable binary logistic regressions that are possible with the variables available in the data frame,
a multivariable binary logistic regression that includes all variables available in the data frame, and
a multivariable binary logistic regression that includes only some of the variables present in the data frame.

We start with all the possible univariable binary logistic regressions:

# load packages
library(tidyverse)
library(gt)
library(finalfit)

# set dependent and independent variables
dependent <- "heart_disease"
independent <- c("age", "sex", "chest_pain", "max_heartrate")

# save results of univariable logistic regressions
glmuni <- dat |>
  glmuni(dependent, independent) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Crude OR",
    estimate_suffix = " (95% CI)"
  )

# print results
glmuni |>
  gt()

Variables	Crude OR (95% CI)
age	1.05 (1.03-1.08, p<0.001)
sexmale	3.57 (2.12-6.18, p<0.001)
chest_painatypical angina	0.51 (0.16-1.66, p=0.255)
chest_painnon-anginal pain	0.63 (0.23-1.86, p=0.384)
chest_painasymptomatic	6.04 (2.39-16.76, p<0.001)
max_heartrate	0.96 (0.94-0.97, p<0.001)

A few remarks regarding this code:

glmuni() is used because we want to run univariable GLM.
explanatory_name = "Variables" is used to rename the first column (by default it is “explanatory”).
estimate_name = "Crude OR" is used to rename the second column and inform the reader that we are in the univariable case. In the univariable case, OR are often called crude OR because they are not adjusted for the effects of the other independent variables.
estimate_suffix = " (95% CI)" is used to specify that it is the 95% confidence intervals which are inside the parentheses.
The gt() layer at the end of the code is not compulsory. It is just to make the output appears in a nice table instead of the usual format of R outputs. See more information about the {gt} package in its documentation.

Here is how to report results of a multivariable binary logistic regression which includes all variables:

# save results of full model
glmmulti_full <- dat |>
  glmmulti(dependent, independent) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Adjusted OR - full model",
  )

# print results
glmmulti_full |>
  gt()

Variables	Adjusted OR - full model
age	1.04 (1.01-1.08, p=0.024)
sexmale	5.40 (2.78-10.97, p<0.001)
chest_painatypical angina	0.89 (0.25-3.19, p=0.851)
chest_painnon-anginal pain	0.88 (0.29-2.81, p=0.828)
chest_painasymptomatic	7.13 (2.51-22.03, p<0.001)
max_heartrate	0.97 (0.95-0.99, p<0.001)

A few remarks regarding this code:

glmmulti() is used because we want to run multivariable GLM.
estimate_name = "Adjusted OR - full model" is used to remind the reader that we are in the multivariable case with all variables included. In the multivariable case, OR are often called adjusted OR because they are adjusted for the effects of the other independent variables.

Here is how to report results of a multivariable binary logistic regression which includes only a selection of variables:

# select the variables to be included in the final model
independent_final <- c("age", "sex", "chest_pain")

# save results of final model
glmmulti_final <- dat |>
  glmmulti(dependent, independent_final) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Adjusted OR - final model",
    estimate_suffix = " (95% CI)"
  )

# print results
glmmulti_final |>
  gt()

Variables	Adjusted OR - final model (95% CI)
age	1.07 (1.03-1.11, p<0.001)
sexmale	5.52 (2.88-11.07, p<0.001)
chest_painatypical angina	0.82 (0.24-2.82, p=0.743)
chest_painnon-anginal pain	0.93 (0.32-2.90, p=0.903)
chest_painasymptomatic	9.48 (3.47-28.53, p<0.001)

Note that you have to manually select the variables (the package will not choose for you unfortunately). The selection could be done preliminary thanks to a stepwise procedure for example.

Now the most interesting part of this package:

we can combine all these results together,
in addition to some descriptive statistics for each level of the dependent variable!

Here are all results combined together and displayed in a table:

# save descriptive statistics
summary <- dat |>
  summary_factorlist(dependent, independent, fit_id = TRUE)

# save results of regressions
output <- summary |>
  finalfit_merge(glmuni) |>
  finalfit_merge(glmmulti_full) |>
  finalfit_merge(glmmulti_final)

# print all results
output |>
  dplyr::select(-fit_id, -index) |>
  dplyr::rename(
    Variables = label,
    " " = levels
  ) |>
  gt()

Variables		no disease	disease	Crude OR (95% CI)	Adjusted OR - full model	Adjusted OR - final model (95% CI)
age	Mean (SD)	52.6 (9.6)	56.8 (7.9)	1.05 (1.03-1.08, p<0.001)	1.04 (1.01-1.08, p=0.024)	1.07 (1.03-1.11, p<0.001)
sex	female	71 (44.4)	25 (18.2)	-	-	-
	male	89 (55.6)	112 (81.8)	3.57 (2.12-6.18, p<0.001)	5.40 (2.78-10.97, p<0.001)	5.52 (2.88-11.07, p<0.001)
chest_pain	typical angina	16 (10.0)	7 (5.1)	-	-	-
	atypical angina	40 (25.0)	9 (6.6)	0.51 (0.16-1.66, p=0.255)	0.89 (0.25-3.19, p=0.851)	0.82 (0.24-2.82, p=0.743)
	non-anginal pain	65 (40.6)	18 (13.1)	0.63 (0.23-1.86, p=0.384)	0.88 (0.29-2.81, p=0.828)	0.93 (0.32-2.90, p=0.903)
	asymptomatic	39 (24.4)	103 (75.2)	6.04 (2.39-16.76, p<0.001)	7.13 (2.51-22.03, p<0.001)	9.48 (3.47-28.53, p<0.001)
max_heartrate	Mean (SD)	158.6 (19.0)	139.1 (22.7)	0.96 (0.94-0.97, p<0.001)	0.97 (0.95-0.99, p<0.001)	-

A few remarks regarding this code:

summary_factorlist(dependent, independent, fit_id = TRUE) is used to compute the descriptive statistics by group of the dependent variable.
finalfit_merge() is used to merge results together.
dplyr::select(-fit_id, -index) is used to remove unnecessary columns.
dplyr::rename(Variables = label, " " = levels) is used to renames some columns.

And finally, a few remarks regarding the resulting table:

The first column gives the name of the variables.
The second column specifies:
- for qualitative variables: the levels
- for quantitative variables: that it is the mean and the standard deviation (SD) which will be computed in the next two columns
The third and fourth columns give the descriptive statistics for each level of the dependent variable:
- for qualitative variables: the number of cases, and in parentheses the frequencies by column
- for quantitative variables: the mean, and in parentheses the standard deviation
The last three columns give the OR, and in parentheses the 95% CI of the OR and the $p$-value (for the univariable and the two multivariable models, respectively).

For your convenience, here is the full code so you can copy paste it easily in case you want to reproduce the process:

# load packages
library(tidyverse)
library(gt)
library(finalfit)

# set variables
dependent <- "heart_disease"
independent <- c("age", "sex", "chest_pain", "max_heartrate")
independent_final <- c("age", "sex", "chest_pain")

# save descriptive statistics
summary <- dat |>
  summary_factorlist(dependent, independent, fit_id = TRUE)

# save results of univariable logistic regressions
glmuni <- dat |>
  glmuni(dependent, independent) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Crude OR",
    estimate_suffix = " (95% CI)"
  )

# save results of full model
glmmulti_full <- dat |>
  glmmulti(dependent, independent) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Adjusted OR - full model",
  )

# save results of final model
glmmulti_final <- dat |>
  glmmulti(dependent, independent_final) |>
  fit2df(
    explanatory_name = "Variables",
    estimate_name = "Adjusted OR - final model",
    estimate_suffix = " (95% CI)"
  )

# save merged results
output <- summary |>
  finalfit_merge(glmuni) |>
  finalfit_merge(glmmulti_full) |>
  finalfit_merge(glmmulti_final)

# print all results
output |>
  dplyr::select(-fit_id, -index) |>
  dplyr::rename(
    Variables = label,
    " " = levels
  ) |>
  gt()

Last but not least, the or_plot() function, also available from the {finalfit} package, is useful to visualize all odds ratio and their 95% confidence intervals:

dat |> or_plot(dependent, independent,
  table_text_size = 3.5 # reduce text size
)

Here is how to read this plot:

The squares represent the OR, and the whiskers their 95% CI.
When the 95% CI crosses the vertical dashed line, it means that the OR is not significantly different from 1 (at the 5% significance level). In these cases, we cannot reject the hypothesis of no association with the dependent variable.
When the 95% CI does not cross the vertical dashed line, it means that the OR is significantly different from 1. In these cases:
- if the square is located to the right of the vertical dashed line, there is a positive relationship between the outcome and the independent variable (known as a risk factor), and
- if the square is located to the left of the vertical dashed line, there is a negative relationship between the outcome and the independent variable (known as a protective factor).

The plot confirms what was obtained above:

age is a risk factor for heart disease,
maximum heart rate achieved is a protective factor for heart disease, and
being a male and suffering from asymptomatic chest pain are both risk factors of heart disease.

Be careful that sometimes the square is too big to see the whiskers of the 95% CI. This is the case for the variables max_heartrate and age. In these cases, it is better to check the significance of the OR thanks to their 95% CI or the $p$-values printed in parentheses.

Conditions of application

For results to be valid and interpretable, a binary logistic regression requires:

the dependent variable to be binary,
independence of the observations: no repeated measurements or matched data, otherwise generalize linear mixed effect models (GLMM) should be used,
linearity of continuous independent variables and the log-odds outcome: take age and heart disease as an example. If heart disease is more frequent or less frequent as age rises, the model will work well. However, if children and the elderly are at high risk of having a heart disease, but those in middle years are not, then the relationship is not linear, or not monotonic, meaning that the response does not only go in one direction,
a sufficiently large sample size (for confidence intervals and hypothesis tests to be valid), and
no multicollinearity: independent variables should not be highly correlated with each other, otherwise coefficients and OR can become unstable.

Here is how to verify each of them:

this is obvious; check if the dependent variable has indeed only two levels,
this is often not tested formally, but verified through the design of the experiment,
quantitative independent variables should have a linear relationship between their log-odds and their observed values. A visual check is sufficient, see below with age, maximum heart rate achieved and model m3 as example:

# linearity to the log-odds?
dat |>
  dplyr::select(age, max_heartrate) |>
  mutate(log_odds = predict(m3)) |>
  pivot_longer(-log_odds) |>
  ggplot(aes(log_odds, value)) +
  geom_point() +
  geom_smooth(method = "lm") +
  facet_wrap(~name)

in practice, it is recommended to have at least 10 times as many events as parameters in the model, and
the variance inflation factors (VIF) is a well known measure of multicollinearity. It should be below 10 or 5, depending on the field of research. VIF can be computed with the vif() function, available in the {car} package:

# load package
library(car)

# compute VIF for model m3
vif(m3)

##                   GVIF Df GVIF^(1/(2*Df))
## age           1.205246  1        1.097837
## sex           1.155071  1        1.074742
## chest_pain    1.113010  3        1.018005
## max_heartrate 1.143125  1        1.069170

Conclusion

Thanks for reading.

In this relatively long and detailed post, we covered several important points about binary logistic regression. First, when to use such models and what is the difference with linear models, how to implement it in R, and how to interpret and report results. We ended by discussing about model selection, how to judge the quality of fit of a logistic regression, and its underlying assumptions.

I now hope that (univariable and multivariable) binary logistic regressions in R no longer hold any secrets for you.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Thanks to Claire from DellaData.fr for introducing me to this package.↩︎

What is the probability that two persons have the same initials?

Wed, 06 Dec 2023 00:00:00 +0000

Introduction

Last week, I joined a team to work on a collaborative project. The team was already established for a few months, with several scientists working together on the project. For simplicity, they used to sign documents, mention colleagues in emails, etc. with their initials (the first letter of their first name followed by the first letter of their last name).

A couple of days after joining the project, when I needed to sign my first document with my initials, we realized that another person in the team had the exact same initials than me.

This was not really an issue, as we decided that I would write my initials backward, that is, “SA” instead of “AS”, and the other person would keep signing with “AS” as usual.

It could have stopped here. However, the idea to write a post about this rather trivial anecdote came to me when the team leader claimed, in the middle of a meeting: “That’s very unfortunate that you two have the same initials! What are the chances of this happening to us?!”.

We spent a couple of minutes trying to estimate this probability, which in the end were mostly based on our intuitions rather than on a formal calculation. This piqued my curiosity.

Given that the project we are working on requires the use of simulations, I decided to focus on answering this question via simulations in R. That being said, as for most simulations, it is a good practice to verify these results. This is done using probability theory. This comparison will allow to assess the truthfulness of results obtained through simulations.

Furthermore, I thought that it would be a nice way to illustrate methods not often presented in my posts: for loops, replications and writing functions in R.

How likely is it?

Before answering the question raised by the team leader, there are three things to note:

Although the team leader was curious to know the probability that exactly two persons have the same initials, we are actually more interested in the probability that at least two persons have the same initials (as the problem also occurs if more than two persons within a team have the same initials).
The team consists of 8 people.
We restrict ourselves to two-letters initials (the first letter being the first letter of the first name, the second letter being the first letter of the last name). This means that middle names are not taken into account, and only the first letter is considered for compound names.

In this post, we will show how to compute this probability:

in our context, that is, for a team of 8 persons, and
for completeness, for teams of all sizes from 2 to 100 persons.

As stated in the introduction, we will compute these probabilities first through simulations and then through probability theory.

For our team

We start by creating a vector of size 8, corresponding to the initials of a team of 8 persons randomly sampled among all 26 letters of the Latin alphabet:

# number of persons
n_persons <- 8

# create vector of initials
initials <- replicate(
  n = n_persons, # number of replications
  paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)) # sample letters
)

# display initials
initials

## [1] "UJ" "MN" "XD" "CY" "BB" "ZB" "CU" "HQ"

# are there duplicates?
any(duplicated(initials))

## [1] FALSE

As we can see, everyone has different initials in this simulated team of 8 persons, but this will not always be the case.

To estimate, via simulations, how likely is that at least two persons have the same initials among the team, we need to replicate this vector of 8 sampled initials a large number of times (say 1,000 replications):¹

# number of replications
reps <- 1000

# create and save replications
dat <- replicate(
  n = reps, # number of replications
  replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
)

# dimensions
dim(dat)

## [1]    8 1000

# display first 4 simulated teams
dat[, 1:4]

##      [,1] [,2] [,3] [,4]
## [1,] "VA" "BU" "LU" "PT"
## [2,] "JG" "SM" "HM" "OL"
## [3,] "BY" "NA" "VJ" "OT"
## [4,] "RT" "CM" "WT" "YT"
## [5,] "PS" "CT" "NB" "QJ"
## [6,] "MG" "KR" "SV" "US"
## [7,] "PL" "SN" "PN" "XW"
## [8,] "NJ" "BR" "DD" "ZC"

The result is a matrix of 8 rows and 1000 columns, where:

each rows corresponds to the sampled initials of a person, and
each column corresponds to one simulated team of 8 people.

For better readability, we rename:

the row names as M1 to M8, corresponding to persons 1 to 8, and
the column names as T1 to T1000, corresponding to teams 1 to 1000.

# rename rows
rownames(dat) <- paste0("M", 1:n_persons)

# rename columns
colnames(dat) <- paste0("T", 1:reps)

# display first 4 simulated teams
dat[, 1:4]

##    T1   T2   T3   T4  
## M1 "VA" "BU" "LU" "PT"
## M2 "JG" "SM" "HM" "OL"
## M3 "BY" "NA" "VJ" "OT"
## M4 "RT" "CM" "WT" "YT"
## M5 "PS" "CT" "NB" "QJ"
## M6 "MG" "KR" "SV" "US"
## M7 "PL" "SN" "PN" "XW"
## M8 "NJ" "BR" "DD" "ZC"

We now need to compute, among the 1000 teams simulated, how many of them have at least two persons with the same initials:

# transform to data frame
dat <- as.data.frame(dat)

# save which teams have duplicates
duplicates <- rep(NA, reps) # create empty vector
for (i in 1:reps) { # for loop over i from 1 to 1,000
  duplicates[i] <- any(duplicated(dat[, i])) # save results TRUE/FALSE in duplicates vector
}

# count how many teams have duplicates
sum(duplicates)

## [1] 41

Here, for each column of our data frame dat (from the first to the 1000th column), we ask whether there are duplicates or not. This is done repeatedly over all columns thanks to a for loop. For each column, the result is TRUE if there are duplicates, otherwise it is FALSE. The result of each iteration is saved in the duplicates vector. As TRUE = 1 and FALSE = 0 in R, we can then count how many columns (and thus teams) have duplicates by summing the number of TRUE in the duplicates vector.

As we can see from the output above, among the 1000 simulated teams, 41 of them have duplicates, that is, 41 of them have at least two persons with the same initials.

Therefore, based on the simulations, we can expect the probability that at least two persons with the same initials in a team of 8 persons to be close to 4.1%.

This is a good starting point. Notice, however, that I wrote close to 4.1% because this probability will vary each time it is computed via simulations.

For instance, if we repeat the exact same process a second time:

# create and save replications
dat <- replicate(
  n = reps, # number of replications
  replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
)

# transform to data frame
dat <- as.data.frame(dat)

# save which teams have duplicates
duplicates <- rep(NA, reps) # create empty vector
for (i in 1:reps) { # for loop over i from 1 to 1,000
  duplicates[i] <- any(duplicated(dat[, i])) # save results in the duplicates vector (as TRUE/FALSE)
}

# count how many teams have duplicates
sum(duplicates)

## [1] 44

We now find a probability of 4.4%. This is not an error, but it is due to randomness when sampling initials.

Luckily, we can make the computation of this probability more robust thanks to replications. Intuitively, it works as follows. We repeat the same computation multiple times, giving us a range of possible probabilities. This allows us to assess the uncertainty of our result, and understand how the probability might vary due to taking different random samples of initials.

So the goal is to compute our probability multiple times (say 100 times), and see its distribution.

To repeat the same computation multiple times, it is best to write a function in order to avoid copy pasting the same code over and over. So we first write a function (called initials) which computes the probability that at least two persons share the same initials among a team of $n$ people:

initials <- function(n_persons, reps = 1000) {
  # simulate data
  dat <- as.data.frame(replicate(
    reps,
    replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
  ))

  # save which teams have duplicates
  duplicates <- rep(NA, reps)
  for (i in 1:reps) {
    duplicates[i] <- any(duplicated(dat[, i]))
  }

  # proportion of teams with duplicates
  return(mean(duplicates))
}

A function in R requires to include:

the parameters inside (), and
the computation inside {}.

We can then use our function to compute the probability that at least two persons share the same initials among a team of 8 people. And we combine it with the replicate() function to compute this probability 100 times.

# compute and save probabilities
probs <- replicate(100, initials(n_persons = 8))

# display probabilities
probs

##   [1] 0.032 0.037 0.040 0.043 0.033 0.042 0.039 0.047 0.045 0.038 0.052 0.042
##  [13] 0.042 0.040 0.023 0.044 0.041 0.039 0.036 0.048 0.041 0.037 0.027 0.030
##  [25] 0.052 0.038 0.043 0.035 0.038 0.045 0.047 0.044 0.030 0.036 0.036 0.048
##  [37] 0.038 0.045 0.044 0.034 0.031 0.043 0.045 0.034 0.049 0.047 0.051 0.036
##  [49] 0.051 0.040 0.043 0.044 0.038 0.049 0.043 0.050 0.035 0.043 0.048 0.038
##  [61] 0.041 0.044 0.039 0.045 0.033 0.057 0.036 0.043 0.041 0.041 0.041 0.041
##  [73] 0.038 0.044 0.031 0.034 0.049 0.041 0.040 0.034 0.032 0.036 0.049 0.047
##  [85] 0.048 0.038 0.038 0.037 0.036 0.037 0.043 0.040 0.026 0.049 0.046 0.044
##  [97] 0.048 0.038 0.026 0.029

Finally, we visualize the distribution of these 100 probabilities thanks to a histogram and a boxplot (with the {ggplot2} package):

# visualize distribution of the computed probabilities
# build and save plots
library(ggplot2)

p1 <- ggplot(mapping = aes(x = probs)) +
  geom_histogram(color = "black", fill = "steelblue", bins = 8) +
  labs(
    x = "Probabilities",
    y = "Frequencies"
  ) +
  scale_x_continuous(labels = scales::percent) # format x-axis in %

p2 <- ggplot(mapping = aes(x = probs)) +
  geom_boxplot(color = "black", fill = "steelblue") +
  labs(x = "Probabilities") +
  theme(
    axis.text.y = element_blank(),
    axis.ticks.y = element_blank()
  ) +
  scale_x_continuous(labels = scales::percent) # format x-axis in %

# combine plots
library(patchwork)

p1 + p2

These two plots show that the probability that at least two persons share the same initials among a team of 8 people is most likely between 3.5% and 4.5%.

For the record, during the meeting at the root of all this thinking, most of us thought that it was much less likely. Indeed, I believe we were tempted to compute the probability that someone who joins the team has “AS” as initials. This is indeed much less likely, as the probability is only $\frac{1}{26} \times \frac{1}{26} \simeq 0.15\%$.

However, this does not take into account the fact:

that the newcomer can have the same initials as any other person, and
that it is not only the newcomer who can have the same initials as another person (2 people already working in the team when the newcomer arrives could have the same initials as well).

If you are puzzled by this finding, I recommend reading about the birthday’s paradox. The birthday’s paradox states that the probability of two people sharing the same birthday becomes surprisingly high with a relatively small group of individuals. In practice, in a group of just 23 people, there is a greater than 50% chance that at least two individuals share the same birthday, illustrating our counterintuitive intuitions about the likelihood of such coincidences. This phenomenon arises due to the multitude of possible birthday pairs within the group, similar to the multitude of possible pairs if initials within a team.

For teams of different sizes

We are now interested in computing this probability not just for a team of 8 persons, but for teams of different sizes. We can do this with the help of our function defined earlier.

For the illustration, let’s compute the probability that at least two persons have the same initials, for teams of 2 and up to 100 persons:

# set lower and upper bounds of number of persons
min_persons <- 2
max_persons <- 100

# create empty vector of probabilities
probs <- rep(NA, length(min_persons:max_persons))

# compute and save probabilities for teams of size 2 to 100
for (i in min_persons:max_persons) {
  probs[i] <- initials(n_persons = i)
}

# display probabilities
probs

##   [1]    NA 0.001 0.005 0.013 0.012 0.019 0.036 0.040 0.047 0.057 0.074 0.083
##  [13] 0.103 0.128 0.158 0.166 0.178 0.215 0.232 0.260 0.275 0.296 0.300 0.329
##  [25] 0.357 0.392 0.405 0.405 0.439 0.478 0.495 0.536 0.535 0.563 0.578 0.599
##  [37] 0.653 0.656 0.686 0.693 0.715 0.711 0.767 0.760 0.786 0.784 0.814 0.817
##  [49] 0.825 0.826 0.842 0.845 0.867 0.893 0.901 0.920 0.919 0.911 0.917 0.942
##  [61] 0.950 0.951 0.946 0.947 0.969 0.959 0.965 0.964 0.977 0.984 0.977 0.977
##  [73] 0.985 0.978 0.986 0.981 0.989 0.991 0.989 0.988 0.992 0.993 0.994 0.996
##  [85] 0.997 0.995 0.994 0.999 0.999 0.999 1.000 0.999 0.997 1.000 0.999 0.999
##  [97] 0.999 1.000 1.000 0.999

We are left with storing these probabilities together with the number of persons in the team in a data frame:

# create data frame with saved probabilities and number of persons
dat_plot_sim <- data.frame(
  n_persons = (min_persons - 1):max_persons,
  prob = probs
)

# display first 6 rows
head(dat_plot_sim)

##   n_persons  prob
## 1         1    NA
## 2         2 0.001
## 3         3 0.005
## 4         4 0.013
## 5         5 0.012
## 6         6 0.019

Of course, two people having the same initials in a team of 1 (if we can call this a team…) is impossible.

An event which is impossible has a probability equal to 0. We thus impute this probability in our data frame, in the first row:

# set proba = 0 when n_person = 1
dat_plot_sim[1, 2] <- 0

# display first 6 rows
head(dat_plot_sim)

##   n_persons  prob
## 1         1 0.000
## 2         2 0.001
## 3         3 0.005
## 4         4 0.013
## 5         5 0.012
## 6         6 0.019

Finally, we visualize these probabilities in function of the number of persons in the team:

# visualize probabilities
ggplot(dat_plot_sim) +
  aes(x = n_persons, y = probs) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

From the plot above, we see that the probability that at least two persons have the same initials reaches 50% when the team exceeds around 30 people.

Moreover, notice that this probability becomes close to 100% when the team reaches around 75 people.

Verification

For the sake of completeness, we now compare results obtained through simulations with results obtained from probability theory.²

We first define the function that will be used to compare results found above:

# define function
have_same <- function(s, n) {
  sample_space <- s
  probability <- 1
  for (i in 0:(n - 1)) {
    probability <- probability * (sample_space - i) / sample_space
  }
  1 - probability
}

For our team

# number of possible two-letter initials
n_initials <- 26^2

# apply function
have_same(n_initials, n_persons)

## [1] 0.0407218

The probability that at least two persons have the same initials in a team of 8 is 4.07%. This is close to the probability found with simulations, and within the range of 3.5%–4.5%.

For teams of different sizes

We now compute the probability for teams between 1 and 100 persons:

# compute and save probabilities for teams between 1 and 100 persons
probs <- vector(length = max_persons)
for (i in 1:max_persons) {
  probs[i] <- have_same(n_initials, i)
}

# create data frame with saved probabilities and number of persons
dat_plot_theory <- data.frame(
  n_persons = (min_persons - 1):max_persons,
  prob = probs
)

# display first 6 rows
head(dat_plot_theory)

##   n_persons        prob
## 1         1 0.000000000
## 2         2 0.001479290
## 3         3 0.004433493
## 4         4 0.008851688
## 5         5 0.014716471
## 6         6 0.022004071

Finally, we visualize these probabilities in function of the number of persons in the team:

# visualize probabilities
ggplot(dat_plot_theory) +
  aes(x = n_persons, y = probs) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

For an easier comparison, we plot probabilities found thanks to simulations and thanks to probability theory on the same plot:

# combine the two data frames into one and add the method as variable
dat_plot_sim$Method <- "Simulations"
dat_plot_theory$Method <- "Theory"
dat_plot_all <- rbind(dat_plot_sim, dat_plot_theory)

# visualize probabilities on same plot
ggplot(dat_plot_all) +
  aes(x = n_persons, y = prob, color = Method) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

The plot above shows that results using probability theory are relatively similar to results obtained through simulations, indicating that the simulations are trustworthy.

Conclusion

The initial question, raised during a meeting, was “What is the probability that, among our team consisting of 8 persons, two have the same initials?”.

In this post, we first showed how to compute this probability through simulations in R. Secondly, we verified the veracity of the simulations thanks to probability theory. Furthermore, we illustrated how for loops, replications and writing a function can be used in R to answer a probability problem.

As a side note, it is important to keep in mind that in this post, we assumed the following:

All letters of the alphabet had the same probability of occurring, meaning that all pairs of initials were equally probable. This is probably not the case in reality, as a first and last name starting both with X is not as probable as a first and last name starting respectively with M and K. This bias could be limited by specifying different weights when sampling initials.
We restricted ourselves to two-letters initials. Therefore, for compound first or last names, only the first letter is considered. Middle names are also not taken into account.

Last but not least, note that you will find slightly different results than mine, even if you use the exact same code. This is due to randomness. To replicate results as shown in this post, use set.seed(6).

Thanks for reading.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

You can always use a larger number of replications, but in our case the final result is similar with more replications, and the aim of the post is more to show the development than the final answer.↩︎
I thank Richard for writing the first version of the code used for the verifications.↩︎

Introduction to data manipulation in R with {dplyr}

Mon, 27 Nov 2023 00:00:00 +0000

Introduction

In a previous post, we showed how to manipulate data in R. In particular, we illustrated how to create and manipulate vectors, factors, lists and data frames. This served as an introduction to R and was aimed at beginners. Moreover, as long as it was possible, all manipulations were made in base R, that is, without having to load any package.

In this post, we would like to show again how to manipulate data in R, but this time using the {dplyr} package.

The {dplyr} package, developed by Hadley Wickham and colleagues at Posit, provides a complete set of functions that help you solve the most common data manipulation challenges such as:

filtering observations based on their values
extracting observations based on their values or positions
sampling observations based on a specific number or fraction of rows
sorting observations based on one or several variables
selecting variables based on their names or positions
renaming variables
adding new variables based on existing ones
summarizing observations or variables to a single descriptive measure
performing any operation by group
categorizing observations into two or more groups
etc.

More information about the package can be found at dplyr.tidyverse.org.

In this post, we will present the most common functions for data manipulation and data management using the {dplyr} package (illustrated on a data frame). This is however not an exhaustive list! It is likely that you will need other functions that the ones presented here. For the interested readers, see the end of this post for further resources.

A question I am often asked is whether it is best to first learn data manipulation with base R and then with {dplyr}, or directly learn {dplyr}.

Everyone may not agree, but here is what I tend to answer. The interest in terms of efficiency and clarity/readability of the code that {dplyr} brings is obvious. So even when I give a training targeted to statisticians, I present the {dplyr} package. However, I still do believe that learning data manipulation with base R is important for two reasons:

Everyone will, at some point, be confronted with code written in base R (either from collaboration with other R users, or from code found in textbooks or online).
Through the feedback I receive from my students, I notice that {dplyr} is relatively easy to learn when you are familiar with base R (which is quite rewarding for those who struggled at the beginning).

For these reasons, I tend to teach data manipulation with base R first and then slowly switch to {dplyr} (up to the point that for some advanced training courses, I hardly use base R at all by the end of the course). As a side note, this is the approach I follow for data visualization in R as well: I teach first how to plot data with base R, then I gradually teach them how to use {ggplot2}.

With this approach, some students may have the impression that they wasted their time learning base R. At least, some may have this impression during the training. However, as soon as the training is done and they have to learn R by themselves or work on real projects, they are grateful of having learned both.

I am curious to hear from other teachers regarding their approach, so feel free to share your opinion.

Data

To present the different functions, we will use the data frame penguins, available within the {palmerpenguins} package (Horst et al. 2020). Data are available by CC-0 license and can be downloaded from CRAN:

# install.packages("palmerpenguins")
library(palmerpenguins)

Before going further, we rename the data frame as dat:

dat <- penguins

I like to call data frames I am working on with a generic name such as dat for two reasons:

Every time I need to write the name of the data frame, it is usually shorter to write dat than to write the name of the data frame (which is in this case penguins).
If I need to do similar analyses or plots on different data frames, the code I wrote in the past can be reused with only a few modifications. With this very simple trick, most of the time I only have to edit the names of the variables, but the name of the data frame does not need to be changed (which saves me a lot of time).

The data frame contains data for 344 penguins and 8 variables describing the species, the island, some measurements of the size of the bill, flipper and body mass, the sex and the study year. More information about the data frame can be found by running ?penguins (after loading the {palmerpenguins} package).

For this post, we will focus only on the variables species, body_mass_g, sex and year.

Before proceeding with the different data manipulation techniques, let’s first inspect the data by displaying its structure, the first 6 rows and a summary of it:

str(dat) # structure of the data

## tibble [344 × 4] (S3: tbl_df/tbl/data.frame)
##  $ species    : Factor w/ 3 levels "Adelie","Chinstrap",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ body_mass_g: int [1:344] 3750 3800 3250 NA 3450 3650 3625 4675 3475 4250 ...
##  $ sex        : Factor w/ 2 levels "female","male": 2 1 1 NA 1 2 1 2 NA NA ...
##  $ year       : int [1:344] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...

head(dat) # display first 6 rows

## # A tibble: 6 × 4
##   species body_mass_g sex     year
##   <fct>         <int> <fct>  <int>
## 1 Adelie         3750 male    2007
## 2 Adelie         3800 female  2007
## 3 Adelie         3250 female  2007
## 4 Adelie           NA <NA>    2007
## 5 Adelie         3450 female  2007
## 6 Adelie         3650 male    2007

summary(dat) # summary

##       species     body_mass_g       sex           year     
##  Adelie   :152   Min.   :2700   female:165   Min.   :2007  
##  Chinstrap: 68   1st Qu.:3550   male  :168   1st Qu.:2007  
##  Gentoo   :124   Median :4050   NA's  : 11   Median :2008  
##                  Mean   :4202                Mean   :2008  
##                  3rd Qu.:4750                3rd Qu.:2009  
##                  Max.   :6300                Max.   :2009  
##                  NA's   :2

{dplyr} package

Without further ado, let’s illustrate the different functions for data manipulation available in the {dplyr} package in the following sections.

As for any package, we first need to install and load it before using it:

# install.packages("dplyr")
library(dplyr)

Note that all functions presented below requires tidy data, which means that:

each variable is in its own column,
each observation, or case, is in its own row, and
each value is in its own cell.

Rules of a tidy data frame: variables are columns, observations are rows, and values are cells. Source: R for Data Science (2e) by H. Wickham, M. Çetinkaya-Rundel and G. Grolemund.

Filter observations

Filtering observations based on their values can be done with the filter() function. This function works on both quantitative and qualitative variables:

# filter observations based on a quantitative variable
filter(dat, body_mass_g > 4000)

## # A tibble: 172 × 4
##    species body_mass_g sex    year
##    <fct>         <int> <fct> <int>
##  1 Adelie         4675 male   2007
##  2 Adelie         4250 <NA>   2007
##  3 Adelie         4400 male   2007
##  4 Adelie         4500 male   2007
##  5 Adelie         4200 male   2007
##  6 Adelie         4150 male   2007
##  7 Adelie         4650 male   2007
##  8 Adelie         4400 male   2007
##  9 Adelie         4600 male   2007
## 10 Adelie         4150 male   2007
## # ℹ 162 more rows

# filter observations based on a qualitative variable
filter(dat, sex == "female")

## # A tibble: 165 × 4
##    species body_mass_g sex     year
##    <fct>         <int> <fct>  <int>
##  1 Adelie         3800 female  2007
##  2 Adelie         3250 female  2007
##  3 Adelie         3450 female  2007
##  4 Adelie         3625 female  2007
##  5 Adelie         3200 female  2007
##  6 Adelie         3700 female  2007
##  7 Adelie         3450 female  2007
##  8 Adelie         3325 female  2007
##  9 Adelie         3400 female  2007
## 10 Adelie         3800 female  2007
## # ℹ 155 more rows

You can combine several conditions with & (if the conditions must be cumulative) or | (if the conditions are alternatives), for instance:

# filter observations based on 2 cumulative conditions
filter(dat, body_mass_g > 4000 & sex == "female")

## # A tibble: 58 × 4
##    species body_mass_g sex     year
##    <fct>         <int> <fct>  <int>
##  1 Gentoo         4500 female  2007
##  2 Gentoo         4450 female  2007
##  3 Gentoo         4550 female  2007
##  4 Gentoo         4800 female  2007
##  5 Gentoo         4400 female  2007
##  6 Gentoo         4650 female  2007
##  7 Gentoo         4650 female  2007
##  8 Gentoo         4200 female  2007
##  9 Gentoo         4150 female  2007
## 10 Gentoo         4800 female  2007
## # ℹ 48 more rows

Notice that variable names do not have to be written inside single nor double quotation marks ('' or ""). This is the case for all functions presented below.

The pipe operator

As you can see from the code above, the filter() functions requires the name of the data frame as first argument, then the condition (with the usual logical operators >, <, >=, <=, ==, !=, %in%, etc.) as second argument.

Specifying the name of the data frame as first argument is required for all functions presented in this list. However, there is a workaround to specifying the data frame’s name inside the functions: the pipe operator (|> or %>%).¹

The pipe operator allows to perform a sequence of several operations, that is, chain a sequence of calculations together. It is particularly useful when you are performing several operations on a data frame, and you do not want to save the output at each intermediate step. We will see below how to use the pipe operator with several operations, but for now I would like to introduce it with only one operation at a time.

As you can see with the filter() function, the pipe operator is not compulsory. However, I recommend it so much (even to beginners) for its easy of use, convenience, code readability and popularity that from now on functions available in {dplyr} will be presented together with the pipe operator.

So with the pipe operator, the code above becomes:

# filter observations based on a quantitative variable
dat |>
  filter(body_mass_g > 4000)

## # A tibble: 172 × 4
##    species body_mass_g sex    year
##    <fct>         <int> <fct> <int>
##  1 Adelie         4675 male   2007
##  2 Adelie         4250 <NA>   2007
##  3 Adelie         4400 male   2007
##  4 Adelie         4500 male   2007
##  5 Adelie         4200 male   2007
##  6 Adelie         4150 male   2007
##  7 Adelie         4650 male   2007
##  8 Adelie         4400 male   2007
##  9 Adelie         4600 male   2007
## 10 Adelie         4150 male   2007
## # ℹ 162 more rows

# filter observations based on a qualitative variable
dat |>
  filter(sex == "female")

## # A tibble: 165 × 4
##    species body_mass_g sex     year
##    <fct>         <int> <fct>  <int>
##  1 Adelie         3800 female  2007
##  2 Adelie         3250 female  2007
##  3 Adelie         3450 female  2007
##  4 Adelie         3625 female  2007
##  5 Adelie         3200 female  2007
##  6 Adelie         3700 female  2007
##  7 Adelie         3450 female  2007
##  8 Adelie         3325 female  2007
##  9 Adelie         3400 female  2007
## 10 Adelie         3800 female  2007
## # ℹ 155 more rows

# filter observations based on 2 cumulative conditions
dat |>
  filter(body_mass_g > 4000 & sex == "female")

## # A tibble: 58 × 4
##    species body_mass_g sex     year
##    <fct>         <int> <fct>  <int>
##  1 Gentoo         4500 female  2007
##  2 Gentoo         4450 female  2007
##  3 Gentoo         4550 female  2007
##  4 Gentoo         4800 female  2007
##  5 Gentoo         4400 female  2007
##  6 Gentoo         4650 female  2007
##  7 Gentoo         4650 female  2007
##  8 Gentoo         4200 female  2007
##  9 Gentoo         4150 female  2007
## 10 Gentoo         4800 female  2007
## # ℹ 48 more rows

The pipe operator simply takes the results of one operation into the next operation below it, making the code extremely easy to write and read.

This way, instead of specifying the data frame’s name as first argument in the filter() function (or any other function within the {dplyr} package), we simply specify the data frame’s name and then the desired function, combined together thanks to the pipe operator.

Extract observations

It is possible to extract observations based on:

their positions, or
their values.

Based on their positions

Extracting observations based on their positions can be done with the slice() function:

# extract rows 2, 5 and 37
dat |>
  slice(c(2, 5, 37))

## # A tibble: 3 × 4
##   species body_mass_g sex     year
##   <fct>         <int> <fct>  <int>
## 1 Adelie         3800 female  2007
## 2 Adelie         3450 female  2007
## 3 Adelie         3950 male    2007

Moreover, extracting the first or last rows can be done with slice_head() and slice_tail():

# extract first 3 rows
dat |>
  slice_head(n = 3)

## # A tibble: 3 × 4
##   species body_mass_g sex     year
##   <fct>         <int> <fct>  <int>
## 1 Adelie         3750 male    2007
## 2 Adelie         3800 female  2007
## 3 Adelie         3250 female  2007

# extract last 3 rows
dat |>
  slice_tail(n = 3)

## # A tibble: 3 × 4
##   species   body_mass_g sex     year
##   <fct>           <int> <fct>  <int>
## 1 Chinstrap        3775 male    2009
## 2 Chinstrap        4100 male    2009
## 3 Chinstrap        3775 female  2009

Based on their values

To extract observations based on values of a variable, use:

slice_min() to select rows with the lowest values (with a defined proportion), and
slice_max() to select rows with the highest values (with a defined proportion).

# extract observations with 25% lowest body mass
dat |>
  slice_min(body_mass_g, prop = 0.25)

## # A tibble: 89 × 4
##    species   body_mass_g sex     year
##    <fct>           <int> <fct>  <int>
##  1 Chinstrap        2700 female  2008
##  2 Adelie           2850 female  2008
##  3 Adelie           2850 female  2008
##  4 Adelie           2900 female  2008
##  5 Adelie           2900 female  2008
##  6 Adelie           2900 female  2009
##  7 Chinstrap        2900 female  2007
##  8 Adelie           2925 female  2009
##  9 Adelie           2975 <NA>    2007
## 10 Adelie           3000 female  2007
## # ℹ 79 more rows

# extract observations with 25% highest body mass
dat |>
  slice_max(body_mass_g, prop = 0.25)

## # A tibble: 90 × 4
##    species body_mass_g sex    year
##    <fct>         <int> <fct> <int>
##  1 Gentoo         6300 male   2007
##  2 Gentoo         6050 male   2007
##  3 Gentoo         6000 male   2008
##  4 Gentoo         6000 male   2009
##  5 Gentoo         5950 male   2008
##  6 Gentoo         5950 male   2009
##  7 Gentoo         5850 male   2007
##  8 Gentoo         5850 male   2007
##  9 Gentoo         5850 male   2009
## 10 Gentoo         5800 male   2008
## # ℹ 80 more rows

Sample observations

Sampling observations can be done in two ways:

Random sample of a number of rows with sample_n()
Random sample of a fraction of rows with sample_frac()

# random sample of 3 rows
dat |>
  sample_n(size = 3)

## # A tibble: 3 × 4
##   species   body_mass_g sex     year
##   <fct>           <int> <fct>  <int>
## 1 Adelie           3450 female  2007
## 2 Chinstrap        3675 female  2009
## 3 Gentoo           4500 female  2007

# random sample of half of the rows
dat |>
  sample_frac(size = 1 / 2)

## # A tibble: 172 × 4
##    species   body_mass_g sex     year
##    <fct>           <int> <fct>  <int>
##  1 Adelie           4150 male    2008
##  2 Gentoo           5800 male    2008
##  3 Adelie           3650 male    2009
##  4 Adelie           3500 male    2009
##  5 Adelie           3450 female  2007
##  6 Adelie           4300 male    2009
##  7 Chinstrap        3400 female  2008
##  8 Adelie           3950 male    2007
##  9 Chinstrap        3325 female  2009
## 10 Adelie           3950 male    2008
## # ℹ 162 more rows

Note that, as with the sample() function within base R, size can be greater than the size of the data frame. In this case, some rows will be duplicated, and you will need to specify the argument replace = TRUE.

Alternatively, it is possible to obtain a random sample of a number of rows or fraction or rows with slice_sample(). For this, use:

the argument n to select a number of rows, or
the argument prop to select a fraction of rows.

# random sample of 3 rows
dat |>
  slice_sample(n = 3)

## # A tibble: 3 × 4
##   species body_mass_g sex     year
##   <fct>         <int> <fct>  <int>
## 1 Adelie         3200 female  2007
## 2 Adelie         3800 female  2007
## 3 Gentoo         4800 female  2007

# random sample of half of the rows
dat |>
  slice_sample(prop = 1 / 2)

## # A tibble: 172 × 4
##    species body_mass_g sex     year
##    <fct>         <int> <fct>  <int>
##  1 Adelie         3900 male    2009
##  2 Adelie         3275 female  2009
##  3 Gentoo         5050 male    2008
##  4 Gentoo         4700 female  2009
##  5 Gentoo         4600 female  2008
##  6 Gentoo         4875 <NA>    2009
##  7 Adelie         3700 <NA>    2007
##  8 Gentoo         3950 female  2008
##  9 Gentoo         4550 female  2007
## 10 Adelie         3500 female  2008
## # ℹ 162 more rows

Sort observations

Sorting observations can be done with the arrange() function:

# sort observations based on body mass (ascending order)
dat |>
  arrange(body_mass_g)

## # A tibble: 344 × 4
##    species   body_mass_g sex     year
##    <fct>           <int> <fct>  <int>
##  1 Chinstrap        2700 female  2008
##  2 Adelie           2850 female  2008
##  3 Adelie           2850 female  2008
##  4 Adelie           2900 female  2008
##  5 Adelie           2900 female  2008
##  6 Adelie           2900 female  2009
##  7 Chinstrap        2900 female  2007
##  8 Adelie           2925 female  2009
##  9 Adelie           2975 <NA>    2007
## 10 Adelie           3000 female  2007
## # ℹ 334 more rows

By default, arrange() uses the ascending order. To sort in descending order, use desc() inside arrange():

# sort observations based on body mass (descending order)
dat |>
  arrange(desc(body_mass_g))

## # A tibble: 344 × 4
##    species body_mass_g sex    year
##    <fct>         <int> <fct> <int>
##  1 Gentoo         6300 male   2007
##  2 Gentoo         6050 male   2007
##  3 Gentoo         6000 male   2008
##  4 Gentoo         6000 male   2009
##  5 Gentoo         5950 male   2008
##  6 Gentoo         5950 male   2009
##  7 Gentoo         5850 male   2007
##  8 Gentoo         5850 male   2007
##  9 Gentoo         5850 male   2009
## 10 Gentoo         5800 male   2008
## # ℹ 334 more rows

As with filter(), arrange() can be used for several variables and works both on quantitative and qualitative variables:

# sort observations based on two variables
dat |>
  arrange(sex, body_mass_g)

## # A tibble: 344 × 4
##    species   body_mass_g sex     year
##    <fct>           <int> <fct>  <int>
##  1 Chinstrap        2700 female  2008
##  2 Adelie           2850 female  2008
##  3 Adelie           2850 female  2008
##  4 Adelie           2900 female  2008
##  5 Adelie           2900 female  2008
##  6 Adelie           2900 female  2009
##  7 Chinstrap        2900 female  2007
##  8 Adelie           2925 female  2009
##  9 Adelie           3000 female  2007
## 10 Adelie           3000 female  2009
## # ℹ 334 more rows

The code above sorts the observations first based on the sex (in alphabetical order) and then based on the body mass (in ascending order, so from lowest to highest).

Note that if the qualitative variable is defined as an ordered factor, the sorting is based on level order, not alphabetical order!

Select variables

Selecting variables can be done with the select() function, based on:

the position of the variable(s), or
the name(s) of the variable(s).

# select variables by their positions
dat |>
  select(c(2, 4))

## # A tibble: 344 × 2
##    body_mass_g  year
##          <int> <int>
##  1        3750  2007
##  2        3800  2007
##  3        3250  2007
##  4          NA  2007
##  5        3450  2007
##  6        3650  2007
##  7        3625  2007
##  8        4675  2007
##  9        3475  2007
## 10        4250  2007
## # ℹ 334 more rows

# select variables by their names
dat |>
  select(body_mass_g, year)

## # A tibble: 344 × 2
##    body_mass_g  year
##          <int> <int>
##  1        3750  2007
##  2        3800  2007
##  3        3250  2007
##  4          NA  2007
##  5        3450  2007
##  6        3650  2007
##  7        3625  2007
##  8        4675  2007
##  9        3475  2007
## 10        4250  2007
## # ℹ 334 more rows

Note that it is also possible to remove variables. For this, use the - sign in front of their positions or names:

# remove variables by their positions
dat |>
  select(-c(2, 4))

## # A tibble: 344 × 2
##    species sex   
##    <fct>   <fct> 
##  1 Adelie  male  
##  2 Adelie  female
##  3 Adelie  female
##  4 Adelie  <NA>  
##  5 Adelie  female
##  6 Adelie  male  
##  7 Adelie  female
##  8 Adelie  male  
##  9 Adelie  <NA>  
## 10 Adelie  <NA>  
## # ℹ 334 more rows

# remove variables by their names
dat |>
  select(-c(body_mass_g, year))

## # A tibble: 344 × 2
##    species sex   
##    <fct>   <fct> 
##  1 Adelie  male  
##  2 Adelie  female
##  3 Adelie  female
##  4 Adelie  <NA>  
##  5 Adelie  female
##  6 Adelie  male  
##  7 Adelie  female
##  8 Adelie  male  
##  9 Adelie  <NA>  
## 10 Adelie  <NA>  
## # ℹ 334 more rows

It is also possible to select variables with a sequence of names:

# select all variables from species to sex
dat |>
  select(species:sex)

## # A tibble: 344 × 3
##    species body_mass_g sex   
##    <fct>         <int> <fct> 
##  1 Adelie         3750 male  
##  2 Adelie         3800 female
##  3 Adelie         3250 female
##  4 Adelie           NA <NA>  
##  5 Adelie         3450 female
##  6 Adelie         3650 male  
##  7 Adelie         3625 female
##  8 Adelie         4675 male  
##  9 Adelie         3475 <NA>  
## 10 Adelie         4250 <NA>  
## # ℹ 334 more rows

Last but not least, select() can also be used as an easy way to rearrange columns in the desired order:

# put sex as first column, then all the others
dat |>
  select(sex, species:year)

## # A tibble: 344 × 4
##    sex    species body_mass_g  year
##    <fct>  <fct>         <int> <int>
##  1 male   Adelie         3750  2007
##  2 female Adelie         3800  2007
##  3 female Adelie         3250  2007
##  4 <NA>   Adelie           NA  2007
##  5 female Adelie         3450  2007
##  6 male   Adelie         3650  2007
##  7 female Adelie         3625  2007
##  8 male   Adelie         4675  2007
##  9 <NA>   Adelie         3475  2007
## 10 <NA>   Adelie         4250  2007
## # ℹ 334 more rows

Rename variables

To rename variables, use the rename() function:

# rename variables
dat |>
  rename(
    body_mass = body_mass_g, # rename body_mass_g into body_mass
    study_year = year # rename year into study_year
  )

## # A tibble: 344 × 4
##    species body_mass sex    study_year
##    <fct>       <int> <fct>       <int>
##  1 Adelie       3750 male         2007
##  2 Adelie       3800 female       2007
##  3 Adelie       3250 female       2007
##  4 Adelie         NA <NA>         2007
##  5 Adelie       3450 female       2007
##  6 Adelie       3650 male         2007
##  7 Adelie       3625 female       2007
##  8 Adelie       4675 male         2007
##  9 Adelie       3475 <NA>         2007
## 10 Adelie       4250 <NA>         2007
## # ℹ 334 more rows

This might not be intuitive (at least it was not for me at the time of learning this package), so bear in mind that you always need to write first the new name and then the old name (separated with the = sign).

Create or modify variables

You can create or modify certain variables of the data frame with mutate(), based on:

another variable, or
a vector of your choice.

# create a new variable based on an existing one
dat |>
  mutate(
    body_mass_kg = body_mass_g / 1000
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_kg
##    <fct>         <int> <fct>  <int>        <dbl>
##  1 Adelie         3750 male    2007         3.75
##  2 Adelie         3800 female  2007         3.8 
##  3 Adelie         3250 female  2007         3.25
##  4 Adelie           NA <NA>    2007        NA   
##  5 Adelie         3450 female  2007         3.45
##  6 Adelie         3650 male    2007         3.65
##  7 Adelie         3625 female  2007         3.62
##  8 Adelie         4675 male    2007         4.68
##  9 Adelie         3475 <NA>    2007         3.48
## 10 Adelie         4250 <NA>    2007         4.25
## # ℹ 334 more rows

# create a new variable from a vector of your choice
dat |>
  mutate(
    ID = 1:nrow(dat)
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year    ID
##    <fct>         <int> <fct>  <int> <int>
##  1 Adelie         3750 male    2007     1
##  2 Adelie         3800 female  2007     2
##  3 Adelie         3250 female  2007     3
##  4 Adelie           NA <NA>    2007     4
##  5 Adelie         3450 female  2007     5
##  6 Adelie         3650 male    2007     6
##  7 Adelie         3625 female  2007     7
##  8 Adelie         4675 male    2007     8
##  9 Adelie         3475 <NA>    2007     9
## 10 Adelie         4250 <NA>    2007    10
## # ℹ 334 more rows

Note that if you create a variable with a name which already exists in the data frame, the old variable will be erased and replaced by the new one.

Like rename(), mutate() requires the argument to be written as name = expression, where name is name of the column created or modified and expression is the formula for calculating the values.

Summarize observations

Often, you will want to summarize the data with some descriptive statistics. This can be done with the summarize() function, in addition to most functions used for descriptive statistics (mean(), median(), min(), max(), sd(), var(), etc.):

# compute mean and sd of body mass
dat |>
  summarize(
    body_mass_mean = mean(body_mass_g, na.rm = TRUE),
    body_mass_sd = sd(body_mass_g, na.rm = TRUE)
  )

## # A tibble: 1 × 2
##   body_mass_mean body_mass_sd
##            <dbl>        <dbl>
## 1          4202.         802.

Note that:

The na.rm = TRUE argument is used to omit missing values in the computation of the summary statistics.
summarize() and summarise() give the exact same results.

Identify distinct values

Identifying distinct values of a variable can be done with distinct():

# find the distinct species
dat |>
  distinct(species)

## # A tibble: 3 × 1
##   species  
##   <fct>    
## 1 Adelie   
## 2 Gentoo   
## 3 Chinstrap

Identifying distinct values is mostly done on qualitative or quantitative discrete variables, but it can be done on any type of variable and with several variables at the same time. If more than one variables is specified, it returns all the combinations of values of the variables.

For instance, with species and study year:

# combination of distinct species and year
dat |>
  distinct(species, year)

## # A tibble: 9 × 2
##   species    year
##   <fct>     <int>
## 1 Adelie     2007
## 2 Adelie     2008
## 3 Adelie     2009
## 4 Gentoo     2007
## 5 Gentoo     2008
## 6 Gentoo     2009
## 7 Chinstrap  2007
## 8 Chinstrap  2008
## 9 Chinstrap  2009

Connected operations

Another advantage of using the {dplyr} package is that several operations can be connected all at once, with great readability of the code. This can easily be done with the pipe operator (|> or %>%) introduced earlier.

Until now, we have always seen the same structure: we call a data frame, and then we apply an operation on that data frame. From now on, we will see how to combine more operations into one single chain of operations.

Group by

group_by() allows to modify the way the basic functions are performed. Instead of covering all the rows of the data frame, the operations will cover each of the groups of rows defined by the grouping command. In this way, aggregation operations, using summarize(), will produce statistics for each group rather than for all observations.

For example, we might be interested in computing some descriptive statistics of a quantitative variable, for each level of a qualitative variable (so by group).

In our case, suppose we would like to compute the mean and standard deviation of the body mass, but this time separately for each species:

# compute mean and sd of body mass by species
dat |>
  group_by(species) |> # group by species
  summarize(
    mean = mean(body_mass_g, na.rm = TRUE), # compute mean
    sd = sd(body_mass_g, na.rm = TRUE) # compute sd
  )

## # A tibble: 3 × 3
##   species    mean    sd
##   <fct>     <dbl> <dbl>
## 1 Adelie    3701.  459.
## 2 Chinstrap 3733.  384.
## 3 Gentoo    5076.  504.

Applied to the example above, here is how the pipe operator works:

The first operation (group_by()) groups observations by species.
Then the output of the first operation is used as the input for the second operation (summarize()): mean and standard deviation are computed on body mass.

As a result, we have the mean and standard deviation of body mass by group. As you can imagine, all previous operations can be connected to match your needs.

Also note that group_by() can be used for several grouping variables at the same time:

# compute mean and sd of body mass by species and sex
dat |>
  group_by(species, sex) |> # group by species and sex
  summarize(
    mean = mean(body_mass_g, na.rm = TRUE), # compute mean
    sd = sd(body_mass_g, na.rm = TRUE) # compute sd
  )

## # A tibble: 8 × 4
## # Groups:   species [3]
##   species   sex     mean    sd
##   <fct>     <fct>  <dbl> <dbl>
## 1 Adelie    female 3369.  269.
## 2 Adelie    male   4043.  347.
## 3 Adelie    <NA>   3540   477.
## 4 Chinstrap female 3527.  285.
## 5 Chinstrap male   3939.  362.
## 6 Gentoo    female 4680.  282.
## 7 Gentoo    male   5485.  313.
## 8 Gentoo    <NA>   4588.  338.

(Note that, as for all connected operations, the name of the data frame needs to be specified only in the first operation.)

Number of observations

Some operations can only be performed inside other operations.

This is the case with the number of observations n(), which can only be used inside summarize():

# number of observations
dat |>
  summarize(n_obs = n())

## # A tibble: 1 × 1
##   n_obs
##   <int>
## 1   344

To refer with the previous connected operation group_by(), we can compute the number of observations by group using the two operations n() and group_by() separated by the pipe operator:

# number of observations by species
dat |>
  group_by(species) |>
  summarize(n_obs = n())

## # A tibble: 3 × 2
##   species   n_obs
##   <fct>     <int>
## 1 Adelie      152
## 2 Chinstrap    68
## 3 Gentoo      124

Note that n() accepts no parameters and is therefore always written with empty parentheses.

Moreover, note also that the count() function is equivalent to summarize(n = n()):

# number of observations
dat |>
  count()

## # A tibble: 1 × 1
##       n
##   <int>
## 1   344

# number of observations by species
dat |>
  count(species)

## # A tibble: 3 × 2
##   species       n
##   <fct>     <int>
## 1 Adelie      152
## 2 Chinstrap    68
## 3 Gentoo      124

Number of distinct values

n_distinct(), which can also be used only inside summarize(), computes the number of different values/levels of a variable or combination of variables:

# number of distinct species
dat |>
  summarize(n_species = n_distinct(species))

## # A tibble: 1 × 1
##   n_species
##       <int>
## 1         3

# number of distinct species and year of study
dat |>
  summarize(n_species_year = n_distinct(species, year))

## # A tibble: 1 × 1
##   n_species_year
##            <int>
## 1              9

Note that you do not have to specify a name for the output. In that case, the name of the operation will be used. For example:

# number of distinct species
dat |>
  summarize(n_distinct(species))

## # A tibble: 1 × 1
##   `n_distinct(species)`
##                   <int>
## 1                     3

First, last or nth value

Also only available inside summarize(), the first, last or nth value can be found with the following commands:

# first value of the variable sex
dat |>
  summarize(first(sex))

## # A tibble: 1 × 1
##   `first(sex)`
##   <fct>       
## 1 male

# last value of the variable sex
dat |>
  summarize(last(sex))

## # A tibble: 1 × 1
##   `last(sex)`
##   <fct>      
## 1 female

# 37th value of the variable sex
dat |>
  summarize(nth(sex, n = 37))

## # A tibble: 1 × 1
##   `nth(sex, n = 37)`
##   <fct>             
## 1 male

Several interesting arguments exist within this function to deal with missing values. For the interested reader, see more information in the documentation of the function (run ?nth()).

If else

A very common data transformation is the well known “if else” technique. This technique is usually used to create, from an existing variable, another variable which can take two levels.

Suppose that we want to create a new variable called body_mass_cat, which takes the value “High” when body_mass_g is equal or greater than a certain threshold, “Low” otherwise. This transformation can be performed with the combination of mutate() and if_else():

# if else
dat |>
  mutate(
    body_mass_cat = if_else(body_mass_g >= 4000, # condition
      "High", # output if condition is true
      "Low" # output if condition is false
    )
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_cat
##    <fct>         <int> <fct>  <int> <chr>        
##  1 Adelie         3750 male    2007 Low          
##  2 Adelie         3800 female  2007 Low          
##  3 Adelie         3250 female  2007 Low          
##  4 Adelie           NA <NA>    2007 <NA>         
##  5 Adelie         3450 female  2007 Low          
##  6 Adelie         3650 male    2007 Low          
##  7 Adelie         3625 female  2007 Low          
##  8 Adelie         4675 male    2007 High         
##  9 Adelie         3475 <NA>    2007 Low          
## 10 Adelie         4250 <NA>    2007 High         
## # ℹ 334 more rows

The if_else() function works with 3 arguments:

The condition (in our case: body_mass_g >= 4000)
The output value when the condition is true (High in our case).
The output value when the conditions is false (Low in our case).

As you can see from the table above, when body mass is missing, if_else() also returns a missing value, which is often a good thing to prevent observations being classified erroneously.

Case when

If you want to categorize a variable into more than two levels, an if else is not the most appropriate tool. In these cases, a “case when” is more appropriate.

For your information, when I learned R, I used to write nested if else functions, that is, a secondary if else inside a primary if else. Most of the time it worked (with very often a waste of time trying to debug my code), but it is very easy to make a mistake. And even if you managed to make it work, the code is not easy to read at all!

So I highly recommend using this case when technique instead of several if else functions nested within each other.

Suppose we want to classify body mass into 3 categories: low, medium and high. For this illustration, we arbitrarily decide that body mass is low when it is strictly lower than 3500, high when it is strictly higher than 4750 and medium otherwise.

With nested if else functions, here is the code we would need to write:

# nested if else
dat |>
  mutate(
    body_mass_cat = if_else(body_mass_g < 3500, # first condition
      "Low", # output if first condition is true
      if_else(body_mass_g > 4750, # second condition when first condition is false
        "High", # output when second condition is true
        "Medium" # output when second condition is false
      )
    )
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_cat
##    <fct>         <int> <fct>  <int> <chr>        
##  1 Adelie         3750 male    2007 Medium       
##  2 Adelie         3800 female  2007 Medium       
##  3 Adelie         3250 female  2007 Low          
##  4 Adelie           NA <NA>    2007 <NA>         
##  5 Adelie         3450 female  2007 Low          
##  6 Adelie         3650 male    2007 Medium       
##  7 Adelie         3625 female  2007 Medium       
##  8 Adelie         4675 male    2007 Medium       
##  9 Adelie         3475 <NA>    2007 Low          
## 10 Adelie         4250 <NA>    2007 Medium       
## # ℹ 334 more rows

This code works as follows:

It evaluates the first condition body_mass_g < 3500.
If it is true, body_mass_cat is Low. On the contrary, if it is false, it evaluates the second condition body_mass_g > 4750.
If this second condition is true, body_mass_cat is High, otherwise it is Medium.

As you can see from the results above, it works. However, you will concede that it is easy to make coding mistakes, and that the code is not easy to write nor to read.

To improve this workflow, we now use the case when technique:

# case when, without a default option
dat |>
  mutate(
    body_mass_cat = case_when(
      body_mass_g < 3500 ~ "Low",
      body_mass_g >= 3500 & body_mass_g <= 4750 ~ "Medium",
      body_mass_g > 4750 ~ "High"
    )
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_cat
##    <fct>         <int> <fct>  <int> <chr>        
##  1 Adelie         3750 male    2007 Medium       
##  2 Adelie         3800 female  2007 Medium       
##  3 Adelie         3250 female  2007 Low          
##  4 Adelie           NA <NA>    2007 <NA>         
##  5 Adelie         3450 female  2007 Low          
##  6 Adelie         3650 male    2007 Medium       
##  7 Adelie         3625 female  2007 Medium       
##  8 Adelie         4675 male    2007 Medium       
##  9 Adelie         3475 <NA>    2007 Low          
## 10 Adelie         4250 <NA>    2007 Medium       
## # ℹ 334 more rows

This workflow is much simpler to code and read!

If there are no missing values in the variable(s) used for the condition(s), it can even be simplified to:

# case when, with a default option
dat |>
  mutate(
    body_mass_cat = case_when(
      body_mass_g < 3500 ~ "Low",
      body_mass_g > 4750 ~ "High",
      .default = "Medium" # default output
    )
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_cat
##    <fct>         <int> <fct>  <int> <chr>        
##  1 Adelie         3750 male    2007 Medium       
##  2 Adelie         3800 female  2007 Medium       
##  3 Adelie         3250 female  2007 Low          
##  4 Adelie           NA <NA>    2007 Medium       
##  5 Adelie         3450 female  2007 Low          
##  6 Adelie         3650 male    2007 Medium       
##  7 Adelie         3625 female  2007 Medium       
##  8 Adelie         4675 male    2007 Medium       
##  9 Adelie         3475 <NA>    2007 Low          
## 10 Adelie         4250 <NA>    2007 Medium       
## # ℹ 334 more rows

As you can see, a default output can be specified with .default for observations that do not match any of the conditions.

However, be careful if there are missing values! Indeed, if there is at least one missing value (as in our case), the code above is not correct because observations with missing body_mass_g will be misclassified as Medium.

Therefore, if you really want to specify a default output, I recommend using the code below which keeps missing values as NA:

# case when, with a default option and missing values
dat |>
  mutate(
    body_mass_cat = case_when(
      body_mass_g < 3500 ~ "Low",
      body_mass_g > 4750 ~ "High",
      is.na(body_mass_g) ~ NA, # keep missing values as NA
      .default = "Medium" # default output
    )
  )

## # A tibble: 344 × 5
##    species body_mass_g sex     year body_mass_cat
##    <fct>         <int> <fct>  <int> <chr>        
##  1 Adelie         3750 male    2007 Medium       
##  2 Adelie         3800 female  2007 Medium       
##  3 Adelie         3250 female  2007 Low          
##  4 Adelie           NA <NA>    2007 <NA>         
##  5 Adelie         3450 female  2007 Low          
##  6 Adelie         3650 male    2007 Medium       
##  7 Adelie         3625 female  2007 Medium       
##  8 Adelie         4675 male    2007 Medium       
##  9 Adelie         3475 <NA>    2007 Low          
## 10 Adelie         4250 <NA>    2007 Medium       
## # ℹ 334 more rows

I personally prefer to write all categories and not write a default option for improved code readability and robustness of my code, but it is more a personal opinion.

In all cases, no matter if you used an if else or a case when, it is a good practice to check the variable you just created to make sure that you obtain the intended results.

Conclusion and other resources

Thanks for reading.

This article introduced and illustrated the most common data manipulation and data management tools in R, using the {dplyr} package. We also introduced the pipe operator, well known to users of modern R packages.

The {dplyr} package offers many more functions for data manipulation (in particular to merge data frames with the family of join() functions). If you would like to learn more about this package, I recommend starting with the following resources:

dplyr.tidyverse.org
Chapter “Data transformation” in the book “R for Data Science”
Cheatsheet
Vignette
For those who are used to base R, a vignette comparing {dplyr} functions to their base R equivalents

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Horst, Allison Marie, Alison Presmanes Hill, and Kristen B Gorman. 2020. Palmerpenguins: Palmer Archipelago (Antarctica) Penguin Data. https://doi.org/10.5281/zenodo.3960218.

The keyboard shortcut for the pipe operator is ctrl + shift + m (Windows) or cmd + shift + m (Mac). It will print %>%, unless you specified to use the native pipe operator |> in the settings of RStudio.↩︎

Scrape Yahoo search engine results with R

Thu, 24 Aug 2023 00:00:00 +0000

Note: This is a guest post by Manthan Koolwal, founder of Scrapingdog.

Introduction

Web scraping is the process of extracting data from websites. It is usually done in an automated manner to obtain large amounts of data through various websites, without the need to gather data by hand.

In a previous post, we introduced this method and illustrated it with a Wikipedia page. Although there are a lot of use cases of web scraping, in this blog post, we are restricting ourselves to scraping search results from Yahoo using R. Scraping search engine results can help you with SEO analysis, competitor analysis, keyword research, trend analysis, etc.

Scraping Yahoo search engine results with R

After installing R and RStudio, we first need to load the necessary packages by running the following commands:¹

# install.packages("rvest")
# install.packages("jsonlite")
# install.packages("purrr")

library(rvest)
library(jsonlite)
library(purrr)

The {rvest} package is for web scraping, the {jsonlite} package is for working with JSON data and the {purrr} package is for working with functions and vectors.

It is always better to decide in advance what exactly we are going to scrape. For this tutorial, we are going to scrape search results from this URL:

We are going to scrape the following data points from this page:

Link
Title
Description

For this, we define the URL of the Yahoo search results page that we want to scrape. In this case, we are searching for the word “pizza”.

# URL of the Yahoo search results page
url <- "https://search.yahoo.com/search?p=pizza"

We then use the read_html() function from the {rvest} package to read the HTML content of the provided URL:

# Read the HTML content of the page
page <- read_html(url)

This creates an HTML document object that we can work with:

str(page)

## List of 2
##  $ node:<externalptr> 
##  $ doc :<externalptr> 
##  - attr(*, "class")= chr [1:2] "xml_document" "xml_node"

Here’s where we start the process of extracting the search results:

# Extract search results
results <- page %>%
  html_nodes(".algo-sr") %>% # Selector for search result elements
  html_nodes("a") %>% # Select the <a> elements within the search results
  # Extract link, title, and description attributes
  map_df(~ data.frame(
    link = .x %>% html_attr("href"),
    title = .x %>% html_text(),
    description = .x %>% html_attr("title"),
    stringsAsFactors = FALSE
  ))

In the code above:

We use %>% (pipe operator) to chain multiple operations together for clarity.
First, we use html_nodes(".algo-sr") to select the search result elements with the class .algo-sr. These elements contain the links to the search results.
Within each search result element, we further select the <a> elements using html_nodes("a"). These <a> elements contain the link, title, and description information.
Using map_df(), we iterate through each <a> element and extract the link, title, and description attributes.
We create a data frame with these attributes for each search result.

Finally, we convert the results data frame into JSON format using the toJSON() function from the {jsonlite} package. The pretty = TRUE argument adds indentation for better readability. We use cat() to print the JSON-formatted results to the console.

# Print the results in JSON format
cat(toJSON(results, pretty = TRUE))

## [
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIASPJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzEEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.yelp.com%2fsearch%3ffind_desc%3dpizza%26find_loc%3dBrussels%252C%2bR%25C3%25A9gion%2bde%2bBruxelles-Capitale/RK=2/RS=xW.v1O1REkKKLsO7DRIWoIGq4bA-",
##     "title": "yelp.comhttps://www.yelp.com › searchTop 10 Best pizza Near Brussels, Région de Bruxelles-Capitale"
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIASfJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzIEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.yelp.com%2fsearch%3fcflt%3dpizza%26find_loc%3dBrussels%252C%2bR%25C3%25A9gion%2bde%2bBruxelles-Capitale/RK=2/RS=zynV6ypY6PoaPj4cg5yD9a1x.LU-",
##     "title": "yelp.comhttps://www.yelp.com › searchTHE BEST 10 PIZZA PLACES IN BRUSSELS, RÉGION DE ... - Yelp"
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIASvJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.dominos.com%2fen/RK=2/RS=darBKTs8REd78kRRFhZb8jRT4jY-",
##     "title": "Domino'shttps://www.dominos.com › enPizza Delivery & Carryout, Pasta, Chicken & More | Domino's"
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAS_JXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.bing.com%2fimages%2fsearch%3fview%3ddetailV2%26ccid%3ds2TCGJY8%26id%3d7236CCFBE7C0B5849BA9DE100BE4F1F32FCD231D%26thid%3dOIP.s2TCGJY8bjV7G5kps1FEwQHaHa%26mediaurl%3dhttps%3a%2f%2fwww.dominos.com%2fstatic%2f1.98.1%2fimages%2ftiles%2fmixAndMatchDeal%2fhero.webp%26exph%3d530%26expw%3d530%26q%3dpizza%26ck%3d8B813BA6312D3F558421CB691D0C6657%26idpp%3drc%26idpview%3dsingleimage%26form%3drc2idp/RK=2/RS=xsbqmguwIFBhAOzGrm.qrjLykcA-",
##     "title": ""
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIATPJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.bing.com%2fimages%2fsearch%3fview%3ddetailV2%26ccid%3d%2f6Qs28Cq%26id%3d7236CCFBE7C0B5849BA9E9CE4472C5692CE3F420%26thid%3dOIP._6Qs28CqWWCt4S-yapYAXwHaHa%26mediaurl%3dhttps%3a%2f%2fwww.dominos.com%2fstatic%2f1.116.0%2fimages%2ftiles%2fmixAndMatchDealPSC%2fhero.webp%26exph%3d530%26expw%3d530%26q%3dpizza%26ck%3d758519383F5F3AF0EA83707449E54C87%26idpp%3drc%26idpview%3dsingleimage%26form%3drc2idp/RK=2/RS=UHqcKBQXuq8AtTptzc.7rkut3rY-",
##     "title": ""
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIATfJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.bing.com%2fimages%2fsearch%3fview%3ddetailV2%26ccid%3dZ33s1urJ%26id%3d7236CCFBE7C0B5849BA9D44071E0F214A85C5EAF%26thid%3dOIP.Z33s1urJ3ZHwJtgKZtWHJAHaDS%26mediaurl%3dhttps%3a%2f%2fwww.dominos.com%2fcms%2fassets%2f58a8e864-d1aa-457a-9d37-fcc1912b10ad%3fim%3dCrop%2crect%3d%28774%2c1015%2c4331%2c1920%29%3bResize%3d%281700%29%2callowExpansion%26exph%3d754%26expw%3d1700%26q%3dpizza%26ck%3d1274A2028C287B1E2E3A8B0EB2F9DE7D%26idpp%3drc%26idpview%3dsingleimage%26form%3drc2idp/RK=2/RS=d9hH9.gzp8cqI8s0XYrAtGSVGXg-",
##     "title": ""
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIATvJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.bing.com%2fimages%2fsearch%3fview%3ddetailV2%26ccid%3deK8nC663%26id%3d7236CCFBE7C0B5849BA99FA63EED83A3990E2BF8%26thid%3dOIP.eK8nC6636jB9TYTyjaHNGgHaFj%26mediaurl%3dhttps%3a%2f%2fwww.dominos.com%2fstatic%2f1.83.3%2fimages%2ftiles%2fperfectComboDeal%2fside.webp%26exph%3d458%26expw%3d610%26q%3dpizza%26ck%3dCD805B6ADACFFC2697D3BE3D36098B7E%26idpp%3drc%26idpview%3dsingleimage%26form%3drc2idp/RK=2/RS=KY2oqWHmG3Y6yqMzcVPRlshaVsg-",
##     "title": ""
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAT_JXNyoA;_ylu=Y29sbwNiZjEEcG9zAzMEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.bing.com%2fimages%2fsearch%3fview%3ddetailV2%26ccid%3d%2bf29F0Yv%26id%3d7236CCFBE7C0B5849BA985F6EAF3041631C46CA1%26thid%3dOIP.-f29F0YvYvZaKklmamQDSwHaDS%26mediaurl%3dhttps%3a%2f%2fwww.dominos.com%2fcms%2fassets%2f43778453-e8b1-4db6-83fa-00f2dc43d6e3%3fim%3dCrop%2crect%3d%28374%2c325%2c3250%2c1441%29%3bResize%3d%281700%29%2callowExpansion%26exph%3d754%26expw%3d1700%26q%3dpizza%26ck%3d66B92DE33E5F605B85BE7C83BE11836C%26idpp%3drc%26idpview%3dsingleimage%26form%3drc2idp/RK=2/RS=15I_QuI3QQbRtWZj5auaBfXOKWQ-",
##     "title": ""
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAYfJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzQEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.albacioixelles.com%2f/RK=2/RS=GTpw.m6z.2eGOKxyo3K8xVrqSPE-",
##     "title": "Albacioixelleshttps://www.albacioixelles.comAuthentic | Albacioixelles | Elsene"
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAYvJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzUEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.nonalife.com%2f/RK=2/RS=d8sI_7Fy4oweSN3IQQ32zJSfgRY-",
##     "title": "Nonahttps://www.nonalife.comNona"
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAY_JXNyoA;_ylu=Y29sbwNiZjEEcG9zAzYEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=https%3a%2f%2fwww.tripadvisor.com%2fRestaurant_Review-g188644-d2480305-Reviews-Pizza_Saco-Brussels.html/RK=2/RS=52RUI7KZf9vt_R13PVY7sv04RMQ-",
##     "title": "Tripadvisorhttps://www.tripadvisor.com › Restaurant_Review-g188644-dPIZZA SACO, Brussels - Avenue Milcamps 154 - Restaurant ..."
##   },
##   {
##     "link": "https://r.search.yahoo.com/_ylt=AwrEteLKS0ZqawIAZPJXNyoA;_ylu=Y29sbwNiZjEEcG9zAzcEdnRpZAMEc2VjA3Ny/RV=2/RE=1784201418/RO=10/RU=http%3a%2f%2fwww.pizzasaco.com%2f/RK=2/RS=Zi2tEoNCQKejdFUdm0zsZwuqoig-",
##     "title": "pizzasaco.comhttp://www.pizzasaco.comSaco Pizza Bar"
##   }
## ]

Conclusion

Thanks for reading.

This was a simple tutorial in which we scraped Yahoo search results with R. Following the same process, you can create your own web crawler which can scrape search results from Yahoo for any web query.

Of course, you can scrape other search engines with almost the same technique. Also, you can check out this tutorial on web scraping Google search results using Python.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Note that, as for any R package, it must first be installed (with the install.packages() function) before being loaded (with the library() function).↩︎

Two-way ANOVA in R

Mon, 19 Jun 2023 00:00:00 +0000

Introduction

The two-way ANOVA (analysis of variance) is a statistical method that allows to evaluate the simultaneous effect of two categorical variables on a quantitative continuous variable.

The two-way ANOVA is an extension of the one-way ANOVA since it allows to evaluate the effects on a numerical response of two categorical variables instead of one.

The advantage of a two-way ANOVA over a one-way ANOVA is that we test the relationship between two variables, while taking into account the effect of a third variable. Moreover, it also allows to include the possible interaction of the two categorical variables on the response to evaluate whether or not they act jointly on the response variable.

The advantage of a two-way over a one-way ANOVA is quite similar to the advantage of a multiple linear regression over a correlation:

The correlation measures the relationship between two quantitative variables. The multiple linear regression also measures the relationship between two variables, but this time taking into account the potential effect of other covariates.
The one-way ANOVA tests whether a quantitative variable is different between groups. The two-way ANOVA also tests whether a quantitative variable is different between groups, but this time taking into account the effect of another qualitative variable.

Previously, we have discussed about one-way ANOVA in R. Now, we show when, why and how to perform a two-way ANOVA in R.

Before going further, I would like to mention and briefly describe some related statistical methods and tests in order to avoid any confusion:

A Student’s t-test is used to evaluate the effect of one categorical variable on a quantitative continuous variable, when the categorical variable has exactly 2 levels:
- Student’s t-test for independent samples if the observations are independent (for example: if we compare the age between women and men)
- Student’s t-test for paired samples if the observations are dependent, that is, when they come in pairs (it is the case when the same subjects are measured twice, at two different points in time, before and after a treatment for example)
To evaluate the effect of one categorical variable on a quantitative variable, when the categorical variable has 3 or more levels:¹
- one-way ANOVA (often simply referred as ANOVA) if the groups are independent (for example a group of patients who received treatment A, another group of patients who received treatment B, and the last group of patients who received no treatment or a placebo)
- repeated measures ANOVA if the groups are dependent (when the same subjects are measured three times, at three different points in time, before, during and after a treatment for example)
A two-way ANOVA is used to evaluate the effects of 2 categorical variables (and their potential interaction) on a quantitative continuous variable. This is the topic of the post.
Linear regression is used to evaluate the relationship between a quantitative continuous dependent variable and one or several independent variables:
- simple linear regression if there is only one independent variable (which can be quantitative or qualitative)
- multiple linear regression if there is at least two independent variables (which can be quantitative, qualitative, or a mix of both)
An ANCOVA (analysis of covariance) is used to evaluate the effect of a categorical variable on a quantitative variable, while controlling for the effect of another quantitative variable (known as covariate). ANCOVA is actually a special case of multiple linear regression with a mix of one qualitative and one quantitative independent variable.
A mixed ANOVA is used to test differences between two or more groups whilst subjecting participants to repeated measures: one factor (a fixed effects factor) is a between-subjects variable (for example, treatment A and B, with patients receiving only one of the two treatments) and the other (a random effects factor) is a within-subjects variable (for example, measurements are made on day 1, day 2 and day 3 on all subjects).

In this post, we start by explaining when and why a two-way ANOVA is useful, we then do some preliminary descriptive analyses and present how to conduct a two-way ANOVA in R. Finally, we show how to interpret and visualize the results. We also briefly mention and illustrate how to verify the underlying assumptions.

Data

To illustrate how to perform a two-way ANOVA in R, we use the penguins dataset, available from the {palmerpenguins} package.

We do not need to import the dataset, but we need to load the package first and then call the dataset:

# install.packages("palmerpenguins")
library(palmerpenguins)

dat <- penguins # rename dataset
str(dat) # structure of dataset

## tibble [344 × 8] (S3: tbl_df/tbl/data.frame)
##  $ species          : Factor w/ 3 levels "Adelie","Chinstrap",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ island           : Factor w/ 3 levels "Biscoe","Dream",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ bill_length_mm   : num [1:344] 39.1 39.5 40.3 NA 36.7 39.3 38.9 39.2 34.1 42 ...
##  $ bill_depth_mm    : num [1:344] 18.7 17.4 18 NA 19.3 20.6 17.8 19.6 18.1 20.2 ...
##  $ flipper_length_mm: int [1:344] 181 186 195 NA 193 190 181 195 193 190 ...
##  $ body_mass_g      : int [1:344] 3750 3800 3250 NA 3450 3650 3625 4675 3475 4250 ...
##  $ sex              : Factor w/ 2 levels "female","male": 2 1 1 NA 1 2 1 2 NA NA ...
##  $ year             : int [1:344] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...

The dataset contains 8 variables for 344 penguins, summarized below:

summary(dat)

##       species          island    bill_length_mm  bill_depth_mm  
##  Adelie   :152   Biscoe   :168   Min.   :32.10   Min.   :13.10  
##  Chinstrap: 68   Dream    :124   1st Qu.:39.23   1st Qu.:15.60  
##  Gentoo   :124   Torgersen: 52   Median :44.45   Median :17.30  
##                                  Mean   :43.92   Mean   :17.15  
##                                  3rd Qu.:48.50   3rd Qu.:18.70  
##                                  Max.   :59.60   Max.   :21.50  
##                                  NA's   :2       NA's   :2      
##  flipper_length_mm  body_mass_g       sex           year     
##  Min.   :172.0     Min.   :2700   female:165   Min.   :2007  
##  1st Qu.:190.0     1st Qu.:3550   male  :168   1st Qu.:2007  
##  Median :197.0     Median :4050   NA's  : 11   Median :2008  
##  Mean   :200.9     Mean   :4202                Mean   :2008  
##  3rd Qu.:213.0     3rd Qu.:4750                3rd Qu.:2009  
##  Max.   :231.0     Max.   :6300                Max.   :2009  
##  NA's   :2         NA's   :2

In this post, we will focus on the following three variables:

species: the species of the penguin (Adelie, Chinstrap or Gentoo)
sex: sex of the penguin (female and male)
body_mass_g: body mass of the penguin (in grams)

If needed, more information about this dataset can be found by running ?penguins in R.

body_mass_g is the quantitative continuous variable and will be the dependent variable, whereas species and sex are both qualitative variables.

Those two last variables will be our independent variables, also referred as factors. Make sure that they are read as factors by R. If it is not the case, they will need to be transformed to factors.

Aim and hypotheses of a two-way ANOVA

As mentioned above, a two-way ANOVA is used to evaluate simultaneously the effect of two categorical variables on one quantitative continuous variable.

It is referred as two-way ANOVA because we are comparing groups which are formed by two independent categorical variables.

Here, we would like to know if body mass depends on species and/or sex. In particular, we are interested in:

measuring and testing the relationship between species and body mass,
measuring and testing the relationship between sex and body mass, and
potentially check whether the relationship between species and body mass is different for females and males (which is equivalent than checking whether the relationship between sex and body mass depends on the species)

The first two relationships are referred as main effects, while the third point is known as the interaction effect.

The main effects test whether at least one group is different from another one (while controlling for the other independent variable). On the other hand, the interaction effect aims at testing whether the relationship between two variables differs depending on the level of a third variable. In other words, if the evolution between the response and the first categorical variable does not depend on the modalities of the second categorical variable, then there is no interaction between the two variables. If, on the contrary, there is a modification of this evolution, either by an increase in the effect of the first variable, or by a decrease, then there is an interaction.

When performing a two-way ANOVA, testing the interaction effect is not mandatory. However, omitting an interaction effect may lead to erroneous conclusions if the interaction effect is present.

If we go back to our example, we have the following hypothesis tests:

Main effect of sex on body mass:
- $H_0$: mean body mass is equal between females and males
- $H_1$: mean body mass is different between females and males
Main effect of species on body mass:
- $H_0$: mean body mass is equal between all 3 species
- $H_1$: mean body mass is different for at least one species
Interaction between sex and species:
- $H_0$: there is no interaction between sex and species, meaning that the relationship between species and body mass is the same for females and males (similarly, the relationship between sex and body mass is the same for all 3 species)
- $H_1$: there is an interaction between sex and species, meaning that the relationship between species and body mass is different for females than for males (similarly, the relationship between sex and body mass depends on the species)

Assumptions of a two-way ANOVA

Most statistical tests require some assumptions for the results to be valid, and a two-way ANOVA is not an exception.

Assumptions of a two-way ANOVA are similar than for a one-way ANOVA. To summarize:

Variable type: the dependent variable must be quantitative continuous, while the two independent variables must be categorical (with at least two levels).
Independence: the observations should be independent between groups and within each group.
Normality:
- For small samples, data should follow approximately a normal distribution
- For large samples (usually $n \ge 30$ in each group/sample), normality is not required (thanks to the central limit theorem)
Equality of variances: variances should be equal across groups.
Outliers: There should be no significant outliers in any group.

More details about these assumptions can be found in the assumptions of a one-way ANOVA.

Now that we have seen the underlying assumptions of the two-way ANOVA, we review them specifically for our dataset before applying the test and interpreting the results.

Variable type

The dependent variable body mass is quantitative continuous, while both independent variables sex and species are qualitative variables (with at least 2 levels).

Therefore, this assumption is met.

If your dependent variable is quantitative discrete, this is count data, which, strictly speaking, should be analyzed using a generalized linear model, not an ANOVA.

Independence

Independence is usually checked based on the design of the experiment and how data have been collected.²

To keep it simple, observations are usually:

independent if each experimental unit (here a penguin) has been measured only once and the observations are collected from a representative and randomly selected portion of the population, or
dependent if each experimental unit has been measured at least twice (as it is often the case in the medical field for example, with two measurements on the same subjects; one before and one after the treatment).

In our case, body mass has been measured only once on each penguin, and on a representative and random sample of the population, so the independence assumption is met.

Note that if your data correspond to observations made several times on the same experimental units (for example, if one of the factors is a treatment (A or B) and the second factor is time (day 1, day 2 and day 3), and measurements are made at each time point on the same subjects), a two-way mixed ANOVA should be used.

Normality

We have a large sample in all subgroups (each combination of the levels of the two factors, called cell):

table(dat$species, dat$sex)

##            
##             female male
##   Adelie        73   73
##   Chinstrap     34   34
##   Gentoo        58   61

so normality does not need to be checked.

For completeness, we still show how to verify normality, as if we had a small samples.

There are several methods to test the normality assumption. The most common methods being:

a QQ-plot by group or on the residuals, and/or
a histogram by group or on the residuals, and/or
a normality test (Shapiro-Wilk test for instance) by group or on the residuals.

The easiest/shortest way is to verify the normality with a QQ-plot on the residuals. To draw this plot, we first need to save the model:

# save model
mod <- aov(body_mass_g ~ sex * species,
  data = dat
)

This piece of code will be explained further.

Now we can draw the QQ-plot on the residuals. We show two ways to do so, first with the plot() function and second with the qqPlot() function from the {car} package:

# method 1
plot(mod, which = 2)

# method 2
library(car)

qqPlot(mod$residuals,
  id = FALSE # remove point identification
)

Code for method 1 is slightly shorter, but it misses the confidence interval around the reference line.

If points follow the straight line (called Henry’s line) and fall within the confidence band, we can assume normality. This is the case here.

If you prefer to verify the normality based on a histogram of the residuals, here is the code:

# histogram
hist(mod$residuals)

The histogram of the residuals show a gaussian distribution, which is in line with the conclusion from the QQ-plot.

Although the QQ-plot and histogram is largely enough to verify the normality, if you want to test it more formally with a statistical test, the Shapiro-Wilk test can be applied on the residuals as well:

# normality test
shapiro.test(mod$residuals)

## 
## 	Shapiro-Wilk normality test
## 
## data:  mod$residuals
## W = 0.99776, p-value = 0.9367

$\Rightarrow$ We do not reject the null hypothesis that the residuals follow a normal distribution ($p$-value = 0.937).

From the QQ-plot, histogram and Shapiro-Wilk test, we conclude that we do not reject the null hypothesis of normality of the residuals.

The normality assumption is thus verified, we can now check the equality of the variances.

Note that if the normality assumption is not met, many transformations can be applied on the dependent variable to improve it, the most common ones being the logarithmic (log() function in R) and the Box-Cox transformations. If the normality assumption is still not met on the transformed data, the non-parametric version of the two-way ANOVA, the Scheirer–Ray–Hare test, can be used. Alternatively, a permutation test can also be used.

Homogeneity of variances

Equality of variances, also referred as homogeneity of variances or homoscedasticity, can be verified visually with the plot() function:

plot(mod, which = 3)

Since the spread of the residuals is constant, the red smooth line is horizontal and flat, so it looks like the constant variance assumption is satisfied here.

The diagnostic plot above is sufficient, but if you prefer it can also be tested more formally with the Levene’s test (also from the {car} package):³

leveneTest(mod)

## Levene's Test for Homogeneity of Variance (center = median)
##        Df F value Pr(>F)
## group   5  1.3908 0.2272
##       327

$\Rightarrow$ We do not reject the null hypothesis that the variances are equal ($p$-value = 0.227).

Both the visual and formal approaches give the same conclusion; we do not reject the hypothesis of homogeneity of the variances.

Note that, as for the normality, the logarithmic and Box-Cox transformations may improve homogeneity of the residuals.

Outliers

The easiest and most common way to detect outliers is visually thanks to boxplots by groups.

For females and males:

library(ggplot2)

# boxplots by sex
ggplot(dat) +
  aes(x = sex, y = body_mass_g) +
  geom_boxplot()

For the three species:

# boxplots by species
ggplot(dat) +
  aes(x = species, y = body_mass_g) +
  geom_boxplot()

There are, as defined by the interquartile range criterion, two outliers for the species Chinstrap. These points are, nonetheless, not extreme enough to bias results.

Therefore, we consider that the assumption of no significant outliers is met.

Two-way ANOVA

We have shown that all assumptions are met, so we can now proceed to the implementation of the two-way ANOVA in R.

This will allow us to answer the following research questions:

Controlling for the species, is body mass significantly different between the two sexes?
Controlling for the sex, is body mass significantly different for at least one species?
Is the relationship between species and body mass different between female and male penguins?

Preliminary analyses

Before performing any statistical test, it is a good practice to make some descriptive statistics in order to have a first overview of the data, and perhaps, have a glimpse of the results to be expected.

This can be done via descriptive statistics or plots.

Descriptive statistics

If we want to keep it simple, we can compute only the mean for each subgroup:

# mean by group
aggregate(body_mass_g ~ species + sex,
  data = dat,
  FUN = mean
)

##     species    sex body_mass_g
## 1    Adelie female    3368.836
## 2 Chinstrap female    3527.206
## 3    Gentoo female    4679.741
## 4    Adelie   male    4043.493
## 5 Chinstrap   male    3938.971
## 6    Gentoo   male    5484.836

Or eventually, the mean and standard deviation for each subgroup using the {dplyr} package:

# mean and sd by group
library(dplyr)

group_by(dat, sex, species) %>%
  summarise(
    mean = round(mean(body_mass_g, na.rm = TRUE)),
    sd = round(sd(body_mass_g, na.rm = TRUE))
  )

## # A tibble: 8 × 4
## # Groups:   sex [3]
##   sex    species    mean    sd
##   <fct>  <fct>     <dbl> <dbl>
## 1 female Adelie     3369   269
## 2 female Chinstrap  3527   285
## 3 female Gentoo     4680   282
## 4 male   Adelie     4043   347
## 5 male   Chinstrap  3939   362
## 6 male   Gentoo     5485   313
## 7 <NA>   Adelie     3540   477
## 8 <NA>   Gentoo     4588   338

Plots

If you are a frequent reader of the blog, you know that I like to draw plots to visualize the data at hand before interpreting results of a test.

The most appropriate plot when we have one quantitative and two qualitative variables is a boxplot by group. This can easily be made with the {ggplot2} package:

# boxplot by group
library(ggplot2)

ggplot(dat) +
  aes(x = species, y = body_mass_g, fill = sex) +
  geom_boxplot()

Some observations are missing for the sex, we can remove them to have a more concise plot:

dat %>%
  filter(!is.na(sex)) %>%
  ggplot() +
  aes(x = species, y = body_mass_g, fill = sex) +
  geom_boxplot()

Note that we could also have made the following plot:

dat %>%
  filter(!is.na(sex)) %>%
  ggplot() +
  aes(x = sex, y = body_mass_g, fill = species) +
  geom_boxplot()

But for a more readable plot, I tend to prefer putting the variable with the smallest number of levels as color (which is in fact the argument fill in the aes() layer) and the variable with the largest number of categories on the x-axis (i.e., the argument x in the aes() layer).

From the means and the boxplots by subgroup, we can already see that, in our sample:

female penguins tend to have a lower body mass than males, and that is the case for all the considered species, and
body mass is higher for Gentoo penguins than for the other two species.

Bear in mind that these conclusions are only valid within our sample! To generalize these conclusions to the population, we need to perform the two-way ANOVA and check the significance of the explanatory variables. This is the aim of the next section.

Two-way ANOVA in R

As mentioned earlier, including an interaction effect in a two-way ANOVA is not compulsory. However, in order to avoid flawed conclusions, it is recommended to first check whether the interaction is significant or not, and depending on the results, include it or not.

If the interaction is not significant, it is safe to remove it from the final model. On the contrary, if the interaction is significant, it should be included in the final model which will be used to interpret results.

We thus start with a model which includes the two main effects (i.e., sex and species) and the interaction:

# Two-way ANOVA with interaction
# save model
mod <- aov(body_mass_g ~ sex * species,
  data = dat
)

# print results
summary(mod)

##              Df    Sum Sq  Mean Sq F value   Pr(>F)    
## sex           1  38878897 38878897 406.145  < 2e-16 ***
## species       2 143401584 71700792 749.016  < 2e-16 ***
## sex:species   2   1676557   838278   8.757 0.000197 ***
## Residuals   327  31302628    95727                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 11 observations deleted due to missingness

Similar to a one-way ANOVA, the principle of a two-way ANOVA is based on the total dispersion of the data, and its decomposition into four components:

the share attributable to the first factor
the share attributable to the second factor
the share attributable to the interaction of the 2 factors
the unexplained, or residual portion.

The sum of squares (column Sum Sq) shows these four components. The two-way ANOVA consists of using a statistical test to determine whether each of the dispersion component (attributable to the 2 factors studied and to their interaction) is significantly greater than the residual component. If this is the case, we conclude that the effect considered (factor A, factor B or the interaction) is significant.

We see that the species explain a large part of the variability of body mass. It is the most important factor in explaining this variability.

The $p$-values are displayed in the last column of the output above (Pr(>F)). From these $p$-values, we conclude that, at the 5% significance level:

controlling for the species, body mass is significantly different between the two sexes,
controlling for the sex, body mass is significantly different for at least one species, and
the interaction between sex and species (displayed at the line sex:species in the output above) is significant.

So from the significant interaction effect, we have just seen that the relationship between body mass and species is different between males and females. Since it is significant, we have to keep it in the model and we should interpret results from that model.

If, on the contrary, the interaction was not significant (that is, if the $p$-value $\ge$ 0.05) we would have removed this interaction effect from the model. For illustrative purposes, below the code for a two-way ANOVA without interaction, referred as an additive model:

# Two-way ANOVA without interaction
aov(body_mass_g ~ sex + species,
  data = dat
)

For the readers who are used to perform linear regressions in R, you will notice that the structure of the code for a two-way ANOVA is in fact similar:

the formula is dependent variable ~ independent variables
the + sign is used to include independent variables without an interaction⁴
the * sign is used to include independent variables with an interaction

The resemblance with a linear regression is not a surprise because a two-way ANOVA, like all ANOVA, is actually a linear model.

Note that the following code works as well, and give the same results:⁵

# method 2
mod2 <- lm(body_mass_g ~ sex * species,
  data = dat
)

Anova(mod2)

## Anova Table (Type II tests)
## 
## Response: body_mass_g
##                Sum Sq  Df F value    Pr(>F)    
## sex          37090262   1 387.460 < 2.2e-16 ***
## species     143401584   2 749.016 < 2.2e-16 ***
## sex:species   1676557   2   8.757 0.0001973 ***
## Residuals    31302628 327                      
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Note that the aov() function assumes a balanced design, meaning that we have equal sample sizes within levels of our independent grouping variables. Moreover, aov() uses the type I sums of squares, so we can obtain different $p$-values when we write y ~ A * B and y ~ B * A.

For unbalanced design, that is, unequal numbers of subjects in each subgroup, the recommended methods are:

the type II ANOVA when there is no significant interaction, which can be done in R with Anova(mod, type = "II") or Anova(mod, type = 2), where mod is the name of your saved model, and
the type III ANOVA when there is a significant interaction, which can be done in R with Anova(mod, type = "III") or Anova(mod, type = 3).

This is beyond the scope of the post and we assume a balanced design here. For the interested reader, see this detailed discussion about type I, type II and type III ANOVA.

Pairwise comparisons

Through the two main effects being significant, we concluded that:

controlling for the species, body mass is different between females and males, and
controlling for the sex, body mass is different for at least one species.

If body mass is different between the two sexes, given that there are exactly two sexes, it must be because body mass is significantly different between females and males.

If one wants to know which sex has the highest body mass, it can be deduced from the means and/or boxplots by subgroup. Here, it is clear that males have a significantly higher body mass than females.

However, it is not so straightforward for the species. Let me explain why it is not as easy as for the sexes.

There are three species (Adelie, Chinstrap and Gentoo), so there are 3 pairs of species:

Adelie and Chinstrap
Adelie and Gentoo
Chinstrap and Gentoo

If body mass is significantly different for at least one species, it could be that:

body mass is significantly different between Adelie and Chinstrap but not significantly different between Adelie and Gentoo, and not significantly different between Chinstrap and Gentoo, or
body mass is significantly different between Adelie and Gentoo but not significantly different between Adelie and Chinstrap, and not significantly different between Chinstrap and Gentoo, or
body mass is significantly different between Chinstrap and Gentoo but not significantly different between Adelie and Chinstrap, and not significantly different between Adelie and Gentoo.

Or, it could also be that:

body mass is significantly different between Adelie and Chinstrap, and between Adelie and Gentoo, but not significantly different between Chinstrap and Gentoo, or
body mass is significantly different between Adelie and Chinstrap, and between Chinstrap and Gentoo, but not significantly different between Adelie and Gentoo, or
body mass is significantly different between Chinstrap and Gentoo, and between Adelie and Gentoo, but not significantly different between Adelie and Chinstrap.

Last, it could also be that body mass is significantly different between all species.

As for a one-way ANOVA, we cannot, at this stage, know precisely which species is different from which one in terms of body mass. To know this, we need to compare each species two by two thanks to post-hoc tests (also known as pairwise comparisons).

There are several post-hoc tests, the most common ones being the Tukey HSD which tests all possible pairs of groups, and the Dunett’s test which compares all groups to a reference group. As mentioned earlier, these tests should not be done on the sex variable because there are only two levels.

In this post, we show only the Tukey HSD test. For the interested reader, the Dunnett’s test is illustrated here.

As for the one-way ANOVA, the Tukey HSD test can be done in R as follows:

# method 1
TukeyHSD(mod,
  which = "species"
)

##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = body_mass_g ~ sex * species, data = dat)
## 
## $species
##                        diff       lwr       upr     p adj
## Chinstrap-Adelie   26.92385  -80.0258  133.8735 0.8241288
## Gentoo-Adelie    1377.65816 1287.6926 1467.6237 0.0000000
## Gentoo-Chinstrap 1350.73431 1239.9964 1461.4722 0.0000000

or using the {multcomp} package:

# method 2
library(multcomp)

res_tukey <- glht(
  aov(body_mass_g ~ sex + species,
    data = dat
  ),
  linfct = mcp(species = "Tukey")
)

summary(res_tukey)

## 
## 	 Simultaneous Tests for General Linear Hypotheses
## 
## Multiple Comparisons of Means: Tukey Contrasts
## 
## 
## Fit: aov(formula = body_mass_g ~ sex + species, data = dat)
## 
## Linear Hypotheses:
##                         Estimate Std. Error t value Pr(>|t|)    
## Chinstrap - Adelie == 0    26.92      46.48   0.579     0.83    
## Gentoo - Adelie == 0     1377.86      39.10  35.236   <1e-05 ***
## Gentoo - Chinstrap == 0  1350.93      48.13  28.067   <1e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)

or using the pairwise.t.test() function using the $p$-value adjustment method of your choice:⁶

# method 3
pairwise.t.test(dat$body_mass_g, dat$species,
  p.adjust.method = "BH"
)

## 
## 	Pairwise comparisons using t tests with pooled SD 
## 
## data:  dat$body_mass_g and dat$species 
## 
##           Adelie Chinstrap
## Chinstrap 0.63   -        
## Gentoo    <2e-16 <2e-16   
## 
## P value adjustment method: BH

Note that when using the second method, it is the model without the interaction that needs to be specified into the glht() function, even if the interaction is significant. Moreover, do not forget to replace mod and species in my code with the name of your model and the name of your independent variable.

Both methods give the same results, that is:

body mass is not significantly different between Chinstrap and Adelie (adjusted $p$-value = 0.83),
body mass is significantly different between Gentoo and Adelie (adjusted $p$-value < 0.001), and
body mass is significantly different between Gentoo and Chinstrap (adjusted $p$-value < 0.001).

Remember that it is the adjusted $p$-values that are reported, to prevent the issue of multiple testing which occurs when comparing several pairs of groups.

If you would like to compare all combinations of groups, it can be done with the TukeyHSD() function and specifying the interaction in the which argument:

# all combinations of sex and species
TukeyHSD(mod,
  which = "sex:species"
)

##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = body_mass_g ~ sex * species, data = dat)
## 
## $`sex:species`
##                                      diff       lwr       upr     p adj
## male:Adelie-female:Adelie        674.6575  527.8486  821.4664 0.0000000
## female:Chinstrap-female:Adelie   158.3703  -25.7874  342.5279 0.1376213
## male:Chinstrap-female:Adelie     570.1350  385.9773  754.2926 0.0000000
## female:Gentoo-female:Adelie     1310.9058 1154.8934 1466.9181 0.0000000
## male:Gentoo-female:Adelie       2116.0004 1962.1408 2269.8601 0.0000000
## female:Chinstrap-male:Adelie    -516.2873 -700.4449 -332.1296 0.0000000
## male:Chinstrap-male:Adelie      -104.5226 -288.6802   79.6351 0.5812048
## female:Gentoo-male:Adelie        636.2482  480.2359  792.2606 0.0000000
## male:Gentoo-male:Adelie         1441.3429 1287.4832 1595.2026 0.0000000
## male:Chinstrap-female:Chinstrap  411.7647  196.6479  626.8815 0.0000012
## female:Gentoo-female:Chinstrap  1152.5355  960.9603 1344.1107 0.0000000
## male:Gentoo-female:Chinstrap    1957.6302 1767.8040 2147.4564 0.0000000
## female:Gentoo-male:Chinstrap     740.7708  549.1956  932.3460 0.0000000
## male:Gentoo-male:Chinstrap      1545.8655 1356.0392 1735.6917 0.0000000
## male:Gentoo-female:Gentoo        805.0947  642.4300  967.7594 0.0000000

Or with the HSD.test() function from the {agricolae} package, which denotes subgroups that are not significantly different from each other with the same letter:

library(agricolae)

HSD.test(mod,
  trt = c("sex", "species"),
  console = TRUE # print results
)

## 
## Study: mod ~ c("sex", "species")
## 
## HSD Test for body_mass_g 
## 
## Mean Square Error:  95726.69 
## 
## sex:species,  means
## 
##                  body_mass_g      std  r  Min  Max
## female:Adelie       3368.836 269.3801 73 2850 3900
## female:Chinstrap    3527.206 285.3339 34 2700 4150
## female:Gentoo       4679.741 281.5783 58 3950 5200
## male:Adelie         4043.493 346.8116 73 3325 4775
## male:Chinstrap      3938.971 362.1376 34 3250 4800
## male:Gentoo         5484.836 313.1586 61 4750 6300
## 
## Alpha: 0.05 ; DF Error: 327 
## Critical Value of Studentized Range: 4.054126 
## 
## Groups according to probability of means differences and alpha level( 0.05 )
## 
## Treatments with the same letter are not significantly different.
## 
##                  body_mass_g groups
## male:Gentoo         5484.836      a
## female:Gentoo       4679.741      b
## male:Adelie         4043.493      c
## male:Chinstrap      3938.971      c
## female:Chinstrap    3527.206      d
## female:Adelie       3368.836      d

If you have many groups to compare, plotting them might be easier to interpret:

# set axis margins so labels do not get cut off
par(mar = c(4.1, 13.5, 4.1, 2.1))

# create confidence interval for each comparison
plot(TukeyHSD(mod, which = "sex:species"),
  las = 2 # rotate x-axis ticks
)

From the outputs and plot above, we conclude that all combinations of sex and species are significantly different, except between female Chinstrap and female Adelie ($p$-value = 0.138) and male Chinstrap and male Adelie ($p$-value = 0.581).

These results, which are by the way in line with the boxplots shown above and which will be confirmed with the visualizations below, concludes the two-way ANOVA in R.

Visualizations

If you would like to visualize results in a different way to what has already been presented in the preliminary analyses, below are some ideas of useful plots.

First, with the mean and standard error of the mean by subgroup using the allEffects() function from the {effects} package:

# method 1
library(effects)

plot(allEffects(mod))

Or using the {ggpubr} package:

# method 2
library(ggpubr)

ggline(subset(dat, !is.na(sex)), # remove NA level for sex
  x = "species",
  y = "body_mass_g",
  color = "sex",
  add = c("mean_se") # add mean and standard error
) +
  labs(y = "Mean of body mass (g)")

Alternatively, using {Rmisc} and {ggplot2}:

library(Rmisc)

# compute mean and standard error of the mean by subgroup
summary_stat <- summarySE(dat,
  measurevar = "body_mass_g",
  groupvars = c("species", "sex")
)

# plot mean and standard error of the mean
ggplot(
  subset(summary_stat, !is.na(sex)), # remove NA level for sex
  aes(x = species, y = body_mass_g, colour = sex)
) +
  geom_errorbar(aes(ymin = body_mass_g - se, ymax = body_mass_g + se), # add error bars
    width = 0.1 # width of error bars
  ) +
  geom_point() +
  labs(y = "Mean of body mass (g)")

Second, if you prefer to draw only the mean by subgroup:

with(
  dat,
  interaction.plot(species, sex, body_mass_g)
)

Last but not least, for those of you who are familiar with GraphPad, you are most likely familiar with plotting means and error bars as follows:

# plot mean and standard error of the mean as barplots
ggplot(
  subset(summary_stat, !is.na(sex)), # remove NA level for sex
  aes(x = species, y = body_mass_g, fill = sex)
) +
  geom_bar(position = position_dodge(), stat = "identity") +
  geom_errorbar(aes(ymin = body_mass_g - se, ymax = body_mass_g + se), # add error bars
    width = 0.25, # width of error bars
    position = position_dodge(.9)
  ) +
  labs(y = "Mean of body mass (g)")

Conclusion

In this post, we started with a few reminders of the different tests that exist to compare a quantitative variable across groups. We then focused on the two-way ANOVA, starting from its goal and hypotheses to its implementation in R, together with the interpretations and some visualizations. We also briefly mentioned its underlying assumptions and one post-hoc test to compare all subgroups.

All this was illustrated with the penguins dataset available from the {palmerpenguins} package.

Thanks for reading.

I hope this article will help you in conducting a two-way ANOVA with your data.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

In theory, a one-way ANOVA can also be used to compare 2 groups, and not only 3 or more. Nonetheless, in practice, it is often the case that a Student’s t-test is performed to compare 2 groups, and a one-way ANOVA to compare 3 or more groups. Conclusions obtained via a Student’s t-test for independent samples and a one-way ANOVA with 2 groups will be similar.↩︎
If you really want to test the independence, you can do so visually with a plot of the residuals vs. fitted values. This plot can be done in R with plot(mod, which = 1), where mod corresponds to the name of your model. Or you can do so with the Durbin-Watson test. In R, it can be done with the durbinWatsonTest() function from the {car} package.↩︎
Note that the Bartlett’s and Fligner-Killeen tests are also appropriate to test the assumption of equal variances.↩︎
An additive model makes the assumption that the 2 explanatory variables are independent; they do not interact with each other.↩︎
To not be confused with the anova() function because it provides sequential results that depend on the order in which the variables appear in the model.↩︎
Here, we use the Benjamini & Hochberg (1995) correction, but you can choose between several methods. See ?p.adjust for more details.↩︎

Top 10 errors in R and how to fix them

Tue, 07 Feb 2023 00:00:00 +0000

Introduction

If you are just starting with R, you will often encounter errors in your code which prevent it to run. I remember when I was just starting to use R, errors in my code were so frequent that I almost gave up learning this programming language. I even recall that I went back to Excel a few times to finish my analyses because I could not find what was causing the issue.

Fortunately, I forced myself to continue despite the difficulties of the beginning. And today, even if I still encounter errors almost every time I write R code, with experience and practice, it takes less and less time to fix them. If you are also struggling at the beginning, rest assured, it is normal: everyone experiences some frustration when learning a new programming language (and this is the case not only with R).

In this post, I highlight the 10 most common errors in R and how to fix them. Of course, errors depend on your code and your analyses, so it is impossible to cover all of them (and Google does it way better than me). However, I would like to focus on some common syntax mistakes that are frequent when learning R, and which can sometimes take a long time to be fixed before realizing that the solution is right in front of our eyes.

This collection is based on my personal experience and the errors encountered by my students when I teach R. This list being non-exhaustive, feel free to comment (at the end of the post) with errors you often face when using R.

For each error, I provide examples and solutions to fix them. I also mention a couple of warnings (which are, strictly speaking, not errors) at the end of the post.

1. Unmatched parentheses, curly braces, square brackets or quotes

One rather trivial but still quite frequent error is a missing parenthesis, curly brace, square bracket or quotation mark.

This type of error is applicable to many programming languages. In R, for instance:

## missing a closing parenthesis:
mean(c(1, 7, 13)
     
## missing a curly brace before "else":
x <- 7 
if(x > 10) {
  print("x > 10")
 else {
  print("x <= 10")
 }
  
## missing a square bracket:
summary(ggplot2::diamonds[, "price")

## missing a closing quote in 2nd element:
grp <- c("Group 1", "Group 2) 
grp

These errors are easy to detect when the code is basic, but can become much harder to spot with a more complex code, for instance:¹

for (i in y) {
  for (j in x) {
    p <- ggboxplot(dat,
      x = colnames(dat[j]), y = colnames(dat[i]),
      color = colnames(dat[j]),
      legend = "none",
      palette = "npg",
      add = "jitter"
    )
    print(
      p + stat_compare_means(aes(label = paste0(..method.., ", p-value = ", ..p.format..),
        method = method1, label.y = max(dat[, i], na.rm = TRUE)
      )
      + stat_compare_means(comparisons = my_comparisons, method = method2, label = "p.format")
    )
  }

Thankfully, if you use RStudio,² a closing parenthesis, curly brace, square bracket or quotation mark will automatically be written when you open one.

Bear in mind that when installing a package, you must use (single or double) quotation marks around the package’s name:

## NOT correct:
install.packages(ggplot2)

## Error in install.packages : object 'ggplot2' not found

Instead, write one of the two following options:

# install.packages("ggplot2")

# install.packages('ggplot2')

Solution

The solution of course is to simply match all opening parentheses, curly braces, square brackets and quotation marks with their closing counterparts:

mean(c(1, 7, 13))

## [1] 7

x <- 7
if (x > 10) {
  print("x > 10")
} else {
  print("x <= 10")
}

## [1] "x <= 10"

summary(ggplot2::diamonds[, "price"])

##      price      
##  Min.   :  326  
##  1st Qu.:  950  
##  Median : 2401  
##  Mean   : 3933  
##  3rd Qu.: 5324  
##  Max.   :18823

grp <- c("Group 1", "Group 2")
grp

## [1] "Group 1" "Group 2"

Also, make sure:

to correctly place commas:

## NOT correct (misplaced comma):
c("Group 1," "Group 2")

## Error: unexpected string constant in "c("Group 1," "Group 2""

## also NOT correct (missing comma):
c("Group 1" "Group 2")

## Error: unexpected string constant in "c("Group 1" "Group 2""

## correct:
c("Group 1", "Group 2")

you do not mix single and double quotation marks for the same element:

## NOT correct:
c("Group 1')

## correct:
c("Group 1")

## also correct:
c('Group 1')

Note that c('Group 1', "Group 2") does not throw an error but for consistency, it is not recommended to mix single and double quotes within the same vector.

2. Using a function that is not installed or loaded

If you encounter the following error: “Error in … : could not find function ‘…’”, for example:

it means you are trying to use a function belonging to a package which is not yet installed or loaded.

Solution

To solve this error, you have to install the package (if it is not installed yet) and load it with the install.packages() and library() functions, respectively:

## install package:
install.packages("ggplot2")

## load package:
library(ggplot2)

If you are not sure about the usage of these two functions, see more details about installing and loading a package in R.

3. Typos in function, variable, dataset, object or package names

Another common mistake is to misspell a function, a variable, a dataset, an object or a package’s name, for example:

## typo in function name:
maen(c(1, 7, 13))

## Error in maen(c(1, 7, 13)) : could not find function "maen"

## typo in variable name:
summary(ggplot2::diamonds[, "detph"])

## Error: Column `detph` doesn't exist

## typo in dataset name:
data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)
summary(dta[, 2])

## Error in summary(dta[, 2]) : object 'dta' not found

## typo in object name:
test <- c(1, 7, 13)
mean(tset)

## Error in mean(tset) : object 'tset' not found

## typo in package name:
library("tydiverse")

## Error in library("tydiverse") : there is no package called ‘tydiverse’

Solution

Make sure that you correctly spell all your functions, variables, datasets, objects and packages:

Note that R is case sensitive; mean() is considered different than Mean() for R!

mean(c(1, 7, 13))

## [1] 7

summary(ggplot2::diamonds[, "depth"])

##      depth      
##  Min.   :43.00  
##  1st Qu.:61.00  
##  Median :61.80  
##  Mean   :61.75  
##  3rd Qu.:62.50  
##  Max.   :79.00

data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)
data[, 2]

##  [1]  1.3048697  2.2866454 -1.3888607 -0.2787888 -0.1333213  0.6359504
##  [7] -0.2842529 -2.6564554 -2.4404669  1.3201133

test <- c(1, 7, 13)
mean(test)

## [1] 7

library(tidyverse)

## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.3     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.0
## ✔ ggplot2   3.4.3     ✔ tibble    3.2.1
## ✔ lubridate 1.9.2     ✔ tidyr     1.3.0
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

If you are sure that you correctly spelled an object, a function or a dataset but you still have an error stating that “object ‘…’ is not found”, make sure that you defined your object/function/dataset before calling it!

It often happens that a student asks me to come to his/her computer because he/she runs the exact same code than me, but cannot make it work. Most of the time, if his/her code is indeed exactly the same than mine, he/she simply has not executed a object/function/dataset before running the code which includes that object/function/dataset. In other words, he/she simply tries to use an undefined object or variable.

Remember that writing code in a R script (contrarily to the console) does not mean it is compiled. You actually have to run it (by clicking on the Run button or using the keyboard shortcut) in order the code to be executed and used later. If you are still struggling with this, see the basics of R and RStudio.

4. Missing, incorrect or misspelled arguments in functions

Most R functions require arguments. For example, the rnorm() function requires at least the number of observations, specified via the argument n.

Your code will not run if you do not specify compulsory arguments, or if incorrectly specify an argument. Moreover, the result might not be what you expect if you misspell an argument:

## missing compulsory argument:
rnorm()

## Error in rnorm() : argument "n" is missing, with no default

## incorrect argument:
rnorm(n = 3, var = 1)

## Error in rnorm(n = 3, var = 1) : unused argument (var = 1)

## misspelled argument:
mean(c(1, 7, 13, NA), narm = TRUE)

## [1] NA

The last piece of code does not throw an error, but the result is not what we want.

Solution

To solve these errors, make sure to specify at least all compulsory arguments of the function, and the correct ones:

In rnorm(), it is the standard deviation, sd, which can be specified in addition to the number of observations n (instead of the variance var).
Removing NA is done with na.rm (instead of narm).

rnorm(n = 3, sd = 1)

## [1] -0.3066386 -1.7813084 -0.1719174

mean(c(1, 7, 13, NA), na.rm = TRUE)

## [1] 7

If you do not know the arguments of a function by heart, you can always check the documentation with ?function_name or help(function_name), for example:

?rnorm()

## or:
help(rnorm)

5. Wrong, inappropriate or inconsistent data types

There are several data types in R, the main ones being:

Numeric
Character
Factor
Logical

You know that some operations and analyses are possible and appropriate only with some specific types of data.

For example, it is not appropriate to compute the mean of a factor or character variable:

gender <- factor(c("female", "female", "male", "female", "male"))

mean(gender)

## Warning in mean.default(gender): argument is not numeric or logical: returning
## NA

## [1] NA

Likewise, although it is technically possible, it makes little sense to draw a barplot of a quantitative continuous variable because in most cases, the frequency will be 1 for each value:

barplot(table(rnorm(10)))

(By the way, if your data is not already displayed in the form of a table, do not forget to add table() inside the barplot() function.)

Solution

Make sure to use the appropriate operation and type of analysis depending on the variable(s) of interest.

For example:

for factor variables, it is more appropriate to compute frequencies and/or relative frequencies, and draw barplots
for quantitative continuous variables, it is more appropriate to compute the mean, median, etc. and draw histograms, boxplots, etc.
for logical variables, the mean,³ a frequency table and a barplot are appropriate
for character variables, word clouds are the most appropriate (unless the variable can be considered as a factor variable because there are not too many different levels)

We now illustrate the examples in R:⁴

## factor:
table(gender)

## gender
## female   male 
##      3      2

prop.table(table(gender))

## gender
## female   male 
##    0.6    0.4

barplot(table(gender))

## quantitative continuous:
x <- rnorm(100)

summary(x)

##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
## -2.99309 -0.74143  0.01809 -0.08570  0.58937  2.70189

par(mfrow = c(1, 2)) ## combine two plots
hist(x)
boxplot(x)

## logical:
x <- c(TRUE, FALSE, FALSE, TRUE, TRUE)

mean(x)

## [1] 0.6

table(x)

## x
## FALSE  TRUE 
##     2     3

barplot(table(x))

For the interested reader, see the most common descriptive statistics in R for different types of data.

Note that, as for descriptive statistics, the choice of the statistical test depends on the variable’s type. See this flowchart to help you in selecting the most appropriate statistical test depending on the number of variables and their types.

An error linked to the one mentioned above is inconsistent data type. See it in practice with the following example:

x <- c(2.4, 3.7, 5.1, 9.8)
class(x)

## [1] "numeric"

y <- c(2.4, 3.7, 5.1, "9.8")
class(y)

## [1] "character"

As you can see, vector x is numerical, whereas vector y is in the form of character. This is due to the fact that the last element of y is surrounded with quotation marks (and thus considered as a string instead of a numerical value), so the entire vector takes the character form.

This can happen when you import a dataset into R and one or several elements of a variable are not encoded correctly. This leads to the entire variable to be considered as a character variable by R.

To avoid this, it is a good practice to check the structure of your dataset (with str()) after importing it to make sure all your variables have the desired format. If not, you can either correct the values in the initial file or change the format in R (with as.numeric()).

6. Forgetting the + sign in ggplot2

If you just learned to use the ggplot2 package for your visualizations (and I highly recommend it!), a common mistake is to forget the + sign.

You know that a visualization made with ggplot2 is constructed by adding several layers:

## load package:
library(ggplot2)

## first layer, the dataset:
ggplot(data = diamonds) +
  ## second layer, the aesthetics:
  aes(x = cut, y = price) +
  ## third layer, the type of plot:
  geom_boxplot() +
  ## add more layers:
  theme_minimal()

Solution

For all your graphics with ggplot2, do not forget to add a + sign after each layer except the last one.

7. Misunderstanding between = and ==

Assignment in R can be done in three ways, from the most to the least common:

<-
=
assign()

The second method, that is =, should not be confused with ==.

Indeed, assigning an object (with any of the three above methods) is used to save something in R. For example, if we want to save the vector (1, 3, 7) and rename that vector x, we can write:

x <- c(1, 3, 7)

## or:
x = c(1, 3, 7)

## or:
assign("x", c(1, 3, 7))

When executing this piece of code, you will see that the vector x of size 3 appears in the tab “Environment” (the top right panel if you use the default view of RStudio):

From now on, we can use that vector simply by calling it by its name:

## [1] 1 3 7

By no means, you can assign an object with ==:

## NOT correct if we want to assign c(1, 3, 7) to x:
x == c(1, 3, 7)

So you are wondering, when would we need to use ==? Actually, it is used when you want to use an equal sign.

I understand that it may be abstract and confusing at the moment, so let’s suppose the following two scenarios as examples (which are the two most common cases when we use ==):

we want to check whether an assigned object or variable respects some conditions, and
we want to subset a dataframe based on one or several conditions.

For these examples, suppose a sample of 5 children:

## create dataframe:
dat <- data.frame(
  Name = c("Mary", "Linda", "James", "John", "Patricia"),
  Age = c(7, 10, 3, 9, 7),
  Gender = c("Girl", "Girl", "Boy", "Boy", "Girl")
)

## print dataframe:
dat

##       Name Age Gender
## 1     Mary   7   Girl
## 2    Linda  10   Girl
## 3    James   3    Boy
## 4     John   9    Boy
## 5 Patricia   7   Girl

Let’s now write different pieces of code for these two scenarios to illustrate them:

We want to check whether the variable Age is equal to the vector (1, 2, 3, 4, 5):

dat$Age == 1:5

## [1] FALSE FALSE  TRUE FALSE FALSE

With this code, we ask whether the first element of the variable Age is equal to 1, the second element of the variable Age is equal to 2, and so on. The answer is of course FALSE, FALSE, TRUE, FALSE and FALSE since only the third child has an age equal to 3 years.

We want to know which of our 5 sampled children are girls:

dat$Gender == "Girl"

## [1]  TRUE  TRUE FALSE FALSE  TRUE

The results show that the first, second and fifth children are girls, while the third and fourth children are not girls.

If you write any of these two lines:

## this overwrites Age and Gender:
dat$Age = 1:5
dat$Gender = "Girl"

You actually overwrite the Age and Gender variables, such that our 5 children will have an age from 1 to 5 (1 year for the first child, up to 5 years for the fifth child) and all of them will be girls.

Now suppose we want to subset our dataframe based on a condition, namely, we want to extract only the children who are 7 years old:

subset(dat, Age == 7)

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

If you do not want to use the subset function, you can also use square brackets:

dat[dat$Age == 7, ]

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

As you can see in the previous examples, we do not want to assign anything. Instead, we are asking “is this variable or vector equal to something else?”. For that specific need, we use ==.

So to sum up, for technical reasons and in order to distinguish between the two concepts, R uses = for assignments, and == for the equality sign. Make sure to understand the difference between the two to avoid any errors.

8. Undefined columns selected

If you are used to subset dataframes with square brackets, [], instead of the subset() or filter() functions, you may have faced the error “Error in [.data.frame(…) : undefined columns selected”.

This occurs when R does not understand the column you want to use while subsetting the dataset.

Considering the same sample of 5 children introduced earlier, the following code will throw an error:

dat[dat$Age == 7]

## Error in `[.data.frame`(dat, dat$Age == 7) : undefined columns selected

because it does not specify the column dimension.

Solution

Remember that dataframes in R have two dimensions:

the rows (one for each experimental unit), and
the columns (one for each variable)

and in that particular order (so row first, then column)!

Since dataframes have two dimensions, R expects two dimensions when you call dat[].

In particular, it expects the first and then the second dimension, separated by a comma:

dat[dat$Age == 7, ]

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

This code means that we are extracting all rows where Age is equal to 7 (first dimension, i.e. before the comma), for all variables of the dataset (since we did not specify any column after the comma).

For the interested reader, see more ways to subset and manipulate data in R.

9. Problem when importing or using the wrong data file

Importing a dataset in R can be quite challenging for beginners, mainly due to the misunderstanding about the working directory.

When importing a file, R will not search for the file in all your folders of your computer. Instead, it will look only in one specific folder. If your dataset is not inside that folder, it will result in an error such as “cannot open file ‘…’: No such file or directory”:

To fix this, you must specify the path to the folder where your dataset is located. In other words, you need to tell R in which folder you want it to work, hence the name working directory.

Setting the working directory can be done with the setwd() function or via the “Files” tab in the lower right panel of RStudio:

Alternatively, you can move the dataset in the folder where R is currently working (this can be found with getwd()). See more details on importing a file into R and about the working directory.

Another related problem is to use the wrong file. This error is different than the previous ones in the sense that you will not encounter an error but your analyses will still be wrong.

It may sound trivial, but make sure to import and use the correct data file! This is particularly the case if you have files for different points in time and which have a common structure (for example weekly or monthly data files with the exact same variables). It happened to me that I reported results for the wrong week (fortunately, without much consequence).

Also, make sure that you actually use all the rows you want to include in your analyses. It happened to me that, in order to test a model (and avoid long computing times), I extracted a random sample of the original dataset, and almost forgot about this sampling when running my final analyses.

It is thus a good practice to remind you to remove sampling and filters after you have tested your code (and before interpreting the final results).

10. Problem when using the $ operator

For the last error of this top 10, I would like to focus on two related errors:

“$ operator is invalid for atomic vectors”, and
“object of type ‘closure’ is not subsettable”.

I gather them in one single section because they are linked to each other in the sense that they both involve the $ operator.

$ operator is invalid for atomic vectors

To understand this error, we first must recall that an atomic vector is a one-dimensional object (usually created with c()). This is different than dataframes or matrices which are two-dimensional (i.e., rows form the first dimension and columns correspond to the second dimension).

The error “$ operator is invalid for atomic vectors” occurs when we try to access an element of an atomic vector using the dollar operator ($):

## define atomic vector:
x <- c(1, 3, 7)

## set names:
names(x) <- LETTERS[1:3]

## print vector:
x

## A B C 
## 1 3 7

## access value of element C:
x$C

## Error in x$C : $ operator is invalid for atomic vectors

Solution

The $ operator cannot be used to extract elements in atomic vectors. Instead, we must use double brackets [[]] notation:

x[["C"]]

## [1] 7

Remember that the $ operator can be used with dataframes, so we can also fix this error by first converting the atomic vector to a dataframe,⁵ and then access an element by its name with the $ operator:

## convert atomic vector to dataframe:
x <- as.data.frame(t(x))

## print x:
x

##   A B C
## 1 1 3 7

## access value of element C:
x$C

## [1] 7

object of type ‘closure’ is not subsettable

Another error (which I must admit is quite obscure and confusing when learning R) is the following: “object of type ‘closure’ is not subsettable”.

This error occurs when we try to subset or access some elements of a function. An example with the well-known mean() function:

mean[1:3]

## Error in mean[1:3] : object of type 'closure' is not subsettable

In R, we can subset lists, vectors, matrices, dataframes, but not functions. So it throws an error because it is impossible to subset an object of type “closure”, and a function is of that type:

typeof(mean)

## [1] "closure"

Most of the times, you will not encounter this error when using a basic function such as the mean() function (because it is unlikely that your goal is really to subset a function…).

Indeed, you will most likely face this error when trying to subset a dataset named data, but this dataset is not defined in the environment (because it has not been imported or created properly for instance).

To understand the concept, see the following examples:

## create dataset:
data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)

## print variable x:
data$x

##  [1]  1.12288964  1.43985574 -1.09711377 -0.11731956  1.20149840 -0.46972958
##  [7] -0.05246948 -0.08610730 -0.88767902 -0.44468400

So far so good. Now suppose we made a mistake when creating the dataset:

## create dataset (with mistake):
data <- data.frame(x = rnorm(10)
                   y = rnorm(10))

You will notice that a comma is missing between variables x and y. As a result, the dataset named data is not created and thus not defined.

Therefore, if we now try to access the variable x from that dataset data, R will actually try to subset the function named data instead of the dataset named data!

data$x

## Error in data$x : object of type 'closure' is not subsettable

This happens because, I repeat, the dataset data does not exist, so R looks for an object named data and find a function with that name:

class(data)

## [1] "function"

Warnings

Warnings are different than errors in the sense that they alert you about something, but it does not prevent you from running the code. It is a good practice to read these warnings as they may give you valuable information.

There are too many warnings to mention them all, but I would like to focus on two common ones:

“NAs introduced by coercion”, and
“Removed … rows containing non-finite values (stat_bin())”.

NAs introduced by coercion

This warning occurs when you try to convert a vector which includes at least one non-numerical value to a numeric vector:

x <- c(1, 3, 7, "Emma")

as.numeric(x)

## Warning: NAs introduced by coercion

## [1]  1  3  7 NA

You do not need to fix it since it is only a warning and not an error. R is simply informing you that at least one element in the initial vector was converted to NA because it could not be converted to a numeric value.

Removed … rows containing non-finite values (stat_bin())

This warning occurs when you use ggplot2 to draw plots. For instance:

ggplot(airquality) +
  aes(x = Ozone) +
  geom_histogram()

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

## Warning: Removed 37 rows containing non-finite values (`stat_bin()`).

Again, as it is a warning you do not need to fix it. It is simply informing you that there are some missing values (NA) in the variable of interest and that these missing values are removed to construct the plot.

Conclusion

Thanks for reading.

I hope that this collection of errors prevented you from making some coding mistakes, or that it helped you in debugging your code.

If you still cannot fix your error, I would recommend to read the documentation of the function (if you struggle with a function in particular), or look online for the solution. Bear in mind that if you encounter an error, it is very likely that someone else posted the answer online (Stack Overflow is usually a good resource).

R has a steep learning curve, in particular if you are not familiar with another programming language. Nonetheless, with practice and time, you will make less and less coding errors, but more importantly, you will be more and more proficient in typing the right keywords in search engines, resulting in less time spent looking for the solution.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

There are 2 mistakes in that piece of code, feel free to try to fix them as an exercise.↩︎
And I strongly recommend using RStudio and not just R. See the differences here.↩︎
Note that mean() applied to a logical variable gives the proportion of TRUE.↩︎
par(mfrow = c(1, 2)) is used to put two plots next to each other.↩︎
Note that we also need to take the transpose of the vector x in order to have it as 1 row, 3 columns.↩︎

Web scraping in R

Mon, 16 Jan 2023 00:00:00 +0000

Note: This post has been written in collaboration with Pietro Zanotta.

Introduction

Almost anyone is familiar with web pages (otherwise you would not be here), but what if we tell you that how you see a site is different from how Google or your browser does?

In fact, when you type any site address in your browser, your browser will download and render the page for you, but for rendering the page it needs some instructions.

There are 3 types of instructions:

HTML: describes a web page’s infrastructure;
CSS: defines the appearance of a site;
JavaScript: decides the behavior of the page.

Web scraping is the art of extracting information from the HTML, CSS and Javascript lines of code. The term usually refers to an automated process, which is less error-prone and faster than gathering data by hand.

It is important to note that web scraping can raise ethical concerns, as it involves accessing and using data from websites without the explicit permission of the website owner. It is a good practice to respect the terms of use for a website, and to seek written permission before scraping large amounts of data.

This article aims to cover the basics of how to do web scraping in R. We will conclude by creating a database on Formula 1 drivers from Wikipedia.

Note that this article doesn’t want to be exhaustive on topic. To learn more, see this section.

HTML and CSS

Before starting it is important to have a basic knowledge of HTML and CSS. This section aims to briefly explain how HTML and CSS work, to learn more we leave you some resources at the bottom of this article.

Feel free to skip this section if you already are knowledgeable in this topic.

Starting from HTML, an HTML file looks like the following piece of code.

<!DOCTYPE html>
<html lang="en">
<body>

<h1 href="https://en.wikipedia.org/wiki/Carl_Friedrich_Gauss"> Carl Friedrich Gauss</h1>
<h2> Biography </h2>
<p> Johann Carl Friedrich Gauss was born on 30 April 1777 in Brunswick. </p>
<h2> Profession </h2>
<p> Gauss is considered as one of the greatest mathematician, statistician and physicist of all time. </p>

</body>
</html>

Those instructions produce the following:

As you read above, HTML is used to describe the infrastructure of a web page, for example we may want to define the headings, the paragraphs, etc.

This infrastructure is represented by what are called tags (for example <h1>...</h1> or <p>...</p> are tags). Tags are the core of an HTML document as they represent the nature of what is inside the tag (for example h1 stands for heading 1). It is important to observe that there are two types of tags:

starting tags (e.g. <h1>)
ending tags (e.g. </h1>)

This is what allows to nest different tags.

Tags can also have attributes, for example in <h1 href="https://en.wikipedia.org/wiki/Carl_Friedrich_Gauss">Carl Friedrich Gauss</h1>, href is an attribute of the tag h1 that specifies an URL.

As the output of the above HTML code is not super elegant, CSS is used to style the final website. For example CSS is used to define the font, the color, the size, the spacing and many more features of a website.

What is important for this article are CSS selectors, which are patterns used to select elements. The most important is the .class selector, which selects all elements with the same class. For example the .xyz selector selects all elements with class="xyz".

Web scraping vs. APIs

Going back to web scraping, you may know that APIs are another way to access data from websites and online services.

In fact an API is a set of rules and protocols that allows two different software systems to communicate with each other. When a website or online service provides an API, it means that they have made it possible for developers to access their data in a structured and controlled way.

Why does web scraping exist if APIs are so powerful and do exactly the same work?

The main difference between web scraping and using APIs is that APIs are typically provided by the website or service to allow access to their data, while web scraping involves accessing data without the explicit permission of the website owner.

This means that using APIs is generally considered more ethical than web scraping, as it is done with the explicit permission of the website or service.

However, there are also some limitations to using APIs:

many APIs have rate limits, which means that they will only allow a certain number of requests to be made within a certain time period, i.e. you may not access large amounts of data;
not all websites or online services provide APIs, which means the only way to access their data is via web scraping.

Web scraping in R

There are several packages for web scraping in R, every package has its strengths and limitations. We will cover only the rvest package since it is the most used.

To get started with web scraping in R you will first need R and RStudio installed (if needed, see here). Once you have R and RStudio installed, you need to install the rvest package:

install.packages("rvest")

rvest

Inspired by beautiful soup and RoboBrowser (two Python libraries for web scraping), rvest has a similar syntax, which makes it the most eligible package for those who come from Python.

rvest provides functions to access a web page and specific elements using CSS selectors and XPath. The library is a part of the Tidyverse collection of packages, i.e. it shares some coding conventions (e.g. the pipes) with other libraries as tibble and ggplot2.

Before the real scraping it is necessary to load the rvest package:

library(rvest)

Now that everything is settled down, we can start the web scraping operation, which is usually made in 3 steps:

HTTP GET request
Parsing HTML content
Getting HTML element attributes

These steps are detailed in the following sections.

HTTP GET request

The HTTP GET method is a method used to send a server a question to get certain data and information. It is important to notice that this method does not change the state of the server.

To send a GET request we need the link (as a character) to the page we want to scrape:

link <- "https://www.nytimes.com/"

Sending the request to the page is simple, rvest provides the read_html function, which returns an object of html_document type:

NYT_page <- read_html(link)

NYT_page

## {html_document}
## <html lang="en" class="  nytapp-vi-homepage  tpl-always-light" xmlns:og="http://opengraphprotocol.org/schema/">
## [1] <head>\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8 ...
## [2] <body>\n    \n    <div id="app">\n<link rel="preload" as="image" href="/v ...

Parsing HTML content

As we saw in the last chunk of code, NYT_page contains the raw HTML code, which is not so easily readable.

In order to make it readable from R it has to be parsed, which means generating a Document Object Model (DOM) from the raw HTML. DOM is what connects scripts and web pages by representing the structure of a document in memory. If you retrieve the HTTP request using Node.js, you can give the raw HTML response to R for parsing and further analysis.

rvest provides 2 ways to select HTML elements:

XPath
CSS selectors

Selecting elements with rvest is simple, for XPath we use the following syntax:

NYT_page %>%
  html_elements(xpath = "")

while for CSS elector we need:

NYT_page %>%
  html_elements(css = "")

CSS selector

Suppose that for a project you need the summaries of the articles of the NYT (note that what is in the following picture is not what you see in the New York Times web page).

Searching in the HTML code, it is not that complex to find <p class="summary-class">, which is the markup of what we are looking for. To parse the HTML using this selector we use the html_element function:

summaries_css <- NYT_page %>%
  html_elements(css = ".summary-class")

head(summaries_css)

## {xml_nodeset (6)}
## [1] <p class="summary-class css-crclbt">At least 18 were killed in Kyiv, offi ...
## [2] <p class="summary-class css-1vqq9lj">Long-delayed funeral ceremonies for  ...
## [3] <p class="summary-class css-1vqq9lj">Senior U.S. officials have said Iran ...
## [4] <p class="summary-class css-1vqq9lj">One of England’s greatest World Cup  ...
## [5] <p class="summary-class css-1vqq9lj">America’s 2-0 win over Bosnia and He ...
## [6] <p class="summary-class css-1vqq9lj">Some data suggest artificial intelli ...

The easiest way to obtain a CSS selector is opening the inspect mode, find the element you desire and right click on it. Then click on copy and copy selector.

XPath

Parsing with XPath is similar to parsing using selectors. In fact, we just need to repeat what we did above using XPath of the element of interest. Moreover, obtaining an element’s XPath is not different form selector: inspector mode -> right click on element of interest -> copy -> copy XPath.

Repeating what we did above with XPath:

summaries_xpath <- NYT_page %>%
  html_elements(xpath = "//*[contains(@class, 'summary-class')]")

head(summaries_xpath)

## {xml_nodeset (6)}
## [1] <p class="summary-class css-crclbt">At least 18 were killed in Kyiv, offi ...
## [2] <p class="summary-class css-1vqq9lj">Long-delayed funeral ceremonies for  ...
## [3] <p class="summary-class css-1vqq9lj">Senior U.S. officials have said Iran ...
## [4] <p class="summary-class css-1vqq9lj">One of England’s greatest World Cup  ...
## [5] <p class="summary-class css-1vqq9lj">America’s 2-0 win over Bosnia and He ...
## [6] <p class="summary-class css-1vqq9lj">Some data suggest artificial intelli ...

Obviously the data we collected with CSS selector and XPath are exactly the same.

Getting attributes

Since the chunk of code above collect all the elements p with the class summary, we render all the elements of NYT_summary_css as a text using the html_text function:

NYT_summaries_css <- html_text(summaries_css)
NYT_summaries_xpath <- html_text(summaries_xpath)

We only print some of them:

head(NYT_summaries_css)

## [1] "At least 18 were killed in Kyiv, officials said. Ukraine’s president had warned of a “massive strike” as his country’s forces hit deeper into Russian territory."
## [2] "Long-delayed funeral ceremonies for Ayatollah Ali Khamenei, killed during U.S.-Israeli strikes at the war’s outset, are set to begin Friday."                    
## [3] "Senior U.S. officials have said Iran would be richly rewarded for changing its stance on America. But Tehran has rejected such a bargain in the past."           
## [4] "One of England’s greatest World Cup moments unfolded on Wednesday, a columnist for The Athletic writes."                                                         
## [5] "America’s 2-0 win over Bosnia and Herzegovina sets up a round-of-16 showdown with Belgium."                                                                      
## [6] "Some data suggest artificial intelligence is already causing job losses. Other sources show the opposite. Why is it so hard to figure out what’s going on?"

A real application of web scraping in R

To conclude this brief introduction to web scraping we want to use the rvest package in a real word application of web scraping. The goal is to scrape data from Formula 1 Wikipedia’s voice and create a CSV file containing the name, the nationality, the number of podiums and some other statistics for every pilot.

The table we are going to scrape is the following:

If you haven’t done so, you need to install the rvest package:

install.packages("rvest")

and then load it:

library(rvest)

HTTP GET request

The GET request is the easiest part of scraping, we just need the following line of code:

link <- "https://en.wikipedia.org/wiki/List_of_Formula_One_drivers"

Parsing HTML content and getting attributes

Again we repeat what we did before with the NYT example:

page <- read_html(link)

Searching in the HTML code we find that the table is a table element with the sortable attribute:

Therefore we run the following lines of code:

drivers_F1 <- html_element(page, "table.sortable") %>%
  html_table()

In the chunk of code above, the html_table function is used to render the HTML code into tables.

To inspect it, we display the first and last observations, and the structure of the dataset:

head(drivers_F1) # first 6 rows

## # A tibble: 6 × 11
##   `Driver name`     Nationality    `Seasons competed` `Drivers' Championships`
##   <chr>             <chr>          <chr>              <chr>                   
## 1 Carlo Abate       Italy          1962–1963          0                       
## 2 George Abecassis  United Kingdom 1951–1952          0                       
## 3 Kenny Acheson     United Kingdom 1983, 1985         0                       
## 4 Andrea de Adamich Italy          1968, 1970–1973    0                       
## 5 Philippe Adams    Belgium        1994               0                       
## 6 Walt Ader         United States  1950               0                       
## # ℹ 7 more variables: `Race entries` <chr>, `Race starts` <chr>,
## #   `Pole positions` <chr>, `Race wins` <chr>, Podiums <chr>,
## #   `Fastest laps` <chr>, `Points[a]` <chr>

tail(drivers_F1) # last 6 rows

## # A tibble: 6 × 11
##   `Driver name`  Nationality `Seasons competed`   `Drivers' Championships`
##   <chr>          <chr>       <chr>                <chr>                   
## 1 Emilio Zapico  Spain       1976                 0                       
## 2 Zhou Guanyu    China       2022–2024            0                       
## 3 Ricardo Zonta  Brazil      1999–2001, 2004–2005 0                       
## 4 Renzo Zorzi    Italy       1975–1977            0                       
## 5 Ricardo Zunino Argentina   1979–1981            0                       
## 6 Driver name    Nationality Seasons competed     Drivers' Championships  
## # ℹ 7 more variables: `Race entries` <chr>, `Race starts` <chr>,
## #   `Pole positions` <chr>, `Race wins` <chr>, Podiums <chr>,
## #   `Fastest laps` <chr>, `Points[a]` <chr>

str(drivers_F1) # structure of the dataset

## tibble [878 × 11] (S3: tbl_df/tbl/data.frame)
##  $ Driver name           : chr [1:878] "Carlo Abate" "George Abecassis" "Kenny Acheson" "Andrea de Adamich" ...
##  $ Nationality           : chr [1:878] "Italy" "United Kingdom" "United Kingdom" "Italy" ...
##  $ Seasons competed      : chr [1:878] "1962–1963" "1951–1952" "1983, 1985" "1968, 1970–1973" ...
##  $ Drivers' Championships: chr [1:878] "0" "0" "0" "0" ...
##  $ Race entries          : chr [1:878] "3" "2" "10" "36" ...
##  $ Race starts           : chr [1:878] "0" "2" "3" "30" ...
##  $ Pole positions        : chr [1:878] "0" "0" "0" "0" ...
##  $ Race wins             : chr [1:878] "0" "0" "0" "0" ...
##  $ Podiums               : chr [1:878] "0" "0" "0" "0" ...
##  $ Fastest laps          : chr [1:878] "0" "0" "0" "0" ...
##  $ Points[a]             : chr [1:878] "0" "0" "0" "6" ...

Now that we have a tibble (a sort of dataframe used in the tidyverse universe), we just need to select the variables of interest and eliminate the last row that contains the name of the variables:

drivers_F1 <- drivers_F1[c(1:4, 7:9)] # select variables

drivers_F1 <- drivers_F1[-nrow(drivers_F1), ] # remove last row

At this point we may want to clean our data. For example, we notice that Drivers' Championships has a small formatting issue: it returns not only the number of championships the driver won, but also the years of the victories. To extract only the number of victories (without the years) we use the substr() function:

drivers_F1$`Drivers' Championships` <- substr(drivers_F1$`Drivers' Championships`,
  start = 1, stop = 1
)

With this code, we actually extract only the first character since we start at 1 and stop at 1. At the moment, the maximum number of championships won by a driver is 7 (Lewis Hamilton & Michael Schumacher), so it is fine to extract only the first digit.

Et voila! With only a few lines of code, we scraped a table and we are now ready to perform our analysis.

If you want to save the dataset, you can always do so:

write.csv(drivers_F1, "F1_drivers.csv", row.names = FALSE)

Analysis on the database

To convince you that this is a real database, we will now answer some simple questions.

First of all, we load the tidyverse package:

library(tidyverse)

Which country has the largest number of wins?

drivers_F1 %>%
  group_by(Nationality) %>%
  summarise(championship_country = sum(as.double(`Drivers' Championships`))) %>%
  arrange(desc(championship_country))

## # A tibble: 48 × 2
##    Nationality    championship_country
##    <chr>                         <dbl>
##  1 United Kingdom                   21
##  2 Germany                          12
##  3 Brazil                            8
##  4 Argentina                         5
##  5 Australia                         4
##  6 Austria                           4
##  7 Finland                           4
##  8 France                            4
##  9 Netherlands                       4
## 10 Italy                             3
## # ℹ 38 more rows

Who has the most Championships?

drivers_F1 %>%
  group_by(`Driver name`) %>%
  summarise(championship_pilot = sum(as.double(`Drivers' Championships`))) %>%
  arrange(desc(championship_pilot))

## # A tibble: 877 × 2
##    `Driver name`       championship_pilot
##    <chr>                            <dbl>
##  1 Lewis Hamilton~                      7
##  2 Michael Schumacher^                  7
##  3 Juan Manuel Fangio^                  5
##  4 Alain Prost^                         4
##  5 Max Verstappen~                      4
##  6 Sebastian Vettel^                    4
##  7 Ayrton Senna^                        3
##  8 Jack Brabham^                        3
##  9 Jackie Stewart^                      3
## 10 Nelson Piquet^                       3
## # ℹ 867 more rows

Sorry Michael, it looks like Lewis dethroned you.

Is there a relation between the number of Championships won and the number of race pole positions?

drivers_F1 %>%
  filter(`Pole positions` > 1) %>%
  ggplot(aes(x = as.double(`Pole positions`), y = as.double(`Drivers' Championships`))) +
  geom_point(position = "jitter") +
  labs(y = "Championships won", x = "Pole positions") +
  theme_minimal()

As expected, there seems to be a positive relationship between the number of pole positions and the number of Championships won. To quantify this relationship, we could build a linear model but this is beyond the scope of the article.

To go further

As you have seen, rvest is a powerful tool. The goal of the article is to show just the tip of the iceberg regarding web scraping in R.

There are many resources online that you can read if you want to know more:

Web scraping: Basics by Paul Bauer
rvest CRAN documentation
xml2 CRAN documentation
httr CRAN documentation
rvest CRAN vignette
httr CRAN vignette
Beautiful Soup documentation
RoboBrowser documentation
HTML documentation
CSS documentation

Conclusion

Thanks for reading.

I hope this article helped you to learn about web scraping in R, and gave you the incentive to use it for your projects. If you are interested in seeing another example, see how to scrape Yahoo search engine results with R.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

What is survival analysis? Examples by hand and in R

Thu, 22 Dec 2022 00:00:00 +0000

Note that this article is inspired from:

the lecture notes of Prof. Van Keilegom and my personal notes as teaching assistant for her course entitled “Analysis of Survival and Duration Data” given at UCLouvain
the lecture notes of Prof. Legrand for her course entitled “Statistics in clinical trials” given at UCLouvain

Introduction

For the last post of the year, I would like to present a rather unknown (yet important) statistical method–survival analysis.

Although survival analysis is a branch of statistics, it is usually not covered in introductory statistics courses and it is rather unknown to the general public. It is mostly taught in biostatistics courses or advanced statistics study programs.

In this article, I will explain what is survival analysis, in which context and how it is used. I will explain the main tools and methods used by biostatisticians to analyze survival data and how to estimate and interpret survival curves.

I will show in detail how to apply these techniques in R with concrete examples. In practice, survival analysis is almost always done via a statistical program and never done by hand. However, as for any statistical concept, I believe that doing it by hand allows to really understand the concepts and what these programs actually do. For this reason, I will also show a brief example on how to perform a basic survival analysis by hand.

What is survival analysis?

Survival analysis (also called time-to-event analysis or duration analysis) is a branch of statistics aimed at analyzing the duration of time from a well-defined time origin until one or more events happen, called survival times or duration times.

In other words, in survival analysis, we are interested in a certain event and want to analyze the time until the event happens.

While the event of interest is often death (in this case we study the time to death for patients having a specific disease) or recurrence (in this case we study the time to relapse of a certain disease), it is not limited to the medical field or epidemiology.

In fact, it can be used in many domains. For example, we may also analyze the time until:

getting cured from a certain disease
finding a new job after a period of unemployment
being arrested again after having been released from jail
the first pregnancy
the failure of a mechanical system or a machine
a bank or a company goes bankrupt
a customer buys a new product or stops its current subscription
a letter is delivered
a taxi picks you up after having called the taxi company
an employee leaves the company
etc.

As you can see, the event of interest does not necessarily have to be the death or the occurrence of a disease, but in all situations we are interested in analyzing the time until a specific event occurs.

Why do we need special methods for survival analysis?

Survival data, also referred as time-to-event data, requires a special set of statistical methods for three main reasons:

Duration times are always positive: the time until an event of interest occurs cannot be less than 0. Moreover, the distribution of survival times is right-skewed.
Different measures are of interest depending on the research question, context, etc. For instance, we could be interested in:
- The probability that a cancer patient survives longer than 5 years after diagnosis?
- The typical waiting time for a cab to arrive after having called the taxi company?
- How many, out of 100 unemployed people, are expected to have a job again after 2 months of unemployment?
Censoring is almost always an issue:
- When the event occurred before the end of the study, the survival time is known.
- However, sometimes, the event is not yet observed at the end of the study. Suppose that we study the time until death of patients with breast cancer. Luckily, some patients will not die before the end of the study.
- It can also happen that the patient withdraws from the study or moves to another country before the end of the study (known as lost to follow up or drop out).
- In all situations, his or her survival time cannot be observed because the event is not observed for the duration of the study.
- Censoring can be seen, in some sense, as a type of missing data.
- For these reasons, many “standard” statistical tools such as descriptive statistics, hypothesis tests and regression models are not appropriate for this kind of data. Specific statistical methods are required to take into account the fact that the exact survival duration for some patients is missing. It is known that they survived a certain amount of time (until the end of the study or until the time of withdrawal), but their exact survival time is unknown.

For your information, there are three types of censoring:

right-censoring (the most frequent),
left-censoring (the least frequent) and
interval-censoring.

When the event is not yet observed at the end of the study (i.e., the survival time is greater than the observed duration), this is referred as right-censoring. Left-censoring occurs if a participant is entered into the study when the event of interest occurred prior to study entry but we do not know exactly when. Interval-censoring implies that the event occurred within a time interval (between two known dates, two visits, etc.); the exact moment of occurrence is not known. The goal is of course to analyze all available data, including information about censored patients.

The goal of survival analysis is thus to model and describe time-to-event data in an appropriate way, taking the particularities of this type of data into account.

Common functions in survival analysis

We are not going to go to much into the details, but it is important to lay the foundation with the most common functions in survival analysis.

Let $T$ be a non-negative continuous random variable, representing the time until the event of interest. We consider the following functions:

Survival function
Cumulative hazard function
Hazard function

Survival function

The most common one is the survival function.

Let $T$ be a non-negative continuous random variable, representing the time until the event of interest. The survival function $S(t)$ is the probability that a randomly chosen individual is still at risk at time $t$, where $0 \le t \le +\infty$. For each $t$, it is given by

\[ \begin{align*} S(t) &= P(T > t)\\ &= 1 - P(T \le t)\\ &= 1 - F(t)\\ &= 1 - \int^t_0 f(u)\text{d}u, \end{align*} \]

where $f(\cdot)$ and $F(\cdot)$ are the density and the cumulative distribution functions of $T$, respectively.

$S(t)$ represents, for each time $t$, the probability that the time until the event is greater than this time $t$. In other words, it models the probability that the event of interest happens after $t$.

In the context of our examples mentioned above, it gives the probability that:

a randomly selected patient will survive beyond time $t$ or the proportion of patients still alive after time $t$,
a cab takes more than $t$ minutes to arrive, or
an unemployed person take more than $t$ months to find a new job.

The survival function $S(t)$ is:

a decreasing function,
taking values in $[0, 1]$ (since it is a probability), and
equal to 1 at $t = 0$ (i.e., $S(0) = 1$) and 0 at $t = \infty$ (i.e., $S(\infty) = 0$).

Visually we have:

The curve shows the proportion of individuals (or experimental units) who, as time goes on, have not experienced the event of interest. As time progresses, events occur, so the proportion who have not experienced the event decreases.

Cumulative hazard function

The cumulative hazard function, which is the total hazard experienced up to time $t$, is defined as:

\[H(t) = -log\left(S(t)\right)\]

and has the following properties:

increasing function,
taking value in $[0, +\infty]$, and
$S(t) = exp(-H(t))$.

Hazard function

The hazard function $h(t)$, or hazard rate, defines the instantaneous event rate at time $t$ for an individual still at risk at that time. It can be obtained by

\[ \begin{align*} h(t) &= \lim_{\Delta t \rightarrow 0} \frac{P(t \le T < t + \Delta t | T \ge t)}{\Delta t}\\ &= \frac{d}{dt} H(t)\\ &= \frac{f(t)}{S(t)}. \end{align*} \]

and has the following properties:

positive function (not necessarily increasing or decreasing)
the hazard function $h(t)$ can have many different shapes and is therefore a useful tool to summarize survival data

In the context of cancer research when death is the event of interest, $h(t)$ measures the instantaneous risk of dying right after time $t$ given the individual is alive at time $t$.

To link the hazard rate with the survival function; the survival curve represents the hazard rates. A steeper slope indicates a higher hazard rate because events happen more frequently, reducing the proportion of individuals who have not experienced the event at a faster rate. On the contrary, a gradual and flatter slope indicates a lower hazard rate because events occur less frequently, reducing the proportion of individuals who have not experiences the event at a slower rate. More formally:

\[S(t) = \exp\left(-\int^t_0 h(u) \text{d}u\right).\]

Note that, in contrast to the survival function which focuses on not having an event, the hazard function focuses on the event occurring.

Estimation

To estimate the survival function, we need to use an estimator which is able to deal with censoring. The most common one is the nonparametric Kaplan and Meier (1958) estimator (also sometimes referred as the product-limit estimator, or more simply, the K-M estimator).

The advantages of the Kaplan-Meier estimator are that:

it is simple and straightforward to use and interpret
it is a nonparametric estimator, so it constructs a survival curve from the data and no assumptions is made about the shape of the underlying distribution
it gives a graphical representation of the survival function(s), useful for illustrative purposes

The principle behind this estimator is that surviving beyond time $t_i$ implies surviving beyond time $t_{i-1}$ and surviving at time $t_i$. Note that an important assumption for the estimation to hold is that censoring is independent of the occurrence of events. We say that censoring is non-informative, that is, censored subjects have the same survival prospects as subjects who are not censored and who continue to be followed.

By hand

To understand how it works, let’s first estimate it by hand on the following dataset:¹

subject	time	event
1	3	0
2	5	1
3	7	1
4	2	1
5	18	0
6	16	1
7	2	1
8	9	1
9	16	1
10	5	0

where:

subject is the individual’s identifier
time is the time to event (in years)²
event is the event status (0 = censored, 1 = event happened)

Remember that for each subject, we need to know at least 2 pieces of information:

the time until the event of interest or the time until the censoring, and
whether we have observed the event of interest or if we have observed censoring.

We first need to count the number of distinct event times. Ignoring censored observations, we have 5 distinct event times:

2, 5, 7, 9 and 16

The easiest way to do the calculation by hand is by filling the following table (a table with 5 rows since there are 5 distinct event times):

$j$	$y_{(j)}$	$d_{(j)}$	$R_{(j)}$	$1 - \frac{d_{(j)}}{R_{(j)}}$
1
2
3
4
5

We fill columns one by one:

$y_{(j)}$ = the ordered distinct event times:

2, 5, 7, 9 and 16

So the table becomes:

$j$	$y_{(j)}$	$d_{(j)}$	$R_{(j)}$	$1 - \frac{d_{(j)}}{R_{(j)}}$
1	2
2	5
3	7
4	9
5	16

$d_{(j)}$ = the number of observations for each distinct event time. For this, the frequency for each distinct event time is useful:

## time
##  2  5  7  9 16 
##  2  1  1  1  2

The table becomes:

$j$	$y_{(j)}$	$d_{(j)}$
1	2	2
2	5	1
3	7	1
4	9	1
5	16	2

$R_{(j)}$ = the remaining number of individuals at risk. For this, the distribution of time (censored and not censored) is useful:

## time
##  2  3  5  7  9 16 18 
##  2  1  2  1  1  2  1

We see that:

At the beginning there are 10 subjects
Just before time $t = 5$, there are 7 subjects left (10 subjects - 2 who had the event - 1 who is censored)
Just before time $t = 7$, there are 5 subjects left (= 10 - 2 - 1 - 2)
Just before time $t = 9$, there are 4 subjects left (= 10 - 2 - 1 - 2 - 1)
Just before time $t = 16$, there are 3 subjects left (= 10 - 2 - 1 - 2 - 1 - 1)

The table becomes:

$j$	$y_{(j)}$	$d_{(j)}$	$R_{(j)}$
1	2	2	10
2	5	1	7
3	7	1	5
4	9	1	4
5	16	2	3

$1 - \frac{d_{(j)}}{R_{(j)}}$ is straightforward, so the table becomes:

$j$	$y_{(j)}$	$d_{(j)}$	$R_{(j)}$	$1 - \frac{d_{(j)}}{R_{(j)}}$
1.00	2.00	2.00	10.00	0.80
2.00	5.00	1.00	7.00	0.86
3.00	7.00	1.00	5.00	0.80
4.00	9.00	1.00	4.00	0.75
5.00	16.00	2.00	3.00	0.33

The Kaplan-Meier estimator is:

\[\hat{S}_{KM}(t) = \prod_{j:y_{(j)} \le t} \left(1 - \frac{d_{(j)}}{R_{(j)}} \right)\]

For each $j$, we thus take the cumulative product:

$j_1 = 0.8$
$j_2 = 0.8 \cdot 0.857 = 0.6856$
$j_3 = 0.6856 \cdot 0.8 = 0.54848$
$j_4 = 0.54848 \cdot 0.75 = 0.41136$
$j_5 = 0.41136 \cdot 0.333 = 0.1369829$

So finally, we have the survival probabilities (rounded to 3 digits):

$j$	$1 - \frac{d_{(j)}}{R_{(j)}}$	$\hat{S}_{KM}(t)$
1.00	0.80	0.80
2.00	0.86	0.69
3.00	0.80	0.55
4.00	0.75	0.41
5.00	0.33	0.14

We can now represent graphically the Kaplan-Meier estimator:

To draw this survival curve, remember that:

the x-axis corresponds to the time variable in the initial dataset, and
the y-axis corresponds to the survival probabilities found above.

In R

We now compare our results with the results found in R.

We first create the dataset with the time and event variables:

# create dataset
dat <- data.frame(
  time = c(3, 5, 7, 2, 18, 16, 2, 9, 16, 5),
  event = c(0, 1, 1, 1, 0, 1, 1, 1, 1, 0)
)

We then run the Kaplan-Meier estimator with the survfit() and Surv() functions:

# KM
library(survival)

km <- survfit(Surv(time, event) ~ 1,
  data = dat
)

Notice that the Surv() function accepts two arguments:

the time variable, and
the event variable.

The ~ 1 in the survfit() function indicates that we estimate the Kaplan-Meier without any grouping. See more on this later in the post.

Finally, we display the results and draw the Kaplan-Meier plot:

# results
summary(km)

## Call: survfit(formula = Surv(time, event) ~ 1, data = dat)
## 
##  time n.risk n.event survival std.err lower 95% CI upper 95% CI
##     2     10       2    0.800   0.126       0.5868        1.000
##     5      7       1    0.686   0.151       0.4447        1.000
##     7      5       1    0.549   0.172       0.2963        1.000
##     9      4       1    0.411   0.176       0.1782        0.950
##    16      3       2    0.137   0.126       0.0225        0.834

# plot
plot(km,
  xlab = "Time",
  ylab = "Survival probability",
  conf.int = FALSE
)

The survival probabilities can be found in the survival column. Remark that results by hand and in R are similar (any difference with the results by hand is due to rounding).

Alternatively, we can use the ggsurvplot() function within the {survminer} package:

library(survminer)

# plot
ggsurvplot(km,
  conf.int = FALSE,
  legend = "none"
)

Note that the crosses on the survival curve denote the censored observations.

The advantage with the ggsurvplot() function is that it is easy to draw the median survival directly on the plot:³

ggsurvplot(km,
  conf.int = FALSE,
  surv.median.line = "hv",
  legend = "none"
)

To find the median survival:⁴

summary(km)$table["median"]

## median 
##      9

# or more simply
km

## Call: survfit(formula = Surv(time, event) ~ 1, data = dat)
## 
##       n events median 0.95LCL 0.95UCL
## [1,] 10      7      9       5      NA

Suppose that the event of interest is death:

At time zero, the survival probability is 1 (100% of the subjects are alive).
The median indicates that the median survival time is 9 years.⁵ This is the time at which the survival $S(t)$ is 50%. In other words, is the time after which half of the subjects are expected to have died.
From the plot, we also see that $S(5) = P(T > 5 \text{ years}) =$ Probability of survival of more than 5 years for these subjects = 68.6%. This means that 68.6% of all subjects survive longer than 5 years, and that 31.4% of all subjects die within the first 5 years.

For the sake of completeness, let’s do another example with a much larger dataset; the tongue dataset within the {KMsurv} package.⁶

# load data
library(KMsurv)
data(tongue)

# preview data
head(tongue)

##   type time delta
## 1    1    1     1
## 2    1    3     1
## 3    1    3     1
## 4    1    4     1
## 5    1   10     1
## 6    1   13     1

type is the tumor DNA profile (1 = aneuploid tumor, 2 = diploid tumor)
time is the time to death or on-study time (in weeks)
delta is the death indicator (0 = alive, 1 = dead)

For this example, we focus on the aneuploid type:

anaploid <- subset(tongue, type == 1)

We can now plot the estimated survival function and estimate the median time to death. Since it is an estimator, we can also construct a confidence interval for the estimated survival at each time $t$ and for the estimated median survival time.⁷

# results
fit <- survfit(Surv(time, delta) ~ 1,
  data = anaploid,
  conf.type = "log-log"
)

fit

## Call: survfit(formula = Surv(time, delta) ~ 1, data = anaploid, conf.type = "log-log")
## 
##       n events median 0.95LCL 0.95UCL
## [1,] 52     31     93      65     157

# plot
ggsurvplot(fit,
  surv.median.line = "hv",
  legend = "none"
)

The median survival time is estimated to be 93 weeks, with a 95% confidence interval between 65 and 157 weeks.

Kaplan-Meier curves can be seen as descriptive statistics for survival data. We now focus on the second branch of statistics, hypothesis testing which allows to draw conclusions on the population based on a sample (see a quick reminder about the difference between population and sample if you need).

Hypothesis testing

Hypothesis testing in the field of survival analysis mostly concerns:

The hazard function of one population: in this case we test whether a censored sample comes from a population with a known hazard function $h_0(t)$. For example, we may be interested to compare survival in a sample of patients to the survival in the overall population (derived from the life tables).
The comparison of the hazard function of two or more populations: in this case we are interested in assessing whether there are differences in survival among different groups of subjects. For example:
- 2 groups: we are interested in comparing survival for female and male colon cancer patients
- 3 groups or more: we are interested in comparing survival for melanoma cancer patients according to their treatments (with treatments A, B and C for example)⁸

Log-rank test

In this article, we focus on comparing survival between two groups using the log-rank test (also known as Mantel-Cox test). This test is the most common hypothesis test to compare survival between two groups.

The intuition behind the test is that if the two groups have different hazard rates, the two survival curves (so their slopes) will differ. More precisely, the log-rank test compares the observed number of events in each group to what would be expected if the survival curves were identical (i.e., if the null hypothesis were true).

Note that, as for the Kaplan-Meier estimator, the log-rank test is a nonparametric test, which makes no assumptions about the survival distributions.

For this example, consider the following dataset:

##    patient group time event
## 1        1     1  4.1     1
## 2        2     1  7.8     0
## 3        3     1 10.0     1
## 4        4     1 10.0     1
## 5        5     1 12.3     0
## 6        6     1 17.2     1
## 7        7     2  9.7     1
## 8        8     2 10.0     1
## 9        9     2 11.1     0
## 10      10     2 13.1     0
## 11      11     2 19.7     1
## 12      12     2 24.1     0

where:

patient is the patient’s identifier
group is the group (group 1 or 2)
time is the time to death (in years)⁹
event is the event status (0 = censored, 1 = death)

Suppose we are interested in comparing group 1 and 2 in terms of survival, that is, we compare survival curves between the 2 groups:

$H_0 : S_1(t) = S_2(t)$ for all $t$
$H_1 : S_1(t) \ne S_2(t)$ for some $t$

It is a statistical test, so if the $p$-value < $\alpha$ (usually 0.05), we reject the null hypothesis and we conclude that survival (or the time to event) is significantly different between the two groups considered.

To perform the log-rank test, the following test statistic will be useful:

\[\begin{eqnarray} U &=& \sum_{j=1}^r w(y_{(j)})\left(O_j - E_j\right) \\ &=& \sum_{j=1}^r w(y_{(j)})\left( d_{(j)1} - \frac{d_{(j)}R_{(j)1}}{R_{(j)}}\right) \end{eqnarray}\]

with $U^{obs} = \frac{U}{\sqrt{Var(U)}} \sim N(0,1)$ and¹⁰

\[\begin{eqnarray} Var(U) &=& \sum_{j=1}^r w^2(y_{(j)}) \frac{N_{(j)}}{ D_{(j)} }\\ &=& \sum_{j=1}^r w^2(y_{(j)}) \frac{ d_{(j)} \frac{R_{(j)1}}{R_{(j)} } \left( 1 - \frac{R_{(j)1}}{R_{(j)} } \right) \left( R_{(j)} - d_{(j)}\right) }{ R_{(j)} - 1 } \end{eqnarray}\]

By hand

As for the Kaplan-Meier estimator by hand, it is best to also fill in a table for the log-rank test by hand.

Let’s present the final table and comment below on how to fill it, column by column:

$j$	$y_{(j)}$	$d_{(j)1}$	$R_{(j)1}$	$d_{(j)2}$	$R_{(j)2}$	$d_{(j)}$	$R_{(j)}$	$E_{j}$	$O_{j}$	$O_{j} - E_{j}$	$N_{(j)}$	$D_{(j)}$	$N_{(j)}/D_{(j)}$
1	4.1	1	6	0	6	1	12	0.5	1	0.5	2.75	11	0.25
2	9.7	0	4	1	6	1	10	0.4	0	-0.4	2.16	9	0.24
3	10	2	4	1	5	3	9	1.333	2	0.667	4.44	8	0.555
4	17.2	1	1	0	2	1	3	0.333	1	0.667	0.44	2	0.22
5	19.7	0	0	1	2	1	2	0	0	0	0.00	1	0.00
$Total$		4		3		7		2.566		1.433			1.265

Column $j$ is the number of distinct event times. We see that there are 5 (ignoring censored observations), so we write 1 to 5 in the table.

Column $y_{(j)}$ is the ordered distinct event times:

4.1, 9.7, 10, 17.2 and 19.7

Column $d_{(j)1}$ is the number of observations for each distinct event time, for group 1:

## time
##  4.1   10 17.2 
##    1    2    1

When there is no event, we simply write 0 in the table.

Column $R_{(j)1}$ is the remaining number of patients at risk, for group 1. For this, the distribution of time (censored and not censored, for group 1) is useful:

## time
##  4.1  7.8   10 12.3 17.2 
##    1    1    2    1    1

We see that:

At the beginning, there are 6 patients
Before time 9.7, there are 4 patients left (6 - 1 who had the event at time 4.1 - 1 who was censored at time 7.8)
Before time 10, there are 4 patients left (6 - 2)
Before time 17.2, there are 1 patient left (6 - 5)
Before time 19.7, there are 0 patient left (6 - 6)

Columns $d_{(j)2}$ and $R_{(j)2}$ follow the same principle, but for group 2 this time. So we have, respectively for $d_{(j)2}$ and $R_{(j)2}$:

## time
##  9.7   10 19.7 
##    1    1    1

## time
##  9.7   10 11.1 13.1 19.7 24.1 
##    1    1    1    1    1    1

Columns $d_{(j)}$ and $R_{(j)}$ also follow the same principle, but this time considering both groups. So we have, respectively for $d_{(j)}$ and $R_{(j)}$:

## time
##  4.1  9.7   10 17.2 19.7 
##    1    1    3    1    1

## time
##  4.1  7.8  9.7   10 11.1 12.3 13.1 17.2 19.7 24.1 
##    1    1    1    3    1    1    1    1    1    1

Column $E_{j}$ is the expected number of events in the first group assuming that $h_1 \equiv h_2$. It is obtained as follows

\[ E_{j} = \frac{d_{(j)}R_{(j)1}}{R_{(j)}}\]

Column $O_{j}$ is the observed number of events in the first group, so it is equal to the $d_{(j)1}$ column.

Column $O_{j} - E_{j}$ is straightforward.

Column $N_{(j)}$ is defined as follows

\[N_{(j)} = d_{(j)} \frac{R_{(j)1}}{R_{(j)} } \left( 1 - \frac{R_{(j)1}}{R_{(j)} } \right) \left( R_{(j)} - d_{(j)}\right)\]

Column $D_{(j)}$ is $R_{(j)} - 1$.

Column $N_{(j)}/D_{(j)}$ is straightforward.

Since $w(y_{(j)}) = w^2(y_{(j)}) = 1$ for a log-rank test, we have¹¹

\[ U^{obs} = \frac{U}{\sqrt{Var(U)}} = \frac{1.434}{\sqrt{1.265}} = 1.275.\]

We reject $H_0$ if $|U^{obs}|>z_{1-\alpha/2}$, so at the 5% significance level we reject $H_0$ if $|U^{obs}|>z_{0.975}=1.96$.

We have $|U^{obs}| = 1.275 < z_{0.975}=1.96$. Hence, at the 5% significance level we do not reject $H_0$. This means that, based on the data, we are not able to conclude that survival is different between the two groups (which is equivalent than saying that we do not reject the hypothesis that survival is equal between the two groups).

If you are interested in computing the $p$-value:

$p$-value $= 2\times P(Z>1.275) = 2 \times 0.101 = 0.202 > 0.05$.

In R

We now compare our results in R with the survdiff() function:

dat <- data.frame(
  group = c(rep(1, 6), rep(2, 6)),
  time = c(4.1, 7.8, 10, 10, 12.3, 17.2, 9.7, 10, 11.1, 13.1, 19.7, 24.1),
  event = c(1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0)
)

dat

##    group time event
## 1      1  4.1     1
## 2      1  7.8     0
## 3      1 10.0     1
## 4      1 10.0     1
## 5      1 12.3     0
## 6      1 17.2     1
## 7      2  9.7     1
## 8      2 10.0     1
## 9      2 11.1     0
## 10     2 13.1     0
## 11     2 19.7     1
## 12     2 24.1     0

survdiff(Surv(time, event) ~ group,
  data = dat
)

## Call:
## survdiff(formula = Surv(time, event) ~ group, data = dat)
## 
##         N Observed Expected (O-E)^2/E (O-E)^2/V
## group=1 6        4     2.57     0.800      1.62
## group=2 6        3     4.43     0.463      1.62
## 
##  Chisq= 1.6  on 1 degrees of freedom, p= 0.2

Alternatively, we can use the ggsurvplot() function to draw the survival curves and perform the log-rank test at the same time:

fit <- survfit(Surv(time, event) ~ group, data = dat)

ggsurvplot(fit,
  pval = TRUE,
  pval.method = TRUE
)

As we can see, the $p$-values and the conclusions are the same (any difference with the results by hand is due to rounding).

As for the Kaplan-Meier estimation, we do another example on a larger dataset. Consider the data on the times until staphylococcus infection of burn patients, also available in the {KMsurv}:¹²

# load data
data(burn)

# preview data
head(burn)

##   Obs Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9 Z10 Z11 T1 D1 T2 D2 T3 D3
## 1   1  0  0  0 15  0  0  1  1  0   0   2 12  0 12  0 12  0
## 2   2  0  0  1 20  0  0  1  0  0   0   4  9  0  9  0  9  0
## 3   3  0  0  1 15  0  0  0  1  1   0   2 13  0 13  0  7  1
## 4   4  0  0  0 20  1  0  1  0  0   0   2 11  1 29  0 29  0
## 5   5  0  0  1 70  1  1  1  1  0   0   2 28  1 31  0  4  1
## 6   6  0  0  1 20  1  0  1  0  0   0   4 11  0 11  0  8  1

Using the log-rank test, we want to test the hypothesis of difference in the time to staphylococcus infection (T3 variable) between patients whose burns were cared for with a routine bathing care method (Z1 = 0) versus those whose body cleansing was initially performed using 4% chlorhexidine gluconate (Z1 = 1). The event indicator is in variable D3.

For this test, we use a two-sided alternative and a 5% significance level.

# fit
fit <- survfit(Surv(T3, D3) ~ Z1, data = burn)

# plot with log-rank test
ggsurvplot(fit,
  pval = TRUE,
  pval.method = TRUE
)

In the sample, it seems that the time to infection for patients with routine bathing (Z1 = 0) is smaller than for patients with body cleansing (Z1 = 1). This is the case because the percentage of patients who have not experienced the infection decreases more quickly, so the hazard rate is greater.

However, this conclusion cannot be generalized to the population without performing a sound statistical test. And based on the result of the log-rank test, we do not reject the hypothesis that time to infection is the same between the two groups of patients ($p$-value = 0.051).

To go further

In this article, we have presented what is survival analysis, when, why and how to use it. We discussed about censoring and survival curves. We showed how to estimate the survival function via the Kaplan-Meier estimator and how to test survival between two groups via the log-rank test. We illustrated these approaches both by hand and in R.

As you noticed, we did not show how to model survival data. There are several regression models that can be applied to survival data, the most common one being the semiparametric Cox Proportional Hazards model (1972). It originated from the medical area to investigate and assess the relationship between the survival times of patients and their corresponding predictor variables.

We have seen that the Kaplan-Meier estimator is useful to visualize survival between groups and the log-rank test to test whether survival significantly differs between groups (so both approaches use a categorical variable as predictor). However, it does not work well for assessing the effect of quantitative predictor. The Cox model has the advantage that it works for both quantitative as well as for categorical predictors, and for several risk factors at the same time (so it can model the effect of multiple variables at once).

With the Cox model, we model the impact of different factors $X_1, X_2, \ldots, X_q$ on survival via their impact on the hazard function:

\[h(t|\textbf{X}) = h_0 (t) exp(\beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_q X_q),\]

where:

$h(t|\textbf{X})$ is the instantaneous death rate conditional on having survived up to time $t$.
$h_0 (t)$ is the population-level baseline hazard – the underlying hazard function. It describes how the average person’s risk evolves over time.
$exp(\beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_q X_q)$ describes how covariates affect the hazard. In particular, a unit increase in $x_i$ leads to an increase of the hazard by a factor of $\exp(\beta_i)$.

This post aimed at presenting the introductory concepts in survival analysis, so this model will be developed in another post. In the meantime, if you would like to learn more about modeling survival data (thanks to the Cox model and other models), see this post from Joseph Rickert.

Thanks for reading.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Cox, David R. 1972. “Regression Models and Life-Tables.” Journal of the Royal Statistical Society: Series B (Methodological) 34 (2): 187–202.

Kaplan, Edward L, and Paul Meier. 1958. “Nonparametric Estimation from Incomplete Observations.” Journal of the American Statistical Association 53 (282): 457–81.

Note that in survival analysis, the precision of the estimators (and the power of the tests) does not depend on the number of patients but on the number of events. So it is best to have many observations where the event does occur for the analyses to be effective. Here we work on a small sample for the sake of illustration.↩︎
Note that the time variable can be expressed in other units, such as seconds, days, weeks, months, etc.↩︎
Median is preferred over mean in survival analysis because survival functions are often skewed to the right. The mean is often influenced by outliers, whereas the median is not. See a discussion comparing the two in this section.↩︎
Note that if the survival curve does not cross 50% (because survival is greater than 50% at the last time point), then the median survival cannot be computed and is simply undefined.↩︎
Note that the median survival is expressed in the same unit than the unit of the time variable in the initial dataset. So if the time unit was months, the median survival time would be 9 months.↩︎
More information about the dataset can be found on CRAN or with ?tongue.↩︎
See the reason we use log-log for the confidence interval in this thread.↩︎
Note that if the groups to compare have a natural ordering (such as the educational level; none, low, medium, high), tests that take it into consideration have more power to detect significant effects. These tests are referred as tests for trend.↩︎
Remember that the time unit can be different than years.↩︎
This is the case for large samples. The example described here does not meet this condition, but we still show it as an illustration.↩︎
Note that other weights can be considered, but this is beyond the scope of this article.↩︎
More information about the dataset can be found on CRAN or with ?burn.↩︎

Google Analytics in R: Review of 2022

Fri, 16 Dec 2022 00:00:00 +0000

Introduction

It is almost the end of the year, which means it is time to do a review of Stats and R and look back on the past year.

This year’s review will be much shorter compared to previous years because there are already a lot of examples in those previous reviews.

Note that beginning of September 2022, I moved from Universal Analytics (known as UA) to Google Analytics 4 (known as GA4). As you may know if you often use Google Analytics; on July 1, 2023, standard Universal Analytics properties will stop processing new hits. This means that you will need to move to GA4 if you want to continue using it after this date.

This year, I will be brief about the different metrics and how to analyze your Google Analytics (GA) data. However, this post will perhaps be helpful for people who also moved (or will move) to GA4 and still want to analyze their GA data from both UA and GA4.

I have to admit that it is quite time consuming to combine data from both, in particular since metrics and dimensions are not the same between UA and GA4. So if you have not moved yet, I recommend doing it at the end or beginning of a year. This will prevent you from having to fetch data from two GA IDs, and then having to combine both data in R.

As usual, I analyze my GA data in R thanks to the amazing {googleAnalyticsR} package from Mark Edmondson.

Note that the actual number presented below will not be very useful for you. Every blog and website is different, so every audience is different. This post is more about illustrating the process of analyzing GA data in R (and this year, also to illustrate the process of combining GA data from two GA IDs), rather than about showing my numbers.

Prerequisites

Before anything else, you have to specify the GA ID for which you want to analyze data. Mine are as follows:

ga_id <- c("208126346", "250795226")

You will find your(s) in the admin section of your GA account.

Analytics

In this section, I present some visualizations that might be useful when analyzing your GA data. Feel free to comment at the end of the post if you use other interesting visualizations.

Note that I focus on the page views metrics in this post, but you can edit my code to show other metrics as well. See the metrics available for UA and the ones for GA4.

Page views over time

Let’s start with the evolution of the number of page views over the past year, so from December 16, 2021 to December 15, 2022.

library(googleAnalyticsR)

# set date range
start_date <- as.Date("2021-12-16")
end_date <- as.Date("2022-12-15")

# extract data from both IDs
dat1 <- google_analytics(ga_id[1],
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = "date",
  anti_sample = TRUE # avoid sampling
)

dat2 <- ga_data(ga_id[2],
  date_range = c(start_date, end_date),
  metrics = "screenPageViews",
  dimensions = "date",
  limit = -1 # return all data (no limit)
)

# combine data from both IDs
library(dplyr)
dat <- full_join(dat1, dat2, by = "date")
dat$page_views <- rowSums(select(dat, pageviews, screenPageViews),
  na.rm = TRUE
)


# scatter plot with a trend line
library(ggplot2)
dat %>%
  ggplot(aes(x = date, y = page_views)) +
  geom_point(size = 1L, color = "gray") + # change size and color of points
  geom_smooth(color = "steelblue", alpha = 0.25) + # change color of smoothed line and transparency of confidence interval
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Evolution of daily page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  theme(plot.margin = unit(c(5.5, 17.5, 5.5, 5.5), "pt")) + # to avoid the plot being cut on the right edge
  scale_y_continuous(labels = scales::comma) # better y labels

As you can see in the code above, we need to:

extract data from two different IDs: this is done with google_analytics() for UA and ga_data() for GA4 (with each function having its own arguments)
join data from the two IDs (with a full_join() because even after moving to GA4, there were still some hits on UA, so I need to keep data from the 2 IDs for the entire year)
sum up the metric for both IDs (with rowSums())

This process will be done repeatedly for each visualization.

From the plot above, we see that the number of page views follows a cyclical evolution (with a decrease during summer). This trend follows the same pattern than last year, but the increase in the last quarter of the year is larger in 2022.

Note that there seems to be consistently some days with a lower number of page views than the rest. This is actually the weekends. See more on that in this section. Note also the presence of an outlier with a number of page views above 5,000. No post has been published on that day, so it probably comes from an old post that was shared to a large audience.

Finally, summing all days give a total page views of 938,630. Last year, it was 876,280, so 2022 saw an increase of 7.12%.

Page views per month and year

Comparison of the number of page views per month for all previous years can also be useful. Below two different types of visualizations.

# set new date range to include previous years
start_date_launch <- as.Date("2019-12-16")

# extract data from both IDs
dat1 <- google_analytics(ga_id[1],
  date_range = c(start_date_launch, end_date),
  metrics = "pageviews",
  dimensions = "date",
  anti_sample = TRUE # avoid sampling
)

dat2 <- ga_data(ga_id[2],
  date_range = c(start_date_launch, end_date),
  metrics = "screenPageViews",
  dimensions = "date",
  limit = -1 # return all data (no limit)
)

# combine data from both IDs
dat <- full_join(dat1, dat2, by = "date")
dat$page_views <- rowSums(select(dat, pageviews, screenPageViews),
  na.rm = TRUE
)


# add year and month columns to dataframe
dat$month <- format(dat$date, "%m")
dat$year <- format(dat$date, "%Y")

# page views by month by year using dplyr then graph using ggplot2 barplot
dat %>%
  filter(year != 2019) %>% # remove 2019 because there are data for December only
  group_by(year, month) %>%
  summarize(page_views = sum(page_views)) %>%
  ggplot(aes(x = month, y = page_views, fill = year)) +
  geom_bar(position = "dodge", stat = "identity") +
  theme_minimal() +
  labs(
    y = "Page views",
    x = "Month",
    title = "Page views per month and year",
    subtitle = paste0(format(start_date_launch, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com",
    fill = "" # remove legend title
  ) +
  theme(legend.position = "top") + # change legend position
  scale_y_continuous(labels = scales::comma) # better y labels

Another possibility is as follows:

dat2 <- dat

library(lubridate)
dat2$day <- day(dat2$date)
dat2$day_month <- as.Date(paste0(dat2$month, "-", dat2$day), format = "%m-%d")

dat2 %>%
  filter(year != 2019 & page_views < 7500) %>% # remove 2019 and outliers
  ggplot(aes(x = day_month, y = page_views, color = year)) +
  geom_point(size = 1L, alpha = 0.25) + # change size and alpha of points
  geom_smooth(se = FALSE) + # remove confidence interval
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Evolution of daily page views",
    subtitle = paste0(format(start_date_launch, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com",
    color = "" # remove legend title
  ) +
  theme(legend.position = "top") + # change legend position
  scale_y_continuous(labels = scales::comma) + # better y labels
  scale_x_date(date_labels = "%b") # show only months

From the 2 plots above, we see that the blog globally performed better in terms of page views compared to 2020-2021 (with an exception in the first quarter for which it performed better in 2021).

Note that a comparison between 3 years is fine, but with more years to compare the plots would quickly become unreadable!

Top performing pages

In case you are interested to know the top performing pages (still in terms of page views):

# extract data from both IDs
dat1 <- google_analytics(ga_id[1],
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = "pageTitle",
  anti_sample = TRUE # avoid sampling
)

dat2 <- ga_data(ga_id[2],
  date_range = c(start_date, end_date),
  metrics = "screenPageViews",
  dimensions = "pageTitle",
  limit = -1 # return all data (no limit)
)

# combine data from both IDs
dat <- full_join(dat1, dat2, by = "pageTitle")
dat$page_views <- rowSums(select(dat, pageviews, screenPageViews),
  na.rm = TRUE
)

## Create a table of the most viewed posts
library(lubridate)
library(reactable)
library(stringr)


most_viewed_posts <- dat %>%
  mutate(Title = str_sub(pageTitle, start = 1, end = -nchar(" - Stats and R"))) %>% # remove blog site in pageTitle
  count(Title, wt = page_views, sort = TRUE) %>%
  mutate(Title = str_trunc(Title, width = 30)) # keep maximum 30 characters

# plot
top_n(most_viewed_posts, n = 7, n) %>% # edit n for more or less pages to display
  ggplot(., aes(x = reorder(Title, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Page title",
    title = "Top performing pages",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

Top performing posts are quite similar than last year, that is:

Since they are the most read posts for several years, I have made them available to download via Gumroad.

Page views by country

Knowing from which country your readers come from can also be useful, in particular for e-commerce or blogs that sell physical products in addition to writing posts.

# extract data from both IDs
dat1 <- google_analytics(ga_id[1],
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = "country",
  anti_sample = TRUE # avoid sampling
)

dat2 <- ga_data(ga_id[2],
  date_range = c(start_date, end_date),
  metrics = "screenPageViews",
  dimensions = "country",
  limit = -1 # return all data (no limit)
)

# combine data from both IDs
dat <- full_join(dat1, dat2, by = "country")
dat$page_views <- rowSums(select(dat, pageviews, screenPageViews),
  na.rm = TRUE
)

# table
countries <- dat %>%
  mutate(Country = str_trunc(country, width = 30)) %>% # keep maximum 30 characters
  count(Country, wt = pageviews, sort = TRUE)

# plot
top_n(countries, n = 10, n) %>% # edit n for more or less countries to display
  ggplot(., aes(x = reorder(Country, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Country",
    title = "Top performing countries",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  theme(plot.margin = unit(c(5.5, 7.5, 5.5, 5.5), "pt")) # to avoid the plot being cut on the right edge

For Stats and R, most readers are from the US, and to a large extent since the second top country (UK) is quite far in terms of page views.

This trend is the same than last years, except that:

UK has surpassed India, and
Belgium (which used to rank 3rd and 6th in 2020 and 2021, respectively) is now only 8th. Since I am from Belgium, this indicates that the blog is attracting more and more international readers through the years (and proportionally less and less from my country).

Page views per day of week

As discussed in the section about the evolution of daily page views (view this section), there seem to be some days which consistently perform worse than other days.

The following plot will show what are these days:

# extract data from both IDs
dat1 <- google_analytics(ga_id[1],
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = c("date"),
  anti_sample = TRUE # avoid sampling
)

dat2 <- ga_data(ga_id[2],
  date_range = c(start_date, end_date),
  metrics = "screenPageViews",
  dimensions = c("date"),
  limit = -1 # return all data (no limit)
)

# combine data from both IDs
dat <- full_join(dat1, dat2, by = "date")
dat$page_views <- rowSums(select(dat, pageviews, screenPageViews),
  na.rm = TRUE
)
# find day of week
dat$weekday <- wday(dat$date, label = TRUE, abbr = TRUE)

## Reordering dat$weekday so Monday is first
dat$weekday <- factor(dat$weekday,
  levels = c("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun")
)

# boxplot
library(scales)
dat %>%
  ggplot(aes(x = weekday, y = page_views)) +
  geom_boxplot(fill = "steelblue", outlier.colour = alpha(0.25)) +
  geom_jitter(alpha = 0.25) + # adds transparency
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Page views per day of week",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

As expected for a technical blog, there are much more readers during the week than during the weekend.

Thank you note

Thank you to all readers who came to Stats and R this year. You made this journey incredibly more enriching. For next year and the many more to come, I will keep writing about topics for which I am familiar and interested in. So stay tuned!

Thanks for reading. I hope this article helped you to analyze your Google Analytics data in R, or helped you to combine your Universal Analytics and Google Analytics 4 data. For more examples of visualizations or summaries of your GA data in R, see also previous years’ reviews.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Paper: 'EpiLPS: A fast and flexible Bayesian tool for estimation of the time-varying reproduction number'

Wed, 19 Oct 2022 00:00:00 +0000

Introduction

A colleague (and friend) of mine recently published a research paper entitled “EpiLPS: A fast and flexible Bayesian tool for estimation of the time-varying reproduction number” in PLoS Computational Biology.

I am not in the habit of sharing research paper to which I did not contribute. Nevertheless, I would like to make an exception with this one because I strongly believe that the method developed in the paper deserves to be known, especially for anyone working in epidemiology.

Below is the motivation behind the article, as well as an illustration on simulated and real data (US hospitalization data). More information can be found in the paper and on the accompanying website.

Motivation

EpiLPS (Gressani et al. 2022) is a methodology for flexible Bayesian inference of the time-varying reproduction number $\mathcal{R}_t$; the average number of secondary cases generated by an infected agent at time $t$. This is a key epidemiological parameter that informs about the transmission potential of an infectious disease and can be used by public health authorities to gauge the effectiveness of interventions and propose an orientation for future control strategies.

This metric has gained in popularity during the SARS-CoV-2 pandemic with wide media coverage as its meaning is easily and intuitively grasped. Put simply, when $\mathcal{R} < 1$, the signal is encouraging as the epidemic is under control and will eventually vanish. On the contrary, a value of $\mathcal{R} > 1$ means that the disease keeps spreading and infections are witnessing an expansionary impact. Having a robust and reliable tool to compute the reproduction number from infectious disease data is therefore crucial.

A group of researchers in the EpiPose team from Hasselt University (Belgium), Leiden University (The Netherlands), and the University of Bern (Switzerland) have recently developed a new methodology for estimating the instantaneous reproduction number from incidence time series data for a given serial interval distribution (the time elapsed between the onset of symptoms in an infector and the onset of symptoms of secondary cases).

They termed their approach EpiLPS for “Epidemiological modeling with Laplacian-P-Splines” as Laplace approximations and P-splines smoothers are key ingredients that form the backbone of the proposed methodology.

The EpiLPS model assumes that the observed reported cases (by reporting date or date of symptom onset) are governed by a negative binomial distribution. As such, it allows to take the feature of overdispersion into account, contrary to a Poisson model. The epidemic curve is smoothed with P-splines (where posterior estimates of latent variables are computed via Laplace approximations) in a first step and a renewal equation model is used in a second step as a bridge between the reproduction number and the estimated spline coefficients through a “plug-in” method.

The authors also explain the main difference between EpiLPS and EpiEstim, a well established approached for estimating $\mathcal{R}_t$ in real-time developed by Cori et al. (2013) and make extensive comparisons between the two approaches under different epidemic scenarios.

An interesting feature of EpiLPS is that the user can choose between a fully “sampling-free” path, where model hyperparameters are fixed at their maximum a posteriori (LPSMAP) or a fully stochastic path (LPSMALA) based on a Metropolis-adjusted Langevin algorithm (LPSMALA). Talking about efficiency, routines for Laplace approximations and B-splines evaluations have been coded in C++ and integrated in R via the Rcpp package, so that the underlying algorithm can be executed in negligible time.

Below, we provide a short example of how to use the EpiLPS routines to estimate $\mathcal{R}_t$.

Getting started

The EpiLPS package is available from CRAN (see https://cran.r-project.org/web/packages/EpiLPS/index.html) and can be installed from the R console by typing:

install.packages("EpiLPS")

The package can then be loaded as follows:

library("EpiLPS")

The EpiLPS package structure is fairly simple as it consists in a few routines:

The function epilps() is the core routine for fitting the reproduction number.
With plot.epilps(), the user can plot the estimated epidemic curve and $\mathcal{R}_t$.
Finally, two ancillary routines, episim() and perfcheck() have been developed to essentially reproduce the simulation results of the associated paper.

A simulated example

A set of epidemic data can be simulated with the episim() routine by specifying a serial interval distribution and by choosing among a set of available patterns for the true reproduction number curve (here we choose pattern number 5 corresponding to a rather wiggly curve).

The simulated outbreak is for a duration of 40 days as specified in the endepi option. By setting the option plotsim = TRUE, the routine returns a figure summarizing the incidence time series, a bar plot for the specified serial interval distribution and the true underlying reproduction number curve.

set.seed(1234)

SI <- c(0.344, 0.316, 0.168, 0.104, 0.068)
simepidemic <- episim(
  serial_interval = SI,
  Rpattern = 5,
  plotsim = TRUE,
  verbose = TRUE,
  endepi = 40
)

## Chosen scenario: 5 'Wiggly then stable Rt'.
## Incidence of cases generated from a Poisson distribution. 
## Total number of days of epidemic: 40.

If you want to have an overview of the generated incidence time series, just type:

simepidemic$y

##  [1]  10   6  15  24  37  43  54  46  47  28  20   8  10  10   3   5   3   4   6
## [20]   6  15  21  44  75 135 217 329 409 453 487 457 443 297 290 255 246 246 339
## [39] 395 573

Smoothing the epidemic curve and estimating $\mathcal{R}_t$

Let us now use the epilps() routine to smooth the epidemic curve and estimate the reproduction number.

We will do this through LPSMAP (a fully sampling-free approach) and via LPSMALA (a fully stochastic approach relying on a MCMC algorithm with Langevin dynamics), where we specify a chain of length 10000 and a burn-in of size 4000.

LPSMAP_fit <- epilps(
  incidence = simepidemic$y,
  serial_interval = SI,
  tictoc = TRUE
)

## Inference method chosen: LPSMAP. 
## CI for LPSMAP computed via lognormal posterior approx. of Rt.Total number of days: 40. 
## Mean Rt discarding first 7 days: 1.327.
## Mean 95% CI of Rt discarding first 7 days: (1.164,1.527) 
## Elapsed real time (wall clock time): 0.261 seconds.

LPSMALA_fit <- epilps(
  incidence = simepidemic$y, serial_interval = SI,
  method = "LPSMALA", chain_length = 10000, burn = 4000
)

## Inference method chosen: LPSMALA with chain length 10000 and warmup 4000.
## MCMC acceptance rate: 56.41%. 
## Geweke z-score < 2.33 for:  32 / 33  variables. 
## Total number of days: 40. 
## Mean Rt discarding first 7 days: 1.326.
## Mean 95% CI of Rt discarding first 7 days: (1.117,1.555). 
## Timing of routine not requested.

After execution, each routine prints in the console a brief summary of the method that has been requested by the user.

For LPSMALA, it summarizes the chain length, the acceptance rate (should be around 56%) and other basic information. As can be seen from the printed output, the mean reproduction number for the simulated epidemic is around 1.33.

We can now use, say, the LPSMALA_fit object together with the plot() routine to obtain the smoothed epidemic curve and the estimated reproduction number (by default the credible interval is at a 5% level of significance but this can be changed by the user).

days <- seq(8, 40)

#--- Smoothed epidemic curve
gridExtra::grid.arrange(
  plot(LPSMALA_fit,
    plotout = "epicurve", incibars = TRUE, themetype = "light",
    epicol = "darkgreen", cicol = rgb(0.3, 0.73, 0.3, 0.2),
    epititle = "Smoothed epidemic curve", titlesize = 13, barwidth = 0.25
  ),

  #--- Estimated reproduction number
  plot(LPSMALA_fit,
    plotout = "rt", theme = "light", rtcol = "black",
    titlesize = 13, Rtitle = "Estimated R (LPSMALA)"
  ),
  nrow = 1, ncol = 2
)

The figure can be customized in various ways:

Users can specify the theme under themetype. Available options are gray (the default), classic, light and dark.
Other choices, such as whether or not to show the incidence bars, the color of the credible interval envelope, the color of the smoothed epidemic curve and the estimated reproduction number are also available.

The figure above was generated within the ggplot2 package, but there is also another way of extracting information directly from the LPSMAP_fit and LPSMALA_fit objects. In fact, the estimated reproduction number values and their associated credible interval for each day can be extracted and plotted.

Below, we make the exercise and plot the estimated $\mathcal{R}_t$ obtained with LPSMAP and LPSMALA, respectively and compare it with the true underlying reproduction number curve. The fit is quite good.

par(mfrow = c(1, 2))

#--- LPSMAP vs target R
plot(days, sapply(days, simepidemic$Rtrue),
  type = "l", lwd = 2, ylim = c(0, 4),
  ylab = "Estimated R", xlab = "Time"
)
polygon(
  x = c(days, rev(days)), y = c(
    LPSMAP_fit$epifit$R95CI_low[8:40],
    rev(LPSMAP_fit$epifit$R95CI_up[8:40])
  ),
  col = rgb(0.23, 0.54, 1, 0.3), border = NA
)
lines(days, LPSMAP_fit$epifit$R_estim[8:40], type = "l", col = "cornflowerblue", lwd = 2)
lines(days, sapply(days, simepidemic$Rtrue), type = "l", lwd = 2)

grid(nx = 10, ny = 10)
legend("topright",
  lty = c(1, 1), lwd = c(2, 2),
  col = c("black", "blue", rgb(0.23, 0.54, 1, 0.3)),
  c("Target R", "LPSMAP", "LPSMAP 95% CI"), bty = "n", cex = 0.9
)

#--- LPSMALA vs target R
plot(days, sapply(days, simepidemic$Rtrue),
  type = "l", lwd = 2, ylim = c(0, 4),
  ylab = "Estimated R", xlab = "Time"
)
polygon(
  x = c(days, rev(days)), y = c(
    LPSMALA_fit$epifit$R95CI_low[8:40],
    rev(LPSMALA_fit$epifit$R95CI_up[8:40])
  ),
  col = rgb(1, 0.23, 0.31, 0.3), border = NA
)
lines(days, LPSMALA_fit$epifit$R_estim[8:40], type = "l", col = "red", lwd = 2)
lines(days, sapply(days, simepidemic$Rtrue), type = "l", lwd = 2)

grid(nx = 10, ny = 10)
legend("topright",
  lty = c(1, 1), lwd = c(2, 2),
  col = c("black", "red", rgb(1, 0.23, 0.31, 0.3)),
  c("Target R", "LPSMALA", "LPSMALA 95% CI"), bty = "n", cex = 0.9
)

You can access, say, the results of the last week of the epidemic by typing:

# Estimated R of the last week (with LPSMAP)
round(tail(LPSMAP_fit$epifit[, 1:4], 7), 3)

##    Date R_estim R95CI_low R95CI_up
## 34   34   0.708     0.663    0.756
## 35   35   0.724     0.676    0.775
## 36   36   0.809     0.755    0.868
## 37   37   0.971     0.908    1.039
## 38   38   1.205     1.135    1.279
## 39   39   1.461     1.384    1.541
## 40   40   1.671     1.557    1.794

# Estimated mean number of cases of the last week (with LPSMAP)
round(tail(LPSMAP_fit$epifit[, 5:7], 7))

##    mu_estim mu95CI_low mu95CI_up
## 34      284        225       357
## 35      254        202       319
## 36      248        196       312
## 37      267        211       338
## 38      319        253       404
## 39      411        325       520
## 40      552        394       774

USA hospitalization data

To illustrate EpiLPS on real data, we download hospitalization data from the COVID19 package for the USA in the period ranging from 2021-09-01 to 2022-09-01 and apply the epilps() routine to estimate the reproduction number.

install.packages("COVID19")
library("COVID19")

# Get data and specify serial interval distribution
USADat <- COVID19::covid19(
  country = "US", level = 1, start = "2021-09-01",
  end = "2022-09-01", verbose = FALSE
)

si <- c(0.344, 0.316, 0.168, 0.104, 0.068)

inciUSA <- USADat$hosp
dateUSA <- USADat$date

We use the epilps() routine with method LPSMAP (default) and plot the smoothed epidemic curve and the estimated reproduction number with a 95% credible interval.

epifit <- epilps(incidence = inciUSA, serial_interval = si, K = 20)

## Inference method chosen: LPSMAP. 
## CI for LPSMAP computed via lognormal posterior approx. of Rt.Total number of days: 366. 
## Mean Rt discarding first 7 days: 0.994.
## Mean 95% CI of Rt discarding first 7 days: (0.983,1.005) 
## Timing of routine not requested.

gridExtra::grid.arrange(
  plot(epifit,
    dates = dateUSA, datelab = "3m",
    plotout = "epicurve", incibars = FALSE, themetype = "light",
    epicol = "darkgreen", cicol = rgb(0.3, 0.73, 0.3, 0.2),
    epititle = "USA smoothed epidemic curve", titlesize = 13
  ),
  plot(epifit,
    dates = dateUSA, datelab = "3m",
    plotout = "rt", theme = "light", rtcol = "black",
    titlesize = 13, Rtitle = "USA Estimated R"
  ),
  nrow = 1, ncol = 2
)

Conclusion

Thanks for reading.

I hope you will find the method developed in the paper as useful as I do. Feel free to reach out to me and to the authors if you happen to use it for your own research.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Cori, Anne, Neil M Ferguson, Christophe Fraser, and Simon Cauchemez. 2013. “A New Framework and Software to Estimate Time-Varying Reproduction Numbers During Epidemics.” American Journal of Epidemiology 178 (9): 1505–12.

Gressani, Oswaldo, Jacco Wallinga, Christian L Althaus, Niel Hens, and Christel Faes. 2022. “EpiLPS: A Fast and Flexible Bayesian Tool for Estimation of the Time-Varying Reproduction Number.” PLoS Computational Biology 18 (10): e1010618. https://doi.org/10.1371/journal.pcbi.1010618.

How to keep yourself updated with the latest R news?

Thu, 13 Oct 2022 00:00:00 +0000

Introduction

At the end of one of the training sessions I gave on R, a student asked me the following question:

How do you keep yourself updated with the latest R news?

It is true that R, being open source (meaning that everyone can contribute), is evolving rapidly. This means that even if I am using R for several years and on a daily basis, I like to stay informed in order to stay up to date with the program and the latest coding practices.

In fact, I learn about new packages, new functions and new features almost everyday. Most of them are not particularly useful for my research or my teaching tasks, but sometimes I discover such a nice package or function that I replace my code with new one.¹

The training was an advanced one, so the student had a good knowledge of R and was not looking for more tutorials or courses. She was interested in knowing where to look for updates about current and new R packages and functions.

After sharing my sources with all students following the training, I thought it would be useful to others. In this article, I share my sources—from where I get the latest R updates and news.

The sources are divided into two main categories: Twitter and newsletters.

How do I keep track?

Twitter

To be honest, I mostly use Twitter to keep up to date with R news.

Twitter allows me to follow discussions about statistical methods or approaches, and to keep me informed about publications of new blog posts.

What I particularly like with Twitter is that there is a mix between:

short messages about new functions or packages (most of the time with an illustration or an example), and
announcements of new blog posts that cover specific subjects in details.

For this, I follow people (researchers, professors, bloggers, statisticians, data scientists, etc.) that are working in my domains of interest. For instance, I am mostly interested in the application of statistics in R, data science, biostatistics and data visualization. I thus follow accounts which regularly post about these topics. I also avoid following accounts that cover topics I am not interested in, so that my Twitter feed really shows information I am most likely to be interested in.

Below, you will see a list of some of the accounts I follow, classified by themes. Of course, this is a non-exhaustive list! There are plenty of very inspiring and intelligent people that are not in the list, simply because I cannot afford to put them all.

If you follow people that post regularly about the themes covered below, feel free to add them in the comments. I am always looking for new inspiring accounts to follow.

Note that the accounts are displayed in alphabetical order and the table is searchable.

You will also find many news when exploring #rstats on Twitter.

Newsletters

Besides Twitter, I also read new blog posts and stay informed through these newsletters:

The 2 first newsletters are actually blog aggregators, so collections of many many blogs. If you have your own blog, don’t hesitate to submit it to make it accessible to more people.

You can always subscribe to a blog you like, but if it is related to R, it will most likely be shared via R-Bloggers or R Weekly.

Conclusion

Thanks for reading.

I hope this article will help you to keep track of great blogs, tutorials and other resources about R. Feel free to follow me on Twitter (@statsandr), where I tweet my new articles and retweet everything I find interesting or worth mentioning.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

See for example this article. The package I discovered was so useful and interesting (to me), that I added a new section to the post and now use it instead of the older package.↩︎

One-sample Wilcoxon test in R

Thu, 07 Jul 2022 00:00:00 +0000

Introduction

In a previous article, we showed how to do a two-sample Wilcoxon test in R. Remember that there are actually two versions of this test:

The Mann-Whitney-Wilcoxon test (also referred as Wilcoxon rank sum test or Mann-Whitney U test), used to compare two independent samples. This test is the non-parametric version of the Student’s t-test for independent samples.
The Wilcoxon signed-rank test (also referred as Wilcoxon test for paired samples), used to compare two paired samples. This test is the non-parametric version of the Student’s t-test for paired samples.

In another article, we also showed how to do a one-sample t-test by hand and in R. This test is used to determine whether the mean of a measurement variable is different from a specified value (a value that you specify based on your beliefs or a theoretical expectation for example). Since it is a parametric test, the data should follow a normal distribution (or sample size should be large enough (i.e., above 30), thanks to the central limit theorem) for the results to be valid.

Unlike the one-sample t-test, the one-sample Wilcoxon test (also referred as the one-sample Wilcoxon signed-rank test) is a non-parametric test, meaning that it does not rely on data belonging to any particular parametric family of probability distributions. Non-parametric tests usually have the same goal as their parametric counterparts (in this case, compare data to a given value). Nonetheless, they do not require the assumption of normality and they can deal with outliers and Likert scales.

In this article, we show when to perform the one-sample Wilcoxon test, how to do it in R and how to interpret its results. We will also briefly show some appropriate visualizations.

When?

The one-sample Wilcoxon test is used to compare our observations to a given default value—a value that you specify based on your beliefs or a theoretical expectation for example. In other words, it is used to determine if a group is significantly different from a known or hypothesized population value on the variable of interest.

Since the test statistic is computed based on the ranks of the difference between the observed values and the default value (making it a non-parametric test), the one-sample Wilcoxon test is more appropriate than a one-sample t-test when the observations do not follow a normal distribution.

The goal of this test is to verify whether the observations are significantly different from our default value. In terms of null and alternative hypotheses, we have (for a two-tailed test):

$H_0:$ location of the data is equal to the chosen value
$H_1:$ location of the data is different from the chosen value

In other words, a significant result (i.e., a rejection of the null hypothesis) suggests that the location of the data is different from the chosen value.

Note that some authors suggest that this test is a test of the median, that is (for a two-tailed test):

$H_0:$ the median is equal to the chosen value
$H_1:$ the median is different from the chosen value

However, this is the case only if the data are symmetric. Without further assumptions about the distribution of the data, the one-sample Wilcoxon test is not a test of the median but a test about the location of the data.¹

Note that although the normality assumption is not required, the independence assumption must still be verified. This means that observations must be independent of one another (usually, random sampling is sufficient to have independence).

Data

For our illustration, suppose we want to test whether the scores at an exam differ from 10, that is:

$H_0:$ scores at the exam $= 10$
$H_1:$ scores at the exam $\ne 10$

To verify this, we have a sample of 15 students and their score at the exam:

dat

##    Student_ID Score
## 1           1    17
## 2           2     5
## 3           3     1
## 4           4    10
## 5           5     4
## 6           6    18
## 7           7    17
## 8           8    15
## 9           9     7
## 10         10     4
## 11         11     5
## 12         12    14
## 13         13    20
## 14         14    18
## 15         15    15

Scores between students are assumed to be independent (a student’s score is not impacted or influenced by the score of another student). Therefore, the independence assumption is met.

Moreover, sample size is small (n < 30) and based on the histogram the data do not follow a normal distribution:

# histogram
hist(dat$Score)

Note that we refrain from verifying the normality via a normality test (such as the Shapiro-Wilk test for instance) because for small sample sizes, normality tests have little power to reject the null hypothesis and therefore small samples most often pass normality tests (Öztuna et al. 2006; Ghasemi and Zahediasl 2012).

Note also that although we use a quantitative variable for the illustration, the one-sample Wilcoxon test is also appropriate for interval data and Likert scales.

How?

The one-sample Wilcoxon test can be done in R with the wilcox.test() function.

But first, it is a good practice to visualize our data in a boxplot and compute some descriptive statistics to compare our observations with our default value:

# boxplot
boxplot(dat$Score,
  ylab = "Score"
)

If like me you prefer to use the {ggplot2} package for your plots:

# boxplot
library(ggplot2)

ggplot(dat, aes(y = Score)) +
  geom_boxplot() +
  labs(y = "Score") +
  theme( # remove axis text and ticks
    axis.text.x = element_blank(),
    axis.ticks = element_blank()
  )

Some basic descriptive statistics (rounded to two decimals):

round(summary(dat$Score),
  digits = 2
)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    1.00    5.00   14.00   11.33   17.00   20.00

From the boxplot and the descriptive statistics above, we see that the mean and median of the scores in our sample are respectively 11.33 and 14.

The one-sample Wilcoxon test will tell us whether the scores are significantly different from 10 or not (and thus whether they are different from 10 in the population or not):

wilcox.test(dat$Score,
  mu = 10 # default value
)

## 
## 	Wilcoxon signed rank test with continuity correction
## 
## data:  dat$Score
## V = 67, p-value = 0.3779
## alternative hypothesis: true location is not equal to 10

The output presents several information such as the:

title of the test
data
test statistic
$p$-value
alternative hypothesis

We focus on the $p$-value to interpret and conclude the test.

Interpretation:

Based on the results of the test, (at the significance level of 0.05) we do not reject the null hypothesis, so we do not reject the hypothesis that the scores at this exam are equal to 10, and we cannot conclude that the scores are significantly different from 10 ($p$-value = 0.378).

By default, it is a two-tailed test that is done. As for the t.test() function, we can specify that a one-sided test is required by using either the alternative = "greater" or alternative = "less argument in the wilcox.test() function.

For example, if we want to test that the scores are higher than 10:

wilcox.test(dat$Score,
  mu = 10, # default value
  alternative = "greater" # H1: scores > 10
)

## 
## 	Wilcoxon signed rank test with continuity correction
## 
## data:  dat$Score
## V = 67, p-value = 0.189
## alternative hypothesis: true location is greater than 10

Interpretation:

In this case, we still do not reject the hypothesis that scores are equal to 10 and we cannot conclude that scores are significantly higher than 10 ($p$-value = 0.189).

For more information about the arguments available in the function, see ?wilcox.test.

Note that you may encounter the following warnings when using wilcox.test():

Warning messages:
1: In wilcox.test.default(dat$Score, mu = 10) :
  cannot compute exact p-value with ties
2: In wilcox.test.default(dat$Score, mu = 10) :
  cannot compute exact p-value with zeroes

It is a warning rather than an indication that your results are incorrect. R is informing you that it is reporting a $p$-value based on a normal approximation rather than an exact $p$-value based on the data because there are ties (some values are the same). Use the exact = FALSE option if you want to remove the warning.

Combine statistical test and plot

If you are a frequent user of the blog, you know that I like to present results of a test directly on a plot. This allows me to visualize the data and conclude the test in a concise manner.

This is possible thanks to the gghistostats() function within the {ggstatsplot} package:

# load package
library(ggstatsplot)

# combine plot and test
gghistostats(
  data = dat, # dataframe
  x = Score, # variable
  type = "nonparametric", # nonparametric = Wilcoxon, parametric = t-test
  test.value = 10 # default value
)

The histogram² shows the distribution of the scores and results of the test is shown in the title of the plot.

As you can see, results of the test are the same, that is, there is not enough evidence in the data to conclude that scores are significantly different from 10 ($p$-value = 0.378).

Conclusion

Thanks for reading.

I hope this article helped you to understand the one-sample Wilcoxon test and how to do it in R.

As always, if you have any question related to the topic covered in this paper, please add it as a comment so other readers can benefit from the discussion.

References

Ghasemi, Asghar, and Saleh Zahediasl. 2012. “Normality Tests for Statistical Analysis: A Guide for Non-Statisticians.” International Journal of Endocrinology and Metabolism 10 (2): 486.

Öztuna, Derya, Atilla Halil Elhan, and Ersöz Tüccar. 2006. “Investigation of Four Different Normality Tests in Terms of Type 1 Error Rate and Power Under Different Distributions.” Turkish Journal of Medical Sciences 36 (3): 171–76.

See more information in this article.↩︎
This histogram looks different than the previous one because the number of bins is different (4 versus 5 bins).↩︎

Koh-Lanta 2022: the ambassadors probability problem

Mon, 16 May 2022 00:00:00 +0000

Introduction

There is a popular TV show broadcasted in France and the french-speaking part of Belgium called “Koh-Lanta”.

In this show, several adventurers are dropped off on a desert island with almost no food nor equipment (just a personal backpack with their clothes and a small portion of rice). They must learn to survive on the hostile island by building their hut, finding water and food, etc.

Each season, adventurers are divided into teams (called tribes), and the teams compete against each other in games involving ability, strength, thinking and endurance. Every other game, the winning tribe receives some food or survival equipment (something to fish or something to make a fire, for instance). The losing tribe receives nothing. For the other half of the games, each member of the losing tribe has to elect an adventurer. The adventurer with the most votes leaves the show definitely. The winning tribe goes back to its island with all of its members.

At some point during the show, the two competing tribes are grouped together into one single tribe, and it continues this time with each adventurer competing against each other (so they play individually). The winner of the show is the last one to “survive”.

The 2022 season started with 24 adventurers. Just before being grouped together, each tribe has to select an adventurer in the opposing tribe. The two selected adventurers become ambassadors of their tribe. The two ambassadors must then go to another island in order to choose the adventurer who is going to leaves the show definitely. The adventurer selected by the two ambassadors will not be part of the reunification of the two tribes and her adventure stops there.

Of course, both ambassadors want to eliminate a member of the opposing tribe (to arrive at the reunification with the most allies). If the two ambassadors cannot agree, they have to play a game that will determine which of the 2 ambassadors must leave the show. This game is entirely based on luck. For the rest of the article, we call this game the ambassadors’ game.

Before 2022

Here were the rules of the ambassadors’ game before the 2022 season:

There are two identical urns, one in front of each ambassador. Each urn contains exactly one black ball and one white ball. Each ambassador has to draw a ball among the 2 from his urn. Both urns are of course closed, so no one sees which ball is picked (nor which one is not picked). The winner of the game (remember that the winner stays in the show, the loser has to leave definitely) is the one who picks a white ball while the other ambassador draws a black ball. If both ambassadors draw the same ball (both black or both white), the balls are put back in the urns and the game start over (with the exact same conditions) until the two ambassadors draw a ball of different color.

For each draw, there are thus four possible results:

Ambassador from tribe A draws a white ball and ambassador from tribe B draws a black ball: ambassador from tribe A wins.
Ambassador from tribe A draws a black ball and ambassador from tribe B draws a white ball: ambassador from tribe B wins.
Both ambassadors draw a black ball: the game start over.
Both ambassadors draw a white ball: the game start over.

Let’s compute the probability for each result to occur. Since the two events are independent (the fact that ambassador A draws a white ball does not change the probability for ambassador B to draw a white ball makes the two events independent), we can multiply the probabilities to compute the joint probability of the two events.

With $P_A$ ($P_B$) denoting the probability that ambassador from tribe A (B) draws a white ball, we have:

$P_A \cdot (1 - P_B) = 0.5 \cdot 0.5 = 0.25$
$(1 - P_A) \cdot P_B = 0.5 \cdot 0.5 = 0.25$
$(1 - P_A) \cdot (1 - P_B) = 0.5 \cdot 0.5 = 0.25$
$P_A \cdot P_B = 0.5 \cdot 0.5 = 0.25$

The sum of the 4 probabilities gives 1 (i.e., 100%), which makes sense since it covers all possible outcomes.

This means that, on the first draw, each ambassador has a probability of 25% to win the game (outcome 1 for ambassador A, outcome 2 for ambassador B).

Of course, since the game is repeated until there is a winner, probabilities for outcomes 3 and 4 tend, in the long run, to decrease until it becomes null (0%). If this statement is not straightforward to you, think about it like this: if you play that game with your friend up to 100 times, what is the probability that there is still no winner, meaning that you and your friend drew the same ball (never a different color) 100 times in a row. You conceive that it is highly unlikely.

In this context, since both urns are identical and the game is played indefinitely until there is a winner, ambassadors have exactly the same probability of winning that game. It is indeed a 50-50 chance for each of them.

In 2022

Of course, I would not write an article about that ambassadors’ game if it is was this easy and straightforward.

The 2022 season differs from the previous ones in the sense that for each game, the losing tribe receives an additional punishment (called a curse in the show). As a consequence, this year, the two urns at the ambassadors’ game were not identical.

To give you some context, the red tribe won against the yellow tribe in the last game before the ambassadors’ negotiation. The punishment for the yellow tribe was the following:

The yellow tribe (with its ambassador Colin) had an urn with 2 black balls and 1 white ball.
The red tribe (with its ambassador Louana) had an urn with 1 black ball and 1 white ball.

This is indeed a punishment for the yellow tribe since the game is not fair anymore: the ambassador of the red tribe clearly has a higher chance of winning that game compared to the ambassador of the yellow tribe.

For the curious among you, here is how it happened:

The two ambassadors were informed by the presenter Denis Brogniart about the composition of the urns for each tribe.
Knowing that the odds were not in his favour, Colin (ambassador of the yellow tribe), chose not to play the game.¹
They finally agreed on the name of the adventurer who would leave the show (an adventurer of the yellow tribe).

You guessed it by now, the reason for writing this article is of course not to explain you what actually happened—the news sites do it better and much faster than me. The reason is that I wanted to compute the chance of winning the game for each ambassador if they had not agreed on an adventurer to eliminate.²

Moreover, I wanted to compute these probabilities in R through simulation. And why not, reuse the code in case organizers of Koh-Lanta decide to change the rules again in the future.

Even if you do not watch Koh-Lanta (because it is not broadcasted in your country, or you do not like the show), it could be of interest to those of you who want to see:

how a real life example can be transferred into R,
and how a for loop and a function can be used to answer the initial question.

Probabilities computation in R

For the remaining of this article, we denote:

$p_c$, the probability that Colin (ambassador of the yellow tribe) draws a white ball,
$p_l$, the probability that Louana (ambassador of the red tribe) draws a white ball,
$q_c$, the probability that Colin draws a black ball,
$q_l$, the probability that Louana draws a black ball.

Based on the composition of the urns given by Denis Brogniart, we have:

p_c <- 1 / 3
p_l <- 1 / 2
q_c <- (1 - p_c)
q_l <- (1 - p_l)

First draw

To start easy, let’s first compute the winning probabilities for each ambassador on the first draw only. Remember that to have a winner, balls must be of different colors.

Louana wins if and only if Colin draws a black ball and Louana draws a white ball:

# Louana winning on first draw
l_win <- q_c * p_l
l_win

## [1] 0.3333333

Louana has a 33.33% chance of winning on the first draw.

On the other hand, Colin wins if and only if Louana draws a black ball and Colin draws a white ball:

# Colin winning on first draw
c_win <- q_l * p_c
c_win

## [1] 0.1666667

Colin has a 16.67% chance of winning on the first draw.

You can already see that the game is in favour of Louana, as expected. Let’s see now how it evolves when playing several times.

Second draw

To win exactly on the second draw, it must be a tie on the first draw.

# Louana winning on second draw
tie <- (p_c * p_l) + (q_c * q_l)

l_win2 <- tie * l_win
l_win2

## [1] 0.1666667

# Colin winning on second draw
c_win2 <- tie * c_win
c_win2

## [1] 0.08333333

Louana has a 16.67% chance of winning on the second draw.
Colin has a 8.33% chance of winning on the second draw.

Third draw

To win exactly on the third draw, it must be a tie on the first two draws.

# Louana winning on third draw
l_win3 <- (tie^2) * l_win
l_win3

## [1] 0.08333333

# Colin winning on third draw
c_win3 <- (tie^2) * c_win
c_win3

## [1] 0.04166667

Louana has a 8.33% chance of winning on the third draw.
Colin has a 4.17% chance of winning on the third draw.

A pattern seems to emerge in the code.

Game limited to 3 draws

We can already compute the probabilities of winning for each ambassador as if the game was limited to three draws, by summing the probabilities for each of the first three draws:

# Louana winning on draw 1, 2 or 3
l_win_tot <- l_win + l_win2 + l_win3
l_win_tot

## [1] 0.5833333

# Colin winning on draw 1, 2 or 3
c_win_tot <- c_win + c_win2 + c_win3
c_win_tot

## [1] 0.2916667

Louana has a 58.33% chance of winning if the game is limited to three draws.
Colin has a 29.17% chance of winning if the game is limited to three draws.

Game limited to 5 draws

Now if we compute the probabilities as if the game was limited to 5 draws and generalize the computation to see the pattern even more clearly, we have:

# Louana winning
l_win_tot <- ((tie^0) * l_win) +
  ((tie^1) * l_win) +
  ((tie^2) * l_win) +
  ((tie^3) * l_win) +
  ((tie^4) * l_win)

l_win_tot

## [1] 0.6458333

# Colin winning
c_win_tot <- ((tie^0) * c_win) +
  ((tie^1) * c_win) +
  ((tie^2) * c_win) +
  ((tie^3) * c_win) +
  ((tie^4) * c_win)

c_win_tot

## [1] 0.3229167

Louana has a 64.58% chance of winning if the game is limited to 5 draws.
Colin has a 32.29% chance of winning if the game is limited to 5 draws.

Game limited to 100 draws

We could continue like this for a long time, but for now let’s compute it for up to 100 draws, using a for loop. Using a for loop is necessary here in order to avoid to copy-paste our computations a hundred times.

For the ease of illustration, we compute only the probability of winning for Louana. We will show later on how the probability for Colin can easily be computed.

n_draws <- 100 # number of draws
l_win_tot <- c() # set empty vector

for (i in 1:n_draws) {
  l_win_tot[i] <- ((tie^(i - 1)) * l_win) # prob of Louana winning up to n_draws
  print(paste0("Draw ", i, ": ", sum(l_win_tot))) # print sum of winning up to n_draws
}

## [1] "Draw 1: 0.333333333333333"
## [1] "Draw 2: 0.5"
## [1] "Draw 3: 0.583333333333333"
## [1] "Draw 4: 0.625"
## [1] "Draw 5: 0.645833333333333"
## [1] "Draw 6: 0.65625"
## [1] "Draw 7: 0.661458333333333"
## [1] "Draw 8: 0.6640625"
## [1] "Draw 9: 0.665364583333333"
## [1] "Draw 10: 0.666015625"
## [1] "Draw 11: 0.666341145833333"
## [1] "Draw 12: 0.66650390625"
## [1] "Draw 13: 0.666585286458333"
## [1] "Draw 14: 0.6666259765625"
## [1] "Draw 15: 0.666646321614583"
## [1] "Draw 16: 0.666656494140625"
## [1] "Draw 17: 0.666661580403646"
## [1] "Draw 18: 0.666664123535156"
## [1] "Draw 19: 0.666665395100911"
## [1] "Draw 20: 0.666666030883789"
## [1] "Draw 21: 0.666666348775228"
## [1] "Draw 22: 0.666666507720947"
## [1] "Draw 23: 0.666666587193807"
## [1] "Draw 24: 0.666666626930237"
## [1] "Draw 25: 0.666666646798452"
## [1] "Draw 26: 0.666666656732559"
## [1] "Draw 27: 0.666666661699613"
## [1] "Draw 28: 0.66666666418314"
## [1] "Draw 29: 0.666666665424903"
## [1] "Draw 30: 0.666666666045785"
## [1] "Draw 31: 0.666666666356226"
## [1] "Draw 32: 0.666666666511446"
## [1] "Draw 33: 0.666666666589056"
## [1] "Draw 34: 0.666666666627862"
## [1] "Draw 35: 0.666666666647264"
## [1] "Draw 36: 0.666666666656965"
## [1] "Draw 37: 0.666666666661816"
## [1] "Draw 38: 0.666666666664241"
## [1] "Draw 39: 0.666666666665454"
## [1] "Draw 40: 0.66666666666606"
## [1] "Draw 41: 0.666666666666364"
## [1] "Draw 42: 0.666666666666515"
## [1] "Draw 43: 0.666666666666591"
## [1] "Draw 44: 0.666666666666629"
## [1] "Draw 45: 0.666666666666648"
## [1] "Draw 46: 0.666666666666657"
## [1] "Draw 47: 0.666666666666662"
## [1] "Draw 48: 0.666666666666664"
## [1] "Draw 49: 0.666666666666666"
## [1] "Draw 50: 0.666666666666666"
## [1] "Draw 51: 0.666666666666666"
## [1] "Draw 52: 0.666666666666667"
## [1] "Draw 53: 0.666666666666667"
## [1] "Draw 54: 0.666666666666667"
## [1] "Draw 55: 0.666666666666667"
## [1] "Draw 56: 0.666666666666667"
## [1] "Draw 57: 0.666666666666667"
## [1] "Draw 58: 0.666666666666667"
## [1] "Draw 59: 0.666666666666667"
## [1] "Draw 60: 0.666666666666667"
## [1] "Draw 61: 0.666666666666667"
## [1] "Draw 62: 0.666666666666667"
## [1] "Draw 63: 0.666666666666667"
## [1] "Draw 64: 0.666666666666667"
## [1] "Draw 65: 0.666666666666667"
## [1] "Draw 66: 0.666666666666667"
## [1] "Draw 67: 0.666666666666667"
## [1] "Draw 68: 0.666666666666667"
## [1] "Draw 69: 0.666666666666667"
## [1] "Draw 70: 0.666666666666667"
## [1] "Draw 71: 0.666666666666667"
## [1] "Draw 72: 0.666666666666667"
## [1] "Draw 73: 0.666666666666667"
## [1] "Draw 74: 0.666666666666667"
## [1] "Draw 75: 0.666666666666667"
## [1] "Draw 76: 0.666666666666667"
## [1] "Draw 77: 0.666666666666667"
## [1] "Draw 78: 0.666666666666667"
## [1] "Draw 79: 0.666666666666667"
## [1] "Draw 80: 0.666666666666667"
## [1] "Draw 81: 0.666666666666667"
## [1] "Draw 82: 0.666666666666667"
## [1] "Draw 83: 0.666666666666667"
## [1] "Draw 84: 0.666666666666667"
## [1] "Draw 85: 0.666666666666667"
## [1] "Draw 86: 0.666666666666667"
## [1] "Draw 87: 0.666666666666667"
## [1] "Draw 88: 0.666666666666667"
## [1] "Draw 89: 0.666666666666667"
## [1] "Draw 90: 0.666666666666667"
## [1] "Draw 91: 0.666666666666667"
## [1] "Draw 92: 0.666666666666667"
## [1] "Draw 93: 0.666666666666667"
## [1] "Draw 94: 0.666666666666667"
## [1] "Draw 95: 0.666666666666667"
## [1] "Draw 96: 0.666666666666667"
## [1] "Draw 97: 0.666666666666667"
## [1] "Draw 98: 0.666666666666667"
## [1] "Draw 99: 0.666666666666667"
## [1] "Draw 100: 0.666666666666667"

The probabilities found up to draw 1, 3 and 5 are coherent with what we found in the previous sections. Moreover, we see that from draw 52 onwards, the probability of Louana winning remains constant at 66.67%. This is referred as the limit.

Game limited to the number of necessary draws

Without printing the results of the above for loop, we do not know how many draws are necessary to reach the limit.

Let’s now try to include the information about the number of necessary draws in order to avoid computing unnecessary draws:

n_draws <- 9999 # initial number of draws, set intentionally to a high number
p_tie <- c() # set empty vector for prob of ties
l_win_tot <- c() # set empty vector for prob of Louana winning

# find number of necessary draws:
for (i in 1:n_draws) {
  p_tie[i] <- tie^i # prob of tie for each draw
  limit_ndraws <- sum(p_tie > 2.2e-16) # number of necessary draws
}

# compute Louana winning probabilities with the smallest number of necessary draws
for (i in 1:limit_ndraws) {
  l_win_tot[i] <- ((tie^(i - 1)) * l_win) # prob of Louana winning up to limited number of draws
  print(paste0("Draw ", i, ": ", sum(l_win_tot))) # sum of winning up to limited number of draws
}

## [1] "Draw 1: 0.333333333333333"
## [1] "Draw 2: 0.5"
## [1] "Draw 3: 0.583333333333333"
## [1] "Draw 4: 0.625"
## [1] "Draw 5: 0.645833333333333"
## [1] "Draw 6: 0.65625"
## [1] "Draw 7: 0.661458333333333"
## [1] "Draw 8: 0.6640625"
## [1] "Draw 9: 0.665364583333333"
## [1] "Draw 10: 0.666015625"
## [1] "Draw 11: 0.666341145833333"
## [1] "Draw 12: 0.66650390625"
## [1] "Draw 13: 0.666585286458333"
## [1] "Draw 14: 0.6666259765625"
## [1] "Draw 15: 0.666646321614583"
## [1] "Draw 16: 0.666656494140625"
## [1] "Draw 17: 0.666661580403646"
## [1] "Draw 18: 0.666664123535156"
## [1] "Draw 19: 0.666665395100911"
## [1] "Draw 20: 0.666666030883789"
## [1] "Draw 21: 0.666666348775228"
## [1] "Draw 22: 0.666666507720947"
## [1] "Draw 23: 0.666666587193807"
## [1] "Draw 24: 0.666666626930237"
## [1] "Draw 25: 0.666666646798452"
## [1] "Draw 26: 0.666666656732559"
## [1] "Draw 27: 0.666666661699613"
## [1] "Draw 28: 0.66666666418314"
## [1] "Draw 29: 0.666666665424903"
## [1] "Draw 30: 0.666666666045785"
## [1] "Draw 31: 0.666666666356226"
## [1] "Draw 32: 0.666666666511446"
## [1] "Draw 33: 0.666666666589056"
## [1] "Draw 34: 0.666666666627862"
## [1] "Draw 35: 0.666666666647264"
## [1] "Draw 36: 0.666666666656965"
## [1] "Draw 37: 0.666666666661816"
## [1] "Draw 38: 0.666666666664241"
## [1] "Draw 39: 0.666666666665454"
## [1] "Draw 40: 0.66666666666606"
## [1] "Draw 41: 0.666666666666364"
## [1] "Draw 42: 0.666666666666515"
## [1] "Draw 43: 0.666666666666591"
## [1] "Draw 44: 0.666666666666629"
## [1] "Draw 45: 0.666666666666648"
## [1] "Draw 46: 0.666666666666657"
## [1] "Draw 47: 0.666666666666662"
## [1] "Draw 48: 0.666666666666664"
## [1] "Draw 49: 0.666666666666666"
## [1] "Draw 50: 0.666666666666666"
## [1] "Draw 51: 0.666666666666666"
## [1] "Draw 52: 0.666666666666667"

Final winning probabilities

Remember that the initial question was:

What is the probability of winning the game for each ambassador?

The probability that Louana wins the game can easily be extracted as follows:

sum(l_win_tot)

## [1] 0.6666667

And since the game only stops when there is a winner, the sum of the winning probabilities for Louana and Colin must be equal to 1, so the probability that Colin wins the game is:

c_win_tot <- 1 - sum(l_win_tot)
c_win_tot

## [1] 0.3333333

To summarize:

Louana has 66.67% chance to win the ambassadors’ game.
Colin has 33.33% chance to win the ambassadors’ game.

Visual representations

To visualize the probabilities for each ambassador, we miss the probabilities of winning for Colin so let’s compute them first:

c_win_tot <- c() # set empty vector for prob of Colin winning

# compute Colin winning probabilities with the smallest number of necessary draws
for (i in 1:limit_ndraws) {
  c_win_tot[i] <- ((tie^(i - 1)) * c_win) # prob of Colin winning
  print(paste0("Draw ", i, ": ", sum(c_win_tot))) # print sum of winning
}

## [1] "Draw 1: 0.166666666666667"
## [1] "Draw 2: 0.25"
## [1] "Draw 3: 0.291666666666667"
## [1] "Draw 4: 0.3125"
## [1] "Draw 5: 0.322916666666667"
## [1] "Draw 6: 0.328125"
## [1] "Draw 7: 0.330729166666667"
## [1] "Draw 8: 0.33203125"
## [1] "Draw 9: 0.332682291666667"
## [1] "Draw 10: 0.3330078125"
## [1] "Draw 11: 0.333170572916667"
## [1] "Draw 12: 0.333251953125"
## [1] "Draw 13: 0.333292643229167"
## [1] "Draw 14: 0.33331298828125"
## [1] "Draw 15: 0.333323160807292"
## [1] "Draw 16: 0.333328247070312"
## [1] "Draw 17: 0.333330790201823"
## [1] "Draw 18: 0.333332061767578"
## [1] "Draw 19: 0.333332697550456"
## [1] "Draw 20: 0.333333015441895"
## [1] "Draw 21: 0.333333174387614"
## [1] "Draw 22: 0.333333253860474"
## [1] "Draw 23: 0.333333293596904"
## [1] "Draw 24: 0.333333313465118"
## [1] "Draw 25: 0.333333323399226"
## [1] "Draw 26: 0.33333332836628"
## [1] "Draw 27: 0.333333330849806"
## [1] "Draw 28: 0.33333333209157"
## [1] "Draw 29: 0.333333332712452"
## [1] "Draw 30: 0.333333333022892"
## [1] "Draw 31: 0.333333333178113"
## [1] "Draw 32: 0.333333333255723"
## [1] "Draw 33: 0.333333333294528"
## [1] "Draw 34: 0.333333333313931"
## [1] "Draw 35: 0.333333333323632"
## [1] "Draw 36: 0.333333333328483"
## [1] "Draw 37: 0.333333333330908"
## [1] "Draw 38: 0.333333333332121"
## [1] "Draw 39: 0.333333333332727"
## [1] "Draw 40: 0.33333333333303"
## [1] "Draw 41: 0.333333333333182"
## [1] "Draw 42: 0.333333333333258"
## [1] "Draw 43: 0.333333333333295"
## [1] "Draw 44: 0.333333333333314"
## [1] "Draw 45: 0.333333333333324"
## [1] "Draw 46: 0.333333333333329"
## [1] "Draw 47: 0.333333333333331"
## [1] "Draw 48: 0.333333333333332"
## [1] "Draw 49: 0.333333333333333"
## [1] "Draw 50: 0.333333333333333"
## [1] "Draw 51: 0.333333333333333"
## [1] "Draw 52: 0.333333333333333"

We create a dataset with the probabilities for both ambassadors:

dat <- data.frame(
  Draw = rep(1:limit_ndraws, 2),
  Probability = c(cumsum(l_win_tot), cumsum(c_win_tot)),
  Ambassador = c(rep("Louana", limit_ndraws), rep("Colin", limit_ndraws))
)

We can now visualize these probabilities up to the number of necessary draws:

# load package
library(ggplot2)

# plot
ggplot(dat) +
  aes(x = Draw, y = Probability, colour = Ambassador) +
  geom_line(linewidth = 2L) +
  labs(
    y = "Probability of winning",
    caption = "Source: Koh-Lanta 2022"
  ) +
  scale_y_continuous(labels = scales::percent_format(accuracy = 1), limits = c(0, 1)) +
  theme_minimal()

But I believe the most appropriate plot to answer the initial question is with a barplot:

# create dataset
dat_barplot <- data.frame(
  Ambassador = c("Louana", "Colin"),
  Probability = c(sum(l_win_tot), sum(c_win_tot))
)

# plot
ggplot(data = dat_barplot, aes(x = Ambassador, y = Probability)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  labs(
    y = "Probability of winning",
    caption = "Source: Koh-Lanta 2022"
  ) +
  scale_y_continuous(labels = scales::percent_format(accuracy = 1), limits = c(0, 1)) +
  theme_minimal()

Coded into a function

Let’s try to implement this problem in a function to be able to reuse it with other initial probabilities.

With:

p_a and p_b denoting, respectively, the probability that ambassador A and ambassador B draw a white ball,
n_draws denoting the maximum number of draws that is allowed (default = 9999),

we have the following function:

ambassadors_game <- function(p_a, p_b, n_draws = 9999) {
  q_a <- (1 - p_a)
  q_b <- (1 - p_b)

  a_win <- q_b * p_a
  b_win <- q_a * p_b

  tie <- (p_a * p_b) + (q_a * q_b)

  p_tie <- c() # set empty vector for prob of ties
  a_win_tot <- c() # set empty vector for prob of A winning
  b_win_tot <- c() # set empty vector for prob of B winning

  # find number of necessary draws:
  for (i in 1:n_draws) {
    p_tie[i] <- tie^i # prob of tie for each draw
    limit_ndraws <- sum(p_tie > 2.2e-16) # number of necessary draws
  }

  # compute A and B winning probabilities with the smallest number of necessary draws:
  for (i in 1:limit_ndraws) {
    a_win_tot[i] <- ((tie^(i - 1)) * a_win) # prob of A winning up to limited number of draws
    b_win_tot[i] <- ((tie^(i - 1)) * b_win) # prob of B winning up to limited number of draws
  }

  # save P(A), P(B) and number of necessary draws:
  res <- list(
    "p_a" = sum(a_win_tot),
    "p_b" = sum(b_win_tot),
    "ndraws" = limit_ndraws
  )
  # print results:
  return(res)
}

We test the function to see if it matches results found above.

First, if both ambassadors have identical urns as it was the case before the 2022 season:

ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 2
)

## $p_a
## [1] 0.5
## 
## $p_b
## [1] 0.5
## 
## $ndraws
## [1] 52

The game is indeed fair, with a 50% chance of winning for each ambassador.

Second, with the urns presented to Louana and Colin, but for the first draw only (setting arbitrarily that Louana is ambassador A and Colin is ambassador B):

# first draw only
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 1
)

## $p_a
## [1] 0.3333333
## 
## $p_b
## [1] 0.1666667
## 
## $ndraws
## [1] 1

Third, still with the urns presented to Louana and Colin, but for a game limited to exactly 3 and 5 draws:

# up to 3 draws
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 3
)

## $p_a
## [1] 0.5833333
## 
## $p_b
## [1] 0.2916667
## 
## $ndraws
## [1] 3

# up to 5 draws
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 5
)

## $p_a
## [1] 0.6458333
## 
## $p_b
## [1] 0.3229167
## 
## $ndraws
## [1] 5

And now, the final verification with the real situation of Louana and Colin in Koh-Lanta 2022:

# Koh-Lanta 2022 situation
out <- ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3
)

out

## $p_a
## [1] 0.6666667
## 
## $p_b
## [1] 0.3333333
## 
## $ndraws
## [1] 52

All results match the ones presented above.

(Note that the probabilities for each ambassador can be extracted as follows:)

# prob ambassador A
out$p_a

## [1] 0.6666667

# prob ambassador B
out$p_b

## [1] 0.3333333

Conclusion

The initial question, coming from the television show Koh-Lanta, was “what is the probability of winning for each participant if they play the ambassadors’ game?”.

In this article, we have shown how to compute these probabilities. Moreover, we have illustrated the process of how a simple probability problem could be generalized to suit many real life situations, and how a for loop and a function could be used to implement a real life situation into R.

Last but not least, I would like to focus on something Denis Brogniart (the well-known presenter of the show) said just after the two ambassadors came back to the island to announce their choice to the other adventurers. He mentioned that, due to the punishment afflicted to the yellow tribe, there was a difference of chances of 16% between Louana and Colin.

This comes naturally from the following two situations:

First situation:

Louana has a 50% chance of picking a white ball.
Colin has a 33.33% chance of picking a white ball.
The difference is 50 - 33.33 = 16.67%, truncated to 16%.

Or, if the game is limited to the first draw only:

Second situation:

Louana has 33.33% chance of winning the game.
Colin has a 16.67% chance of winning the game.
The difference is 33.33 - 16.67 = 16.66%, truncated to 16%.

It is true that, seeing the problem from those two angles, the difference is ~16%. Denis Brogniart is right in mentioning this 16% difference.

However, the difference of chances between the two ambassadors is larger if we see the game from a broader perspective. The rules of the game say that it stops only when there is a winner. From that point of view, as demonstrated above:

Louana has 66.67% chance of winning the game.
Colin has 33.33% chance of winning the game.

In this situation, the difference in probability of winning between the 2 ambassadors is 66.67 - 33.33 = 33.34%!

I must admit that I was not expecting such a large difference between the two ambassadors. Seeing the ambassadors’ game from that perspective looks different to what we were told, or to what we thought before actually computing the probabilities. (Denis Brogniart, if you happen to read this, feel free to let me know whether this relatively large difference was intended or not.)

For those of you who do not watch the show: upon the return of the ambassadors on the island with all adventurers, Colin has been heavily criticized by the members of his tribe. His decision not to play the ambassadors’ game (and the decision to eliminate one member of his tribe with the aim of saving himself) was seen as a betrayal. As a consequence of this, he was eliminated by the reunited tribe directly after that episode.

I am going to conclude this article with the following question:

If you were an ambassador in the 2022 Koh-Lanta season and given that now you know the exact probabilites of winning for each tribe, what would you have done?

Thanks for reading.

As always, if you have any question related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

We will never know if he would have played the game if the chances of winning were equal for both ambassadors. Since I started to watch this television show, I have never seen any ambassadors’ negotiation leading to the ambassadors’ game, they all ended with the designation of an adventurer.↩︎
And to be honest, I wanted to compute these probabilities because while I was watching the show with my girlfriend, she looked at me and asked “what are the probabilities for each of them?”. I am now able to give her a precise answer.↩︎

Kruskal-Wallis test, or the nonparametric version of the ANOVA

Thu, 24 Mar 2022 00:00:00 +0000

Introduction

In a previous article, we showed how to do an ANOVA in R to compare three or more groups.

Remember that, as for many statistical tests, the one-way ANOVA requires that some assumptions are satisfied in order to be able to use and interpret the results. In particular, the ANOVA requires that residuals follow approximately a normal distribution.¹

Luckily, if the normality assumption is not satisfied, there is the nonparametric version of the ANOVA: the Kruskal-Wallis test.

In the rest of the article, we show how to perform the Kruskal-Wallis test in R and how to interpret its results. We will also briefly show how to do post-hoc tests and how to present all necessary statistical results directly on a plot.

Data

Data for the present article is based on the penguins dataset (an alternative to the well-known iris dataset), accessible via the {palmerpenguins} package:

# install.packages("palmerpenguins")
library(palmerpenguins)

The original dataset contains data for 344 penguins of 3 different species (Adelie, Chinstrap and Gentoo).

It contains 8 variables, but we focus only on the flipper length and the species for this article, so we keep only those 2 variables:

library(tidyverse)

dat <- penguins %>%
  select(species, flipper_length_mm)

(If you are unfamiliar with the pipe operator (%>%), you can also select variables with penguins[, c("species", "flipper_length_mm")]. Learn more ways to select variables in the article about data manipulation.)

It is always a good practice to do some descriptive statistics for the entire sample and by group before doing the test, so we have a broad overview of the data at hand.

# entire sample
summary(dat)

##       species    flipper_length_mm
##  Adelie   :152   Min.   :172.0    
##  Chinstrap: 68   1st Qu.:190.0    
##  Gentoo   :124   Median :197.0    
##                  Mean   :200.9    
##                  3rd Qu.:213.0    
##                  Max.   :231.0    
##                  NA's   :2

# by group
library(doBy)
summaryBy(flipper_length_mm ~ species,
  data = dat,
  FUN = median,
  na.rm = TRUE
)

## # A tibble: 3 × 2
##   species   flipper_length_mm.median
##   <fct>                        <dbl>
## 1 Adelie                         190
## 2 Chinstrap                      196
## 3 Gentoo                         216

# boxplot by species
ggplot(dat) +
  aes(x = species, y = flipper_length_mm, fill = species) +
  geom_boxplot() +
  theme(legend.position = "none")

Based on the boxplots and the summary statistics, we already see that, in our sample, penguins from the Adelie species seem to have the smallest flippers, while those from the Gentoo species seem to have the biggest flippers. However, only a sound statistical test will tell us whether we can infer this conclusion to our population.

Kruskal-Wallis test

Aim and hypotheses

As mentioned earlier, the Kruskal-Wallis test allows to compare three or more groups. More precisely, it is used to compare three or more groups in terms of a quantitative variable.

It can be seen as the extension to the Mann-Whitney test which allows to compare 2 groups under the non-normality assumption.

In the context of our example, we are going to use the Kruskal-Wallis test to help us answer the following question: “Is the length of the flippers different between the 3 species of penguins?”.

The null and alternative hypotheses of the Kruskal-Wallis test are:

$H_0$: The 3 species are equal in terms of flipper length
$H_1$: At least one species is different from the other 2 species in terms of flipper length

Be careful that, as for the ANOVA, the alternative hypothesis is not that all species are different in terms of flipper length. The opposite of all species being equal ($H_0$) is that at least one species is different from the others ($H_1$).

In this sense, if the null hypothesis is rejected, it means that at least one species is different from the other 2, but not necessarily that all 3 species are different from each other. It could be that flipper length for the species Gentoo is different than for the species Chinstrap and Adelie, but flipper length is similar between Chinstrap and Adelie. Other types of test (known as post-hoc tests and covered later) must be performed to test whether all 3 species differ.

Assumptions

First, the Kruskal-Wallis test compares several groups in terms of a quantitative variable. So there must be one quantitative dependent variable (which corresponds to the measurements to which the question relates) and one qualitative independent variable (with at least 2 levels which will determine the groups to compare).²

Second, remember that the Kruskal-Wallis test is a nonparametric test, so the normality assumption is not required. However, the independence assumption still holds.

This means that the data, collected from a representative and randomly selected portion of the total population, should be independent between groups and within each group. The assumption of independence is most often verified based on the design of the experiment and on the good control of experimental conditions rather than via a formal test. If you are still unsure about independence based on the experiment design, ask yourself if one observation is related to another (if one observation has an impact on another) within each group or between the groups themselves. If not, it is most likely that you have independent samples. If observations between samples (forming the different groups to be compared) are dependent (for example, if three measurements have been collected on the same individuals as it is often the case in medical studies when measuring a metric (i) before, (ii) during and (iii) after a treatment), the Friedman test should be preferred in order to take into account the dependency between the samples.

Regarding the homoscedasticity (i.e., equality of the variances): As long as you use the Kruskal-Wallis test to, in fine, compare groups, homoscedasticity is not required. If you wish to compare medians, the Kruskal-Wallis test requires homoscedasticity.³

In our example, independence is assumed and we do not need to compare medians (we are only interested in comparing groups), so we can proceed to how to do the test in R. Note that the normality assumption may or may not hold, but for this article we assume it is not satisfied.

In R

The Kruskal-Wallis test in R can be done with the kruskal.test() function:

kruskal.test(flipper_length_mm ~ species,
  data = dat
)

## 
## 	Kruskal-Wallis rank sum test
## 
## data:  flipper_length_mm by species
## Kruskal-Wallis chi-squared = 244.89, df = 2, p-value < 2.2e-16

The most important result in this output is the p-value. We show how to interpret it in the next section.

Interpretations

Based on the Kruskal-Wallis test, we reject the null hypothesis and we conclude that at least one species is different in terms of flippers length (p-value < 0.001).

(For the sake of illustration, if the p-value was larger than the significance level $\alpha = 0.05$: we cannot reject the null hypothesis so we cannot reject the hypothesis that the 3 considered species of penguins are equal in terms of flippers length.)

Post-hoc tests

We have just showed that at least one species is different from the others in terms of flippers length. Nonetheless, here comes the limitations of the Kruskal-Wallis test: it does not say which group(s) is(are) different from the others.

To know this, we need to use other types of test, referred as post-hoc tests (in Latin, “after this”, so after obtaining statistically significant Kruskal-Wallis results) or multiple pairwise-comparison tests. For the interested reader, a more detailed explanation of post-hoc tests can be found here.

The most common post-hoc tests after a significant Kruskal-Wallis test are:

Dunn test
Conover test
Nemenyi test
Pairwise Wilcoxon test

The Dunn test being the most common one, here is how to do it in R.⁴

Dunn test

library(FSA)

dunnTest(flipper_length_mm ~ species,
  data = dat,
  method = "holm"
)

##           Comparison          Z      P.unadj        P.adj
## 1 Adelie - Chinstrap  -3.629336 2.841509e-04 2.841509e-04
## 2    Adelie - Gentoo -15.476612 4.990733e-54 1.497220e-53
## 3 Chinstrap - Gentoo  -8.931938 4.186100e-19 8.372200e-19

It is the last column (the adjusted p-values, adjusted for multiple comparisons) that is of interest. These p-values should be compared to your desired significance level (usually 5%).

Based on the output, we conclude that:

Adelie and Chinstrap differ significantly (p < 0.001)
Adelie and Gentoo differ significantly (p < 0.001)
Chinstrap and Gentoo differ significantly (p < 0.001)

Therefore, based on the Dunn test, we can now conclude that all 3 species differ in terms of flipper length.

Combination of statistical results and plot

A very good alternative for performing a Kruskal-Wallis and the post-hoc tests in R is with the ggbetweenstats() function from the {ggstatsplot} package:

library(ggstatsplot)

ggbetweenstats(
  data = dat,
  x = species,
  y = flipper_length_mm,
  type = "nonparametric", # ANOVA or Kruskal-Wallis
  plot.type = "box",
  pairwise.comparisons = TRUE,
  pairwise.display = "significant",
  centrality.plotting = FALSE,
  bf.message = FALSE
)

This method has the advantage that all necessary statistical results are displayed directly on the plot.

The results of the Kruskal-Wallis test are shown in the subtitle above the plot (the p-value is after p =). Moreover, the results of the post-hoc test are displayed between each group via accolades, and the boxplots allow to visualize the distribution for each species.

Summary

In this post, we reviewed the aim and hypotheses of the Kruskal-Wallis test and its underlying assumptions. We then showed how to do the test in R and how to interpret the results.

We also showed the most common post-hoc test after a significant Kruskal-Wallis test—the Dunn test.

Last but not least, we presented a concise way to present both the data by group and all the statistical results on the same plot.

Thanks for reading.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Stevens, James P. 2013. Intermediate Statistics: A Modern Approach. Routledge.

This is the case for small sample sizes. For large sample sizes, normality is not required (this is a common misconception!). By the central limit theorem, sample means of large samples are often well-approximated by a normal distribution even if the data are not normally distributed (Stevens 2013). See more details in this note.↩︎
Note that in theory, Kruskal-Wallis test can also be used for only two groups. However, in practice we use the Mann-Whitney test for two groups and Kruskal-Wallis for three or more groups.↩︎
See more information about the difference in this article.↩︎
Note that there are other p-value adjustment methods. See ?dunnTest for more options.↩︎

Stats and R is 2 years old!

Thu, 16 Dec 2021 00:00:00 +0000

Introduction

Stats and R has been launched exactly two years ago. Like last year, I think it is a good time to do a review of the past 12 months by sharing some figures about the audience of the blog.

This article is not about showing off my numbers, but rather a way to illustrate how to analyze your blog or your website’s traffic using Google Analytics data. Figures regarding the audience of my blog is probably useless to you (and I believe, should not be compared with). However, the code used in this post can be reused for your own blog or website (provided you also use Google Analytics to track your audience).

Note that I use the {googleAnalyticsR} R package to analyze my blog’s Google Analytics data. If you are unfamiliar with this package, see the prerequisites first.

If you have already used that package, you can select your account as followed (make sure to edit the code with your own property name):

library(googleAnalyticsR)

accounts <- ga_account_list()

# select the view ID by property name
view_id <- accounts$viewId[which(accounts$webPropertyName == "statsandr.com")]

On top of that, I also assume that you have basic knowledge of {ggplot2}—a popular R package to draw nice plots and visualizations.

Analytics

Last year, I mainly focused on the number of sessions. This year, I mainly concentrate on the number of page views to illustrate a different metrics.

For your information, a session is a group of user interactions with your website that take place within a given time frame, whereas a page view, as the name suggests, is defined as a view of a page on your site.

You can always change the metrics by editing metrics = c("pageviews") in the code below. See all available metrics provided by Google Analytics in this article.

Users and page views

As for last year’s review, let’s start with some general numbers, such as the number of users and page views for the entire site.

Note that we analyze traffic over the last year only so we extract data from December 16, 2020 to yesterday (December 15, 2021):

# set date range
start_date <- as.Date("2020-12-16")
end_date <- as.Date("2021-12-15")

# get Google Analytics (GA) data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("users", "pageviews"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

gadata

##    users pageviews
## 1 549360    876280

Over this past year, Stats and R has attracted 549,360 users (number of new and returning people who visited the site), who generated a total of 876,280 page views.

That is an average of 2401 page views per day in 2021, compared to 1,531 page views per day in 2020 (an increase of 56.81%).

Page views over time

One of the first interesting metrics to analyze your blog’s audience is the evolution of traffic over time.

The daily number of page views over time can be presented in a scatterplot—together with a smoothed line—to analyze the evolution of the audience of your blog:

# get the Google Analytics (GA) data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews"), # edit for other metrics
  dimensions = c("date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# load required libraries
library(dplyr)
library(ggplot2)

# scatter plot with a trend line
gadata %>%
  ggplot(aes(x = date, y = pageviews)) +
  geom_point(size = 1L, color = "steelblue") + # change size and color of points
  geom_smooth(color = "steelblue", alpha = 0.25) + # change color of smoothed line and transparency of confidence interval
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Evolution of daily page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  theme(plot.margin = unit(c(5.5, 17.5, 5.5, 5.5), "pt")) + # to avoid the plot being cut on the right edge
  scale_y_continuous(labels = scales::comma) # better y labels

Although the number of page views varies quite a bit (with an outlier at more than 5,000 page views in a day and as low as less than 1,000 page views for some days), it seems to be cyclical with a dip during summer. It is worth noting that the same dip appeared last year, probably due to the fact that people are less likely to read posts about statistics and R during summer holidays.

So if you write about technical stuff in your blog, low numbers during summer may be expected and does not necessarily mean something is broken on your website.

Page views per channel

Knowing how people come to your blog is also a pretty important factor.

Here is how to visualize the evolution of daily page views per channel in a line plot:

# Get the data
trend_data <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  dimensions = c("date"),
  metrics = "pageviews",
  pivots = pivot_ga4("medium", "pageviews"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# edit variable names
names(trend_data) <- c("Date", "Total", "Organic", "Referral", "Direct", "Email", "Social")

# Change the data into a long format
library(tidyr)
trend_long <- gather(trend_data, Channel, Page_views, -Date)

# Build up the line plot
trend_long %>%
  filter(Channel != "Total") %>%
  ggplot() +
  aes(x = Date, y = Page_views, group = Channel) +
  theme_minimal() +
  geom_line(aes(colour = Channel)) +
  labs(
    y = "Page views",
    x = "",
    title = "Evolution of daily page views per channel",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

As we can see from the plot above, the large majority of page views come from the organic channel (so from search engines such as Google, Bing, etc.), with some peaks from referral (mainly from R-bloggers and RWeekly) when an article is published. (By the way, you can always subscribe to the newsletter if you want to be informed by email when a new post goes out.)

If you happen to write tutorials, you can also expect that most visitors come from the organic channel. If you are very present on social media, you will most likely attract more visitors from the social channel.

Page views per day of week and month of year

As seen in the previous plot, there are many ups and downs and traffic seems to be cyclical.

To investigate this further, we draw a boxplot of the number of page views for each day of the week:

# get data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = c("dayOfWeek", "date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

## Recoding gadata$dayOfWeek following GA naming conventions
gadata$dayOfWeek <- recode_factor(gadata$dayOfWeek,
  "0" = "Sunday",
  "1" = "Monday",
  "2" = "Tuesday",
  "3" = "Wednesday",
  "4" = "Thursday",
  "5" = "Friday",
  "6" = "Saturday"
)

## Reordering gadata$dayOfWeek to have Monday as first day of the week
gadata$dayOfWeek <- factor(gadata$dayOfWeek,
  levels = c(
    "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday",
    "Sunday"
  )
)

# Boxplot
gadata %>%
  ggplot(aes(x = dayOfWeek, y = pageviews)) +
  geom_boxplot(fill = "steelblue") +
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Page views per day of week",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

We can also compute the sum and the mean number of page views per day to have a numerical summary instead of a plot:

# compute sum
dat_sum <- aggregate(pageviews ~ dayOfWeek,
  data = gadata,
  FUN = sum
)

# compute mean
dat_mean <- aggregate(pageviews ~ dayOfWeek,
  data = gadata,
  FUN = mean
)

# combine both in one table
dat_summary <- cbind(dat_sum, dat_mean[, 2])

# rename columns
names(dat_summary) <- c("Day of week", "Sum", "Mean")

# display table
dat_summary

##   Day of week    Sum     Mean
## 1      Monday 141115 2713.750
## 2     Tuesday 143145 2752.788
## 3   Wednesday 146472 2763.623
## 4    Thursday 140712 2706.000
## 5      Friday 128223 2465.827
## 6    Saturday  85766 1649.346
## 7      Sunday  90847 1747.058

As expected, there are more readers during the week compared to the weekends.

The same analysis can be done for each month of the year instead of days of the week:

# get data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = c("month", "date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# Boxplot
gadata %>%
  ggplot(aes(x = month, y = pageviews)) +
  geom_boxplot(fill = "steelblue") +
  theme_minimal() +
  labs(
    y = "Page views",
    x = "Month",
    title = "Page views per month",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

# compute sum
dat_sum <- aggregate(pageviews ~ month,
  data = gadata,
  FUN = sum
)

# compute mean
dat_mean <- aggregate(pageviews ~ month,
  data = gadata,
  FUN = mean
)

# combine both in one table
dat_summary <- cbind(dat_sum, dat_mean[, 2])

# rename columns
names(dat_summary) <- c("Month", "Sum", "Mean")

# display table
dat_summary

##    Month   Sum     Mean
## 1     01 68534 2210.774
## 2     02 80953 2891.179
## 3     03 92629 2988.032
## 4     04 88679 2955.967
## 5     05 81739 2636.742
## 6     06 64460 2148.667
## 7     07 49772 1605.548
## 8     08 46389 1496.419
## 9     09 68046 2268.200
## 10    10 79237 2556.032
## 11    11 77745 2591.500
## 12    12 78097 2519.258

From the plots and the numerical summaries, it is clear that the number of page views is not the same between the days of the week and the months of the year.

Page views per month and year

If you have data over more than a year, it could be useful to compare your monthly blog’s traffic over the years.

With the following code, we create a barplot of the number of daily page views per month and year:

# set new date range
start_date_launch <- as.Date("2019-12-16")

# get data
df2 <- google_analytics(view_id,
  date_range = c(start_date_launch, end_date),
  metrics = c("pageviews"),
  dimensions = c("date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# add in year month columns to dataframe
df2$month <- format(df2$date, "%m")
df2$year <- format(df2$date, "%Y")

# page views by month by year using dplyr then graph using ggplot2 barplot
df2 %>%
  filter(year != 2019) %>% # remove 2019 because there are data for December only
  group_by(year, month) %>%
  summarize(pageviews = sum(pageviews)) %>%
  # print table steps by month by year
  # print(n = 100) %>%
  # graph data by month by year
  ggplot(aes(x = month, y = pageviews, fill = year)) +
  geom_bar(position = "dodge", stat = "identity") +
  theme_minimal() +
  labs(
    y = "Page views",
    x = "Month",
    title = "Page views per month and year",
    subtitle = paste0(format(start_date_launch, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

This barplot allows to easily see the evolution of the number of page views over the months, but more importantly, compare this evolution across different years.

Top performing pages

Another important factor when measuring the performance of your blog or website is the number of page views per pages.

It is true that the top performing pages in terms of page views over the year can easily be found in Google Analytics (you can access it via Behavior > Site Content > All pages).

However, for the interested reader, here is how to get the data in R (note that you can change n = 7 in the code below to change the number of top performing pages to display):

## Make the request to GA
data_fetch <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews"),
  dimensions = c("pageTitle"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

## Create a table of the most viewed posts
library(lubridate)
library(reactable)
library(stringr)

most_viewed_posts <- data_fetch %>%
  mutate(Title = str_trunc(pageTitle, width = 40)) %>% # keep maximum 40 characters
  count(Title, wt = pageviews, sort = TRUE)

head(most_viewed_posts, n = 7) # edit n for more or less pages to display

##                                      Title      n
## 1    Outliers detection in R - Stats and R 119747
## 2 Descriptive statistics in R - Stats a... 109473
## 3 Variable types and examples - Stats a...  83025
## 4 Correlation coefficient and correlati...  65703
## 5 Chi-square test of independence in R ...  62100
## 6 The complete guide to clustering anal...  40440
## 7                 ANOVA in R - Stats and R  32914

If like me you prefer a visualization over a table, here is how to draw this table of top performing pages in a barplot:

# plot
top_n(most_viewed_posts, n = 7, n) %>% # edit n for more or less pages to display
  ggplot(., aes(x = reorder(Title, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Page title",
    title = "Top performing pages in terms of page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  theme(plot.margin = unit(c(5.5, 17.5, 5.5, 5.5), "pt")) # to avoid the plot being cut on the right edge

This gives me a good first overview on how posts performed in terms of page views, so in some sense, what people find useful. The top 3 articles in the past year were:

Be careful that this ranking is based on the total number of page views over the last 12 months. A recent article may thus be found at the bottom of the list simply because it collected page views over a shorter period of time compared to an old article. So it is best to avoid comparing recent articles with older ones, or you can compare articles after having “time-normalized” the number of page views. See previous year’s review for more details and illustrations of this metrics.

You could also be interested in knowing the worst performing ones (to eventually improve them or include them in higher quality posts):

tail(subset(most_viewed_posts, n > 1000),
  n = 7
)

##                                       Title    n
## 49 A guide on how to read statistical ta... 1477
## 50                      About - Stats and R 1470
## 51 How to embed a Shiny app in blogdown?... 1455
## 52                   About me - Stats and R 1277
## 53 One-proportion and chi-square goodnes... 1226
## 54 Running pace calculator in R Shiny - ... 1049
## 55                      Shiny - Stats and R 1018

Note that I intentionally excluded pages with less than 1000 views to remove deleted or hidden pages from the ranking.

Another issue with this ranking is that it may be biased due to some pages which have been duplicated (if you edited the title for example), but at least you have a broad idea of the worst performing pages.

Page views by country

Knowing the country where your readers come from may also be handy for some content creators or marketers.

Location of my readers is not really important for me because I intend to write for everyone, but this may be completely the opposite if you are selling things or running a business/ecommerce.

# get GA data
data_fetch <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = "country",
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# table
countries <- data_fetch %>%
  mutate(Country = str_trunc(country, width = 40)) %>% # keep maximum 40 characters
  count(Country, wt = pageviews, sort = TRUE)

head(countries, n = 10) # edit n for more or less countries to display

##           Country      n
## 1   United States 244752
## 2           India  71797
## 3  United Kingdom  53272
## 4         Germany  38322
## 5          Canada  33812
## 6         Belgium  26421
## 7       Australia  26225
## 8     Philippines  23193
## 9     Netherlands  21944
## 10         Brazil  17956

Again, if you prefer a plot over a table, you can visualize the top countries in terms of page views in a barplot:

# plot
top_n(countries, n = 10, n) %>% # edit n for more or less countries to display
  ggplot(., aes(x = reorder(Country, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Country",
    title = "Top performing countries in terms of page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  theme(plot.margin = unit(c(5.5, 7.5, 5.5, 5.5), "pt")) # to avoid the plot being cut on the right edge

We see that a large share of readers are from the US—and Belgium (my country) comes only in $6^{th}$ place in terms of number of page views.

Be careful that, as the number of people located in different countries differs widely, this ranking may hide some insights if you are comparing page views by countries in absolute terms. See why in this section of last year’s review.

User engagement by devices

One may also be interested in checking how users are engaged depending on device’s type. To investigate this, we plot 3 charts describing:

How page views are distributed by type of device?
The average time on page (in seconds) by type of device
The number of page views per session by device type

So first, how page views are distributed by device type?

# GA data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews", "avgTimeOnPage"),
  dimensions = c("date", "deviceCategory"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# plot pageviews by deviceCategory
gadata %>%
  ggplot(aes(deviceCategory, pageviews)) +
  geom_bar(aes(fill = deviceCategory), stat = "identity") +
  theme_minimal() +
  labs(
    y = "Page views",
    x = "",
    title = "Page views per device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com",
    fill = "Device" # edit legend title
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

From the above plot, we see that the large majority of readers visited the blog from a desktop and only a very small proportion comes from a tablet.

This makes sense since I guess many visitors are reading my articles or tutorials while using R (which is only available on desktop).

However, this information of total number of page views per device type does not tell me anything about the time spent on each page and thus the engagement by device type. The following plot answers this question:

# add median of average time on page per device
gadata <- gadata %>%
  group_by(deviceCategory) %>%
  mutate(med = median(avgTimeOnPage))

# plot avgTimeOnPage by deviceCategory
ggplot(gadata) +
  aes(x = avgTimeOnPage, fill = deviceCategory) +
  geom_histogram(bins = 30L) +
  scale_fill_hue() +
  theme_minimal() +
  theme(legend.position = "none") +
  facet_wrap(vars(deviceCategory)) +
  labs(
    y = "Frequency",
    x = "Average time on page (in seconds)",
    title = "Average time on page per device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  geom_vline(aes(xintercept = med, group = deviceCategory),
    color = "darkgrey",
    linetype = "dashed"
  ) +
  geom_text(
    aes(
      x = med, y = 125,
      label = paste0("Median = ", round(med), " seconds")
    ),
    angle = 90,
    vjust = 3,
    color = "darkgrey",
    size = 3
  )

From the above plot, we see that:

Most readers coming from a tablet actually leave the page very quickly (see the peak around 0 second in the tablet facet).
Distributions of the average time on page for readers on desktop and mobile were quite similar, with an average time on page mostly between 125 seconds (= 2 minutes and 5 seconds) and 375 seconds (= 6 minutes and 15 seconds).
Quite surprisingly, the median of the average time spent on page is slightly higher for visitors on mobile than on desktop (median = 316 seconds on mobile and 266 seconds on desktop, see the dashed vertical lines representing the medians in the desktop and mobile facets). This indicates that, although more people visit the blog from desktop (as shown by the total number of page views by device), it seems that people on mobile spend more time per page. I find this result quite surprising given that most of my articles include R code and require a computer to run the code. Therefore, I expected that people would spend more time on desktop than on mobile because on mobile they would quickly scan the article, while on desktop they would read the article more carefully and try to reproduce the code on their own computer. At least that is what I do when I read blogs on mobile versus reading them on desktop. What is even more intriguing, is that it was already the case last year. If someone finds similar results and have a possible explanation, I would be glad to hear from her (if possible, in the comments at the end of the article so everyone can benefit from the discussion).
As a side note, we see that these medians are higher this year compared to last year (266, 316 and 114 seconds in 2021 compared to 190, 228 and 106 seconds in 2020 on desktop, mobile and tablet, respectively). This is somewhat encouraging because it indicates that people spend more time on each page (which is an indication, at least partially, of the quality of the blog for Google).¹

Given this result, I also believe it is helpful to illustrate the number of page views during a session by device type.

In fact, it may be the case that visitors on mobile spend, on average, more time on each page but people on desktop visit more pages per session (remember that a session is a set of interactions with your website that take place within a given time frame).

We verify this belief via a density plot, and for better readability we exclude visits from tablet:

# GA data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviewsPerSession"),
  dimensions = c("date", "deviceCategory"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# add median of number of page views/session
gadata <- gadata %>%
  group_by(deviceCategory) %>%
  mutate(med = median(pageviewsPerSession))

## Reordering gadata$deviceCategory
gadata$deviceCategory <- factor(gadata$deviceCategory,
  levels = c("mobile", "desktop", "tablet")
)

# plot pageviewsPerSession by deviceCategory
gadata %>%
  filter(deviceCategory != "tablet") %>% # filter out pageviewsPerSession > 2.5 and visits from tablet
  ggplot(aes(x = pageviewsPerSession, fill = deviceCategory, color = deviceCategory)) +
  geom_density(alpha = 0.5) +
  scale_fill_hue() +
  theme_minimal() +
  labs(
    y = "Frequency",
    x = "Page views per session",
    title = "Page views per session by device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com\nDashed lines represent the medians",
    color = "Device", # edit legend title
    fill = "Device" # edit legend title
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  geom_vline(aes(xintercept = med, group = deviceCategory, color = deviceCategory),
    linetype = "dashed",
    show.legend = FALSE # remove legend
  )

This last plot shows that readers on desktop and mobile visit approximately the same number of pages per session, as indicated by the fact that the two distributions overlap each other and are not distant from each other. It is true that the median is higher for people on desktop than on mobile, but to a very small margin only (and the difference between the two is smaller than in last year’s review).

So to summarize what we learned based on the 3 last plots, we now know that

most readers visited the blog from desktop;
readers on mobile spent more time on each page than readers on desktop (and even more compared to readers on tablet);
users on desktop and mobile seem to have visited approximately the same number of pages per session.

One may wonder why I chose to compare medians instead of means. The main reason is that the median is a more robust way to represent non-normal data. For the interested reader, see a note on the difference between mean and median, and the context in which each measure is more appropriate.

Browser information

From a more technical perspective, you could also be interested in the number of page views by browser. I am personally not really interested in knowing which browser my visitors are using the most (mostly because this blog is available on all common browsers), but the most geeky among you may be so.

This information can be visualized with the following barplot:

# get data
browser_info <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews"),
  dimensions = c("browser"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# table
browser <- browser_info %>%
  mutate(Browser = str_trunc(browser, width = 40)) %>% # keep maximum 40 characters
  count(Browser, wt = pageviews, sort = TRUE)

# plot
top_n(browser, n = 10, n) %>% # edit n for more or less browser to display
  ggplot(., aes(x = reorder(Browser, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Browser",
    title = "Which browsers are our visitors using?",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

Most readers visited the site using Chrome, Safari and Firefox (this was expected since they are the most common browsers).

This was the last metrics presented in this review. Of course, many more are possible depending on your R skills (mainly, data manipulation and {ggplot2}) and your expertise in SEO or analyzing Google Analytics data. Hopefully, thanks to this review and possibly from last year too, you will be able to analyze your own blog or website using R and the {googleAnalyticsR} package. At least, this was the aim of the present article.

For those of you who are interested in a more condensed analysis, see my custom Google Analytics dashboard.²

End note

I would also like to add that these figures are not done to be compared with. Every website or blog is unique, every author is unique (with different priorities and different agendas) and more is not always better. I learn a lot thanks to this blog, I use it for personal purposes and for my students as part of my teaching tasks. I will keep writing on it as long as I enjoy it and as long as I have the time to do so, not matter how low or high the number of clicks.

Thanks to all readers of the past year, and see you in a year for another review! In the meantime, if you maintain a blog I would be really happy to hear how you track its performance.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

More time spend on each page is a favorable factor for Google because it means that people are reading it more carefully. If the blog or the post is of mediocre quality, users tend to bounce back quickly (known as bounce rate) and look for an answer to their question somewhere else (leading ultimately to less time spend on the page or site).↩︎
Thanks to the RStudio blog for the inspiration.↩︎

Multiple linear regression made simple

Mon, 04 Oct 2021 00:00:00 +0000

Introduction

Remember that descriptive statistics is a branch of statistics that allows to describe your data at hand.

Inferential statistics (with the popular hypothesis tests and confidence intervals) is another branch of statistics that allows to make inferences, that is, to draw conclusions about a population based on a sample.

The last branch of statistics is about modeling the relationship between two or more variables.¹ The most common statistical tool to describe and evaluate the link between variables is linear regression.

There are two types of linear regression:

Simple linear regression is a statistical approach that allows to assess the linear relationship between two quantitative variables. More precisely, it enables the relationship to be quantified and its significance to be evaluated.
Multiple linear regression is a generalization of simple linear regression, in the sense that this approach makes it possible to evaluate the linear relationships between a response variable (quantitative) and several explanatory variables (quantitative or qualitative).

In the real world, multiple linear regression is used more frequently than simple linear regression. This is mostly the case because:

Multiple linear regression allows to evaluate the relationship between two variables, while controlling for the effect (i.e., removing the effect) of other variables.
With data collection becoming easier, more variables can be included and taken into account when analyzing data.

Multiple linear regression being such a powerful statistical tool, I would like to present it so that everyone understands it, and perhaps even use it when deemed necessary. However, I cannot afford to write about multiple linear regression without first presenting simple linear regression.

So after a reminder about the principle and the interpretations that can be drawn from a simple linear regression, I will illustrate how to perform multiple linear regression in R. I will also show, in the context of multiple linear regression, how to interpret the output and discuss about its conditions of application. I will then conclude the article by presenting more advanced topics directly linked to linear regression.

Simple linear regression: reminder

Simple linear regression is an asymmetric procedure in which:

one of the variable is considered the response or the variable to be explained. It is also called dependent variable, and is represented on the $y$-axis
the other variable is the explanatory or also called independent variable, and is represented on the $x$-axis

Simple linear regression allows to evaluate the existence of a linear relationship between two variables and to quantify this link. Note that linearity is a strong assumption in linear regression in the sense that it tests and quantifies whether the two variables are linearly dependent.

What makes linear regression a powerful statistical tool is that it allows to quantify by what quantity the response/dependent variable varies when the explanatory/independent variable increases by one unit.

This concept is key in linear regression and helps to answer the following questions:

Is there a link between the amount spent in advertising and the sales during a certain period?
Is the number of years of schooling valued, in financial terms, in the first job?
Will an increase in tobacco taxes reduce its consumption?
What is the most likely price of an apartment, depending on the area?
Does a person’s reaction time to a stimulus depend on gender?

Simple linear regression can be seen as an extension to the analysis of variance (ANOVA) and the Student’s t-test. ANOVA and t-test allow to compare groups in terms of a quantitative variable—2 groups for t-test and 3 or more groups for ANOVA.²

For these tests, the independent variable, that is, the grouping variable forming the different groups to compare must be a qualitative variable. Linear regression is an extension because in addition to be used to compare groups, it is also used with quantitative independent variables (which is not possible with t-test and ANOVA).

In this article, we are interested in assessing whether there is a linear relationship between the distance traveled with a gallon of fuel and the weight of cars. For this example, we use the mtcars dataset (preloaded in R).

The dataset includes fuel consumption and 10 aspects of automotive design and performance for 32 automobiles:³

mpg Miles/(US) gallon (with a gallon $\approx$ 3.79 liters)
cyl Number of cylinders
disp Displacement (cu.in.)
hp Gross horsepower
drat Rear axle ratio
wt Weight (1000 lbs, with 1000 lbs $\approx$ 453.59 kg)
qsec 1/4 mile time (with 1/4 mile $\approx$ 402.34 meters)
vs Engine (0 = V-shaped, 1 = straight)
am Transmission (0 = automatic, 1 = manual)
gear Number of forward gears
carb Number of carburetors

dat <- mtcars

library(ggplot2)
ggplot(dat, aes(x = wt, y = mpg)) +
  geom_point() +
  labs(
    y = "Miles per gallon",
    x = "Car's weight (1000 lbs)"
  ) +
  theme_minimal()

The scatterplot above shows that there seems to be a negative relationship between the distance traveled with a gallon of fuel and the weight of a car. This makes sense, as the heavier the car, the more fuel it consumes and thus the fewer miles it can drive with a gallon.

This is already a good overview of the relationship between the two variables, but a simple linear regression with the miles per gallon as dependent variable and the car’s weight as independent variable goes further. It will tell us by how many miles the distance varies, on average, when the weight varies by one unit (1000 lbs in this case). This is possible thanks to the regression line.

Principle

The principle of simple linear regression is to find the line (i.e., determine its equation) which passes as close as possible to the observations, that is, the set of points formed by the pairs $(x_i, y_i)$.

In the first step, there are many potential lines. Three of them are plotted:

To find the line which passes as close as possible to all the points, we take the square of the vertical distance between each point and each potential line. Note that we take the square of the distances to make sure that a negative gap (i.e., a point below the line) is not compensated by a positive gap (i.e., a point above the line). The line which passes closest to the set of points is the one which minimizes the sum of these squared distances.

The resulting regression line is presented in blue in the following plot, and the dashed gray lines represent the vertical distance between the points and the fitted line. These vertical distances between each observed point and the fitted line determined by the least squares method are called the residuals of the linear regression model and denoted $\epsilon$.

By definition, there is no other line with a smaller total distance between the points and the line. This method is called the least squares method, or OLS for ordinary least squares.

Equation

The regression model can be written in the form of the equation:

\[Y = \beta_0 + \beta_1 X + \epsilon\]

with:

$Y$ the dependent variable
$X$ the independent variable
$\beta_0$ the intercept (the mean value of $Y$ when $x = 0$), also sometimes denoted $\alpha$
$\beta_1$ the slope (the expected increase in $Y$ when $X$ increases by one unit)
$\epsilon$ the residuals (the error term of mean 0 which describes the variations of $Y$ not captured by the model, also referred as the noise)

When we determine the line which passes closest to all the points (we say that we fit a line to the observed data), we actually estimate the unknown parameters $\beta_0$ and $\beta_1$ based on the data at hand. Remember from your geometry classes, to draw a line you only need two parameters—the intercept and the slope.

These estimates (and thus the blue line shown in the previous scatterplot) can be computed by hand with the following formulas:

\[\begin{align} \widehat\beta_1 &= \frac{\sum^n_{i = 1} (x_i - \bar{x})(y_i - \bar{y})}{\sum^n_{i = 1}(x_i - \bar{x})^2} \\ &= \frac{\left(\sum^n_{i = 1}x_iy_i\right) - n\bar{x}\bar{y}}{\sum^n_{i = 1}(x_i - \bar{x})^2} \end{align}\]

and

\[\widehat\beta_0 = \bar{y} - \widehat\beta_1 \bar{x}\]

with $\bar{x}$ and $\bar{y}$ denoting the sample mean of $x$ and $y$, respectively.

(If you struggle to compute $\widehat\beta_0$ and $\widehat\beta_1$ by hand, see this Shiny app which helps you to easily find these estimates based on your data.)

Interpretations of coefficients $\widehat\beta$

The intercept $\widehat\beta_0$ is the mean value of the dependent variable $Y$ when the independent variable $X$ takes the value 0. Its estimation has no interest in evaluating whether there is a linear relationship between two variables. It has, however, an interest if you want to know what the mean value of $Y$ could be when $x = 0$.⁴

The slope $\widehat\beta_1$, on the other hand, corresponds to the expected variation of $Y$ when $X$ varies by one unit. It tells us two important informations:

The sign of the slope indicates the direction of the line—a positive slope ($\widehat\beta_1 > 0$) indicates that there is a positive relationship between the two variables of interest (they vary in the same direction), whereas a negative slope ($\widehat\beta_1 < 0$) means that there is a negative relationship between the two variables (they vary in opposite directions).
The value of the slope provides information on the speed of evolution of the variable $Y$ as a function of the variable $X$. The larger the slope in absolute value, the larger the expected variation of $Y$ for each unit of $X$. Note, however, that a large value does not necessarily mean that the relationship is statistically significant (more on that in the section about significance of the relationship).

This is similar to the correlation coefficient, which gives information about the direction and the strength of the relationship between two variables.

To perform a linear regression in R, we use the lm() function (which stands for linear model). The function requires to set the dependent variable first then the independent variable, separated by a tilde (~).

Applied to our example of weight and car’s consumption, we have:

model <- lm(mpg ~ wt, data = dat)

The summary() function gives the results of the model:

summary(model)

## 
## Call:
## lm(formula = mpg ~ wt, data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
## wt           -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528,	Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

In practice, we usually check the conditions of application before interpreting the coefficients (because if they are not respected, results may be biased).

In this article, however, I present the interpretations before testing the conditions because the point is to show how to interpret the results, and less about finding a valid model.

The results can be summarized as follows (see the column Estimate in the table Coefficients):

The intercept $\widehat\beta_0 =$ 37.29 indicates that, for a hypothetical car weighting 0 lbs, we can expect, on average, a consumption of 37.29 miles/gallon. This interpretation is shown for illustrative purposes, but as a car weighting 0 lbs is impossible, the interpretation has no meaning. In practice, we would therefore refrain from interpreting the intercept in this case. See another interpretation of the intercept when the independent variable is centered around its mean in this section.
The slope $\widehat\beta_1 =$ -5.34 indicates that:
- There is a negative relationship between the weight and the distance a car can drive with a gallon (this was expected given the negative trend of the points in the scatterplot shown previously).
- But more importantly, a slope of -5.34 means that, for an increase of one unit in the weight (that is, an increase of 1000 lbs), the number of miles per gallon decreases, on average, by 5.34 units. In other words, for an increase of 1000 lbs, the number of miles/gallon decreases, on average, by 5.34.

Another interpretation of the intercept

Another useful interpretation of the intercept is when the independent variable is centered around its mean. In this case, the intercept is interpreted as the mean value of $Y$ for individuals who have a value of $X$ equal to the mean of $X$.

Let’s see it in practice.

We first center the wt variable around the mean then rerun a simple linear model with this new variable:

dat_centered <- dat

dat_centered$wt_centered <- dat$wt - mean(dat$wt)

mod_centered <- lm(mpg ~ wt_centered,
  data = dat_centered
)

summary(mod_centered)

## 
## Call:
## lm(formula = mpg ~ wt_centered, data = dat_centered)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.5432 -2.3647 -0.1252  1.4096  6.8727 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  20.0906     0.5384  37.313  < 2e-16 ***
## wt_centered  -5.3445     0.5591  -9.559 1.29e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.046 on 30 degrees of freedom
## Multiple R-squared:  0.7528,	Adjusted R-squared:  0.7446 
## F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

Based on the results, we see that:

The slope has not changed, the interpretation is the same than without the centering (which makes sense since the regression line has simply been shifted to the right or left).
More importantly, the intercept is now $\widehat\beta_0 =$ 20.09, so we can expect, on average, a consumption of 20.09 miles/gallon for a car with an average weight (the mean of weight is 3.22 so 3220 lbs).

This centering is particularly interesting:

when the continuous independent variable has no meaningful value of 0 (which is the case here as a car with a weight of 0 lbs is not meaningful), or
when interpreting the intercept is important.

Note that centering does not have to be done around the mean only. The independent variable can also be centered at some value that is actually in the range of the data. The exact value you center on does not matter as long it’s meaningful and within the range of data (it is not recommended to center it on a value that is not in the range of the data because we are not sure about the type of relationship between the two variables outside that range).

For our example, we may find that choosing the lowest value or the highest value of weight is the best option. So it’s up to us to decide the weight at which it’s most meaningful to interpret the intercept.

Significance of the relationship

As mentioned earlier, the value of the slope does not, by itself, make it possible to assess the significance of the linear relationship.

In other words, a slope different from 0 does not necessarily mean it is significantly different from 0, so it does not mean that there is a significant relationship between the two variables in the population. There could be a slope of 10 that is not significant, and a slope of 2 that is significant.

Significance of the relationship also depends on the variability of the slope, which is measured by its standard error and generally noted $se(\widehat\beta_1)$.

Without going too much into details, to assess the significance of the linear relationship, we divide the slope by its standard error. This ratio is the test statistic and follows a Student distribution with $n - 2$ degrees of freedom:⁵

\[T_{n - 2} = \frac{\widehat\beta_1}{se(\widehat\beta_1)}\]

For a bilateral test, the null and alternative hypotheses are:⁶

$H_0 : \beta_1 = 0$ (there is no (linear) relationship between the two variables)
$H_1 : \beta_1 \ne 0$ (there is a (linear) relationship between the two variables)

Roughly speaking, if this ratio is greater than 2 in absolute value then the slope is significantly different from 0, and therefore the relationship between the two variables is significant (and in that case it is positive or negative depending on the sign of the estimate $\widehat\beta_1$).

The standard error and the test statistic are shown in the column Std. Error and t value in the table Coefficients.

Fortunately, R gives a more precise and easier way to assess to the significance of the relationship. The information is provided in the column Pr(>|t|) of the Coefficients table. This is the p-value of the test. As for any statistical test, if the p-value is greater than or equal to the significance level (usually $\alpha = 0.05$), we do not reject the null hypothesis, and if the p-value is lower than the significance level, we reject the null hypothesis.

If we do not reject the null hypothesis, we do not reject the hypothesis of no relationship between the two variables (because we do not reject the hypothesis of a slope of 0). On the contrary, if we reject the null hypothesis of no relationship, we can conclude that there is a significant linear relationship between the two variables.

In our example, the p-value = 1.29e-10 < 0.05 so we reject the null hypothesis at the significance level $\alpha = 5\%$. We therefore conclude that there is a significant relationship between a car’s weight and its fuel consumption.

Tip: In order to make sure I interpret only parameters that are significant, I tend to first check the significance of the parameters thanks to the p-values, and then interpret the estimates accordingly. For completeness, note that the test is also performed on the intercept. The p-value being smaller than 0.05, we also conclude that the intercept is significantly different from 0.

Correlation does not imply causation

Be careful that a significant relationship between two variables does not necessarily mean that there is an influence of one variable on the other or that there is a causal effect between these two variables!

A significant relationship between $X$ and $Y$ can appear in several cases:

$X$ causes $Y$
$Y$ causes $X$
a third variable cause $X$ and $Y$
a combination of these three reasons

A statistical model alone cannot establish a causal link between two variables. Demonstrating causality between two variables is more complex and requires, among others, a specific experimental design, the repeatability of the results over time, as well as various samples.

This is the reason you will often read “Correlation does not imply causation” and linear regression follows the same principle.

Conditions of application

Unfortunately, linear regression cannot be used in all situations.

In addition to the requirement that the dependent variable must be a continuous quantitative variables, simple linear regression requires that the data satisfy the following conditions:

Linearity: The relationship between the two variables should be linear (at least roughly). For this reason it is always necessary to represent graphically the data with a scatterplot before performing a simple linear regression.⁷

Independence: Observations must be independent. It is the sampling plan and the experimental design that usually provide information on this condition. If the data come from different individuals or experimental units, they are usually independent. On the other hand, if the same individuals are measured at different periods, the data are probably not independent.
Normality of the residuals: For large sample sizes, confidence intervals and tests on the coefficients are (approximately) valid whether the error follows a normal distribution or not (a consequence of the central limit theorem, see more in Ernst and Albers (2017) and Lumley et al. (2002))! For small sample sizes, residuals should follow a normal distribution. This condition can be tested visually (via a QQ-plot and/or a histogram), or more formally (via the Shapiro-Wilk test for instance).
Homoscedasticity of the residuals: The variance of the errors should be constant. There is a lack of homoscedasticity when the dispersion of the residuals increases with the predicted values (fitted values). This condition can be tested visually (by plotting the standardized residuals vs. the fitted values) or more formally (via the Breusch-Pagan test).
No influential points: If the data contain outliers, it is essential to identify them so that they do not, on their own, influence the results of the regression. Note that an outlier is not an issue per se if the point is in the alignment of the regression line for example because it does not influence the regression line. It becomes a problem in the context of linear regression if it influences in a substantial manner the estimates (and in particular the slope of the regression line). This can be tackled by identifying outliers (via the Cook’s distance⁸ or the leverage index⁹ for instance), and comparing the results with and without the potential outliers. Do the results remain the same with the two approaches? If yes, outliers are not really an issue in this case. If results are much different, you can use the Theil-Sen estimator, robust regression or quantile regression which are all more robust to outliers.

Tip: I remember the first 4 conditions thanks to the acronym “LINE”, for Linearity, Independence, Normality and Equality of variance.

If any of the condition is not met, the tests and the conclusions could be erroneous so it is best to avoid using and interpreting the model. If this is the case, sometimes the conditions can be met by transforming the data (e.g., logarithmic transformation, square or square root, Box-Cox transformation, etc.) or by adding a quadratic or cubic (or even a higher-order polynomial) term to the model.

If it does not help, it could be worth thinking about removing some variables or adding other variables, or even considering other types of models such as non-linear models.

Keep in mind that in practice, conditions of application should be verified before drawing any conclusion based on the model. I refrain here from testing the conditions on our data because it will be covered in details in the context of multiple linear regression (see this section).

Visualizations

If you are a frequent reader of the blog, you may know that I like to draw (simple but efficient) visualizations to illustrate my statistical analyses. Linear regression is not an exception.

There are numerous ways to visualize the relationship between the two variables of interest, but the easiest one I found so far is via the visreg() function from the package of the same name:

library(visreg)
visreg(model)

I like this approach for its simplicity—only a single line of code.

However, other elements could be displayed on the regression plot (for example the regression equation and the $R^2$). This can easily be done with the stat_regline_equation() and stat_cor() functions from the {ggpubr} package:

# load necessary libraries
library(ggpubr)

# create plot with regression line, regression equation and R^2
ggplot(dat, aes(x = wt, y = mpg)) +
  geom_smooth(method = "lm") +
  geom_point() +
  stat_regline_equation(label.x = 3, label.y = 32) + # for regression equation
  stat_cor(aes(label = after_stat(rr.label)), label.x = 3, label.y = 30) + # for R^2
  theme_minimal()

Multiple linear regression

Now that you understand the principle behind simple linear regression and you know how to interpret the results, it is time to discuss about multiple linear regression.

We also start with the underlying principle of multiple linear regression, then show how to interpret the results, how to test the conditions of application and finish with more advanced topics.

Principle

Multiple linear regression is a generalization of simple linear regression, in the sense that this approach makes it possible to relate one variable with several variables through a linear function in its parameters.

Multiple linear regression is used to assess the relationship between two variables while taking into account the effect of other variables. By taking into account the effect of other variables, we cancel out the effect of these other variables in order to isolate and measure the relationship between the two variables of interest. This point is the main difference with simple linear regression.

To illustrate how to perform a multiple linear regression in R, we use the same dataset than the one used for simple linear regression (mtcars). Below a short preview:

head(dat)

##                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2
## Valiant           18.1   6  225 105 2.76 3.460 20.22  1  0    3    1

We have seen that there is a significant and negative linear relationship between the distance a car can drive with a gallon and its weight ($\widehat\beta_1 =$ -5.34, $p$-value < 0.001).

However, one may wonder whether there are not in reality other factors that could explain a car’s fuel consumption.

To explore this, we can visualize the relationship between a car’s fuel consumption (mpg) together with its weight (wt), horsepower (hp) and displacement (disp) (engine displacement is the combined swept (or displaced) volume of air resulting from the up-and-down movement of pistons in the cylinders, usually the higher the more powerful the car):

ggplot(dat) +
  aes(x = wt, y = mpg, colour = hp, size = disp) +
  geom_point() +
  scale_color_gradient() +
  labs(
    y = "Miles per gallon",
    x = "Weight (1000 lbs)",
    color = "Horsepower",
    size = "Displacement"
  ) +
  theme_minimal()

It seems that, in addition to the negative relationship between miles per gallon and weight, there is also:

a negative relationship between miles/gallon and horsepower (lighter points, indicating more horsepower, tend to be more present in low levels of miles per gallon)
a negative relationship between miles/gallon and displacement (bigger points, indicating larger values of displacement, tend to be more present in low levels of miles per gallon).

Therefore, we would like to evaluate the relation between the fuel consumption and the weight, but this time by adding information on the horsepower and displacement. By adding this additional information, we are able to capture only the direct relationship between miles/gallon and weight (the indirect effect due to horsepower and displacement is canceled out).

This is the whole point of multiple linear regression! In fact, in multiple linear regression, the estimated relationship between the dependent variable and an explanatory variable is an adjusted relationship, that is, free of the linear effects of the other explanatory variables.

Let’s illustrate this notion of adjustment by adding both horsepower and displacement in our linear regression model:

model2 <- lm(mpg ~ wt + hp + disp,
  data = dat
)

summary(model2)

## 
## Call:
## lm(formula = mpg ~ wt + hp + disp, data = dat)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.891 -1.640 -0.172  1.061  5.861 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.105505   2.110815  17.579  < 2e-16 ***
## wt          -3.800891   1.066191  -3.565  0.00133 ** 
## hp          -0.031157   0.011436  -2.724  0.01097 *  
## disp        -0.000937   0.010350  -0.091  0.92851    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.639 on 28 degrees of freedom
## Multiple R-squared:  0.8268,	Adjusted R-squared:  0.8083 
## F-statistic: 44.57 on 3 and 28 DF,  p-value: 8.65e-11

We can see that now, the relationship between miles/gallon and weight is weaker in terms of slope ($\widehat\beta_1 =$ -3.8 now, against $\widehat\beta_1 =$ -5.34 when only the weight was considered).

The effect of weight on fuel consumption was adjusted according to the effect of horsepower and displacement. This is the remaining effect between miles/gallon and weight after the effects of horsepower and displacement have been taken into account. More detailed interpretations in this section.

Equation

Multiple linear regression models are defined by the equation

\[Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_p X_p + \epsilon\]

It is similar than the equation of simple linear regression, except that there is more than one independent variables ($X_1, X_2, \dots, X_p$).

Estimation of the parameters $\beta_0, \dots, \beta_p$ by the method of least squares is based on the same principle as that of simple linear regression, but applied to $p$ dimensions. It is thus no longer a question of finding the best line (the one which passes closest to the pairs of points ($y_i, x_i$)), but finding the $p$-dimensional plane which passes closest to the coordinate points ($y_i, x_{i1}, \dots, x_{ip}$).

This is done by minimizing the sum of the squares of the deviations of the points on the plane:

Source: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013)

Interpretations of coefficients $\widehat\beta$

The least squares method results in an adjusted estimate of the coefficients. The term adjusted means after taking into account the linear effects of the other independent variables on the dependent variable, but also on the predictor variable.

In other words, the coefficient $\beta_1$ corresponds to the slope of the relationship between $Y$ and $X_1$ when the linear effects of the other explanatory variables ($X_2, \dots, X_p$) have been removed, both at the level of the dependent variable $Y$ but also at the level of $X_1$.

Applied to our model with weight, horsepower and displacement as independent variables, we have:

summary(model2)

## 
## Call:
## lm(formula = mpg ~ wt + hp + disp, data = dat)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.891 -1.640 -0.172  1.061  5.861 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.105505   2.110815  17.579  < 2e-16 ***
## wt          -3.800891   1.066191  -3.565  0.00133 ** 
## hp          -0.031157   0.011436  -2.724  0.01097 *  
## disp        -0.000937   0.010350  -0.091  0.92851    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.639 on 28 degrees of freedom
## Multiple R-squared:  0.8268,	Adjusted R-squared:  0.8083 
## F-statistic: 44.57 on 3 and 28 DF,  p-value: 8.65e-11

The table Coefficients gives the estimate for each parameter (column Estimate), together with the $p$-value of the nullity of the parameter (column Pr(>|t|)).

The hypotheses are the same as for simple linear regression, that is:

$H_0 : \beta_j = 0$
$H_1 : \beta_j \ne 0$

The test of $\beta_j = 0$ is equivalent to testing the hypothesis: is the dependent variable associated with the independent variable studied, all other things being equal, that is to say, at constant level of the other independent variables.

In other words:

the test of $\beta_1 = 0$ corresponds to testing the hypothesis: is fuel consumption associated with a car’s weight, at a constant level of horsepower and displacement
the test of $\beta_2 = 0$ corresponds to testing the hypothesis: is fuel consumption associated with horsepower, at a constant level of weight and displacement
the test of $\beta_3 = 0$ corresponds to testing the hypothesis: is fuel consumption associated with displacement, at a constant level of weight and horsepower
(for the sake of completeness: the test of $\beta_0 = 0$ corresponds to testing the hypothesis: is miles/gallon different from 0 when weight, horsepower and displacement are equal to 0)

In practice, we usually check the conditions of application before interpreting the coefficients (because if they are not respected, results may be biased). In this article, however, I present the interpretations before testing the conditions because the point is to show how to interpret the results, and less about finding a valid model.

Based on the output of our model, we conclude that:

There is a significant and negative relationship between miles/gallon and weight, all else being equal. So for an increase of one unit in the weight (that is, an increase of 1000 lbs), the number of miles/gallon decreases, on average, by 3.8, for a constant level of horsepower and displacement ($p$-value = 0.001).
There is a significant and negative relationship between miles/gallon and horsepower, all else being equal. So for an increase of one unit of horsepower, the distance traveled with a gallon decreases, on average, by 0.03 mile, for a constant level of weight and displacement ($p$-value = 0.011).
We do not reject the hypothesis of no relationship between miles/gallon and displacement when weight and horsepower stay constant (because $p$-value = 0.929 > 0.05).
(For completeness but it should be interpreted only when it makes sense: for a weight, horsepower and displacement = 0, we can expect that a car has, on average, a fuel consumption of 37.11 miles/gallon ($p$-value < 0.001). See a more useful interpretation of the intercept when the independent variables are centered in this section.)

This is how to interpret quantitative independent variables. Interpreting qualitative independent variables is slightly different in the sense that it quantifies the effect of a level in comparison with the reference level, sill all else being equal.

So it compares the different groups (formed by the different levels of the categorical variable) in terms of the dependent variable (this is why linear regression can be seen as an extension to the t-test and ANOVA).

For the illustration, we model the fuel consumption (mpg) on the weight (wt) and the shape of the engine (vs). The variable vs has two levels: V-shaped (the reference level) and straight engine.¹⁰

## Recoding dat$vs
library(forcats)
dat$vs <- as.character(dat$vs)
dat$vs <- fct_recode(dat$vs,
  "V-shaped" = "0",
  "Straight" = "1"
)

model3 <- lm(mpg ~ wt + vs,
  data = dat
)

summary(model3)

## 
## Call:
## lm(formula = mpg ~ wt + vs, data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.7071 -2.4415 -0.3129  1.4319  6.0156 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  33.0042     2.3554  14.012 1.92e-14 ***
## wt           -4.4428     0.6134  -7.243 5.63e-08 ***
## vsStraight    3.1544     1.1907   2.649   0.0129 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.78 on 29 degrees of freedom
## Multiple R-squared:  0.801,	Adjusted R-squared:  0.7873 
## F-statistic: 58.36 on 2 and 29 DF,  p-value: 6.818e-11

Based on the output of our model, we conclude that:

For a V-shaped engine and for an increase of one unit in the weight (that is, an increase of 1000 lbs), the number of miles/gallon decreases, on average, by 4.44 ($p$-value < 0.001).
The distance traveled with a gallon of fuel increases by, on average, 3.15 miles when the engine is straight compared to a V-shaped engine, for a constant weight ($p$-value = 0.013).
(For completeness but it should be interpreted only when it makes sense: for a weight = 0 and a V-shaped engine, we can expect that the car has, on average, a fuel consumption of 33 miles/gallon ($p$-value < 0.001). See a more useful interpretation of the intercept when the independent variables are centered in this section.)

Conditions of application

As for simple linear regression, multiple linear regression requires some conditions of application for the model to be usable and the results to be interpretable. Conditions for simple linear regression also apply to multiple linear regression, that is:

Linearity of the relationships between the dependent and independent variables¹¹
Independence of the observations
Normality of the residuals
Homoscedasticity of the residuals
No influential points (outliers)

But there is one more condition for multiple linear regression:

No multicollinearity: Multicollinearity arises when there is a strong linear correlation between the independent variables, conditional on the other variables in the model. It is important to check it because it may lead to an imprecision or an instability of the estimated parameters when a variable changes. It can be assessed by studying the correlation between each pair of independent variables, or even better, by computing the variance inflation factor (VIF). The VIF measures how much the variance of an estimated regression coefficient increases, relative to a situation in which the explanatory variables are strictly independent. A high value of VIF is a sign of multicollinearity (the threshold is generally admitted at 5 or 10 depending on the domain). The easiest way to reduce the VIF is to remove some correlated independent variables, or eventually to standardize the data.

You will often see that these conditions are verified by running plot(model, which = 1:6) and it is totally correct. However, I recently discovered the check_model() function from the {performance} package which tests these conditions all at the same time (and let’s be honest, in a more elegant way).¹²

Applied on our model2 with miles/gallon as dependent variable, and weight, horsepower and displacement as independent variables, we have:

# install.packages("performance")
# install.packages("see")
library(performance)

check_model(model2)

In addition to testing all conditions at the same time, it also gives insight on how to interpret the different diagnostic plots and what you should expect (see in the subtitles of each plot).

Based on these diagnostic plots, we see that:

Homogeneity of variance (middle left plot) is respected
Multicollinearity (bottom left plot) is not an issue (I tend to use the threshold of 10 for VIF, and all of them are below 10)¹³
There is no influential points (middle right plot)
Normality of the residuals (bottom right plot) is also not perfect due to 3 points deviating from the reference line but it still seems acceptable to me. In any case, the number of observations is large enough given the number of parameters¹⁴ and given the small deviation from normality so tests on the coefficients are (approximately) valid whether the error follows a normal distribution or not
Linearity (top right plot) is not perfect so let’s check each independent variable separately:

# weight
ggplot(dat, aes(x = wt, y = mpg)) +
  geom_point() +
  theme_minimal()

# horsepower
ggplot(dat, aes(x = hp, y = mpg)) +
  geom_point() +
  theme_minimal()

# displacement
ggplot(dat, aes(x = disp, y = mpg)) +
  geom_point() +
  theme_minimal()

It seems that the relationship between miles/gallon and horsepower is not linear, which could be the main component of the slight linearity defect of the model.

To improve linearity, the variable could be removed, a transformation could be applied (logarithmic and/or squared for instance) or a quadratic or cubic (or even a higher-order polynomial) term could be added to the model.¹⁵ If this does not fix the issue of linearity, other types of models could be considered.

If you want to read more about these conditions of applications and how to deal with them, here is a very complete chapter on diagnostics for linear models written by Prof. Dustin Fife.

For the sake of easiness and for illustrative purposes, we assume linearity for the rest of the article.

When the conditions of application are met, we usually say that the model is valid. But not all valid models are good models. The next section deals with model selection.

How to choose a good linear model?

A model which satisfies the conditions of application is the minimum requirement, but you will likely find several models that meet this criteria. So one may wonder how to choose between different models that are all valid?

The three most common tools to select a good linear model are according to:

the $p$-value associated to the model,
the coefficient of determination $R^2$ and
the Akaike Information Criterion

The approaches are detailed in the next sections. Note that the first two are applicable to simple and multiple linear regression, whereas the third is only applicable to multiple linear regression.

$P$-value associated to the model

Before interpreting the estimates of a model, it is a good practice to first check the $p$-value associated to the model. This $p$-value indicates if the model is better than a model with only the intercept.

The hypotheses of the test (called F-test) are:

$H_0: \beta_1 = \beta_2 = \dots = \beta_p = 0$
$H_1:$ at least one coefficient $\beta \ne 0$

This $p$-value can be found at the bottom of the summary() output:

summary(model2)

## 
## Call:
## lm(formula = mpg ~ wt + hp + disp, data = dat)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.891 -1.640 -0.172  1.061  5.861 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.105505   2.110815  17.579  < 2e-16 ***
## wt          -3.800891   1.066191  -3.565  0.00133 ** 
## hp          -0.031157   0.011436  -2.724  0.01097 *  
## disp        -0.000937   0.010350  -0.091  0.92851    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.639 on 28 degrees of freedom
## Multiple R-squared:  0.8268,	Adjusted R-squared:  0.8083 
## F-statistic: 44.57 on 3 and 28 DF,  p-value: 8.65e-11

The $p$-value = 8.65e-11. The null hypothesis is rejected, so we conclude that our model is better than a model with only the intercept because at least one coefficient $\beta$ is significantly different from 0.

If this $p$-value > 0.05 for one of your model, it means that none of the variables you selected help in explaining the dependent variable. In other words, you should completely forget about this model because it cannot do better than simply taking the mean of the dependent variable.

Coefficient of determination $R^2$

The coefficient of determination, $R^2$, is a measure of the goodness of fit of the model. It measures the proportion of the total variability that is explained by the model, or how well the model fits the data.

$R^2$ varies between 0 and 1:

$R^2 = 0$: the model explains nothing
$R^2 = 1$: the model explains everything
$0 < R^2 < 1$: the model explains part of the variability
the higher the $R^2$, the better the model explains the dependent variable. As a rule of thumb, a $R^2 > 0.7$ indicates a good fit of the model¹⁶

Note that in a simple linear regression model, the coefficient of determination is equal to the square of the Pearson correlation coefficient:

\[R^2 = corr(X, Y)^2\]

Applied on our model2 with miles/gallon as dependent variable, and weight, horsepower and displacement as independent variables, we have:

summary(model2)

## 
## Call:
## lm(formula = mpg ~ wt + hp + disp, data = dat)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.891 -1.640 -0.172  1.061  5.861 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 37.105505   2.110815  17.579  < 2e-16 ***
## wt          -3.800891   1.066191  -3.565  0.00133 ** 
## hp          -0.031157   0.011436  -2.724  0.01097 *  
## disp        -0.000937   0.010350  -0.091  0.92851    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.639 on 28 degrees of freedom
## Multiple R-squared:  0.8268,	Adjusted R-squared:  0.8083 
## F-statistic: 44.57 on 3 and 28 DF,  p-value: 8.65e-11

$R^2$ is displayed at the bottom of the summary() output or can be extracted with summary(model2)$r.squared.

$R^2$ for this model is 0.8268, which means that 82.68% of the variability of the distance traveled with a gallon is explained by the weight, horsepower and displacement of the car. The relatively high $R^2$ means that the weight, horsepower and displacement of a car are good characteristics to explain the distance it can drive with a gallon of fuel.

Note that if you want to compare models with different number of independent variables, it is best to refer to the adjusted $R^2$ (= 0.8083 here).

Indeed, adding variables to the model cannot make the $R^2$ to decrease, even if the variables are not related to the dependent variables (so the $R^2$ will artificially increase when adding variables to the model, or at least stay constant). Therefore, the adjusted $R^2$ takes into account the complexity of the model (the number of variables) by penalizing for additional variables, so it is a compromise between goodness of fit and parsimony.

Parsimony

A parsimonious model (few variables) is usually preferred over a complex model (many variables). There are two ways to obtain a parsimonious model from a model with many independent variables:

We can iteratively remove the independent variable least significantly related to the dependent variable (i.e., the one with the highest $p$-value in an analysis of variance table) until all of them are significantly associated to the response variable, or
We can select the model based on the Akaike Information Criterion (AIC). AIC expresses a desire to fit the model with the smallest number of coefficients possible and allows to compare models. According to this criterion, the best model is the one with the lowest AIC. This criterion is based on a compromise between the quality of the fit and its complexity. We usually start from a global model with many independent variables, and the procedure (referred as stepwise algorithm)¹⁷ automatically compares models then selects the best one according to the AIC.

We show how to do the second option in R. For the illustration, we start with a model with all variables in the dataset as independent variables (do not forget to transform the factor variables first):

## vs has already been transformed into factor
## so only am is transformed here

## Recoding dat$vs
library(forcats)
dat$am <- as.character(dat$am)
dat$am <- fct_recode(dat$am,
  "Automatic" = "0",
  "Manual" = "1"
)

model4 <- lm(mpg ~ .,
  data = dat
)

model4 <- step(model4, trace = FALSE)

(Tip: The formula mpg ~ . is a shortcut to consider all variables present in the dataset as independent variables, except the one that has been specified as the dependent variable (mpg here)).

The model that has been selected according to this criterion is the following:

summary(model4)

## 
## Call:
## lm(formula = mpg ~ wt + qsec + am, data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.4811 -1.5555 -0.7257  1.4110  4.6610 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   9.6178     6.9596   1.382 0.177915    
## wt           -3.9165     0.7112  -5.507 6.95e-06 ***
## qsec          1.2259     0.2887   4.247 0.000216 ***
## amManual      2.9358     1.4109   2.081 0.046716 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.459 on 28 degrees of freedom
## Multiple R-squared:  0.8497,	Adjusted R-squared:  0.8336 
## F-statistic: 52.75 on 3 and 28 DF,  p-value: 1.21e-11

Be careful when using an automatic procedure because, even though it is the best model that is selected, it is based:

on a single criterion (AIC in this case), but more importantly;
it is based on some set of mathematical rules, which means that industry knowledge or human expertise is not taken into consideration.

I believe that this kind of automatic procedure for model’s selection is a good starting point, but I also believe that the final model should always be checked and tested against other models to make sure it makes sense in practice (apply common sense).

Last but not least, do not forget to also verify the conditions of application because the stepwise procedure does not guarantee that they are respected.

Visualizations

There are many ways to visualize results of a linear regression. The easiest ones I am aware of are:

visreg() illustrates the relationships between the dependent and independent variables in different plots (one for each independent variable unless you specify which relationship you want to illustrate):

library(visreg)

visreg(model4)

ggcoefstats() illustrates the results in one single plot, with many statistical details:

library(ggstatsplot)

ggcoefstats(model4)

In this plot:

when the solid line does not cross the vertical dashed line, the estimates is significantly different from 0 at the 5% significance level (i.e., $p$-value < 0.05)
furthermore, a point to the right (left) of the vertical dashed line means that there is a positive (negative) relationship between the two variables
the more extreme the point, the stronger the relationship

plot_summs() which also illustrates the results but in a more concise way:

library(jtools)
library(ggstance)

plot_summs(model4,
  omit.coefs = NULL
)

The advantage of this approach is that it is possible to compare coefficients of multiple models simultaneously (particularly interesting when the models are nested):

model4bis <- lm(mpg ~ wt + qsec + am + hp,
  data = dat
)

plot_summs(model4,
  model4bis,
  omit.coefs = NULL
)

To go further

Below some more advanced topics related to linear regression. Feel free to comment at the end of the article if you believe I missed an important one.

Print model’s parameters

Thanks to the model_parameters() function from the {parameters} package, you can print a summary of the model in a nicely formatted way to make the output more readable:

library(parameters)

model_parameters(model4, summary = TRUE)

## Parameter   | Coefficient |   SE |         95% CI | t(28) |      p
## ------------------------------------------------------------------
## (Intercept) |        9.62 | 6.96 | [-4.64, 23.87] |  1.38 | 0.178 
## wt          |       -3.92 | 0.71 | [-5.37, -2.46] | -5.51 | < .001
## qsec        |        1.23 | 0.29 | [ 0.63,  1.82] |  4.25 | < .001
## am [Manual] |        2.94 | 1.41 | [ 0.05,  5.83] |  2.08 | 0.047

And if you are using R Markdown, you can use the print_html() function to get a compact and yet comprehensive summary table in your HTML file:

library(gt)

print_html(model_parameters(model4, summary = TRUE))

Parameter	Coefficient	SE	95% CI	t(28)	p
(Intercept)	9.62	6.96	(-4.64, 23.87)	1.38	0.178
wt	-3.92	0.71	(-5.37, -2.46)	-5.51	< .001
qsec	1.23	0.29	(0.63, 1.82)	4.25	< .001
am (Manual)	2.94	1.41	(0.05, 5.83)	2.08	0.047

Automatic reporting

The report() function from the package of the same name allows to automatically produces reports of models according to best practices guidelines:

library(report)

report(model4)[1]

## [1] "We fitted a linear model (estimated using OLS) to predict mpg with wt, qsec and am (formula: mpg ~ wt + qsec + am). The model explains a statistically significant and substantial proportion of variance (R2 = 0.85, F(3, 28) = 52.75, p < .001, adj. R2 = 0.83). The model's intercept, corresponding to wt = 0, qsec = 0 and am = Automatic, is at 9.62 (95% CI [-4.64, 23.87], t(28) = 1.38, p = 0.178). Within this model:\n\n  - The effect of wt is statistically significant and negative (beta = -3.92, 95% CI [-5.37, -2.46], t(28) = -5.51, p < .001; Std. beta = -0.64, 95% CI [-0.87, -0.40])\n  - The effect of qsec is statistically significant and positive (beta = 1.23, 95% CI [0.63, 1.82], t(28) = 4.25, p < .001; Std. beta = 0.36, 95% CI [0.19, 0.54])\n  - The effect of am [Manual] is statistically significant and positive (beta = 2.94, 95% CI [0.05, 5.83], t(28) = 2.08, p = 0.047; Std. beta = 0.49, 95% CI [7.59e-03, 0.97])\n\nStandardized parameters were obtained by fitting the model on a standardized version of the dataset. 95% Confidence Intervals (CIs) and p-values were computed using a Wald t-distribution approximation."

Note that the function also works for dataframes, statistical tests and other models.

Predictions

Linear regression is also very often used for predictive purposes. Confidence and prediction intervals for new data can be computed with the predict() function.

Suppose we want to predict the miles/gallon for a car with a manual transmission, weighting 3000 lbs and which drives a quarter of a mile (qsec) in 18 seconds:

# confidence interval for new data
predict(model4,
  new = data.frame(wt = 3, qsec = 18, am = "Manual"),
  interval = "confidence",
  level = .95
)

##        fit      lwr    upr
## 1 22.87005 21.09811 24.642

# prediction interval for new data
predict(model4,
  new = data.frame(wt = 3, qsec = 18, am = "Manual"),
  interval = "prediction",
  level = .95
)

##        fit      lwr      upr
## 1 22.87005 17.53074 28.20937

Based on our model, it is expected that this car will drive 22.87 miles with a gallon.

The difference between the confidence and prediction interval is that:

a confidence interval gives the predicted value for the mean of $Y$ for a new observation, whereas
a prediction interval gives the predicted value for an individual $Y$ for a new observation.

The prediction interval is wider than the confidence interval to account for the additional uncertainty due to predicting an individual response, and not the mean, for a given value of $X$.

Linear hypothesis tests

Linear hypothesis tests make it possible to generalize the F-test mentioned in this section, while offering the possibility to perform either tests of comparison of coefficients, or tests of equality of linear combinations of coefficients.

For example, to test the linear constraint:

$H_0: \beta_1 = \beta_2 = 0$
$H_1:$ not $H_0$

we use the linearHypothesis() function of the {car} package as follows:

library(car)
linearHypothesis(model4, c("wt = 0", "qsec = 0"))

## 
## Linear hypothesis test:
## wt = 0
## qsec = 0
## 
## Model 1: restricted model
## Model 2: mpg ~ wt + qsec + am
## 
##   Res.Df    RSS Df Sum of Sq      F   Pr(>F)    
## 1     30 720.90                                 
## 2     28 169.29  2    551.61 45.618 1.55e-09 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

We reject the null hypothesis and we conclude that at least one of $\beta_1$ and $\beta_2$ is different from 0 ($p$-value = 1.55e-09).

Overall effect of categorical variables

When the independent variables are categorical with $k$ categories, the regression table provides $k-1$ $p$-values:

model5 <- lm(mpg ~ vs + am + as.factor(cyl),
  data = dat
)

summary(model5)

## 
## Call:
## lm(formula = mpg ~ vs + am + as.factor(cyl), data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.2821 -1.4402  0.0391  1.8845  6.2179 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       22.809      2.928   7.789 2.24e-08 ***
## vsStraight         1.708      2.235   0.764  0.45135    
## amManual           3.165      1.528   2.071  0.04805 *  
## as.factor(cyl)6   -5.399      1.837  -2.938  0.00668 ** 
## as.factor(cyl)8   -8.161      2.892  -2.822  0.00884 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.097 on 27 degrees of freedom
## Multiple R-squared:  0.7701,	Adjusted R-squared:  0.736 
## F-statistic: 22.61 on 4 and 27 DF,  p-value: 2.741e-08

The variables vs and am have 2 levels so one is displayed in the regression output. The variable cyl has 3 levels (4, 6 and 8) so 2 of them are displayed. The overall effect of vs and am are reported in the Pr(>|t|) column, but not the overall effect of cyl because there are more than 2 levels for this variable.

To get the $p$-value of the overall effect of a categorical variable, we need to get an analysis of variance table via the Anova() function from the {car} package:¹⁸

library(car)
Anova(model5)

## Anova Table (Type II tests)
## 
## Response: mpg
##                 Sum Sq Df F value  Pr(>F)  
## vs               5.601  1  0.5841 0.45135  
## am              41.122  1  4.2886 0.04805 *
## as.factor(cyl)  94.591  2  4.9324 0.01493 *
## Residuals      258.895 27                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

From this analysis of variance table, we conclude that:

vs is not significantly associated with mpg ($p$-value = 0.451)
am and cyl are significantly associated with mpg ($p$-values < 0.05)

Interaction

So far we have covered multiple linear regression without any interaction.

There is an interaction effect between factors A and B if the effect of factor A on the response depends on the level taken by factor B.

In R, interaction can be added as follows:

model6 <- lm(mpg ~ wt + am + wt:am,
  data = dat
)

# Or in a shorter way:
model6 <- lm(mpg ~ wt * am,
  data = dat
)

summary(model6)

## 
## Call:
## lm(formula = mpg ~ wt * am, data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.6004 -1.5446 -0.5325  0.9012  6.0909 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  31.4161     3.0201  10.402 4.00e-11 ***
## wt           -3.7859     0.7856  -4.819 4.55e-05 ***
## amManual     14.8784     4.2640   3.489  0.00162 ** 
## wt:amManual  -5.2984     1.4447  -3.667  0.00102 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.591 on 28 degrees of freedom
## Multiple R-squared:  0.833,	Adjusted R-squared:  0.8151 
## F-statistic: 46.57 on 3 and 28 DF,  p-value: 5.209e-11

From the output we conclude that there is an interaction between the weight and the transmission ($p$-value = 0.00102). This means that the effect of the weight on the distance traveled with a gallon depends on the transmission type.

The easiest way to handle interaction is to visualize the relationship for each level of the categorical variable:

visreg(model6, "wt", by = "am")

We see that the relationship between weight and miles/gallon is stronger (the slope is steeper) for cars with a manual transmission compared to cars with an automatic transmission.

This is a good example to illustrate the point that when studying a relationship between two variables, say $X$ and $Y$, if one also has data for other variables which are potentially associated with both $X$ and $Y$, it is important to include them in the regression and to analyze the relationship conditionally on these variables.

Omitting some variables that should be included in the model may lead to erroneous and misleading conclusions, up to the point that the relationship is completely reversed (a phenomenon referred as Simpson’s paradox).

Summary

In this article, we started with a reminder of simple linear regression and in particular its principle and how to interpret the results.

This laid the foundations for a better understanding of multiple linear regression. After explaining its principle, we showed how to interpret the output and how to choose a good linear model. We then mentioned a couple of visualizations and finished the article with some more advanced topics.

Thanks for reading.

I hope this article helped you to understand better linear regression and gave you the confidence to do your own linear regressions in R. If you need to model a binary variable instead of a quantitative continuous variable, see how to perform a binary logistic regression in R.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Austin, Peter C, and Ewout W Steyerberg. 2015. “The Number of Subjects Per Variable Required in Linear Regression Analyses.” Journal of Clinical Epidemiology 68 (6): 627–36.

Ernst, Anja F, and Casper J Albers. 2017. “Regression Assumptions in Clinical Psychology Research Practice?a Systematic Review of Common Misconceptions.” PeerJ 5: e3323.

James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning. Vol. 112. Springer.

Lumley, Thomas, Paula Diehr, Scott Emerson, and Lu Chen. 2002. “The Importance of the Normality Assumption in Large Public Health Data Sets.” Annual Review of Public Health 23 (1): 151–69.

Schmidt, Amand F, and Chris Finan. 2018. “Linear Regression and the Normality Assumption.” Journal of Clinical Epidemiology 98: 146–51.

Some people see regression analysis as a part of inferential statistics. It is true, as a sample is taken to evaluate the link between two or more variables in a population of interest. I tend to distinguish regression from inferential statistics for the simple reasons that (i) regressions are often used to a broader extent (for predictive analyses, among others), and because (ii) the main goal of linear regression (see this section) differs from the objectives of confidence intervals and hypothesis testing well known in the field of inferential statistics.↩︎
Formally, ANOVA can also be used to compare 2 groups, but in practice we tend to use it for 3 or more groups, leaving the t-test for 2 groups.↩︎
More information about the dataset can be found by executing ?mtcars.↩︎
Note that it best to avoid interpreting the intercept when $X$ cannot be equal to 0 or when it makes no sense in practice.↩︎
$n$ is the number of observations.↩︎
Other values than 0 are accepted as well. In that case, the test statistic becomes $T_{n - 2} = \frac{\widehat\beta - a}{se(\widehat\beta_1)}$ where $a$ is the hypothesized slope.↩︎
Note that linearity can be checked with a scatterplot of the two variables, or via a scatterplot of the residuals and the fitted values. See more about this in this section.↩︎
An observation is considered as an outlier based on the Cook’s distance if its value is > 1.↩︎
An observation has a high leverage value (and thus needs to be investigated) if it is greater than $2p/n$, where $p$ is the number of parameters in the model (intercept included) and $n$ is the number of observations.↩︎
You can always change the reference level with the relevel() function. See more data manipulation techniques.↩︎
Note that linearity can also be tested with a scatterplot of the residuals and the fitted values.↩︎
After installing the {performance} package, you will also need to install the {see} package manually. See how to install a R package if you need more help.↩︎
I use the threshold of 10 because, as shown by James et al. (2013), a value between 5 and 10 indicates a moderate correlation, while VIF values greater than 10 indicate a high and non-tolerable correlation.↩︎
Austin and Steyerberg (2015) showed that two subjects per variable tends to permit accurate estimation of regression coefficients in a linear regression model estimated using ordinary least squares. Moreover, the general rule of thumb says that there should be at least 10 observations per variable (Schmidt and Finan 2018). Our dataset contains 32 observations, above the minimum of 10 subjects per variable.↩︎
If you apply a logarithmic transformation, see two guides on how to interpret the results: in English and in French.↩︎
Note that a high $R^2$ does not guarantee that you selected the best variables or that your model is good. It simply tells that the model fits the data quite well. It is advised to apply common sense when comparing models and not only refer to $R^2$ (in particular when $R^2$ are close).↩︎
There are two main methods; backward and forward. The backward method consists in starting from the model containing all the explanatory variables likely to be relevant, then recursively removing the variable which reduces the information criterion of the model, until no reduction is possible. The forward method is the reverse of the backward method in the sense that we start from a one-variable model with the lowest information criterion and at each step, an explanatory variable is added. By default, the step() function in R combines the backward and forward methods.↩︎
To not be confused with the anova() function because it provides results that depend on the order in which the variables appear in the model.↩︎

Running pace calculator in R Shiny

Mon, 15 Mar 2021 00:00:00 +0000

Introduction

If you are a runner yourself, you are certainly aware of how important preparation is before a race. For the preparation of my first marathon, I used to rely on a training plan.

This running plan was great, but an important information was missing: the running pace. Most of the time, the distance and the time was given, but I needed to figure out the pace myself.

Although the computation is fairly easy, I felt like I was missing a quick way to compute my running pace based on the distance and expected time given by the training plan.

So I started to look for a solution online, but I was never completely satisfied. Some running pace calculators were too detailed (showing way too much information) others were too basic (showing not enough information).

Running pace calculator

I thus decided to create one myself so I could really choose what information would be displayed, and how it would be displayed.

For the runners among you, here is a link to the application:

Running pace calculator

If you follow the blog, you know how much I like R Shiny, so you probably guessed that the calculator is built with this technology.

Note that this running pace calculator is inspired by several calculators I found online. I kept it quite basic so that it goes straight to the point, but most importantly so that it would fit to my needs (which may be different than yours).

How to use it?

I try to keep all my Shiny apps easy to use for everyone. However, here is how to use it in case it is not intuitive enough:

Access the app via this link
Choose the units (kilometers or miles)
Enter the distance you plan to run
Enter the time for which you would like to run that distance

On the right panel (or bottom if you use the app on mobile) you will see:

The pace you will need to respect in order to run the distance within the time you specified
Depending on the units you selected, your pace will be displayed in minutes/km or minutes/miles, and kilometers/hour or miles/hour
The table below displays the splits—the time at each kilometer or mile

Update of January 11, 2023:

Until now, only the conversion from distance and time to pace was possible. For completeness, I have added to following conversions:

Pace and time to distance: enter the pace and the time you plan to run to find the expected running distance.
Pace and distance to time: enter the pace and the distance you plan to run to find the expected running time.

Code

As for all my Shiny apps, the code is available on GitHub. Feel free to open an issue if you find a bug or if you have a suggestion. And if you are proficient in R Shiny, do not hesitate to propose a pull request with your suggestions implemented.

Conclusion

Thanks for reading.

I hope this running pace calculator will be useful if you are a runner, or if you plan to start running.

As always, if you have a question about the app, please add it as a comment so other readers can benefit from the discussion.

Hypothesis test by hand

Wed, 27 Jan 2021 00:00:00 +0000

Descriptive versus inferential statistics

Remember that descriptive statistics is the branch of statistics aiming at describing and summarizing a set of data in the best possible manner, that is, by reducing it down to a few meaningful key measures and visualizations—with as little loss of information as possible. In other words, the branch of descriptive statistics helps to have a better understanding and a clear image about a set of observations thanks to summary statistics and graphics. With descriptive statistics, there is no uncertainty because we describe only the group of observations that we decided to work on and no attempt is made to generalize the observed characteristics to another or to a larger group of observations.

Inferential statistics, one the other hand, is the branch of statistics that uses a random sample of data taken from a population to make inferences, i.e., to draw conclusions about the population of interest (see the difference between population and sample if you need a refresh of the two concepts). In other words, information from the sample is used to make generalizations about the parameter of interest in the population.

The two most important tools used in the domain of inferential statistics are:

hypothesis test (which is the main subject of the present article), and
confidence interval (which is briefly discussed in this section)

Motivations and limitations

Via my teaching tasks, I realized that many students (especially in introductory statistic classes) struggle to perform hypothesis tests and interpret the results. It seems to me that these students often encounter difficulties mainly because hypothesis testing is rather unclear and abstract to them.

One of the reason it looks abstract to them is because they do not understand the final goal of hypothesis testing—the “why” behind this tool. They often do inferential statistics without understanding the reasoning behind it, as if they were following a cooking recipe which does not require any thinking. However, as soon as they understand the principle underlying hypothesis testing, it is much easier for them to apply the concepts and solve the exercises.

For this reason, I though it would be useful to write an article on the goal of hypothesis tests (the “why?”), in which context they should be used (the “when?”), how they work (the “how?”) and how to interpret the results (the “so what?”). Like anything else in statistics, it becomes much easier to apply a concept in practice when we understand what we are testing or what we are trying to demonstrate beforehand.

In this article, I present—as comprehensibly as possible—the different steps required to perform and conclude a hypothesis test by hand.

These steps are illustrated with a basic example. This will build the theoretical foundations of hypothesis testing, which will in turn be of great help for the understanding of most statistical tests.

Hypothesis tests come in many forms and can be used for many parameters or research questions. The steps I present in this article are not applicable to all hypothesis test, unfortunately.

They are however, appropriate for at least the most common hypothesis tests—the tests on:

One mean: $\mu$
Two means:
- independent samples: $\mu_1$ and $\mu_2$
- paired samples: $\mu_D$
One proportion: $p$
Two proportions: $p_1$ and $p_2$
One variance: $\sigma^2$
Two variances: $\sigma^2_1$ and $\sigma^2_2$

The good news is that the principles behind these 6 statistical tests (and many more) are exactly the same. So if you understand the intuition and the process for one of them, all others pretty much follow.

Hypothesis test

Why?

Unlike descriptive statistics where we only describe the data at hand, hypothesis tests use a subset of observations, referred as a sample, to draw conclusions about a population.

One may wonder why we would try to “guess” or make inference about a parameter of a population based on a sample, instead of simply collecting data for the entire population, compute statistics we are interested in and take decisions based upon that.

The main reason we actually use a sample instead of the entire population is because, most of the time, collecting data on the entire population is practically impossible, too complex, too expensive, it would take too long, or a combination of any of these.¹

So the overall objective of a hypothesis test is to draw conclusions in order to confirm or refute a belief about a population, based on a smaller group of observations.

In practice, we take some measurements of the variable of interest—representing the sample(s)—and we check whether our measurements are likely or not given our assumption (our belief). Based on the probability of observing the sample(s) we have, we decide whether we can trust our belief or not.

When?

Hypothesis tests have many practical applications.

Here are different situations illustrating when the 6 tests mentioned above would be appropriate:

One mean: suppose that a health professional would like to test whether the mean weight of Belgian adults is different than 80 kg (176.4 lbs).
Two means:
- Independent samples: suppose that a physiotherapist would like to test the effectiveness of a new treatment by measuring the mean response time (in seconds) for patients in a control group and patients in a treatment group, where patients in the two groups are different.
- Paired samples: suppose that a physiotherapist would like to test the effectiveness of a new treatment by measuring the mean response time (in seconds) before and after a treatment, where patients are measured twice—before and after treatment, so patients are the same in the 2 samples.
One proportion: suppose that a political pundit would like to test whether the proportion of citizens who are going to vote for a specific candidate is smaller than 30%.
Two proportions: suppose that a doctor would like to test whether the proportion of smokers is different between professional and amateur athletes.
One variance: suppose that an engineer would like to test whether a voltmeter has a lower variability than what is imposed by the safety standards.
Two variances: suppose that, in a factory, two production lines work independently from each other. The financial manager would like to test whether the costs of the weekly maintenance of these two machines have the same variance. Note that a test on two variances is also often performed to verify the assumption of equal variances, which is required for several other statistical tests, such as the Student’s t-test for instance.

Of course, this is a non-exhaustive list of potential applications and many research questions can be answered thanks to a hypothesis test.

One important point to remember is that in hypothesis testing we are always interested in the population and not in the sample. The sample is used for the aim of drawing conclusions about the population, so we always test in terms of the population.

Usually, hypothesis tests are used to answer research questions in confirmatory analyses. Confirmatory analyses refer to statistical analyses where hypotheses—deducted from theory—are defined beforehand (preferably before data collection). In this approach, the researcher has a specific idea about the variables under consideration and she is trying to see if her idea, specified as hypotheses, is supported by data.

On the other hand, hypothesis tests are rarely used in exploratory analyses.² Exploratory analyses aims to uncover possible relationships between the variables under investigation. In this approach, the researcher does not have any clear theory-driven assumptions or ideas in mind before data collection. This is the reason exploratory analyses are sometimes referred as hypothesis-generating analyses—they are used to create some hypotheses, which in turn may be tested via confirmatory analyses at a later stage.

How?

There are, to my knowledge, 3 different methods to perform a hypothesis tests:

Although the process for these 3 approaches may slightly differ, they all lead to the exact same conclusions. Using one method or another is, therefore, more often than not a matter of personal choice or a matter of context. See this section to know which method I use depending on the context.

I present the 3 methods in the following sections, starting with, in my opinion, the most comprehensive one when it comes to doing it by hand: comparing the test statistic with the critical value.

For the three methods, I will explain the required steps to perform a hypothesis test from a general point of view and illustrate them with the following situation:³

Suppose a health professional who would like to test whether the mean weight of Belgian adults is different than 80 kg.

Note that, as for most hypothesis tests, the test we are going to use as example below requires some assumptions. Since the aim of the present article is to explain a hypothesis test, we assume that all assumptions are met. For the interested reader, see the assumptions (and how to verify them) for this type of hypothesis test in the article presenting the one-sample t-test.

Method A: Comparing the test statistic with the critical value

Method A, which consists in comparing the test statistic with the critical value, boils down to the following 4 steps:

Stating the null and alternative hypothesis
Computing the test statistic
Finding the critical value
Concluding and interpreting the results

Each step is detailed below.

Step #1: Stating the null and alternative hypothesis

As discussed before, a hypothesis test first requires an idea, that is, an assumption about a phenomenon. This assumption, referred as hypothesis, is derived from the theory and/or the research question.

Since a hypothesis test is used to confirm or refute a prior belief, we need to formulate our belief so that there is a null and an alternative hypothesis. Those hypotheses must be mutually exclusive, which means that they cannot be true at the same time. This is step #1.

In the context of our scenario, the null and alternative hypothesis are thus:

Null hypothesis $H_0: \mu = 80$
Alternative hypothesis $H_1: \mu \ne 80$

When stating the null and alternative hypothesis, bear in mind the following three points:

We are always interested in the population and not in the sample. This is the reason $H_0$ and $H_1$ will always be written in terms of the population and not in terms of the sample (in this case, $\mu$ and not $\bar{x}$).
The assumption we would like to test is often the alternative hypothesis. If the researcher wanted to test whether the mean weight of Belgian adults was less than 80 kg, she would have stated $H_0: \mu = 80$ (or equivalently, $H_0: \mu \ge 80$) and $H_1: \mu < 80$.⁴ Do not mix the null with the alternative hypothesis, or the conclusions will be diametrically opposed!
The null hypothesis is often the status quo. For instance, suppose that a doctor wants to test whether the new treatment A is more efficient than the old treatment B. The status quo is that the new and old treatments are equally efficient. Assuming a larger value is better, she will then write $H_0: \mu_A = \mu_B$ (or equivalently, $H_0: \mu_A - \mu_B = 0$) and $H_1: \mu_A > \mu_B$ (or equivalently, $H_0: \mu_A - \mu_B > 0$). On the opposite, if the lower the better, she would have written $H_0: \mu_A = \mu_B$ (or equivalently, $H_0: \mu_A - \mu_B = 0$) and $H_1: \mu_A < \mu_B$ (or equivalently, $H_0: \mu_A - \mu_B < 0$).

Step #2: Computing the test statistic

The test statistic (often called t-stat) is, in some sense, a metric indicating how extreme the observations are compared to the null hypothesis. The higher the t-stat (in absolute value), the more extreme the observations are.

There are several formulas to compute the t-stat, with one formula for each type of hypothesis test—one or two means, one or two proportions, one or two variances. This means that there is a formula to compute the t-stat for a hypothesis test on one mean, another formula for a test on two means, another for a test on one proportion, etc.⁵

The only difficulty in this second step is to choose the appropriate formula. As soon as you know which formula to use based on the type of test, you simply have to apply it to the data. For the interested reader, see the different formulas to compute the t-stat for the most common tests in this Shiny app.

Luckily, formulas for hypothesis tests on one and two means, and one and two proportions follow the same structure.

Computing the test statistic for these tests is similar than scaling a random variable (a process also knows as “standardization” or “normalization”) which consists in subtracting the mean from that random variable, and dividing the result by the standard deviation:

\[Z = \frac{X - \mu}{\sigma}\]

For these 4 hypothesis tests (one/two means and one/two proportions), computing the test statistic is like scaling the estimator (computed from the sample) corresponding to the parameter of interest (in the population). So we basically subtract the target parameter from the point estimator and then divide the result by the standard error (which is equivalent to the standard deviation but for an estimator).

If this is unclear, here is how the test statistic (denoted $t_{obs}$) is computed in our scenario (assuming that the variance of the population is unknown):

\[t_{obs} = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}}\]

where:

$\bar{x}$ is the sample mean (i.e., the estimator)
$\mu$ is the mean under the null hypothesis (i.e., the target parameter)
$s$ is the sample standard deviation
$n$ is the sample size
($\frac{s}{\sqrt{n}}$ is the standard error)

Notice the similarity between the formula of this test statistic and the formula used to standardize a random variable. This structure is the same for a test on two means, one proportion and two proportions, except that the estimator, the parameter and the standard error are, of course, slightly different for each type of test.

Suppose that in our case we have a sample mean of 71 kg ($\bar{x}$ = 71), a sample standard deviation of 13 kg ($s$ = 13) and a sample size of 10 adults ($n$ = 10). Remember that the population mean (the mean under the null hypothesis) is 80 kg ($\mu$ = 80).

The t-stat is thus:

\[t_{obs} = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}} = \frac{71 - 80}{\frac{13}{\sqrt{10}}} = -2.189\]

Although formulas are different depending on which parameter you are testing, the value found for the test statistic gives us an indication on how extreme our observations are.

We keep this value of -2.189 in mind because it will be used again in step #4.

Step #3: Finding the critical value

Although the t-stat gives us an indication of how extreme our observations are, we cannot tell whether this “score of extremity” is too extreme or not based on its value only.

So, at this point, we cannot yet tell whether our data are too extreme or not. For this, we need to compare our t-stat with a threshold—referred as critical value—given by the probability distribution tables (and which can, of course, also be found with R).

In the same way that the formula to compute the t-stat is different for each parameter of interest, the underlying probability distribution—and thus the statistical table—on which the critical value is based is also different for each target parameter. This means that, in addition to choosing the appropriate formula to compute the t-stat, we also need to select the appropriate probability distribution depending on the parameter we are testing.

Luckily, there are only 4 different probability distributions for the 6 hypothesis tests covered in this article (one/two means, one/two proportions and one/two variances):

Standard Normal distribution:
- test on one and two means with known population variance(s)
- test on two paired samples where the variance of the difference between the 2 samples $\sigma^2_D$ is known
- test on one and two proportions (given that some assumptions are met)
Student distribution:
- test on one and two means with unknown population variance(s)
- test on two paired samples where the variance of the difference between the 2 samples $\sigma^2_D$ is unknown
Chi-square distribution:
- test on one variance
Fisher distribution:
- test on two variances

Each probability distribution also has its own parameters (up to two parameters for the 4 distribution considered here), defining its shape and/or location. Parameter(s) of a probability distribution can be seen as its DNA; meaning that the distribution is entirely defined by its parameter(s).

Taking our initial scenario—a health professional who would like to test whether the mean weight of Belgian adults is different than 80 kg—as example.

The underlying probability distribution of a test on one mean is either the standard Normal or the Student distribution, depending on whether the variance of the population (not sample variance!) is known or unknown:⁶

If the population variance is known $\rightarrow$ the standard Normal distribution is used
If the population variance is unknown $\rightarrow$ the Student distribution is used

If no population variance is explicitly given, you can assume that it is unknown since you cannot compute it based on a sample. If you could compute it, that would mean you have access to the entire population and there is, in this case, no point in performing a hypothesis test (you could simply use some descriptive statistics to confirm or refute your belief).

In our example, no population variance is specified so it is assumed to be unknown. We therefore use the Student distribution.

The Student distribution has one parameter which defines it; the number of degrees of freedom. The number of degrees of freedom depends on the type of hypothesis test. For instance, the number of degrees of freedom for a test on one mean is equal to the number of observations minus one ($n$ - 1). Without going too far into the details, the - 1 comes from the fact that there is one quantity which is estimated (i.e., the mean).⁷ The sample size being equal to 10 in our example, the degrees of freedom is equal to $n$ - 1 = 10 - 1 = 9.

There is only one last element missing to find the critical value: the significance level. The significance level, denoted $\alpha$, is the probability of wrongly rejecting the null hypothesis, so the probability of rejecting the null hypothesis although it is in reality true. In this sense, it is an error (type I error, as opposed to the type II error⁸) that we accept to deal with, in order to be able to draw conclusions about a population based on a subset of it.

As you may have read in many statistical textbooks, the significance level is very often set to 5%.⁹ In some fields (such as medicine or engineering, among others), the significance level is also sometimes set to 1% to decrease the error rate.

It is best to specify the significance level before performing a hypothesis test to avoid the temptation to set the significance level in accordance to the results (the temptation is even bigger when the results are on the edge of being significant). As I always tell my students, you cannot “guess” nor compute the significance level. Therefore, if it is not explicitly specified, you can safely assume it is 5%. In our case, we did not indicate it, so we take $\alpha$ = 5% = 0.05.

Furthermore, in our example, we want to test whether the mean weight of Belgian adults is different than 80 kg. Since we do not specify the direction of the test, it is a two-sided test. If we wanted to test that the mean weight was less than 80 kg ($H_1: \mu <$ 80) or greater than 80 kg ($H_1: \mu >$ 80), we would have done a one-sided test.

Make sure that you perform the correct test (two-sided or one-sided) because it has an impact on how to find the critical value (see more in the following paragraphs).

So now that we know the appropriate distribution (Student distribution), its parameter (degrees of freedom (df) = 9), the significance level ($\alpha$ = 0.05) and the direction (two-sided), we have all we need to find the critical value in the statistical tables:

By looking at the row df = 9 and the column $t_.025$ in the Student’s distribution table, we find a critical value of:

\[t_{n-1; \alpha / 2} = t_{9; 0.025} = 2.262\]

One may wonder why we take $t_{\alpha/2} = t_.025$ and not $t_\alpha = t_.05$ since the significance level is 0.05. The reason is that we are doing a two-sided test ($H_1: \mu \ne$ 80), so the error rate of 0.05 must be divided in 2 to find the critical value to the right of the distribution. Since the Student’s distribution is symmetric, the critical value to the left of the distribution is simply: -2.262.

Visually, the error rate of 0.05 is partitioned into two parts:

0.025 to the left of -2.262 and
0.025 to the right of 2.262

We keep in mind these critical values of -2.262 and 2.262 for the fourth and last step.

Note that the red shaded areas in the previous plot are also known as the rejection regions. More on that in the following section.

These critical values can also be found in R, thanks to the qt() function:

qt(0.025, df = 9, lower.tail = TRUE)

## [1] -2.262157

qt(0.025, df = 9, lower.tail = FALSE)

## [1] 2.262157

The qt() function is used for the Student’s distribution (q stands for quantile and t for Student). There are other functions accompanying the different distributions:

qnorm() for the Normal distribution
qchisq() for the Chi-square distribution
qf() for the Fisher distribution

Step #4: Concluding and interpreting the results

In this fourth and last step, all we have to do is to compare the test statistic (computed in step #2) with the critical values (found in step #3) in order to conclude the hypothesis test.

The only two possibilities when concluding a hypothesis test are:

Rejection of the null hypothesis
Non-rejection of the null hypothesis

In our example of adult weight, remember that:

the t-stat is -2.189
the critical values are -2.262 and 2.262

Also remember that:

the t-stat gives an indication on how extreme our sample is compared to the null hypothesis
the critical values are the threshold from which the t-stat is considered as too extreme

To compare the t-stat with the critical values, I always recommend to plot them:

These two critical values form the rejection regions (the red shaded areas):

from $- \infty$ to -2.262, and
from 2.262 to $\infty$

If the t-stat lies within one of the rejection region, we reject the null hypothesis. On the contrary, if the t-stat does not lie within any of the rejection region, we do not reject the null hypothesis.

As we can see from the above plot, the t-stat is less extreme than the critical value and therefore does not lie within any of the rejection region. In conclusion, we do not reject the null hypothesis that $\mu = 80$.

This is the conclusion in statistical terms but they are meaningless without proper interpretation. So it is a good practice to also interpret the result in the context of the problem:

At the 5% significance level, we do not reject the hypothesis that the mean weight of Belgian adults is 80 kg.

Why don’t we accept $H_0$?

From a more philosophical (but still very important) perspective, note that we wrote “we do not reject the null hypothesis” and “we do not reject the hypothesis that the mean weight of Belgian adults is equal to 80 kg”. We did not write “we accept the null hypothesis” nor “the mean weight of Belgian adults is 80 kg”.

The reason is due to the fact that, in hypothesis testing, we conclude something about the population based on a sample. There is, therefore, always some uncertainty and we cannot be 100% sure that our conclusion is correct.

Perhaps it is the case that the mean weight of Belgian adults is in reality different than 80 kg, but we failed to prove it based on the data at hand. It may be the case that if we had more observations, we would have rejected the null hypothesis (since all else being equal, a larger sample size implies a more extreme t-stat). Or, it may be the case that even with more observations, we would not have rejected the null hypothesis because the mean weight of Belgian adults is in reality close to 80 kg. We cannot distinguish between the two.

So we can just say that we did not find enough evidence against the hypothesis that the mean weight of Belgian adults is 80 kg, but we do not conclude that the mean is equal to 80 kg.

If the difference is still not clear to you, the following example may help. Suppose a person is suspected of having committed a crime. This person is either innocent—the null hypothesis—or guilty—the alternative hypothesis. In the attempt to know if the suspect committed the crime, the police collects as much information and proof as possible. This is similar to the researcher collecting data to form a sample. And then the judge, based on the collected evidence, decides whether the suspect is considered as innocent or guilty. If there is enough evidence that the suspect committed the crime, the judge will conclude that the suspect is guilty. In other words, she will reject the null hypothesis of the suspect being innocent because there are enough evidence that the suspect committed the crime.

This is similar to the t-stat being more extreme than the critical value: we have enough information (based on the sample) to say that the null hypothesis is unlikely because our data would be too extreme if the null hypothesis were true. Since the sample cannot be “wrong” (it corresponds to the collected data), the only remaining possibility is that the null hypothesis is in fact wrong. This is the reason we write “we reject the null hypothesis”.

On the other hand, if there is not enough evidence that the suspect committed the crime (or no evidence at all), the judge will conclude that the suspect is considered as not guilty. In other words, she will not reject the null hypothesis of the suspect being innocent. But even if she concludes that the suspect is considered as not guilty, she will never be 100% sure that he is really innocent.

It may be the case that:

the suspect did not commit the crime, or
the suspect committed the crime but the police was not able to collect enough information against the suspect.

In the former case the suspect is really innocent, whereas in the latter case the suspect is guilty but the police and the judge failed to prove it because they failed to find enough evidence against him. Similar to hypothesis testing, the judge has to conclude the case by considering the suspect not guilty, without being able to distinguish between the two.

This is the main reason we write “we do not reject the null hypothesis” or “we fail to reject the null hypothesis” (you may even read in some textbooks conclusion such as “there is no sufficient evidence in the data to reject the null hypothesis”), and we do not write “we accept the null hypothesis”.

I hope this metaphor helped you to understand the reason why we reject the null hypothesis instead of accepting it.

In the following sections, we present two other methods used in hypothesis testing.

These methods will result in the exact same conclusion: non-rejection of the null hypothesis, that is, we do not reject the hypothesis that the mean weight of Belgian adults is 80 kg. It is thus presented only if you prefer to use these methods over the first one.

Method B: Comparing the p-value with the significance level $\alpha$

Method B, which consists in computing the p-value and comparing this p-value with the significance level $\alpha$, boils down to the following 4 steps:

Stating the null and alternative hypothesis
Computing the test statistic
Computing the p-value
Concluding and interpreting the results

In this second method which uses the p-value, the first and second steps are similar than in the first method.

Step #1: Stating the null and alternative hypothesis

The null and alternative hypotheses remain the same:

$H_0: \mu = 80$
$H_1: \mu \ne 80$

Step #2: Computing the test statistic

Remember that the formula for the t-stat is different depending on the type of hypothesis test (one or two means, one or two proportions, one or two variances). In our case of one mean with unknown variance, we have:

\[t_{obs} = \frac{\bar{x} - \mu}{\frac{s}{\sqrt{n}}} = \frac{71 - 80}{\frac{13}{\sqrt{10}}} = -2.189\]

Step #3: Computing the p-value

The p-value is the probability (so it goes from 0 to 1) of observing a sample at least as extreme as the one we observed if the null hypothesis were true. In some sense, it gives you an indication on how likely your null hypothesis is. It is also defined as the smallest level of significance for which the data indicate rejection of the null hypothesis.

For more information about the p-value, I recommend reading this note about the p-value and the significance level $\alpha$.

Formally, the p-value is the area beyond the test statistic. Since we are doing a two-sided test, the p-value is thus the sum of the area above 2.189 and below -2.189.

Visually, the p-value is the sum of the two blue shaded areas in the following plot:

The p-value can computed with precision in R with the pt() function:

p_val <- pt(-2.189, df = 9, lower.tail = TRUE) + pt(2.189, df = 9, lower.tail = FALSE)
p_val

## [1] 0.05634202

# which is equivalent than:
p_val <- 2 * pt(2.189, df = 9, lower.tail = FALSE)
p_val

## [1] 0.05634202

The p-value is 0.0563, which indicates that there is a 5.63% chance to observe a sample at least as extreme as the one observed if the null hypothesis were true. This already gives us a hint on whether our t-stat is too extreme or not (and thus whether our null hypothesis is likely or not), but we formally conclude in step #4.

Like the qt() function to find the critical value, we use pt() to find the p-value because the underlying distribution is the Student’s distribution.

Use pnorm(), pchisq() and pf() for the Normal, Chi-square and Fisher distribution, respectively. See also this Shiny app to compute the p-value given a certain t-stat for most probability distributions.

If you do not have access to a computer (during exams for example) you will not be able to compute the p-value precisely, but you can bound it using the statistical table referring to your test.

In our case, we use the Student distribution and we look at the row df = 9 (since df = n - 1):

The test statistic is -2.189
We take the absolute value, which gives 2.189
The value 2.189 is between 1.833 and 2.262 (highlighted in blue in the above table)
From the column names $t_{.050}$ and $t_{.025}$ related to 1.833 and 2.262, we know that:
- the area to the right of 1.833 is 0.05
- the area to the right of 2.262 is 0.025
So we know that the area to the right of 2.189 must be between 0.025 and 0.05
Since the Student distribution is symmetric, we know that the area to the left of -2.189 must also be between 0.025 and 0.05
Therefore, the sum of the two areas must be between 0.05 and 0.10
In other words, the p-value is between 0.05 and 0.10 (i.e., 0.05 < p-value < 0.10)

Although we could not compute it precisely, it is enough to conclude our hypothesis test in the last step.

Step #4: Concluding and interpreting the results

The final step is now to simply compare the p-value (computed in step #3) with the significance level $\alpha$. As for all statistical tests:

If the p-value is smaller than $\alpha$ (p-value < 0.05) $\rightarrow H_0$ is unlikely $\rightarrow$ we reject the null hypothesis
If the p-value is greater than or equal to $\alpha$ (p-value $\ge$ 0.05) $\rightarrow H_0$ is likely $\rightarrow$ we do not reject the null hypothesis

No matter if we take into consideration the exact p-value (i.e., 0.0563) or the bounded one (0.05 < p-value < 0.10), it is larger than 0.05, so we do not reject the null hypothesis.¹⁰ In the context of the problem, we do not reject the null hypothesis that the mean weight of Belgian adults is 80 kg.

Remember that rejecting (or not rejecting) a null hypothesis at the significance level $\alpha$ using the critical value method (method A) is equivalent to rejecting (or not rejecting) the null hypothesis when the p-value is lower (equal or greater) than $\alpha$ (method B).

This is the reason we find the exact same conclusion than with method A, and why you should too if you use both methods on the same data and with the same significance level.

Method C: Comparing the target parameter with the confidence interval

Method C, which consists in computing the confidence interval and comparing this confidence interval with the target parameter (the parameter under the null hypothesis), boils down to the following 3 steps:

Stating the null and alternative hypothesis
Computing the confidence interval
Concluding and interpreting the results

In this last method which uses the confidence interval, the first step is similar than in the first two methods.

Step #1: Stating the null and alternative hypothesis

The null and alternative hypotheses remain the same:

$H_0: \mu = 80$
$H_1: \mu \ne 80$

Step #2: Computing the confidence interval

Like hypothesis testing, confidence intervals are a well-known tool in inferential statistics.

Confidence interval is an estimation procedure which produces an interval (i.e., a range of values) containing the true parameter with a certain—usually high—probability.

In the same way that there is a formula for each type of hypothesis test when computing the test statistics, there exists a formula for each type of confidence interval. Formulas for the different types of confidence intervals can be found in this Shiny app.

Here is the formula for a confidence interval on one mean $\mu$ (with unknown population variance):

\[ (1-\alpha)\text{% CI for } \mu = \bar{x} \pm t_{\alpha/2, n - 1} \frac{s}{\sqrt{n}} \]

where $t_{\alpha/2, n - 1}$ is found in the Student distribution table (and is similar to the critical value found in step #3 of method A).

Given our data and with $\alpha$ = 0.05, we have:

\[ \begin{aligned} 95\text{% CI for } \mu &= \bar{x} \pm t_{\alpha/2, n - 1} \frac{s}{\sqrt{n}} \\ &= 71 \pm 2.262 \frac{13}{\sqrt{10}} \\ &= [61.70; 80.30] \end{aligned} \]

The 95% confidence interval for $\mu$ is [61.70; 80.30] kg. But what does a 95% confidence interval mean?

We know that this estimation procedure has a 95% probability of producing an interval containing the true mean $\mu$. In other words, if we construct many confidence intervals (with different samples of the same size), 95% of them will, on average, include the mean of the population (the true parameter). So on average, 5% of these confidence intervals will not cover the true mean.

If you wish to decrease this last percentage, you can decrease the significance level (set $\alpha$ = 0.01 or 0.02 for instance). All else being equal, this will increase the range of the confidence interval and thus increase the probability that it includes the true parameter.

Step #3: Concluding and interpreting the results

The final step is simply to compare the confidence interval (constructed in step #2) with the value of the target parameter (the value under the null hypothesis, mentioned in step #1):

If the confidence interval does not include the hypothesized value $\rightarrow H_0$ is unlikely $\rightarrow$ we reject the null hypothesis
If the confidence interval includes the hypothesized value $\rightarrow H_0$ is likely $\rightarrow$ we do not reject the null hypothesis

In our example:

the hypothesized value is 80 (since $H_0: \mu$ = 80)
80 is included in the 95% confidence interval since it goes from 61.70 to 80.30 kg
So we do not reject the null hypothesis

In the terms of the problem, we do not reject the hypothesis that the mean weight of Belgian adults is 80 kg.

As you can see, the conclusion is equivalent than with the critical value method (method A) and the p-value method (method B). Again, this must be the case since we use the same data and the same significance level $\alpha$ for all three methods.

Which method to choose?

All three methods give the same conclusion. However, each method has its own advantage so I usually select the most convenient one depending on the situation:

Method A (comparing the test statistic with the critical value):
- It is, in my opinion, the easiest and most straightforward method of the three when I do not have access to R.
Method B (comparing the p-value with the significance level $\alpha$):
- In addition to being able to know whether the null hypothesis is rejected or not, computing the exact p-value can be very convenient so I tend to use this method if I have access to R.
Method C (comparing the target parameter with the confidence interval):
- If I need to test several hypothesized values, I tend to choose this method because I can construct one single confidence interval and compare it to as many values as I want. For example, with our 95% confidence interval [61.70; 80.30], I know that any value below 61.70 kg and above 80.30 kg will be rejected, without testing it for each value.

Summary

In this article, we reviewed the goals and when hypothesis testing is used. We then showed how to do a hypothesis test by hand through three different methods (A. critical value, B. p-value and C. confidence interval). We also showed how to interpret the results in the context of the initial problem.

Although all three methods give the exact same conclusion when using the same data and the same significance level (otherwise there is a mistake somewhere), I also presented my personal preferences when it comes to choosing one method over the other two.

Thanks for reading.

I hope this article helped you to understand the structure of a hypothesis by hand. I remind you that, at least for the 6 hypothesis tests covered in this article, the formulas are different, but the structure and the reasoning behind it remain the same. So you basically have to know which formulas to use, and simply follow the steps mentioned in this article.

For the interested reader, I created two accompanying Shiny apps:

Hypothesis testing and confidence intervals: after entering your data, the app illustrates all the steps in order to conclude the test and compute a confidence interval. See more information in this article.
How to read statistical tables: the app helps you to compute the p-value given a t-stat for most probability distributions. See more information in this article.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Suppose a researcher wants to test whether Belgian women are taller than French women. Suppose a health professional would like to know whether the proportion of smokers is different among athletes and non-athletes. It would take way too long to measure the height of all Belgian and French women and to ask all athletes and non-athletes their smoking habits. So most of the time, decisions are based on a representative sample of the population and not on the whole population. If we could measure the entire population in a reasonable time frame, we would not do any inferential statistics.↩︎
Don’t get me wrong, this does not mean that hypothesis tests are never used in exploratory analyses. It is just much less frequent in exploratory research than in confirmatory research.↩︎
You may see more or less steps in other articles or textbooks, depending on whether these steps are detailed or concise. Hypothesis testing should, however, follows the same process regardless of the number of steps.↩︎
For one-sided tests, writing $H_0: \mu = 80$ or $H_0: \mu \ge 80$ are both correct. The point is that the null and alternative hypothesis must be mutually exclusive since you are testing one hypothesis against the other, so both cannot be true at the same time.↩︎
To be complete, there are even different formulas within each type of test, depending on whether some assumptions are met or not. For the interested reader, see all the different scenarios and thus the different formulas for a test on one mean and on two means.↩︎
There are more uncertainty if the population variance is unknown than if it is known, and this greater uncertainty is taken into account by using the Student distribution instead of the standard Normal distribution. Also note that as the sample size increases, the degrees of freedom of the Student distribution increases and the two distributions become more and more similar. For large sample size (usually from $n >$ 30), the Student distribution becomes so close to the standard Normal distribution that, even if the population variance is unknown, the standard Normal distribution can be used.↩︎
For a test on two independent samples, the degrees of freedom is $n_1 + n_2 - 2$, where $n_1$ and $n_2$ are the size of the first and second sample, respectively. Note the - 2 due to the fact that in this case, two quantities are estimated.↩︎
The type II error is the probability of not rejecting the null hypothesis although it is in reality false.↩︎
Whether this is a good or a bad standard is a question that comes up often and is debatable. This is, however, beyond the scope of the article.↩︎
Again, p-values found via a statistical table or via R must be coherent.↩︎

How to track the performance of your blog in R?

Wed, 16 Dec 2020 00:00:00 +0000

Introduction

Stats and R has been launched on December 16, 2019. Since the blog is officially one year old today and after having discussed the main benefits of maintaining a technical blog, I thought it would be a good time to share some numbers and thoughts about it.

In this article, I show how to analyze a blog and its blog posts with the {googleAnalyticsR} R package (see package’s full documentation). After sharing some analytics about the blog, I will also discuss about content creation/distribution and, to a smaller extent, the future plans. This is a way to share my journey as a data science blogger and a way to give you an insight about how Stats and R is doing.

I decided to share with you some numbers through the {googleAnalyticsR} package instead of the regular Google Analytics dashboards for several reasons:

There are plenty of data analysts who are much more experienced than me when it comes to analyzing Google Analytics data via their dedicated platform
I recently discovered the {googleAnalyticsR} package in R and I would like to present its possibilities, and perhaps convince marketing specialists familiar with R to complement their Google Analytics dashboards with some data visualizations made in R (via some ggplot2 visualizations for instance)
I would like to automate the process such in a way that I can easily replicate the same types of analysis across the years. This will allow to see how the blog evolves throughout the years. We know that using R is a pretty good starting point when it comes to automation and replication—especially thanks to R Markdown reports

I am not an expert in the field of digital marketing, but who knows, it may still give some ideas to data analysts, SEO specialists or other bloggers on how to track the performance of their own blog or website using R. For those of you who are interested in a more condensed analysis, see my custom Google Analytics dashboard.¹

Before going further, I would like to remind that I am not making a living from my blog (far from it!) and it is definitely not my goal as I do not believe that I would be the same kind of writer if it was my main occupation.

Prerequisites

As for any package in R, we first need to install it—with install.packages()—and load it—with library():²

# install.packages('googleAnalyticsR', dependencies = TRUE)
library(googleAnalyticsR)

Next, we need to authorize the access of the Google Analytics account using the ga_auth() function:

ga_auth()

Running this code will open a browser window on which you will be able to authorize the access. This step will save an authorization token so you only have to do it once.

Make sure to run the ga_auth() function in a R script and not in a R Markdown document. Follow this procedure if you want to use the package and its functions in a R Markdown report or in a blog post like I did for this article.

Once we have completed the Google Analytics authorization, we will need the ID of the Google Analytics account we want to access. All the available accounts linked to your email address (after authentication) are stored in ga_account_list():

accounts <- ga_account_list()

accounts

## # A tibble: 2 × 10
##   accountId account…¹ inter…² level websi…³ type  webPr…⁴ webPr…⁵ viewId viewN…⁶
##   <chr>     <chr>     <chr>   <chr> <chr>   <chr> <chr>   <chr>   <chr>  <chr>  
## 1 86997981  Antoine … 129397… STAN… https:… WEB   UA-869… Antoin… 13318… All We…
## 2 86997981  Antoine … 218214… STAN… https:… WEB   UA-869… statsa… 20812… All We…
## # … with abbreviated variable names ¹accountName, ²internalWebPropertyId,
## #   ³websiteUrl, ⁴webPropertyId, ⁵webPropertyName, ⁶viewName

accounts$webPropertyName

## [1] "Antoine Soetewey" "statsandr.com"

As you can see I have two accounts linked to my Google Analytics profile: one for my personal website (antoinesoetewey.com) and one for this blog.

Of course, I select the account linked to this blog:

# select the view ID by property name
view_id <- accounts$viewId[which(accounts$webPropertyName == "statsandr.com")]

Make sure to edit the code with your own property name.

We are now finally ready to use our Google Analytics data in R for a better analysis!

Analytics

Users, page views and sessions

Let’s start with some general numbers, such as the number of users, sessions and page views for the entire site. Note that for the present article, we use data over the past year, so from December 16, 2019 to December 15, 2020:

# set date range
start_date <- as.Date("2019-12-16")
end_date <- as.Date("2020-12-15")

# get Google Analytics (GA) data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("users", "sessions", "pageviews"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

gadata

##    users sessions pageviews
## 1 321940   428217    560491

In its first year, Stats and R has attracted 321,940 users, who generated a total of 428,217 sessions and 560,491 page views (that is an average of 1531 page views per day).

For those unfamiliar with Google Analytics data and the difference between these metrics, remember that:

a user is the number of new and returning people who visit your site during a set period of time
a session is a group of user interactions with your website that take place within a given time frame
a page view, as the name suggests, is defined as a view of a page on your site

So if person A reads three blog posts then leave the site and person B reads one blog post, your about page then leave the site, Google Analytics data will show 2 users, 2 sessions and 5 page views.

Sessions over time

In addition to the rather general metrics presented above, it is also interesting to illustrate the daily number of sessions over time in a scatterplot—together with a smoothed line—to analyze the evolution of the blog:

# get the Google Analytics (GA) data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("sessions"), # edit for other metrics
  dimensions = c("date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# load required libraries
library(dplyr)
library(ggplot2)

# scatter plot with a trend line
gadata %>%
  ggplot(aes(x = date, y = sessions)) +
  geom_point(size = 1L, color = "steelblue") + # change size and color of points
  geom_smooth(color = "darkgrey", alpha = 0.25) + # change color of smoothed line and transparency of confidence interval
  theme_minimal() +
  labs(
    y = "Sessions",
    x = "",
    title = "Evolution of daily sessions",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  theme(plot.margin = unit(c(5.5, 15.5, 5.5, 5.5), "pt")) + # to avoid the plot being cut on the right edge
  scale_y_continuous(labels = scales::comma) # better y labels

(See how to draw plots with the {ggplot2} package, or with the {esquisse} addin if you are not familiar with the package.)

As you can see, there was a huge peak of traffic around end of April, with almost 12,000 users in a single day. Yes, you read it well and there is no bug. The blog post “A package to download free Springer books during Covid-19 quarantine” went viral and generated a massive traffic for a few days. The daily number of sessions returned to a more normal level after a couple of days. We also observe an upward trend in the last months (since end of August/beginning of September), which indicates that the blog is growing in terms of number of daily sessions.

Note that I decided to focus on the number of sessions and the number of page views in this section and the following ones, but you can always change to your preferred metrics by editing metrics = c("sessions") in the code. See all available metrics provided by Google Analytics in this article.

Sessions per channel

Knowing how people come to your blog is a pretty important factor. Here is how to visualize the evolution of daily sessions per channel in a line plot:

# Get the data
trend_data <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  dimensions = c("date"),
  metrics = "sessions",
  pivots = pivot_ga4("medium", "sessions"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# edit variable names
names(trend_data) <- c("Date", "Total", "Organic", "Referral", "Direct", "Email", "Social")

# Change the data into a long format
library(tidyr)
trend_long <- gather(trend_data, Channel, Sessions, -Date)

# Build up the line plot
ggplot(trend_long, aes(x = Date, y = Sessions, group = Channel)) +
  theme_minimal() +
  geom_line(aes(colour = Channel)) +
  labs(
    y = "Sessions",
    x = "",
    title = "Evolution of daily sessions per channel",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

We see that a large share of the traffic is from the organic channel, which indicates that most readers visit the blog after a query on search engines (mostly Google). In my case, where most of my posts are tutorials and which help people with specific problems, it is thus not a surprise that most of my traffic comes from organic search.

We also notice some small peaks of sessions generated from the referral and direct channels, which are probably happening on the date of publication of each article.

We also see that there seems to be a recurrent pattern of ups and downs in the number of daily sessions. Those are weekly cycles, with less readers during the weekend and which indicates that people are working on improving their statistical or R knowledge mostly during the week (which actually makes sense!).

Sessions per day of week

As shown above, traffic seems to be different depending on the day of week. To investigate this further, we draw a boxplot of the number of sessions for every day of the week:

# get data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = "sessions",
  dimensions = c("dayOfWeek", "date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

## Recoding gadata$dayOfWeek following GA naming conventions
gadata$dayOfWeek <- recode_factor(gadata$dayOfWeek,
  "0" = "Sunday",
  "1" = "Monday",
  "2" = "Tuesday",
  "3" = "Wednesday",
  "4" = "Thursday",
  "5" = "Friday",
  "6" = "Saturday"
)

## Reordering gadata$dayOfWeek to have Monday as first day of the week
gadata$dayOfWeek <- factor(gadata$dayOfWeek,
  levels = c(
    "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday",
    "Sunday"
  )
)

# Boxplot
gadata %>%
  ggplot(aes(x = dayOfWeek, y = sessions)) +
  geom_boxplot() +
  theme_minimal() +
  labs(
    y = "Sessions",
    x = "",
    title = "Sessions per day of week",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

As you can see there are some outliers, probably due (in part at least) to the article that went viral. For the sake of illustration, below the same plot after removing points considered as potential outliers according to the interquartile range (IQR) criterion (i.e., points above or below the whiskers), and after a couple of visual improvements:

# boxplot
gadata %>%
  filter(sessions <= 3000) %>% # filter out sessions > 3,000
  ggplot(aes(x = dayOfWeek, y = sessions, fill = dayOfWeek)) + # fill boxplot by dayOfWeek
  geom_boxplot(varwidth = TRUE) + # vary boxes width according to n obs.
  geom_jitter(alpha = 0.25, width = 0.2) + # adds random noise and limit its width
  theme_minimal() +
  labs(
    y = "Sessions",
    x = "",
    title = "Sessions per day of week",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com\nPoints > 3,000 excluded"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  theme(legend.position = "none") # remove legend

After excluding data points above 3,000, it is now easier to see that the median number of sessions (represented by the horizontal bold line in the boxes) is the highest on Wednesdays, and lowest on Saturdays and Sundays.

The difference in sessions between weekdays is, however, not as large as expected.

Sessions per day and time

We have seen the traffic per day of week. The example below shows a visualization of traffic, broken down this time by day of week and hour of day in a density plot. In this plot, the device type has also been added for more insights.

## Get data by deviceCategory, day of week and hour
weekly_data <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("sessions"),
  dimensions = c("deviceCategory", "dayOfWeekName", "hour"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

## Manipulation using dplyr
weekly_data_sessions <- weekly_data %>%
  group_by(deviceCategory, dayOfWeekName, hour)

## Reordering weekly_data_sessions$dayOfWeekName to have Monday as first day of the week
weekly_data_sessions$dayOfWeekName <- factor(weekly_data_sessions$dayOfWeekName,
  levels = c(
    "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday",
    "Sunday"
  )
)

## Plotting using ggplot2
weekly_data_sessions %>%
  ggplot(aes(hour, sessions, fill = deviceCategory, group = deviceCategory)) +
  geom_area(position = "stack") +
  labs(
    title = "Sessions per day and time",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com",
    x = "Time",
    y = "Sessions",
    fill = "Device" # edit legend title
  ) +
  theme_minimal() +
  facet_wrap(~dayOfWeekName, ncol = 2, scales = "fixed") +
  theme(
    legend.position = "bottom", # move legend
    axis.text = element_text(size = 7) # change font of axis text
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

The above plot shows that:

traffic increases in the afternoon then declines in the late evening
traffic is the highest from Monday to Thursday, and lowest on Saturday and Sunday

In addition to that, thanks to the additional information on the device category, we also see that:

the number of sessions on tablet is low (it is so low compared to desktop and mobile that it is not visible on the plot), and
the number of sessions on mobile seems to be quite stable during the entire day,
as opposed to sessions on desktop which are highest at the end of the day.

Sessions per month and year

In the following code, we create a barplot of the number of daily sessions per month and year:

# get data
df2 <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("sessions"),
  dimensions = c("date"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# add in year month columns to dataframe
df2$month <- format(df2$date, "%m")
df2$year <- format(df2$date, "%Y")

# sessions by month by year using dplyr then graph using ggplot2 barplot
df2 %>%
  group_by(year, month) %>%
  summarize(sessions = sum(sessions)) %>%
  # print table steps by month by year
  # print(n = 100) %>%
  # graph data by month by year
  ggplot(aes(x = month, y = sessions, fill = year)) +
  geom_bar(position = "dodge", stat = "identity") +
  theme_minimal() +
  labs(
    y = "Sessions",
    x = "Month",
    title = "Sessions per month and year",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

This barplot allows to easily see the evolution of the number of sessions over the months, and compare this evolution across different years.

At the moment, since the blog is online only since December 2019, the year factor is not relevant. However, I still present the visualization for other users who work on older websites, and also to remind my future self to create this interesting barplot when there will be data for more than a year.

Top performing pages

Another important factor when measuring the performance of your blog or website is the number of page views for the different pages. The top performing pages in terms of page views over the year can easily be found in Google Analytics (you can access it via Behavior > Site Content > All pages).

For the interested reader, here is how to get the data in R (note that you can change n = 7 in the code to change the number of top performing pages to display):

## Make the request to GA
data_fetch <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews"),
  dimensions = c("pageTitle"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

## Create a table of the most viewed posts
library(lubridate)
library(reactable)
library(stringr)

most_viewed_posts <- data_fetch %>%
  mutate(Title = str_trunc(pageTitle, width = 40)) %>% # keep maximum 40 characters
  count(Title, wt = pageviews, sort = TRUE)
head(most_viewed_posts, n = 7) # edit n for more or less pages to display

##                                      Title     n
## 1 A package to download free Springer b... 85684
## 2 Variable types and examples - Stats a... 44951
## 3 Descriptive statistics in R - Stats a... 43621
## 4    Outliers detection in R - Stats and R 32560
## 5 The complete guide to clustering anal... 27184
## 6 Correlation coefficient and correlati... 21581
## 7                              Stats and R 16786

Here is how to visualize this table of top performing pages in a barplot:

# plot
top_n(most_viewed_posts, n = 7, n) %>% # edit n for more or less pages to display
  ggplot(., aes(x = reorder(Title, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Title",
    title = "Top performing pages in terms of page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

This gives me a good first overview on how posts performed in terms of page views, so in some sense, what people find useful. For instance, I never thought that the post illustrating the different types of variables that exist in statistics (ranked #2) would be so appreciated when I wrote it.

This is something I learned with this blog: at the time of writing, there are some posts which I think no one will care about (and which I mostly write as a personal note for myself), and others which I think people will find very useful. However, after a couple of weeks after publication, sometimes I realize that it is actually exactly the opposite.

Time-normalized page views

When it comes to comparing blog posts, however, this is not as simple.

Based on the above barplot and without any further analysis, I would conclude that my article about variable types is performing much better than the one about outliers detection in R.

However, if I tell you that the article on outliers detection has been published on August 11 and the one about variable types on December 30, you will agree that the comparison does not make much sense anymore since page views for these articles were counted over a different length of time. One could also argue that a recent article had less time to generate backlinks, so it is unfair to compare it with an old post which had plenty of time to be ranked high by Google.

In order to make the comparison more “fair”, we would need to compare the number of page views for each post since their date of publication. The following code does precisely this:³

it pulls daily data for a bunch of pages,
then tries to detect their publication date,
time-normalizes the traffic for each page based on that presumed publication date,
and finally, plots the daily traffic from the publication date on out, as well as overall cumulative traffic for the top n pages

# figure out when a page actually launched by finding the
# first day where the page had at least 2 unique pageviews
first_day_pageviews_min <- 2

# exclude pages that have total traffic (daily unique pageviews) that are relatively low
total_unique_pageviews_cutoff <- 500

# set how many "days since publication" we want to include in our plot
days_live_range <- 180

# set number of top pages to display
n <- 7

# Create a dimension filter object
# You need to update the "expressions" value to be a regular expression that filters to
# the appropriate set of content on your site
page_filter_object <- dim_filter("pagePath",
  operator = "REGEXP",
  expressions = "/blog/.+"
)

# Now, put that filter object into a filter clause. The "operator" argument can be AND # or OR...but you have to have it be something, even though it doesn't do anything
# when there is only a single filter object.
page_filter <- filter_clause_ga4(list(page_filter_object),
  operator = "AND"
)

# Pull the GA data
ga_data <- google_analytics(
  viewId = view_id,
  date_range = c(start_date, end_date),
  metrics = "uniquePageviews",
  dimensions = c("date", "pagePath"),
  dim_filters = page_filter,
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# Find the first date for each post. This is actually a little tricky, so we're going to write a
# function that takes each page as an input, filters the data to just include those
# pages, finds the first page, and then puts a "from day 1" count on that data and
# returns it.
normalize_date_start <- function(page) {
  # Filter all the data to just be the page being processed
  ga_data_single_page <- ga_data %>% filter(pagePath == page)

  # Find the first value in the result that is greater than first_day_pageviews_min. In many
  # cases, this will be the first row, but, if there has been testing/previews before it
  # actually goes live, some noise may sneak in where the page may have been live, technically,
  # but wasn't actually being considered live.
  first_live_row <- min(which(ga_data_single_page$uniquePageviews > first_day_pageviews_min))

  # Filter the data to start with that page
  ga_data_single_page <- ga_data_single_page[first_live_row:nrow(ga_data_single_page), ]

  # As the content ages, there may be days that have ZERO traffic. Those days won't show up as
  # rows at all in our data. So, we actually need to create a data frame that includes
  # all dates in the range from the "publication" until the last day traffic was recorded. There's
  # a little trick here where we're going to make a column with a sequence of *dates* (date) and,
  # with a slightly different "seq," a "days_live" that corresponds with each date.
  normalized_results <- data.frame(
    date = seq.Date(
      from = min(ga_data_single_page$date),
      to = max(ga_data_single_page$date),
      by = "day"
    ),
    days_live = seq(min(ga_data_single_page$date):
    max(ga_data_single_page$date)),
    page = page
  ) %>%
    # Join back to the original data to get the uniquePageviews
    left_join(ga_data_single_page) %>%
    # Replace the "NAs" (days in the range with no uniquePageviews) with 0s (because
    # that's exactly what happened on those days!)
    mutate(uniquePageviews = ifelse(is.na(uniquePageviews), 0, uniquePageviews)) %>%
    # We're going to plot both the daily pageviews AND the cumulative total pageviews,
    # so let's add the cumulative total
    mutate(cumulative_uniquePageviews = cumsum(uniquePageviews)) %>%
    # Grab just the columns we need for our visualization!
    select(page, days_live, uniquePageviews, cumulative_uniquePageviews)
}

# We want to run the function above on each page in our dataset. So, we need to get a list
# of those pages. We don't want to include pages with low traffic overall, which we set
# earlier as the 'total_unique_pageviews_cutoff' value, so let's also filter our
# list to only include the ones that exceed that cutoff. We also select the top n pages
# in terms of page views to display in the visualization.
library(dplyr)
pages_list <- ga_data %>%
  group_by(pagePath) %>%
  summarise(total_traffic = sum(uniquePageviews)) %>%
  filter(total_traffic > total_unique_pageviews_cutoff) %>%
  top_n(n = n, total_traffic)

# The first little bit of magic can now occur. We'll run our normalize_date_start function on
# each value in our list of pages and get a data frame back that has our time-normalized
# traffic by page!
library(purrr)
ga_data_normalized <- map_dfr(pages_list$pagePath, normalize_date_start)

# We specified earlier -- in the `days_live_range` object -- how many "days since publication" we
# actually want to include, so let's do one final round of filtering to only include those
# rows.
ga_data_normalized <- ga_data_normalized %>% filter(days_live <= days_live_range)

Now that our data is ready, we create two visualizations:

Number of page views by day since publication: this plot shows how quickly interest in a particular piece of content drops off. If it is not declining as rapidly as the other posts, it means you are getting sustained value from it
Cumulative number of page views by day since publication: this plot can be used to compare blog posts on the same ground since number of page views is shown based on the publication date. To see which pages have generated the most traffic over time, simply look from top to bottom (at the right edge of the plot)

Note that both plots use the {plotly} package to make them interactive so that you can mouse over a line and find out exactly what page it is (together with its values). The interactivity of the plot makes it also possible to zoom in to see, for instance, the number of page views in the first days after publication (instead of the default length of 180 days),⁴ or zoom in to see only the evolution of the number of page views below/above a certain threshold.

# Create first plot
library(ggplot2)

gg <- ggplot(ga_data_normalized, mapping = aes(x = days_live, y = uniquePageviews, color = page)) +
  geom_line() + # The main "plot" operation
  scale_y_continuous(labels = scales::comma) + # Include commas in the y-axis numbers
  labs(
    title = "Page views by day since publication",
    x = "Days since publication",
    y = "Page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  theme_minimal() + # minimal theme
  theme(
    legend.position = "none", # remove legend
  )

# Output the plot, wrapped in ggplotly so we will get some interactivity in the plot
library(plotly)
ggplotly(gg, dynamicTicks = TRUE)

The plot above shows again a huge spike for the post that went viral (orange line). If we zoom in to include only page views below 1500, comparison between posts is easier and we see that:

The post on the top 100 R resources on Coronavirus (purple line) generated comparatively more traffic than the other posts in the first 50 days ($\approx$ 1 month and 3 weeks) after publication. However, traffic gradually decreased up to the point that after 180 days ($\approx$ 6 months), it attracted less traffic than other more performing posts.
The post on outliers detection in R (blue line) did not get a lot of attention in the first weeks after its publication. However, it gradually generated more and more traffic up to the point that, after 60 days ($\approx$ 2 months) after publication, it actually generated more traffic than any other post (and by a relatively large margin).
The post on correlation coefficient and correlation test in R (green line) took approximately 100 days ($\approx$ 3 months and 1 week) to take off, but after that it generated quite a lot of traffic. This is interesting to keep in mind when analyzing recent posts, because they may actually follow the same trend in the long run.

# Create second plot: cumulative
gg <- ggplot(ga_data_normalized, mapping = aes(x = days_live, y = cumulative_uniquePageviews, color = page)) +
  geom_line() + # The main "plot" operation
  scale_y_continuous(labels = scales::comma) + # Include commas in the y-axis numbers
  labs(
    title = "Cumulative page views by day since publication",
    x = "Days since publication",
    y = "Cumulative page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  theme_minimal() + # minimal theme
  theme(
    legend.position = "none", # remove legend
  )

# Output the plot, wrapped in ggplotly so we will get some interactivity in the plot
ggplotly(gg, dynamicTicks = TRUE)

Compared to the previous plot, this one shows the cumulative number of page views since the date of publication of the post.

Without taking into consideration the post that went viral and which, by the way, does not really generate much traffic anymore (which makes sense since the incredible campaign from Springer to offer their books for free during the COVID-19 quarantine has ended), we see that:

The post on outliers detection has surpassed the post on top R resources on Coronavirus in terms of cumulative number of page views after around 120 days ($\approx$ 4 months) after publication, which indicates that people are looking at this specific problem. I remember that I wrote this post because, at that time, I had to deal with the problems of outliers in R and I did not found a neat solution online. So I guess, the fact that resources on a specific topic are missing helps to attract visitors looking for an answer to their question.
Among the remaining posts, the ranking of the most performing ones in terms of cumulative number of page views within 180 days ($\approx$ 6 months) after publication is the following:

The analyses so far give you already a good understanding of the performance of your blog. However, for the interested readers, we show other important metrics in the following sections.

Page views by country

In the following we are interested in seeing where the traffic comes from. This is particularly interesting to get to know your audience, and even more important if you are running a business or an ecommerce.

# get GA data
data_fetch <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = "pageviews",
  dimensions = "country",
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# table
countries <- data_fetch %>%
  mutate(Country = str_trunc(country, width = 40)) %>% # keep maximum 40 characters
  count(Country, wt = pageviews, sort = TRUE)
head(countries, n = 10) # edit n for more or less countries to display

##           Country      n
## 1   United States 131306
## 2           India  38800
## 3         Belgium  32983
## 4  United Kingdom  25833
## 5          Brazil  18961
## 6         Germany  17851
## 7           Spain  17355
## 8          Canada  13880
## 9          Mexico  12882
## 10    Philippines  12540

To visualize this table of top countries in terms of page views in a barplot:

# plot
top_n(countries, n = 10, n) %>% # edit n for more or less countries to display
  ggplot(., aes(x = reorder(Country, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Country",
    title = "Top performing countries in terms of page views",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  theme(plot.margin = unit(c(5.5, 7.5, 5.5, 5.5), "pt")) # to avoid the plot being cut on the right edge

We see that readers from the US take the largest share of the number of page views (by quite a lot actually!), and Belgium (my country) comes in third place in terms of number of page views.

Given that the US population is much larger than the Belgian population ($\approx$ 331 million people compared to $\approx$ 11.5 million people, respectively), the above result is not really surprising. Again, for a better comparison, it would be interesting to take into consideration the size of the population when comparing countries.

Indeed, it could be that a large share of the traffic comes from a country with a large population, but that the number of page views per person (or per 100,000 inhabitants) is higher for another country. This information about top performing countries in terms of page views per person could give you insights on which country do the most avid readers come from. This is beyond the scope of this article, but you can see examples of plots which include the information on the population size in these COVID-19 visualizations. I recommend to apply the same methodology to the above plot for a better comparison.

If you are thinking about doing the extra step of including population size when comparing countries, I believe that it would be even better to take into account the information on computer access in each country as well. If we take India as example: at the time of writing this article, its population amounts to almost 1.4 billion. However, the percentage of Indian people having access to a computer is undoubtedly lower than in US or Belgium (again, at least at the moment). It would therefore make more sense to compare countries by comparing the number of page views per people having access to a computer.⁵

Browser information

For more technical aspects, you could also be interested in the number of page views by browser. This can be visualized with the following barplot:

# get data
browser_info <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviews"),
  dimensions = c("browser"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# table
browser <- browser_info %>%
  mutate(Browser = str_trunc(browser, width = 40)) %>% # keep maximum 40 characters
  count(Browser, wt = pageviews, sort = TRUE)

# plot
top_n(browser, n = 10, n) %>% # edit n for more or less browser to display
  ggplot(., aes(x = reorder(Browser, n), y = n)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  theme_minimal() +
  coord_flip() +
  labs(
    y = "Page views",
    x = "Browser",
    title = "Which browsers are our visitors using?",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

Most visits were, as expected, from Chrome, Safari and Firefox browsers.

User engagement by devices

One may also be interested in checking how users are engaged on different types of devices. To do so, we plot 3 charts describing:

How many sessions were made from the different types of devices
The average time on page (in seconds) by type of device
The number of page views per session by device type

# GA data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("sessions", "avgTimeOnPage"),
  dimensions = c("date", "deviceCategory"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# plot sessions by deviceCategory
gadata %>%
  ggplot(aes(deviceCategory, sessions)) +
  geom_bar(aes(fill = deviceCategory), stat = "identity") +
  theme_minimal() +
  labs(
    y = "Sessions",
    x = "",
    title = "Sessions per device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com",
    fill = "Device" # edit legend title
  ) +
  scale_y_continuous(labels = scales::comma) # better y labels

From the above plot, we see that the majority of readers visited the blog from a desktop, and a small number of readers from a tablet.

# add median of average time on page per device
gadata <- gadata %>%
  group_by(deviceCategory) %>%
  mutate(med = median(avgTimeOnPage))

# plot avgTimeOnPage by deviceCategory
ggplot(gadata) +
  aes(x = avgTimeOnPage, fill = deviceCategory) +
  geom_histogram(bins = 30L) +
  scale_fill_hue() +
  theme_minimal() +
  theme(legend.position = "none") +
  facet_wrap(vars(deviceCategory)) +
  labs(
    y = "Frequency",
    x = "Average time on page (in seconds)",
    title = "Average time on page per device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com"
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  geom_vline(aes(xintercept = med, group = deviceCategory),
    color = "darkgrey",
    linetype = "dashed"
  ) +
  geom_text(
    aes(
      x = med, y = 25,
      label = paste0("Median = ", round(med), " seconds")
    ),
    angle = 90,
    vjust = 2,
    color = "darkgrey",
    size = 3
  )

From the above plot, we see that:

some readers on tablet have a very low average time spent on each page (see the peak around 0 second in the tablet facet)
distributions of the average time on page for readers on desktop and mobile were quite similar, with an average time on page mostly between 100 seconds (= 1 minute 40 seconds) and 400 seconds (= 6 minutes 40 seconds)
quite surprisingly, the median of the average time spent on page is slightly higher for visitors on mobile than on desktop (see the dashed vertical lines representing the medians in the desktop and mobile facets). This indicates that, although more people visit the blog from desktop, it seems that people on mobile spend more time per page. I find this result quite surprising given that most of my articles include R code and require a computer to run the code. Therefore, I expected that people would spend more time on desktop than on mobile because on mobile they would quickly scan the article, while on desktop they would read the article carefully and try to reproduce the code on their computer.⁶

Given this result, it would be interesting to also illustrate the number of page views during a session, represented by device type.

Indeed, it may be the case that visitors on mobile spend, on average, more time on each page but people on desktop visit more pages per session. We verify this via a density plot, and for better readability we exclude data points above 2.5 page views/session and we exclude visits from a tablet:

# GA data
gadata <- google_analytics(view_id,
  date_range = c(start_date, end_date),
  metrics = c("pageviewsPerSession"),
  dimensions = c("date", "deviceCategory"),
  anti_sample = TRUE # slows down the request but ensures data isn't sampled
)

# add median of number of page views/session
gadata <- gadata %>%
  group_by(deviceCategory) %>%
  mutate(med = median(pageviewsPerSession))

## Reordering gadata$deviceCategory
gadata$deviceCategory <- factor(gadata$deviceCategory,
  levels = c("mobile", "desktop", "tablet")
)

# plot pageviewsPerSession by deviceCategory
gadata %>%
  filter(pageviewsPerSession <= 2.5 & deviceCategory != "tablet") %>% # filter out pageviewsPerSession > 2.5 and visits from tablet
  ggplot(aes(x = pageviewsPerSession, fill = deviceCategory, color = deviceCategory)) +
  geom_density(alpha = 0.5) +
  scale_fill_hue() +
  theme_minimal() +
  labs(
    y = "Frequency",
    x = "Page views per session",
    title = "Page views/session by device",
    subtitle = paste0(format(start_date, "%b %d, %Y"), " to ", format(end_date, "%b %d, %Y")),
    caption = "Data: Google Analytics data of statsandr.com\nPoints > 2.5 excluded",
    color = "Device", # edit legend title
    fill = "Device" # edit legend title
  ) +
  scale_y_continuous(labels = scales::comma) + # better y labels
  geom_vline(aes(xintercept = med, group = deviceCategory, color = deviceCategory),
    linetype = "dashed",
    show.legend = FALSE # remove legend
  ) +
  geom_text(
    aes(
      x = med, y = 2.75,
      label = paste0("Median = ", round(med, 2), " page views/session"),
      color = deviceCategory
    ),
    angle = 90,
    vjust = 2,
    size = 3,
    show.legend = FALSE # remove legend
  )

This last plot confirms our thoughts, that is, although people on mobile seem to spend more time on each page, people on desktop tend to visit more pages per session.

(One may wonder why I chose to compare medians instead of means. The main reason is that not all distributions considered here are bell-shaped (especially for data on tablet) and there are many outliers. In these cases, the mean is usually not the most appropriate descriptive statistics and the median is a more robust way to represent such data. For the interested reader, see a note on the difference between mean and median, and the context in which each measure is more appropriate.)

This is the end of the analytics section. Of course, many more visualizations and data analyses are possible, depending on the site that is tracked and the marketing expertise of the analyst. This was an overview of what is possible, and I hope it will give you some ideas to explore your Google Analytics data further. Next year, I may also include forecasts and make annual comparisons. See also some examples of other analyses in this GitHub repository and this website.

As a side note, I would like to add the following: even if tracking the performance of your blog is important to understand how you attract visitors and how they engage with your site, I also believe that looking at your Google Analytics stats too often is not optimal, nor sane.

Talking about personal experience: at the beginning of the blog I used to look very often at the number of visitors in real-time and the audience. I was kind of obsessed to know how many people were right now on my blog and I was constantly checking if it performed better than the day before in terms of number of visitors. I remember that I was spending so much time looking at these metrics in the first weeks that I felt I was wasting my time. And the time I was wasting looking at my Google Analytics stats was lost not creating good quality content for the blog, working on my thesis/classes, or other projects.

So when I realized that, I deleted the Google Analytics app from my smartphone and forced myself not to look at my stats more than once a month (just to make sure there is no critical issues that need to be fixed). From that moment onward, I stopped wasting my time on things I cannot control, and I got more satisfaction from writing articles because I was writing them for myself, not for the sake of seeing people reading them. This change was like a relief, and I am now more satisfied with my work on this blog compared to the first weeks or months.

Everyone is different and unique so I am not saying that you should do the same. However, if you feel that you look too much at your stats and sometimes lose motivation in writing, perhaps this is one potential solution.

Content

Now that we have seen how we can analyze Google Analytics data and track the performance of a website or blog in details, I would like to share some thoughts about content creation, content distribution and the future plans.

Finding topics

One of the biggest challenges I face with my blog is creating proper content. In the best case scenario, I would like to:

create useful content
about subjects I am really familiar with,
which I enjoy and
which fit into the blog.

By sharing only articles about topics I am familiar with, the writing process is easier and quicker. Indeed, most of the articles I wrote cover topics I teach at university, so a large part of the preliminary research is already done when I decide to write about it. Also (and this is not negligible), questions and incomprehension from students allow me to see the points I should focus on when writing about it, and how to present it to make it accessible and comprehensible to most people.

Moreover, by making my thoughts public, I often have the chance to confront them with other points of view, which allows me to study the topic even further. This in turn accelerates the writing process even more when writing about a related topic.

I also tend to write only about things I enjoy or I am interested in. I prefer quality over quantity, so writing an article from A to Z takes quite a long time depending on the depth of the subject. Since it takes time (even without taking into account the time spent after publication) and I have a full-time job, I really focus on topics I enjoy in order to keep seeing this blog as a source of pleasure, and not as a work or an obligation.

The fact that I write only about things I am familiar with, which I enjoy and when I have some free time (which mostly depend on the ongoing projects related to my PhD thesis) makes it hard for me to follow a defined pace for posts. This is why my writing schedule has been a bit inconsistent during this first year and is likely to be similar in the future as I do not want to be in the position to force myself to write.⁷

Content distribution

Even if we all agree that bloggers should write for themselves first and not for the sake of having lots of readers, it is still appreciated when your content is being read by others.

So although I do not like abusive self-promotion and I do not feel comfortable sharing my blog posts to all existing Facebook, LinkedIn and Reddit groups, somehow people need to get informed that you have written something.

For this reason, I created a Twitter account where I share new posts immediately after publishing them on the blog. In addition to posting them on Twitter, I also share the link by email to people who subscribed to the newsletter of the blog.

I also managed to get my content published on Medium through the Towards Data Science publication, R-bloggers and R Weekly. A non-negligible part of my audience comes from these referral, especially during the couple of days after publication.

By distributing the content this way, I do not feel pushy (something I want to avoid at all costs!) because people decided by themselves to receive the content I publish (e.g., they subscribed to the newsletter, they followed the blog on Twitter, they chose to read blog aggregators, etc.). So in some sense, I do not distribute my content “without their prior consent” and they can always choose not to see my posts anymore.

However, as you can see from the plot of the number of session per channel, you see that most readers come from the organic channel, so from search engines. And for this channel, apart from creating quality content (and some basic knowledge in SEO), I do not have any control on how well it is presented nor distributed to people. For this channel, it is basically search engine algorithms which decide to promote my content or not, and if they do, how well it is promoted. The only control I have regarding ranking factors is simply to create quality content.

So even if I believe that having a content distribution strategy definitely helps in reaching more people and growing your blog, it does not make everything. Creating quality content is, in my view of a beginner in SEO and marketing, the best strategy for my posts to be read.

For this very specific reason, now that I feel I have settled a decent distribution strategy, I no longer spend my energy and time on this matter.

So I just simply try to enjoy writing on my blog, and the rest will eventually follow. If not, I am still learning a lot from this blog anyway so I do not see it as wasted time.

A small note about ads

I know that an easy way to get money out of your blog is to run ads and I understand that people do it if it is profitable. However, I often find ads too intrusive or annoying and I personally do not like to read blog posts which include many ads.

To keep the reading process as enjoyable as possible, as you can see, I do not display any advertising on my blog. As long as the costs of running this website and the related open source projects (e.g., my Shiny apps, etc.) are not too high, I do not expect to include ads.

If in the future costs were to increase, I will still try to avoid putting ads as much as possible and I will try to rely on sponsorship programs & donations and on paid side projects for people who need help for their statistical analyses.

Future plans

After exactly one year of blogging, I am asking myself the following:

What do I want statsandr.com to be?

As already said, as long as I enjoy writing stuff I am passionate about on this blog, I will continue. This is very important to me.

In addition to that, I would like it to be a place to share knowledge. The field of statistics and R (and data science in general) is evolving at an extremely fast pace—and more and more people have many interesting things to say.

Moreover, I learned a lot since the launch this blog. But I learned even more when working in collaboration with someone else (see for instance these collaborations). I see so many learning opportunities when collaborating that I would love to see this blog as a place to share knowledge, but not only my knowledge.

To be more precise, I would love to hear about other researchers, statisticians, R lovers, data scientists, authors, etc. who want to collaborate with me. This could lead, in the end, to:

a guest post if you have a good idea on what you want to write about but need a way to share it (and since you have access to my Google Analytics data in the past year through this article you have a good idea of the number of visitors who will see your content),
a piece of content written together if you believe our skills and knowledge are complementary (see all articles for an overview of what I like to write about),
a piece of code, a R package, a visualization or a Shiny app you want to create together (or just share),
a book or a course on statistics and/or R,
or anything else you have in mind (I am open to new ideas and challenges).

With this article, you see the figures and the audience of the blog. With all my articles, you can see my strengths and weaknesses.

So I will finish this section by saying that, if anyone is willing to work on something together (which does not need to be huge), you can always contact me or leave a comment at the end of this post. I am looking forward to hearing from you.

Thank you note

Last but not least, I also wanted to leave a short thank you note to the Towards Data Science, R-bloggers and R Weekly teams that have cross-posted most of my articles in their respective publications and therefore allowed me to share my thoughts to a larger—and very knowledgeable—audience.

Also, I would like to thank all active readers for their comments, constructive feedback and support so far. I am looking forward to sharing more quality content through this blog and I hope it will keep being useful to many of you, in parallel to being useful to me.

A special thanks to Mark Edmondson, the author of the {googleAnalyticsR} package and all people who wrote tutorials using the package, which were used as inspiration for this blog post. More broadly, thanks also to the open source community which is of great help in learning R.

Thanks for reading. I hope that you learned how to track the performance of your website or blog in R using the {googleAnalyticsR} package. See you next year for a second review, and in the meantime, if you maintain a blog I would be really happy to hear how you track its performance. Feel free to let me know via the comments below!

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Thanks to the RStudio blog for the inspiration.↩︎
See more ways to install and load R packages.↩︎
Many thanks to dartistics.com for the code. Note that for better readability of the plots, I slightly edited the code: (i) to display only the top n pages in terms of traffic instead of all pages, (ii) to change the theme to theme_minimal() and (iii) to make the axis ticks dynamic when zooming in or out (in the ggplotly() function).↩︎
Note that the default period of 180 days can also be changed in the code.↩︎
This may require some assumptions if the percentage of people having access to a computer is not readily available. I believe, however, that it would still be more appropriate than just taking into account the population size—especially when comparing developed with developing countries.↩︎
At least that is what I do when I read blogs on mobile versus reading them on desktop.↩︎
Note, however, that research has shown that scheduling time for writing is a good way to productive writing (Silvia, 2019). This is why, unlike for my blog, I have set regular writing periods in my calendar (and I try to stick to it no matter how busy I am at that time) dedicated to my PhD thesis.↩︎

ANOVA in R

Mon, 12 Oct 2020 00:00:00 +0000

Introduction

ANOVA (ANalysis Of VAriance) is a statistical test to determine whether two or more population means are different. In other words, it is used to compare two or more groups to see if they are significantly different.

In practice, however, the:

Student t-test is used to compare 2 groups;
ANOVA generalizes the t-test beyond 2 groups, so it is used to compare 3 or more groups.

Note that there are several versions of the ANOVA (e.g., one-way ANOVA, two-way ANOVA, mixed ANOVA, repeated measures ANOVA, etc.). In this article, we present the simplest form only—the one-way ANOVA¹—and we refer to it as ANOVA in the remaining of the article.

Although ANOVA is used to make inference about means of different groups, the method is called “analysis of variance”. It is called like this because it compares the “between” variance (the variance between the different groups) and the variance “within” (the variance within each group). If the between variance is significantly larger than the within variance, the group means are declared to be different. Otherwise, we cannot conclude one way or the other. The two variances are compared to each other by taking the ratio ($\frac{variance_{between}}{variance_{within}}$) and then by comparing this ratio to a threshold from the Fisher probability distribution (a threshold based on a specific significance level, usually 5%).

This is enough theory regarding the ANOVA method for now. In the remaining of this article, we discuss about it from a more practical point of view, and in particular we will cover the following points:

the aim of the ANOVA, when it should be used and the null/alternative hypothesis
the underlying assumptions of the ANOVA and how to check them
how to perform the ANOVA in R
how to interpret results of the ANOVA
understand the notion of post-hoc test and interpret the results
how to visualize results of ANOVA and post-hoc tests

Data

Data for the present article is the penguins dataset (an alternative to the well-known iris dataset), accessible via the {palmerpenguins} package:

# install.packages("palmerpenguins")
library(palmerpenguins)

The dataset contains data for 344 penguins of 3 different species (Adelie, Chinstrap and Gentoo). The dataset contains 8 variables, but we focus only on the flipper length and the species for this article, so we keep only those 2 variables:

library(tidyverse)

dat <- penguins %>%
  select(species, flipper_length_mm)

Below some basic descriptive statistics and a plot (made with the {ggplot2} package) of our dataset before we proceed to the goal of the ANOVA:

summary(dat)

##       species    flipper_length_mm
##  Adelie   :152   Min.   :172.0    
##  Chinstrap: 68   1st Qu.:190.0    
##  Gentoo   :124   Median :197.0    
##                  Mean   :200.9    
##                  3rd Qu.:213.0    
##                  Max.   :231.0    
##                  NA's   :2

Flipper length varies from 172 to 231 mm, with a mean of 200.9 mm. There are respectively 152, 68 and 124 penguins of the species Adelie, Chinstrap and Gentoo.

library(ggplot2)

ggplot(dat) +
  aes(x = species, y = flipper_length_mm, color = species) +
  geom_jitter() +
  theme(legend.position = "none")

Here, the factor is the species variable which contains 3 modalities or groups (Adelie, Chinstrap and Gentoo).

Aim and hypotheses of ANOVA

As mentioned in the introduction, the ANOVA is used to compare groups (in practice, 3 or more groups). More generally, it is used to:

study whether measurements are similar across different modalities (also called levels or treatments in the context of ANOVA) of a categorical variable
compare the impact of the different levels of a categorical variable on a quantitative variable
explain a quantitative variable based on a qualitative variable

In this context and as an example, we are going to use an ANOVA to help us answer the question: “Is the length of the flippers different between the 3 species of penguins?”.

The null and alternative hypothesis of an ANOVA are:

$H_0$: $\mu_{Adelie} = \mu_{Chinstrap} = \mu_{Gentoo}$ ($\Rightarrow$ the 3 species are equal in terms of flipper length)
$H_1$: at least one mean is different ($\Rightarrow$ at least one species is different from the other 2 species in terms of flipper length)

Be careful that the alternative hypothesis is not that all means are different. The opposite of all means being equal ($H_0$) is that at least one mean is different from the others ($H_1$).

In this sense, if the null hypothesis is rejected, it means that at least one species is different from the other 2, but not necessarily that all 3 species are different from each other. It could be that flipper length for the species Gentoo is different than for the species Chinstrap and Adelie, but flipper length is similar between Chinstrap and Adelie. Other types of test (known as post-hoc tests and covered in this section) must be performed to test whether all 3 species differ.

Underlying assumptions of ANOVA

As for many statistical tests, there are some assumptions that need to be met in order to be able to interpret the results. When one or several assumptions are not met, although it is technically possible to perform these tests, it would be incorrect to interpret the results and trust the conclusions.

Below are the assumptions of the ANOVA, how to test them and which other tests exist if an assumption is not met:

Variable type: ANOVA requires a mix of one continuous quantitative dependent variable (which corresponds to the measurements to which the question relates) and one qualitative independent variable (with at least 2 levels which will determine the groups to compare).
Independence: the data, collected from a representative and randomly selected portion of the total population, should be independent between groups and within each group. The assumption of independence is most often verified based on the design of the experiment and on the good control of experimental conditions rather than via a formal test. If you are still unsure about independence based on the experiment design, ask yourself if one observation is related to another (if one observation has an impact on another) within each group or between the groups themselves. If not, it is most likely that you have independent samples. If observations between samples (forming the different groups to be compared) are dependent (for example, if three measurements have been collected on the same individuals as it is often the case in medical studies when measuring a metric (i) before, (ii) during and (iii) after a treatment), the repeated measures ANOVA should be preferred in order to take into account the dependency between the samples.
Normality:
- In case of small samples, residuals² should follow approximately a normal distribution. The normality assumption can be tested visually thanks to a histogram and a QQ-plot, and/or formally via a normality test such as the Shapiro-Wilk or Kolmogorov-Smirnov test. If, even after a transformation of your data (e.g., logarithmic transformation, square root, Box-Cox, etc.), the residuals still do not follow approximately a normal distribution, the Kruskal-Wallis test can be applied (kruskal.test(variable ~ group, data = dat in R). This non-parametric test, robust to non normal distributions, has the same goal than the ANOVA—compare 3 or more groups—but it uses sample medians instead of sample means to compare groups.
- In case of large samples, normality is not required (this is a common misconception!). By the central limit theorem, sample means of large samples are often well-approximated by a normal distribution even if the data are not normally distributed (Stevens 2013).³ It is therefore not required to test the normality assumption when the number of observations in each group/sample is large (usually $n \ge 30$).
Equality of variances: the variances of the different groups should be equal in the populations (an assumption called homogeneity of the variances, or even sometimes referred as homoscedasticity, as opposed to heteroscedasticity if variances are different across groups). This assumption can be tested graphically (by comparing the dispersion in a boxplot or dotplot for instance), or more formally via the Levene’s test (leveneTest(variable ~ group) from the {car} package) or Bartlett’s test, among others. If the hypothesis of equal variances is rejected, another version of the ANOVA can be used: the Welch ANOVA (oneway.test(variable ~ group, var.equal = FALSE)). Note that the Welch ANOVA does not require homogeneity of the variances, but the distributions should still follow approximately a normal distribution. Note that the Kruskal-Wallis test does not require the assumptions of normality nor homoscedasticity of the variances.⁴
Outliers: An outlier is a value or an observation that is distant from the other observations. There should be no significant outliers in the different groups, or the conclusions of your ANOVA may be flawed. There are several methods to detect outliers in your data but in order to deal with them, it is your choice to either:
- use the non-parametric version (i.e., the Kruskal-Wallis test)
- transform your data (logarithmic or Box-Cox transformation, among others)
- or remove them (be careful)

Choosing the appropriate test depending on whether assumptions are met may be confusing so here is a brief summary:

Check that your observations are independent.
Sample sizes:
- In case of small samples, test the normality of residuals:
  - If normality is assumed, test the homogeneity of the variances:
    - If variances are equal, use ANOVA.
    - If variances are not equal, use the Welch ANOVA.
  - If normality is not assumed, use the Kruskal-Wallis test.
- In case of large samples normality is assumed, so test the homogeneity of the variances:
  - If variances are equal, use ANOVA.
  - If variances are not equal, use the Welch ANOVA.

Now that we have seen the underlying assumptions of the ANOVA, we review them specifically for our dataset before applying the appropriate version of the test.

Variable type

The dependent variable flipper_length_mm is a quantitative variable and the independent variable species is a qualitative one (with 3 levels corresponding to the 3 species). So we have a mix of the two types of variable and this assumption is met.

Independence

Independence of the observations is assumed as data have been collected from a randomly selected portion of the population and measurements within and between the 3 samples are not related.

The independence assumption is most often verified based on the design of the experiment and on the good control of experimental conditions, as it is the case here.

If you really want to test it more formally, you can, however, test it via a statistical test—the Durbin-Watson test (in R: durbinWatsonTest(res_lm) where res_lm is a linear model). The null hypothesis of this test specifies an autocorrelation coefficient = 0, while the alternative hypothesis specifies an autocorrelation coefficient $\ne$ 0.

Normality

Since the smallest sample size per group (i.e., per species) is 68, we have large samples. Therefore, we do not need to check normality.

Usually, we would directly test the homogeneity of the variances without testing normality. However, for the sake of illustration, we act as if the sample sizes were small in order to illustrate what would need to be done in that case.

Remember that normality of residuals can be tested visually via a histogram and a QQ-plot, and/or formally via a normality test (Shapiro-Wilk test for instance).

Before checking the normality assumption, we first need to compute the ANOVA (more on that in this section). We then save the results in res_aov :

res_aov <- aov(flipper_length_mm ~ species,
  data = dat
)

We can now check normality visually:

par(mfrow = c(1, 2)) # combine plots

# histogram
hist(res_aov$residuals)

# QQ-plot
library(car)
qqPlot(res_aov$residuals,
  id = FALSE # id = FALSE to remove point identification
)

From the histogram and QQ-plot above, we can already see that the normality assumption seems to be met. Indeed, the histogram roughly form a bell curve, indicating that the residuals follow a normal distribution. Furthermore, points in the QQ-plots roughly follow the straight line and most of them are within the confidence bands, also indicating that residuals follow approximately a normal distribution.

Some researchers stop here and assume that normality is met, while others also test the assumption via a formal normality test. It is your choice to test it (i) only visually, (ii) only via a normality test, or (iii) both visually AND via a normality test. Bear in mind, however, the two following points:

ANOVA is quite robust to small deviations from normality. This means that it is not an issue (from the perspective of the interpretation of the ANOVA results) if a small number of points deviates slightly from the normality,
normality tests are sometimes quite conservative, meaning that the null hypothesis of normality may be rejected due to a limited deviation from normality. This is especially the case with large samples as power of the test increases with the sample size.

In practice, I tend to prefer the (i) visual approach only, but again, this is a matter of personal choice and also depends on the context of the analysis.

Still for the sake of illustration, we also now test the normality assumption via a normality test. You can use the Shapiro-Wilk test or the Kolmogorov-Smirnov test, among others.

Remember that the null and alternative hypothesis of these tests are:

$H_0$: data come from a normal distribution
$H_1$: data do not come from a normal distribution

In R, we can test normality of the residuals with the Shapiro-Wilk test thanks to the shapiro.test() function:

shapiro.test(res_aov$residuals)

## 
## 	Shapiro-Wilk normality test
## 
## data:  res_aov$residuals
## W = 0.99452, p-value = 0.2609

P-value of the Shapiro-Wilk test on the residuals is larger than the usual significance level of $\alpha = 5\%$, so we do not reject the hypothesis that residuals follow a normal distribution (p-value = 0.261).

This result is in line with the visual approach. In our case, the normality assumption is thus met both visually and formally.

Side note: Remind that the p-value is the probability of having observations as extreme as the ones we have observed in the sample(s) given that the null hypothesis is true. If the p-value $< \alpha$ (indicating that it is not likely to observe the data we have in the sample given that the null hypothesis is true), the null hypothesis is rejected, otherwise the null hypothesis is not rejected. See more about p-value and significance level if you are unfamiliar with those important statistical concepts.

Remember that if the normality assumption was not reached, some transformation(s) would need to be applied on the raw data in the hope that residuals would better fit a normal distribution, or you would need to use the non-parametric version of the ANOVA—the Kruskal-Wallis test.

As pointed out by a reader (see comments at the very end of the article), the normality assumption can also be tested on the “raw” data (i.e., the observations) instead of the residuals. However, if you test the normality assumption on the raw data, it must be tested for each group separately as the ANOVA requires normality in each group.

Testing normality on all residuals or on the observations per group is equivalent, and will give similar results. Indeed, saying “The distribution of Y within each group is normally distributed” is the same as saying “The residuals are normally distributed”.

Remember that residuals are the distance between the actual value of Y and the mean value of Y for a specific value of X, so the grouping variable is induced in the computation of the residuals.

So in summary, in ANOVA you actually have two options for testing normality:

Checking normality separately for each group on the “raw” data (Y values)
Checking normality on all residuals (but not per group)

In practice, you will see that it is often easier to just use the residuals and check them all together, especially if you have many groups or few observations per group.

If you are still not convinced: remember that an ANOVA is a special case of a linear model. Suppose your independent variable is a continuous variable (instead of a categorical variable), the only option you have left is to check normality on the residuals, which is precisely what is done for testing normality in linear regression models.

Equality of variances - homogeneity

Assuming residuals follow a normal distribution, it is now time to check whether the variances are equal across species or not. The result will have an impact on whether we use the ANOVA or the Welch ANOVA.

This can again be verified visually—via a boxplot or dotplot—or more formally via a statistical test (Levene’s test, among others).

Visually, we have:

# Boxplot
boxplot(flipper_length_mm ~ species,
  data = dat
)

# Dotplot
library("lattice")

dotplot(flipper_length_mm ~ species,
  data = dat
)

Both the boxplot and the dotplot show a similar variance for the different species. In the boxplot, this can be seen by the fact that the boxes and the whiskers have a comparable size for all species.

There are a couple of outliers as shown by the points outside the whiskers, but this does not change the fact that the dispersion is more or less the same between the different species.

In the dotplot, this can be seen by the fact that points for all 3 species have more or less the same range, a sign of the dispersion and thus the variance being similar.

Like the normality assumption, if you feel that the visual approach is not sufficient, you can formally test for equality of the variances with a Levene’s or Bartlett’s test. Notice that the Levene’s test is less sensitive to departures from normal distribution than the Bartlett’s test.

The null and alternative hypothesis for both tests are:

$H_0$: variances are equal
$H_1$: at least one variance is different

In R, the Levene’s test can be performed thanks to the leveneTest() function from the {car} package:

# Levene's test
library(car)

leveneTest(flipper_length_mm ~ species,
  data = dat
)

## Levene's Test for Homogeneity of Variance (center = median)
##        Df F value Pr(>F)
## group   2  0.3306 0.7188
##       339

The p-value being larger than the significance level of 0.05, we do not reject the null hypothesis, so we cannot reject the hypothesis that variances are equal between species (p-value = 0.719).

This result is also in line with the visual approach, so the homogeneity of variances is met both visually and formally.

Another method to test normality and homogeneity

For your information, it is also possible to test the homogeneity of the variances and the normality of the residuals visually (and both at the same time) via the plot() function:

par(mfrow = c(1, 2)) # combine plots

# 1. Homogeneity of variances
plot(res_aov, which = 3)

# 2. Normality
plot(res_aov, which = 2)

Plot on the left hand side shows that there is no evident relationships between residuals and fitted values (the mean of each group), so homogeneity of variances is assumed. If homogeneity of variances was violated, the red line would not be flat (horizontal).

Plot on the right hand side shows that residuals follow approximately a normal distribution, so normality is assumed. If normality was violated, points would consistently deviate from the dashed line.

Outliers

There are several techniques to detect outliers. In this article, we focus on the most simple one (yet very efficient)—the visual approach via a boxplot:

boxplot(flipper_length_mm ~ species,
  data = dat
)

There is one outlier in the group Adelie, as defined by the interquartile range criterion. This point is, however, not seen as a significant outlier so we can assume that the assumption of no significant outliers is met.

ANOVA

We showed that all assumptions of the ANOVA are met.

We can thus proceed to the implementation of the ANOVA in R, but first, let’s do some preliminary analyses to better understand the research question.

Preliminary analyses

A good practice before actually performing the ANOVA in R is to visualize the data in relation to the research question. The best way to do so is to draw and compare boxplots of the quantitative variable flipper_length_mm for each species.

This can be done with the boxplot() function in base R (same code than the visual check of equal variances):

boxplot(flipper_length_mm ~ species,
  data = dat
)

Or with the {ggplot2} package:

library(ggplot2)

ggplot(dat) +
  aes(x = species, y = flipper_length_mm) +
  geom_boxplot()

The boxplots above show that, at least for our sample, penguins of the species Gentoo seem to have the biggest flipper, and Adelie species the smallest flipper.

Besides a boxplot for each species, it is also a good practice to compute some descriptive statistics such as the mean and standard deviation by species.

This can be done, for instance, with the aggregate() function:

aggregate(flipper_length_mm ~ species,
  data = dat,
  function(x) round(c(mean = mean(x), sd = sd(x)), 2)
)

##     species flipper_length_mm.mean flipper_length_mm.sd
## 1    Adelie                 189.95                 6.54
## 2 Chinstrap                 195.82                 7.13
## 3    Gentoo                 217.19                 6.48

or with the summarise() and group_by() functions from the {dplyr} package:

library(dplyr)

group_by(dat, species) %>%
  summarise(
    mean = mean(flipper_length_mm, na.rm = TRUE),
    sd = sd(flipper_length_mm, na.rm = TRUE)
  )

## # A tibble: 3 × 3
##   species    mean    sd
##   <fct>     <dbl> <dbl>
## 1 Adelie     190.  6.54
## 2 Chinstrap  196.  7.13
## 3 Gentoo     217.  6.48

Mean is also the lowest for Adelie and highest for Gentoo. Boxplots and descriptive statistics are, however, not enough to conclude that flippers are significantly different in the 3 populations of penguins.

ANOVA in R

As you guessed by now, only the ANOVA can help us to make inference about the population given the sample at hand, and help us to answer the initial research question “Is the length of the flippers different between the 3 species of penguins?”.

ANOVA in R can be done in several ways, of which two are presented below:

With the oneway.test() function:

# 1st method:
oneway.test(flipper_length_mm ~ species,
  data = dat,
  var.equal = TRUE # assuming equal variances
)

## 
## 	One-way analysis of means
## 
## data:  flipper_length_mm and species
## F = 594.8, num df = 2, denom df = 339, p-value < 2.2e-16

With the summary() and aov() functions:

# 2nd method:
res_aov <- aov(flipper_length_mm ~ species,
  data = dat
)

summary(res_aov)

##              Df Sum Sq Mean Sq F value Pr(>F)    
## species       2  52473   26237   594.8 <2e-16 ***
## Residuals   339  14953      44                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 2 observations deleted due to missingness

As you can see from the two outputs above, the test statistic (F = in the first method and F value in the second one) and the p-value (p-value in the first method and Pr(>F) in the second one) are exactly the same for both methods, which means that in case of equal variances, results and conclusions will be unchanged.

The advantage of the first method is that it is easy to switch from the ANOVA (used when variances are equal) to the Welch ANOVA (used when variances are unequal). This can be done by replacing var.equal = TRUE by var.equal = FALSE, as presented below:

oneway.test(flipper_length_mm ~ species,
  data = dat,
  var.equal = FALSE # assuming unequal variances
)

## 
## 	One-way analysis of means (not assuming equal variances)
## 
## data:  flipper_length_mm and species
## F = 614.01, num df = 2.00, denom df = 172.76, p-value < 2.2e-16

The advantage of the second method, however, is that:

the full ANOVA table (with degrees of freedom, mean squares, etc.) is printed, which may be of interest in some (theoritical) cases
results of the ANOVA (res_aov) can be saved for later use (especially useful for post-hoc tests)

Interpretations of ANOVA results

Given that the p-value is smaller than 0.05, we reject the null hypothesis, so we reject the hypothesis that all means are equal. Therefore, we can conclude that at least one species is different than the others in terms of flippers length (p-value < 2.2e-16).

(For the sake of illustration, if the p-value was larger than 0.05: we cannot reject the null hypothesis that all means are equal, so we cannot reject the hypothesis that the 3 considered species of penguins are equal in terms of flippers length.)

A nice and easy way to report results of an ANOVA in R is with the report() function from the {report} package:

# install.packages("remotes")
# remotes::install_github("easystats/report") # You only need to do that once
library("report") # Load the package every time you start R

report(res_aov)

## The ANOVA (formula: flipper_length_mm ~ species) suggests that:
## 
##   - The main effect of species is statistically significant and large (F(2, 339)
## = 594.80, p < .001; Eta2 = 0.78, 95% CI [0.75, 1.00])
## 
## Effect sizes were labelled following Field's (2013) recommendations.

As you can see, the function interprets the results for you and indicates a large and significant main effect of the species on the flipper length (p-value < .001).

Note that the report() function can be used for other analyses. See more tips and tricks in R if you find this one useful.

What’s next?

If the null hypothesis is not rejected (p-value $\ge$ 0.05), it means that we do not reject the hypothesis that all groups are equal. The ANOVA more or less stops here.

Other types of analyses can be performed of course, but—given the data at hand—we could not prove that at least one group was different so we usually do not go further with the ANOVA.

On the contrary, if the null hypothesis is rejected (as it is our case since the p-value < 0.05), we proved that at least one group is different. We can decide to stop here if we are only interested to test whether all species are equal in terms of flippers length.

But most of the time, when we showed thanks to an ANOVA that at least one group is different, we are also interested in knowing which one(s) is(are) different. Results of an ANOVA, however, do NOT tell us which group(s) is(are) different from the others.

To test this, we need to use other types of test, referred as post-hoc tests (in Latin, “after this”, so after obtaining statistically significant ANOVA results) or multiple pairwise-comparison tests.⁵

This family of statistical tests is the topic of the following sections.

Post-hoc test

Issue of multiple testing

In order to see which group(s) is(are) different from the others, we need to compare groups 2 by 2. In practice, since there are 3 species, we are going to compare species 2 by 2 as follows:

Chinstrap versus Adelie
Gentoo vs. Adelie
Gentoo vs. Chinstrap

In theory, we could compare species thanks to 3 Student’s t-tests since we need to compare 2 groups and a t-test is used precisely in that case.

However, if several t-tests are performed, the issue of multiple testing (also referred as multiplicity) arises. In short, when several statistical tests are performed, some will have p-values less than $\alpha$ purely by chance, even if all null hypotheses are in fact true.

To demonstrate the problem, consider our case where we have 3 hypotheses to test and a desired significance level of 0.05.

The probability of observing at least one significant result (at least one p-value < 0.05) just due to chance is:

\[\begin{equation} \begin{split} P(\text{at least 1 sig. result}) & = 1 - P(\text{no sig. results}) \\ & = 1 - (1 - 0.05)^3 \\ & = 0.142625 \end{split} \end{equation}\]

So, with as few as 3 tests being considered, we already have a 14.26% chance of observing at least one significant result, even if all of the tests are actually not significant.

And as the number of groups increases, the number of comparisons increases as well, so the probability of having a significant result simply due to chance keeps increasing.

For example, with 10 groups we need to make 45 comparisons and the probability of having at least one significant result by chance becomes $1 - (1 - 0.05)^{45} = 90\%$. So it is very likely to observe a significant result just by chance when comparing 10 groups, and when we have 14 groups or more we are almost certain (99%) to have a false positive!

Post-hoc tests take into account that multiple tests are done and deal with the problem by adjusting $\alpha$ in some way, so that the probability of observing at least one significant result due to chance remains below our desired significance level.⁶

Post-hoc tests in R and their interpretation

Post-hoc tests are a family of statistical tests so there are several of them. The most common ones are:

Tukey HSD, used to compare all groups to each other (so all possible comparisons of 2 groups).
Dunnett, used to make comparisons with a reference group. For example, consider 2 treatment groups and one control group. If you only want to compare the 2 treatment groups with respect to the control group, and you do not want to compare the 2 treatment groups to each other, the Dunnett’s test is preferred.
Bonferroni correction if one has a set of planned comparisons to do.

The Bonferroni correction is simple: you simply divide the desired global $\alpha$ level by the number of comparisons.

In our example, we have 3 comparisons so if we want to keep a global $\alpha = 0.05$, we have $\alpha' = \frac{0.05}{3} = 0.0167$. We can then simply perform a Student’s t-test for each comparison, and compare the obtained $p$-values with this new $\alpha'$.

The other two post-hoc tests are presented in the next sections.

Note that variances are assumed to be equal for all three methods (unless you use the Welch’s t-test instead of the Student’s t-test with the Bonferroni correction). If variances are not equal, you can use the Games-Howell test, among others.

Tukey HSD test

In our case, since there is no “reference” species and we are interested in comparing all species, we are going to use the Tukey HSD test.

In R, the Tukey HSD test is done as follows. This is where the second method to perform the ANOVA comes handy because the results (res_aov) are reused for the post-hoc test:

library(multcomp)

# Tukey HSD test:
post_test <- glht(res_aov,
  linfct = mcp(species = "Tukey")
)

summary(post_test)

## 
## 	 Simultaneous Tests for General Linear Hypotheses
## 
## Multiple Comparisons of Means: Tukey Contrasts
## 
## 
## Fit: aov(formula = flipper_length_mm ~ species, data = dat)
## 
## Linear Hypotheses:
##                         Estimate Std. Error t value Pr(>|t|)    
## Chinstrap - Adelie == 0   5.8699     0.9699   6.052 1.03e-08 ***
## Gentoo - Adelie == 0     27.2333     0.8067  33.760  < 1e-08 ***
## Gentoo - Chinstrap == 0  21.3635     1.0036  21.286  < 1e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)

In the output of the Tukey HSD test, we are interested in the table displayed after Linear Hypotheses:, and more precisely, in the first and last column of the table. The first column shows the comparisons which have been made; the last column (Pr(>|t|)) shows the adjusted⁷ p-values for each comparison (with the null hypothesis being the two groups are equal and the alternative hypothesis being the two groups are different).

It is these adjusted p-values that are used to test whether two groups are significantly different or not, and we can be confident that the entire set of comparisons collectively has an error rate of 0.05.

In our example, we tested:

Chinstrap versus Adelie (line Chinstrap - Adelie == 0)
Gentoo vs. Adelie (line Gentoo - Adelie == 0)
Gentoo vs. Chinstrap (line Gentoo - Chinstrap == 0)

All three ajusted p-values are smaller than 0.05, so we reject the null hypothesis for all comparisons, which means that all species are significantly different in terms of flippers length.

The results of the post-hoc test can be visualized with the plot() function:

par(mar = c(3, 8, 3, 3))
plot(post_test)

We see that the confidence intervals do not cross the zero line, which indicate that all groups are significantly different.

Note that the Tukey HSD test can also be done in R with the TukeyHSD() function:

TukeyHSD(res_aov)

##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = flipper_length_mm ~ species, data = dat)
## 
## $species
##                       diff       lwr       upr p adj
## Chinstrap-Adelie  5.869887  3.586583  8.153191     0
## Gentoo-Adelie    27.233349 25.334376 29.132323     0
## Gentoo-Chinstrap 21.363462 19.000841 23.726084     0

With this code, it is the column p adj (also the last column) which is of interest. Notice that the conclusions are the same than above: all species are significantly different in terms of flippers length.

The results can also be visualized with the plot() function:

plot(TukeyHSD(res_aov))

Dunnett’s test

We have seen in this section that as the number of groups increases, the number of comparisons also increases. And as the number of comparisons increases, the post-hoc analysis must lower the individual significance level even further, which leads to lower statistical power (so a difference between group means in the population is less likely to be detected).

One method to mitigate this and increase the statistical power is by reducing the number of comparisons. This reduction allows the post-hoc procedure to use a larger individual error rate to achieve the desired global error rate.

While comparing all possible groups with a Tukey HSD test is a common approach, many studies have a control group and several treatment groups. For these studies, you may need to compare the treatment groups only to the control group, which reduces the number of comparisons.

Dunnett’s test does precisely this—it only compares a group taken as reference to all other groups, but it does not compare all groups to each others.

So to recap:

the Tukey HSD test allows to compares all groups but at the cost of less power
the Dunnett’s test allows to only make comparisons with a reference group, but with the benefit of more power

Now, again for the sake of illustration, consider that the species Adelie is the reference species and we are only interested in comparing the reference species against the other 2 species. In that scenario, we would use the Dunnett’s test.

In R, the Dunnett’s test is done as follows (the only difference with the code for the Tukey HSD test is in the line linfct = mcp(species = "Dunnett")):

library(multcomp)

# Dunnett's test:
post_test <- glht(res_aov,
  linfct = mcp(species = "Dunnett")
)

summary(post_test)

## 
## 	 Simultaneous Tests for General Linear Hypotheses
## 
## Multiple Comparisons of Means: Dunnett Contrasts
## 
## 
## Fit: aov(formula = flipper_length_mm ~ species, data = dat)
## 
## Linear Hypotheses:
##                         Estimate Std. Error t value Pr(>|t|)    
## Chinstrap - Adelie == 0   5.8699     0.9699   6.052 7.59e-09 ***
## Gentoo - Adelie == 0     27.2333     0.8067  33.760  < 1e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)

The interpretation is the same as for the Tukey HSD test’s except that in the Dunett’s test we only compare:

Chinstrap versus Adelie (line Chinstrap - Adelie == 0)
Gentoo vs. Adelie (line Gentoo - Adelie == 0)

Both adjusted p-values (displayed in the last column) are below 0.05, so we reject the null hypothesis for both comparisons.

This means that both the species Chinstrap and Gentoo are significantly different from the reference species Adelie in terms of flippers length. (Nothing can be said about the comparison between Chinstrap and Gentoo though.)

Again, the results of the post-hoc test can be visualized with the plot() function:

par(mar = c(3, 8, 3, 3))
plot(post_test)

We see that the confidence intervals do not cross the zero line, which indicate that both the species Gentoo and Chinstrap are significantly different from the reference species Adelie.

Note that in R, by default, the reference category for a factor variable is the first category in alphabetical order. This is the reason that, by default, the reference species is Adelie.

The reference category can be changed with the relevel() function (or with the {questionr} addin). Considering that we want Gentoo as the reference category instead of Adelie:

# Change reference category:
dat$species <- relevel(dat$species, ref = "Gentoo")

# Check that Gentoo is the reference category:
levels(dat$species)

## [1] "Gentoo"    "Adelie"    "Chinstrap"

Gentoo now being the first category of the three, it is indeed considered as the reference level.

In order to perform the Dunnett’s test with the new reference we first need to rerun the ANOVA to take into account the new reference:

res_aov2 <- aov(flipper_length_mm ~ species,
  data = dat
)

We can then run the Dunett’s test with the new results of the ANOVA:

# Dunnett's test:
post_test <- glht(res_aov2,
  linfct = mcp(species = "Dunnett")
)

summary(post_test)

## 
## 	 Simultaneous Tests for General Linear Hypotheses
## 
## Multiple Comparisons of Means: Dunnett Contrasts
## 
## 
## Fit: aov(formula = flipper_length_mm ~ species, data = dat)
## 
## Linear Hypotheses:
##                         Estimate Std. Error t value Pr(>|t|)    
## Adelie - Gentoo == 0    -27.2333     0.8067  -33.76   <1e-10 ***
## Chinstrap - Gentoo == 0 -21.3635     1.0036  -21.29   <1e-10 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)

par(mar = c(3, 8, 3, 3))
plot(post_test)

From the results above we conclude that Adelie and Chinstrap species are significantly different from Gentoo species in terms of flippers length (adjusted p-values < 1e-10).

Note that even if your study does not have a reference group which you can compare to the other groups, it is still often better to do multiple comparisons determined by some research questions than to do all-pairwise tests. By reducing the number of post-hoc comparisons to what is necessary only, and no more, you maximize the statistical power.⁸

Other p-values adjustment methods

For the interested readers, note that you can use other p-values adjustment methods by using the pairwise.t.test() function:

pairwise.t.test(dat$flipper_length_mm, dat$species,
  p.adjust.method = "holm"
)

## 
## 	Pairwise comparisons using t tests with pooled SD 
## 
## data:  dat$flipper_length_mm and dat$species 
## 
##           Gentoo  Adelie 
## Adelie    < 2e-16 -      
## Chinstrap < 2e-16 3.8e-09
## 
## P value adjustment method: holm

By default, the Holm method is applied but other methods exist. See ?p.adjust for all available options.

Visualization of ANOVA and post-hoc tests on the same plot

If you are interested in including results of ANOVA and post-hoc tests on the same plot (directly on the boxplots), here are two pieces of code which may be of interest to you.

The first one is edited by me based on the code found in this article:

# Edit from here
x <- which(names(dat) == "species") # name of grouping variable
y <- which(
  names(dat) == "flipper_length_mm" # names of variables to test
)
method1 <- "anova" # one of "anova" or "kruskal.test"
method2 <- "t.test" # one of "wilcox.test" or "t.test"
my_comparisons <- list(c("Chinstrap", "Adelie"), c("Gentoo", "Adelie"), c("Gentoo", "Chinstrap")) # comparisons for post-hoc tests
# Edit until here


# Edit at your own risk
library(ggpubr)
for (i in y) {
  for (j in x) {
    p <- ggboxplot(dat,
      x = colnames(dat[j]), y = colnames(dat[i]),
      color = colnames(dat[j]),
      legend = "none",
      palette = "npg",
      add = "jitter"
    )
    print(
      p + stat_compare_means(aes(label = paste0(after_stat(method), ", p-value = ", after_stat(p.format))),
        method = method1, label.y = max(dat[, i], na.rm = TRUE)
      )
      + stat_compare_means(comparisons = my_comparisons, method = method2, label = "p.format") # remove if p-value of ANOVA or Kruskal-Wallis test >= alpha
    )
  }
}

And the second method is from the {ggstatsplot} package:

library(ggstatsplot)

ggbetweenstats(
  data = dat,
  x = species,
  y = flipper_length_mm,
  type = "parametric", # ANOVA or Kruskal-Wallis
  var.equal = TRUE, # ANOVA or Welch ANOVA
  plot.type = "box",
  pairwise.comparisons = TRUE,
  pairwise.display = "significant",
  centrality.plotting = FALSE,
  bf.message = FALSE
)

As you can see on the above plot, boxplots by species are presented together with p-values of the ANOVA (after p = in the subtitle of the plot) and p-values of the post-hoc tests (above each comparison).

Besides the fact that these methods can be used to combine a visual representation and statistical results on the same plot, they also have the advantage that you can perform multiple ANOVA tests at once. See more information in this article.

Summary

In this article, we reviewed the goals and hypotheses of an ANOVA, what are the assumptions which need to be verified before being able to trust the results (namely, independence, normality and homogeneity), we then showed how to do an ANOVA in R and how to interpret the results.

An article about ANOVA would not be complete without discussing about post-hoc tests, and in particular, the Tukey HSD—to compare all groups—and the Dunnett’s test—to compare a reference group to all other groups.

Last but not least, we showed how to visualize the data and the results of the ANOVA and post-hoc tests in the same plot.

Thanks for reading. See this tutorial if you would like to learn how to do an ANOVA by hand.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

(Note that this article is available for download on my Gumroad page.)

References

Hsu, Jason. 1996. Multiple Comparisons: Theory and Methods. CRC Press.

Stevens, James P. 2013. Intermediate Statistics: A Modern Approach. Routledge.

Note that it is called one-way or one-factor ANOVA because the means relate to the different modalities of a single independent variable, or factor.↩︎
Residuals (denoted $\epsilon$) are the differences between the observed values of the dependent variable ($y$) and the predicted values ($\hat{y}$). In the context of ANOVA, residuals correspond to the differences between the observed values and the mean of all values for that group.↩︎
Stevens (2013) wrote, in p. 57, “Numerous studies have examined the effect of violations of assumptions in ANOVA, and an excellent summary of this literature has been provided by Glass, Peckham, and Sanders (1972). Their review indicates that non normality has only a slight effect on the type I error rate, even for very skewed or kurtotic distributions. For example, the actual $\alpha$s for some very non-normal populations were only .055 or .06: very minor deviations from the nominal level of .05. […] The basic reason is the Central Limit Theorem, which states that the sum of independent observations having any distribution whatsoever approaches a normal distribution as the number of observations increases. To be somewhat more specific, Bock (1975) notes,”even for distributions which depart markedly from normality, sums of 50 or more observations approximate to normality. For moderately non-normal distributions the approximation is good with as few as 10 to 20 observations” (p. 111). Now since the sums of independent observations approach normality rapidly, so do the means, and the sampling distribution of F is based on means. Thus the sampling distribution of F is only slightly affected, and therefore the critical values when sampling from normal and non-normal distributions will not differ by much. Lack of normality due to skewness also has only a slight effect on power (a few hundredths).”↩︎
As long as you use the Kruskal-Wallis test to, in fine, compare groups, homoscedasticity is not required. If you wish to compare medians, the Kruskal-Wallis test requires homoscedasticity. See more information about the difference in this article.↩︎
Note that, as discussed in the comments at the end of the article, post-hoc tests can under some circumstances be done directly (without an ANOVA). See the comments or Hsu (1996) for more details.↩︎
Note that you could in principle apply the Bonferroni correction to all tests. For example, in the example above, with 3 tests and a global desired significance level of $\alpha$ = 0.05, we would only reject a null hypothesis if the p-value is less than $\frac{0.05}{3}$ = 0.0167. This method is, however, known to be quite conservative, leading to a potentially high rate of false negatives.↩︎
The p-values are adjusted to keep the global significance level to the desired level.↩︎
Thanks Michael Friendly for this suggestion.↩︎

Graphics in R with ggplot2

Fri, 21 Aug 2020 00:00:00 +0000

Note that this article is inspired from the lecture notes of Prof. Anouar El Ghouch and my personal notes as teaching assistant for his course entitled “Statistics and data sciences with R: Advanced programming” given at UCLouvain.

Introduction

R is known to be a really powerful programming language when it comes to graphics and visualizations (in addition to statistics and data science of course!).

To keep it short, graphics in R can be done in three ways, via the:

{graphics} package (the base graphics in R, loaded by default)
{lattice} package which adds more functionalities to the base package
{ggplot2} package (which needs to be installed and loaded beforehand)

The {graphics} package comes with a large choice of plots (such as plot, hist, barplot, boxplot, pie, mosaicplot, etc.) and additional related features (e.g., abline, lines, legend, mtext, rect, etc.). It is often the preferred way to draw plots for most R users, and in particular for beginners to intermediate users.

Since its creation in 2005 by Hadley Wickham, {ggplot2} has grown in use to become one of the most popular R packages and the most popular package for graphics and data visualizations. The {ggplot2} package is a much more modern approach to creating professional-quality graphics. More information about the package can be found at ggplot2.tidyverse.org.

In this article, we will see how to create common plots such as scatter plots, line plots, histograms, boxplots, barplots, density plots in R with this package. If you are unfamiliar with any of these types of graph, you will find more information about each one (when to use it, its purpose, what does it show, etc.) in my article about descriptive statistics in R.

Data

To illustrate plots with the {ggplot2} package we will use the mpg dataset available in the package.

The dataset contains observations collected by the US Environmental Protection Agency on fuel economy from 1999 to 2008 for 38 popular models of cars (run ?mpg for more information about the data):

library(ggplot2)
dat <- ggplot2::mpg

Before going further, let’s transform the cyl, drv, fl, year and class variables in factor with the transform() function:

dat <- transform(dat,
  cyl = factor(cyl),
  drv = factor(drv),
  fl = factor(fl),
  year = factor(year),
  class = factor(class)
)

For the interested reader, see more data manipulation techniques in R.

Basic principles of `{ggplot2}`

The {ggplot2} package is based on the principles of “The Grammar of Graphics” (hence “gg” in the name of {ggplot2}), that is, a coherent system for describing and building graphs. The main idea is to design a graphic as a succession of layers.

The main layers are:

The dataset that contains the variables that we want to represent. This is done with the ggplot() function and comes first.
The variable(s) to represent on the x and/or y-axis, and the aesthetic elements (such as color, size, fill, shape and transparency) of the objects to be represented. This is done with the aes() function (abbreviation of aesthetic).
The type of graphical representation (scatter plot, line plot, barplot, histogram, boxplot, etc.). This is done with the functions geom_point(), geom_line(), geom_bar(), geom_histogram(), geom_boxplot(), etc.
If needed, additional layers (such as labels, annotations, scales, axis ticks, legends, themes, facets, etc.) can be added to personalize the plot.

To create a plot, we thus first need to specify the data in the ggplot() function and then add the required layers such as the variables, the aesthetic elements and the type of plot:

ggplot(data) +
  aes(x = var_x, y = var_y) +
  geom_x()

data in ggplot() is the name of the data frame which contains the variables var_x and var_y.
The + symbol is used to indicate the different layers that will be added to the plot. Make sure to write the + symbol at the end of the line of code and not at the beginning of the line, otherwise R throws an error.
The layer aes() indicates what variables will be used in the plot and more generally, the aesthetic elements of the plot.
Finally, x in geom_x() represents the type of plot.
Other layers are usually not required unless we want to personalize the plot further.

Note that it is a good practice to write one line of code per layer to improve code readability.

Create plots with `{ggplot2}`

In the following sections we will show how to draw the following plots:

scatter plot
line plot
histogram
density plot
dotplot
boxplot
barplot
raincloud plot

In order to focus on the construction of the different plots and the use of {ggplot2}, we will restrict ourselves to drawing basic (yet beautiful) plots without unnecessary layers. For the sake of completeness, we will briefly discuss and illustrate different layers to further personalize a plot at the end of the article (see this section).

Note that if you still struggle to create plots with {ggplot2} after reading this tutorial, you may find the {esquisse} addin useful. This addin allows you to interactively (that is, by dragging and dropping variables) create plots with the {ggplot2} package. Give it a try!

Scatter plot

We start by creating a scatter plot using geom_point. Remember that a scatter plot is used to visualize the relation between two quantitative variables.

We start by specifying the data:

ggplot(dat) # data

Then we add the variables to be represented with the aes() function:

ggplot(dat) + # data
  aes(x = displ, y = hwy) # variables

Finally, we indicate the type of plot:

ggplot(dat) + # data
  aes(x = displ, y = hwy) + # variables
  geom_point() # type of plot

You will also sometimes see the aesthetic elements (aes() with the variables) inside the ggplot() function in addition to the dataset:

ggplot(mpg, aes(x = displ, y = hwy)) +
  geom_point()

This second method gives the exact same plot than the first method. I tend to prefer the first method over the second for better readability, but this is more a matter of taste so the choice is up to you.

Line plot

Line plots, particularly useful in time series or finance, can be created similarly but by using geom_line():

ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_line()

(Note that this might not be the most appropriate plot since there are multiple points for each value of displ, but this is just an example to show how to create a line plot.)

Combination of line and points

An advantage of {ggplot2} is the ability to combine several types of plots and its flexibility in designing it. For instance, we can add a line to a scatter plot by simply adding a layer to the initial scatter plot:

ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point() +
  geom_line() # add line

Histogram

A histogram (useful to visualize distributions and detect potential outliers) can be plotted using geom_histogram():

ggplot(dat) +
  aes(x = hwy) +
  geom_histogram()

By default, the number of bins is equal to 30. You can change this value using the bins argument inside the geom_histogram() function:

ggplot(dat) +
  aes(x = hwy) +
  geom_histogram(bins = round(sqrt(nrow(dat))))

Here I specify the number of bins to be equal to the square root of the number of observations (following the square-root rule) but you can specify any integer number.

Density plot

Density plots can be created using geom_density():

ggplot(dat) +
  aes(x = hwy) +
  geom_density()

Combination of histogram and densities

We can also superimpose a histogram and a density curve on the same plot:

ggplot(dat) +
  aes(x = hwy, y = after_stat(density)) +
  geom_histogram() +
  geom_density()

Or superimpose several densities:

ggplot(dat) +
  aes(x = hwy, color = drv, fill = drv) +
  geom_density(alpha = 0.25) # add transparency

The argument alpha = 0.25 has been added for some transparency. More information about this argument can be found in this section.

Dotplot

A dotplot in {ggplot2} can be built with geom_dotplot():

# Dotplot for one variable
ggplot(dat) +
  aes(x = "", y = hwy) +
  geom_dotplot(binaxis = "y", stackdir = "center")

# Dotplot by factor
ggplot(dat) +
  aes(x = drv, y = hwy) +
  geom_dotplot(
    binaxis = "y", stackdir = "center",
    dotsize = 0.75 # decrease dot size
  )

Dotplots are more appropriate with small samples (because the plot may be hard to read with too many points). For large samples, a boxplot may be used.

For the interested reader, see many personalization that is possible with a dotplot in this tutorial.

Boxplot

A boxplot (also very useful to visualize distributions and detect potential outliers) can be plotted using geom_boxplot():

# Boxplot for one variable
ggplot(dat) +
  aes(x = "", y = hwy) +
  geom_boxplot()

# Boxplot by factor
ggplot(dat) +
  aes(x = drv, y = hwy) +
  geom_boxplot()

It is also possible to plot the points on the boxplot with geom_jitter(), and to vary the width of the boxes according to the size (i.e., the number of observations) of each level with varwidth = TRUE:

ggplot(dat) +
  aes(x = drv, y = hwy) +
  geom_boxplot(varwidth = TRUE) + # vary boxes width according to n obs.
  geom_jitter(alpha = 0.25, width = 0.2) # adds random noise and limit its width

The geom_jitter() layer adds some random variation to each point in order to prevent them from overlapping (an issue known as overplotting).¹ Moreover, the alpha argument adds some transparency to the points (see more in this section) to keep the focus on the boxes and not on the points.

Finally, it is also possible to divide boxplots into several panels according to the levels of a qualitative variable:

ggplot(dat) +
  aes(x = drv, y = hwy) +
  geom_boxplot(varwidth = TRUE) + # vary boxes width according to n obs.
  geom_jitter(alpha = 0.25, width = 0.2) + # adds random noise and limit its width
  facet_wrap(~year) # divide into 2 panels

For a visually more appealing plot, it is also possible to use some colors for the boxes depending on the x variable:

ggplot(dat) +
  aes(x = drv, y = hwy, fill = drv) + # add color to boxes with fill
  geom_boxplot(varwidth = TRUE) + # vary boxes width according to n obs.
  geom_jitter(alpha = 0.25, width = 0.2) + # adds random noise and limit its width
  facet_wrap(~year) + # divide into 2 panels
  theme(legend.position = "none") # remove legend

In that case, it best to remove the legend as it becomes redundant. See more information about the legend in this section.

If you are unhappy with the default colors provided in {ggplot2}, you can change them manually with the scale_fill_manual() layer:

ggplot(dat) +
  aes(x = drv, y = hwy, fill = drv) + # add color to boxes with fill
  geom_boxplot(varwidth = TRUE) + # vary boxes width according to n obs.
  geom_jitter(alpha = 0.25, width = 0.2) + # adds random noise and limit its width
  facet_wrap(~year) + # divide into 2 panels
  theme(legend.position = "none") + # remove legend
  scale_fill_manual(values = c("darkred", "darkgreen", "steelblue")) # change fill color manually

Barplot

A barplot (useful to visualize qualitative variables) can be plotted using geom_bar():

ggplot(dat) +
  aes(x = drv) +
  geom_bar()

Bars’ heights correspond to the observed frequencies (i.e., the number of observations) for each level of the variable of interest (drv in our case).

Tip: by default, the order of the bars follows the initial order (by alphabetical order or numerical order if you did not change it). If you want to order the levels by frequency (largest first), use the fct_infreq() function from the {forcats} package.

library(forcats)

ggplot(dat) +
  aes(x = fct_infreq(drv)) + # order by frequency
  geom_bar()

If you want to order levels in an increasing order (i.e., category with the smallest frequency first), use the fct_rev() in addition to the fct_infreq() function:

ggplot(dat) +
  aes(x = fct_rev(fct_infreq(drv))) + # order by frequency
  geom_bar()

(Label for the x-axis can then easily be edited with the labs() function. See below for more information.)

Again, for a more appealing plot, we can add some colors to the bars with the fill argument:

ggplot(dat) +
  aes(x = drv, fill = drv) + # add colors to bars
  geom_bar() +
  theme(legend.position = "none") # remove legend

We can also create a barplot with two qualitative variables:

ggplot(dat) +
  aes(x = drv, fill = year) + # fill by years
  geom_bar()

In order to compare proportions across groups, it is best to make each bar the same height using position = "fill":

ggplot(dat) +
  geom_bar(aes(x = drv, fill = year), position = "fill")

To draw the bars next to each other for each group, use position = "dodge":

ggplot(dat) +
  geom_bar(aes(x = drv, fill = year), position = "dodge")

Raincloud plot

A raincloud plot is a graph that combines 3 visualizations:

a density plot,
a boxplot,
and the raw data in the form of a dotplot or jittered points.

The advantage of this plot is that it illustrates, all at once, the distribution (with the density curve), the summary measures (first, second and third quartiles, and maximum/mininum without outliers thanks to the boxplot) and the number of observations (either via a dotplot or via jittered points).

Let’s illustrate the raincloud plot, first with jittered points (more appropriate with large samples):²

library(tidyverse)
library(ggdist)

# density plot:
dat %>%
  ggplot(aes(x = drv, y = hwy, fill = drv)) +
  stat_halfeye(
    adjust = 0.5, # set the smoothing parameter
    width = 0.5, # set the height of the curves
    justification = -0.2, # move curves to the right
    .width = 0, point_colour = NA # remove interval present by default
  ) +
  # boxplot:
  geom_boxplot(
    width = 0.12, # width of boxes
    outlier.color = NA, # remove color of outliers
    alpha = 0.5 # add transparency
  ) +
  # jittered points:
  geom_point(aes(colour = drv), # add color on points
    size = 1.3, # size of points
    alpha = .3, # add transparency
    position = position_jitter( # obtain shifted points
      seed = 1, # set seed for same random representation
      width = .09 # manage the width of the offset
    )
  ) +
  # further personalization:
  coord_flip() + # rotate plot
  theme(legend.position = "none") # remove legend

Now the same chart but with dotplots this time (more appropriate with small samples):

# density plot:
dat %>%
  sample_n(100) %>% # random sample of size 100
  ggplot(aes(x = drv, y = hwy, fill = drv)) +
  stat_halfeye(
    adjust = 0.5, # set the smoothing parameter
    width = 0.5, # set the height of the curves
    justification = -0.2, # move curves to the right
    .width = 0, point_colour = NA # remove interval present by default
  ) +
  # boxplot:
  geom_boxplot(
    width = 0.12, # width of boxes
    outlier.color = NA, # remove color of outliers
    alpha = 0.5 # add transparency
  ) +
  # dotplot:
  stat_dots(
    dotsize = 0.5, # size of points
    side = "left", # place points on opposite side of density curve
    justification = 1.1, # move points away from boxplot
    binwidth = 1 # group points together
  ) +
  # further personalization:
  coord_flip() + # rotate plot
  theme(legend.position = "none") # remove legend

The code is much longer compared to the other plots, but the only line(s) to edit to adapt to your dataset is the aesthetics (aes()). The rest is mainly adjustments that should not be changed.

You may have notice that at the end of the code, there are some personalization which allow to improve the plot even further. The most common personalization are presented in the next section.

Further personalization

Title and axis labels

The first things to personalize in a plot is the labels to make the plot more informative to the audience. We can easily add a title, subtitle, caption and edit axis labels with the labs() function:

p <- ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point()

p + labs(
  title = "Fuel efficiency for 38 popular models of car",
  subtitle = "Period 1999-2008",
  caption = "Data: ggplot2::mpg. See more at statsandr.com",
  x = "Engine displacement (litres)",
  y = "Highway miles per gallon (mpg)"
)

As you can see in the above code, you can save one or more layers of the plot in an object for later use.

This way, you can save your “main” plot, and add more layers of personalization until you get the desired output. Here we saved the main scatter plot in an object called p and we will refer to it for the subsequent personalization.

You can also edit the alignment, the size and the shape of the title and subtitle via the theme() layer and the element_text() function:

p + labs(
  title = "Fuel efficiency for 38 popular models of car",
  subtitle = "Period 1999-2008",
  caption = "Data: ggplot2::mpg. See more at statsandr.com",
  x = "Engine displacement (litres)",
  y = "Highway miles per gallon (mpg)"
) +
  theme(
    plot.title = element_text(
      hjust = 0.5, # center
      size = 12,
      color = "steelblue",
      face = "bold"
    ),
    plot.subtitle = element_text(
      hjust = 0.5, # center
      size = 10,
      color = "gray",
      face = "italic"
    )
  )

If the title or subtitle is long and you want to divide it into multiple lines, use \n:

p + labs(
  title = "Fuel efficiency for 38 popular \n models of car",
  subtitle = "Period 1999-2008",
  caption = "Data: ggplot2::mpg. See more at statsandr.com",
  x = "Engine displacement (litres)",
  y = "Highway miles per gallon (mpg)"
) +
  theme(
    plot.title = element_text(
      hjust = 0.5, # center
      size = 12,
      color = "steelblue",
      face = "bold"
    ),
    plot.subtitle = element_text(
      hjust = 0.5, # center
      size = 10,
      color = "gray",
      face = "italic"
    )
  )

Axis ticks

Axis ticks can be adjusted using scale_x_continuous() and scale_y_continuous() for the x and y-axis, respectively:

# Adjust ticks
p + scale_x_continuous(breaks = seq(from = 1, to = 7, by = 0.5)) + # x-axis
  scale_y_continuous(breaks = seq(from = 10, to = 45, by = 5)) # y-axis

Log transformations

In some cases, it is useful to plot the log transformation of the variables. This can be done with the scale_x_log10() and scale_y_log10() functions:

p + scale_x_log10() +
  scale_y_log10()

Limits

The most convenient way to control the limits of the plot is to use again the scale_x_continuous() and scale_y_continuous() functions in addition to the limits argument:

p + scale_x_continuous(limits = c(3, 6)) +
  scale_y_continuous(limits = c(20, 30))

It is also possible to simply take a subset of the dataset with the subset() or filter() function. See how to subset a dataset if you need a reminder.

Scales for better axis formats

Depending on your data, it is possible to format axes in a certain way with the {scales} package. The format I use the most is comma which formats large numbers in a more-readable way.

For this example, we multiply both variables by 10000 to have larger numbers and then we apply the format to the y-axis (only to the y-axis so we can see the difference with the x-axis which is not formatted):

ggplot(dat) +
  aes(x = displ * 10000, y = hwy * 10000) +
  geom_point() +
  scale_y_continuous(labels = scales::comma) # format y-axis

As you can see, numbers on the y-axis are displayed as 200,000, 300,000, etc. instead of 200000, 300000, etc., which makes it more readable.

Another common format is percent to display numbers as percentages:

ggplot(dat) +
  aes(x = displ, y = hwy / 100) +
  geom_point() +
  scale_y_continuous(labels = scales::percent) # format y-axis

These two formats make large numbers and percentages easier to read. Other formats are possible such as using dollar signs, dates etc. See more information in the package’s documentation.

Legend

By default, the legend is located to the right side of the plot (when there is a legend to be displayed of course).

To control the position of the legend, we need to use the theme() function in addition to the legend.position argument:

p + aes(color = class) +
  theme(legend.position = "top")

Replace "top" by "left" or "bottom" to change its position and by "none" to remove it.

The title of the legend can be edited with the labs() layer:

p + aes(color = class) +
  labs(color = "Car's class")

Note that the argument inside labs() must match the one inside the aes() layer (in this case: color).

The title of the legend can also be removed with legend.title = element_blank() inside the theme() layer:

p + aes(color = class) +
  theme(
    legend.title = element_blank(),
    legend.position = "bottom"
  )

The legend now appears at the bottom of the plot, without the legend title.

Shape, color, size and transparency

There are a very large number of options to improve the quality of the plot or to add additional information. These include:

shape,
size,
color, and
alpha (transparency).

We can for instance change the shape of all points in a scatter plot by adding shape to geom_point(), or vary the shape according to the values taken by another variable (in that case, the shape argument must be inside aes()):³

# Change shape of all points
ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point(shape = 4)

# Change shape of points based on a categorical variable
ggplot(dat) +
  aes(x = displ, y = hwy, shape = drv) +
  geom_point()

Following the same principle, we can modify the color, size and transparency of the points based on a qualitative or quantitative variable. Here are some examples:

p <- ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point()

# Change color for all points
p + geom_point(color = "steelblue")

# Change color based on a qualitative variable
p + aes(color = drv)

# Change color based on a quantitative variable
p + aes(color = cty)

# Change color based on a criterion (median of cty variable)
p + aes(color = cty > median(cty))

# Change size of all points
p + geom_point(size = 4)

# Change size of points based on a quantitative variable
p + aes(size = cty)

# Change transparency based on a quantitative variable
p + aes(alpha = cty)

We can of course mix several options (shape, color, size, alpha) to build more complex graphics:

p + geom_point(size = 0.5) +
  aes(color = drv, shape = year, alpha = cty)

If you are unhappy with the default colors, you can change them manually with the scale_colour_manual() layer (for qualitative variables) and the scale_colour_gradient2() layer (for quantitative variables):

# Change color based on a qualitative variable
p + aes(color = drv) +
  scale_colour_manual(values = c("red", "blue", "green"))

# Change color based on a quantitative variable
p + aes(color = cty) +
  scale_colour_gradient2(
    low = "green",
    mid = "gray",
    high = "red",
    midpoint = median(dat$cty)
  )

For your information, you can emulate {ggplot2} default color palette for a desired number of colors and produce a character vector of HEX colors. For example, with 4 colors:

library(scales)
show_col(hue_pal()(4))

Text and labels

To add a label on a point (for example the row number), we can use the geom_text() and aes() functions:

p + geom_text(aes(label = rownames(dat)),
  check_overlap = TRUE,
  size = 2,
  vjust = -1
)

To add text on the plot, we use the annotate() function:

p + annotate("text",
  x = 6,
  y = 40,
  label = "hwy and displ are \n negatively correlated \n (rho = -0.77, p-value < 0.001)",
  size = 3
)

Read the article on correlation coefficient and correlation test in R to see how I computed the correlation coefficient (rho) and the p-value of the correlation test.

Smooth and regression lines

In a scatter plot, it is possible to add a smooth line fitted to the data:

p + geom_smooth()

In the context of simple linear regression, it is often the case that the regression line is displayed on the plot.

This can be done by adding method = lm (lm stands for linear model) in the geom_smooth() layer:

p + geom_smooth(method = lm)

It is also possible to draw a regression line for each level of a categorical variable:

p + aes(color = drv, shape = drv) +
  geom_smooth(method = lm, se = FALSE)

The se = FALSE argument removes the confidence interval around the regression lines.

Facets

facet_grid allows you to divide the same graphic into several panels according to the values of one or two qualitative variables:

# According to one variable
p + facet_grid(. ~ drv)

# According to 2 variables
p + facet_grid(drv ~ year)

It is then possible to add a regression line to each facet:

p + facet_grid(. ~ drv) +
  geom_smooth(method = lm)

facet_wrap() can also be used, as illustrated in this section.

Themes

Several functions are available in the {ggplot2} package to change the theme of the plot.

The most common themes after the default theme (i.e., theme_gray()) are the black and white (theme_bw()), minimal (theme_minimal()) and classic (theme_classic()) themes:

# Black and white theme
p + theme_bw()

# Minimal theme
p + theme_minimal()

# Classic theme
p + theme_classic()

I tend to use the minimal theme for most of my R Markdown reports as it brings out the patterns and points and not the layout of the plot, but again this is a matter of personal taste. See more themes at ggplot2.tidyverse.org/reference/ggtheme.html and in the {ggthemes} package.

In order to avoid having to change the theme for each plot you create, you can change the theme for the current R session using the theme_set() function as follows:

theme_set(theme_minimal())

Interactive plot with `{plotly}`

You can easily make your plots created with {ggplot2} interactive with the {plotly} package:

library(plotly)
ggplotly(p + aes(color = year))

You can now hover over a point to display more information about that point. There is also the possibility to zoom in and out, to download the plot, to select some observations, etc. More information about {plotly} for R can be found here.

Combine plots with `{patchwork}`

There are several ways to combine plots made in {ggplot2}. In my opinion, the most convenient way is with the {patchwork} package using symbols such as +, / and parentheses.

We first need to create some plots and save them:

p_a <- ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point()

p_b <- ggplot(dat) +
  aes(x = hwy) +
  geom_histogram()

p_c <- ggplot(dat) +
  aes(x = drv, y = hwy) +
  geom_boxplot()

Now that we have 3 plots saved in our environment, we can combine them. To have plots next to each other simply use the + symbol:

library(patchwork)
p_a + p_b + p_c

To display them above each other simply use the / symbol:

p_a / p_b / p_c

And finally, to combine them above and next to each other, mix +, / and parentheses:

p_a + p_b / p_c

(p_a + p_b) / p_c

See more ways to combine plots with:

grid.arrange() from the {gridExtra} package
plot_grid() from the {cowplot} package

Flip coordinates

Flipping coordinates of your plot is useful to create horizontal boxplots, or when labels of a variable are so long that they overlap each other on the x-axis. See with and without flipping coordinates below:

# without flipping coordinates
p1 <- ggplot(dat) +
  aes(x = class, y = hwy) +
  geom_boxplot()

# with flipping coordinates
p2 <- ggplot(dat) +
  aes(x = class, y = hwy) +
  geom_boxplot() +
  coord_flip()

library(patchwork)
p1 + p2 # left: without flipping, right: with flipping

This can be done with many types of plot, not only with boxplots. For instance, if a categorical variable has many levels or the labels are long, it is usually best to flip the coordinates for a better visual:

ggplot(dat) +
  aes(x = class) +
  geom_bar() +
  coord_flip()

Save plot

The ggsave() function will save the most recent plot in your current working directory unless you specify a path to another folder:

ggplot(dat) +
  aes(x = displ, y = hwy) +
  geom_point()

ggsave("plot1.pdf")

You can also specify the width, height and resolution (dpi) as follows:

ggsave("plot1.pdf",
  width = 12,
  height = 12,
  units = "cm",
  dpi = 300
)

Managing dates

If the time variable in your dataset is in date format, the {ggplot2} package recognizes the date format and automatically uses a specific type for the axis ticks.

There is no time variable with a date format in our dataset, so let’s create a new variable of this type thanks to the as.Date() function:

dat$date <- as.Date("2020-08-21") - 0:(nrow(dat) - 1)

See the first 6 observations of this date variable and its class:

head(dat$date)

## [1] "2020-08-21" "2020-08-20" "2020-08-19" "2020-08-18" "2020-08-17"
## [6] "2020-08-16"

str(dat$date)

##  Date[1:234], format: "2020-08-21" "2020-08-20" "2020-08-19" "2020-08-18" "2020-08-17" ...

The new variable date is correctly specified in a date format.

Most of the time, with a time variable, we want to create a line plot with the date on the X-axis and another continuous variable on the Y-axis, like the following plot for example:

p <- ggplot(dat) +
  aes(x = date, y = hwy) +
  geom_line()
p

As soon as the time variable is recognized as a date, we can use the scale_x_date() layer to change the format displayed on the X-axis. The following table shows the most frequent date formats:

Source: www.statmethods.net

Run ?strptime() to see many more date formats available in R.

For this example, let’s add the year in addition to the unabbreviated month:

p + scale_x_date(date_labels = "%B %Y")

It also possible to control the breaks to display on the X-axis with the date_breaks argument. For this example, let’s say we want to display the day as number and the abbreviated month for each interval of 10 days:

p + scale_x_date(date_breaks = "10 days", date_labels = "%d %b")

If labels displayed on the X-axis are unreadable because they overlap each other, you can rotate them with the theme() layer and the angle argument:

p + scale_x_date(date_breaks = "10 days", date_labels = "%d %b") +
  theme(axis.text.x = element_text(angle = 60, hjust = 1))

Highlight data with `{gghighlight}`

The {gghighlight} package allows, as its name suggests, to highlight some data directly on your ggplot. The highlighted data (that you define) are shown in bright color and the rest in gray.

Below example of how it works for a scatter plot, boxplot, barplot and histogram.

library(gghighlight)

# scatter plot
ggplot(mpg, aes(x = displ, y = hwy, color = cyl)) +
  geom_point() +
  gghighlight(cyl == "8") +
  theme(legend.position = "none") # remove legend

# boxplot
ggplot(dat) +
  aes(x = drv, y = hwy, fill = drv) +
  geom_boxplot() +
  gghighlight(drv %in% c("r", "4")) +
  theme(legend.position = "none") # remove legend

# barplot
ggplot(dat) +
  aes(x = drv, fill = drv) +
  geom_bar() +
  gghighlight(drv == "f")

As you can see, the gghighlight() layer accepts different types of conditions, but also several of them at the same time:

# histogram
ggplot(dat) +
  aes(x = hwy, fill = year) +
  geom_histogram() +
  gghighlight(displ > 2 & year == "1999")

Tip

I recently learned a tip very useful when drawing plots with {ggplot2}. If like me, you often comment and uncomment some lines of code in your plot, you know that you cannot transform the last line into a comment without removing the + sign in the line just above.

Adding a line NULL at the end of your plots will avoid an error if you forget to remove the + sign in the last line of your code. See with this basic example:

ggplot(dat) +
  aes(x = date, y = hwy) +
  geom_line() + # I do not have to remove the + sign
  # theme_minimal() + # this line is a comment
  NULL # adding this line doesn't change anything to the plot

This trick saves me a lot of time as I do not need to worry about making sure to remove the last + sign after commenting some lines of code in my plots.

If you find this trick useful, you may like these other tips and tricks in RStudio and R Markdown.

To go further

By now you have seen that {ggplot2} is a very powerful and complete package to create plots in R. This article illustrated only the tip of the iceberg, and you will find many tutorials on how to create more advanced plots and visualizations with {ggplot2} online. If you want to learn more than what is described in the present article, I highly recommend starting with:

the chapters Data visualisation and Graphics for communication from the book R for Data Science from Garrett Grolemund and Hadley Wickham
the book ggplot2: Elegant Graphics for Data Analysis from Hadley Wickham
the book R Graphics Cookbook from Winston Chang
the ggplot2 extensions guide which lists many of the packages that extend {ggplot2}
the {ggplot2} cheat sheet
this presentation by Garrick Aden-Buie
a detailed tutorial by Cédric Scherer

Conclusion

Thanks for reading.

I hope this article helped you to create your first plots with the {ggplot2} package. As a reminder, for simple graphs, it is sometimes easier to draw them via the {esquisse} addin. After some time, you will quickly learn how to create them by yourselves and in no time you will be able to build complex and sophisticated data visualizations.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Use the geom_jitter() layer with caution because, although it makes a plot more revealing at large scales, it also makes it slightly less accurate at small scales since some randomness is added to the points.↩︎
Code inspired from Claire Della Vedova (DellaData) and Cédric Scherer. Thanks to both of them for this nice plot!↩︎
There are (at the time of writing) 26 shapes accepted in the shape argument. See this documentation for all available shapes.↩︎

Mortgage calculator in R Shiny

Fri, 14 Aug 2020 00:00:00 +0000

Introduction

I recently moved out and bought my first apartment. Of course, I could not pay it entirely with my own savings, so I had to borrow money from the bank. I visited a couple of banks operating in my country and asked for a mortgage.

If you already bought your house or apartment in the past, you know how it goes: the bank analyzes your financial and personal situation and make an offer based on your propensity to repay the bank. You then either accept the offer if you are satisfied with the rate and conditions, or visit another bank if you believe you could receive a better offer. Mortgages and loans are more complicated than that of course, but let’s keep it simple here.

As I kind of like to control and keep a close eye on my personal finances (sometimes a bit too close I must admit), I knew precisely how much I could spend for my monthly mortgage repayment while still being able to cover my living expenses. However, I had no clue how much I could borrow in total for my new apartment given these housing repayments.

I knew I was not the first person in this case, so I looked online if I could find a R script which would answer my question (and potentially also give me the total cost of the housing loan, including the loan amount and the accumulated interests). I finally found a R script created a while ago by Prof. Thomas Girke.

Mortgage calculator

The function in the script was functional and solved my main issue, but I wanted to be able to play more easily with the different settings such as the amount, the duration and the interest rate of the loan.

For this reason, I created a R Shiny app which is available here:

Mortgage calculator

Mortgage calculator in R Shiny

In the meantime, I received an Excel file from a friend working in a Belgian bank which does precisely the same task. I am not an actuary nor an expert in mortgage loan, so with his file I was able to cross check the results and edit the code accordingly.

The app greatly helped me to know the maximum amount I could borrow from the bank by playing with the three main settings of a mortgage, so it gave me a precise price limit when looking for apartments online.

Note that the app can of course be used for any loan, not only for mortgage.

How to use the mortgage calculator?

First, you can find the mortgage calculator here.

I try to keep all my Shiny apps easy to use for everyone. However, here is how to use it in case it is not intuitive enough:

Enter the amount of the loan (i.e., the amount you would like to borrow, do not include downpayment)
Enter the annual interest rate in %
Enter the duration of the loan in years

On the right panel (or bottom if you use the app on mobile) you will see:

a summary repeating the settings you entered,
the total cost of the loan (principal and interests included), and more importantly
the amount of the monthly payments

A plot representing the percentage attributed to the repayment of the interests and the capital is also displayed. You see that (especially in the first years of the loan), the higher the interest rate and the duration of the loan, the higher the percentage of the monthly repayments is attributed to the repayments of the interests.

Finally, the amortization table showing the remaining balance month by month is displayed after the summary and the plot. You can copy, export (in PDF, CSV or Excel) or print this amortization table for further use.

Code of the app

Here is the entire code (or see the last version on GitHub) in case you would like to enhance it (feel free to send me your app if you happen to improve it!).

Conclusion

Thanks for reading.

I hope this mortgage calculator helped you to play with the different settings of a mortgage, and who knows, helped you to decide which house or apartment to buy.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Disclosure: Note that this application does not include investment advice or recommendations, nor a financial analysis. This application is intended for information only and you invest at your own risks. I cannot be held liable for any decision made based on the information contained in this application, nor for its use by third parties.

Outliers detection in R

Tue, 11 Aug 2020 00:00:00 +0000

Introduction

An outlier is a value or an observation that is distant from other observations, that is to say, a data point that differs significantly from other data points. Enderlein (1987) goes even further as the author considers outliers as values that deviate so much from other observations one might suppose a different underlying sampling mechanism.

An observation must always be compared to other observations made on the same phenomenon before actually calling it an outlier. Indeed, someone who is 200 cm tall (6’7” in US) will most likely be considered as an outlier compared to the general population, but that same person may not be considered as an outlier if we measured the height of basketball players.

An outlier may be due to the variability inherent in the observed phenomenon. For example, it is often the case that there are outliers when collecting data on salaries, as some people make much more money than the rest.

Outliers can also arise due to an experimental, measurement or encoding error. For instance, a human weighting 786 kg (1733 pounds) is clearly an error when encoding the weight of the subject. Her or his weight is most probably 78.6 kg (173 pounds) or 7.86 kg (17 pounds) depending on whether weights of adults or babies have been measured.

For this reason, it sometimes makes sense to formally distinguish two classes of outliers: (i) extreme values and (ii) mistakes. Extreme values are statistically and philosophically more interesting, because they are possible but unlikely responses.¹

In this article, I present several approaches to detect outliers in R, from simple techniques such as descriptive statistics (including minimum, maximum, histogram, boxplot and percentiles) to more formal techniques such as the Hampel filter, the Grubbs, the Dixon and the Rosner tests for outliers.

Although there is no strict or unique rule whether outliers should be removed or not from the dataset before doing statistical analyses, it is quite common to, at least, remove or impute outliers that are due to an experimental or measurement error (like the weight of 786 kg (1733 pounds) for a human). Some statistical tests require the absence of outliers in order to draw sound conclusions, but removing outliers is not recommended in all cases and must be done with caution.

This article will not tell you whether you should remove outliers or not (nor if you should impute them with the median, mean, mode or any other value), but it will help you to detect them in order to, as a first step, verify them. After their verification, it is then your choice to exclude or include them for your analyses (and this usually requires a thoughtful reflection on the researcher’s side).

Removing or keeping outliers mostly depend on three factors:

The domain/context of your analyses and the research question. In some domains, it is common to remove outliers as they often occur due to a malfunctioning process. In other fields, outliers are kept because they contain valuable information. It also happens that analyses are performed twice, once with and once without outliers to evaluate their impact on the conclusions. If results change drastically due to some influential values, this should caution the researcher to make overambitious claims.
Whether the tests you are going to apply are robust to the presence of outliers or not. For instance, the slope of a simple linear regression may significantly varies with just one outlier, whereas non-parametric tests such as the Wilcoxon test are usually robust to outliers.
How distant are the outliers from other observations? Some observations considered as outliers (according to the techniques presented below) are actually not really extreme compared to all other observations, while other potential outliers may be really distant from the rest of the observations.

The dataset mpg from the {ggplot2} package will be used to illustrate the different approaches of outliers detection in R, and in particular we will focus on the variable hwy (highway miles per gallon). We will also use some simulated data for the presentation of the outlier tests.

Descriptive statistics

Several methods using descriptive statistics exist. We present the most common ones below.

Minimum and maximum

The first step to detect outliers in R is to start with some descriptive statistics, and in particular with the minimum and maximum.

In R, this can easily be done with the summary() function:

dat <- ggplot2::mpg
summary(dat$hwy)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   12.00   18.00   24.00   23.44   27.00   44.00

where the minimum and maximum are respectively the first and last values in the output above.

Alternatively, they can also be computed with the min() and max(), or range() functions:²

min(dat$hwy)

## [1] 12

max(dat$hwy)

## [1] 44

range(dat$hwy)

## [1] 12 44

Some clear encoding mistake like a weight of 786 kg (1733 pounds) for a human will already be easily detected by this very simple technique.

Histogram

Another basic way to detect outliers is to draw a histogram of the data.

Using R base (with the number of bins corresponding to the square root of the number of observations in order to have more bins than the default option):

hist(dat$hwy,
  xlab = "hwy",
  main = "Histogram of hwy",
  breaks = sqrt(length(dat$hwy)) # set number of bins
)

or using ggplot2 (learn how to create plots with this package via the esquisse addin or via this tutorial):

library(ggplot2)

ggplot(dat) +
  aes(x = hwy) +
  geom_histogram(
    bins = round(sqrt(length(dat$hwy))), # set number of bins
    fill = "steelblue", color = "black"
  ) +
  theme_minimal()

From the histograms, we see that there seems to be a couple of observations larger than all other observations (see the bars on the right side of the plot).

Boxplot

In addition to histograms, boxplots are also useful to detect potential outliers.

Using R base:

boxplot(dat$hwy,
  ylab = "hwy"
)

or using ggplot2:

ggplot(dat) +
  aes(x = "", y = hwy) +
  geom_boxplot(fill = "steelblue") +
  theme_minimal()

A boxplot helps to visualize a quantitative variable by displaying five common location summary (minimum, median, first and third quartiles and maximum) and any observation that was classified as a suspected outlier using the interquartile range (IQR) criterion.

The IQR criterion means that all observations above $q_{0.75} + 1.5 \cdot IQR$ or below $q_{0.25} - 1.5 \cdot IQR$ (where $q_{0.25}$ and $q_{0.75}$ correspond to first and third quartile respectively, and IQR is the difference between the third and first quartile) are considered as potential outliers by R.

In other words, all observations outside of the following interval will be considered as potential outliers:

\[I = [q_{0.25} - 1.5 \cdot IQR; q_{0.75} + 1.5 \cdot IQR]\]

Observations considered as potential outliers by the IQR criterion are displayed as points in the boxplot. Based on this criterion, there are 2 potential outliers (see the 2 points above the vertical line, at the top of the boxplot).

Remember that it is not because an observation is considered as a potential outlier by the IQR criterion that you should remove it. Removing or keeping an outlier depends on:

the context of your analysis,
whether the tests you are going to perform on the dataset are robust to outliers or not, and
how extreme is the outlier (so how far is the outlier from other observations).

It is also possible to extract the values of the potential outliers based on the IQR criterion thanks to the boxplot.stats()$out function:

boxplot.stats(dat$hwy)$out

## [1] 44 44 41

As you can see, there are actually 3 points considered as potential outliers: 2 observations with a value of 44 and 1 observation with a value of 41.

Thanks to the which() function it is possible to extract the row number corresponding to these outliers:

out <- boxplot.stats(dat$hwy)$out
out_ind <- which(dat$hwy %in% c(out))
out_ind

## [1] 213 222 223

With this information you can now easily go back to the specific rows in the dataset to verify them, or print all variables for these outliers:

dat[out_ind, ]

## # A tibble: 3 × 11
##   manufacturer model      displ  year   cyl trans  drv     cty   hwy fl    class
##   <chr>        <chr>      <dbl> <int> <int> <chr>  <chr> <int> <int> <chr> <chr>
## 1 volkswagen   jetta        1.9  1999     4 manua… f        33    44 d     comp…
## 2 volkswagen   new beetle   1.9  1999     4 manua… f        35    44 d     subc…
## 3 volkswagen   new beetle   1.9  1999     4 auto(… f        29    41 d     subc…

Another method to display these specific rows is with the identify_outliers() function from the {rstatix} package:³

library(rstatix)

identify_outliers(
  data = dat,
  variable = "hwy"
)

## # A tibble: 3 × 13
##   manufacturer model      displ  year   cyl trans  drv     cty   hwy fl    class
##   <chr>        <chr>      <dbl> <int> <int> <chr>  <chr> <int> <int> <chr> <chr>
## 1 volkswagen   jetta        1.9  1999     4 manua… f        33    44 d     comp…
## 2 volkswagen   new beetle   1.9  1999     4 manua… f        35    44 d     subc…
## 3 volkswagen   new beetle   1.9  1999     4 auto(… f        29    41 d     subc…
## # ℹ 2 more variables: is.outlier <lgl>, is.extreme <lgl>

It is also possible to print the values of the outliers directly on the boxplot with the mtext() function:

boxplot(dat$hwy,
  ylab = "hwy",
  main = "Boxplot of highway miles per gallon"
)
mtext(paste("Outliers: ", paste(out, collapse = ", ")))

Percentiles

This method of outliers detection is based on the percentiles.

With the percentiles method, all observations that lie outside the interval formed by the 2.5 and 97.5 percentiles will be considered as potential outliers. Other percentiles such as the 1 and 99, or the 5 and 95 percentiles can also be considered to construct the interval.

The values of the lower and upper percentiles (and thus the lower and upper limits of the interval) can be computed with the quantile() function:

lower_bound <- quantile(dat$hwy, 0.025)
lower_bound

## 2.5% 
##   14

upper_bound <- quantile(dat$hwy, 0.975)
upper_bound

##  97.5% 
## 35.175

According to this method, all observations below 14 and above 35.175 will be considered as potential outliers. The row numbers of the observations outside of the interval can then be extracted with the which() function:

outlier_ind <- which(dat$hwy < lower_bound | dat$hwy > upper_bound)
outlier_ind

##  [1]  55  60  66  70 106 107 127 197 213 222 223

Then their values of highway miles per gallon can be printed:

dat[outlier_ind, "hwy"]

## # A tibble: 11 × 1
##      hwy
##    <int>
##  1    12
##  2    12
##  3    12
##  4    12
##  5    36
##  6    36
##  7    12
##  8    37
##  9    44
## 10    44
## 11    41

Alternatively, all variables for these outliers can be printed:

dat[outlier_ind, ]

## # A tibble: 11 × 11
##    manufacturer model      displ  year   cyl trans drv     cty   hwy fl    class
##    <chr>        <chr>      <dbl> <int> <int> <chr> <chr> <int> <int> <chr> <chr>
##  1 dodge        dakota pi…   4.7  2008     8 auto… 4         9    12 e     pick…
##  2 dodge        durango 4…   4.7  2008     8 auto… 4         9    12 e     suv  
##  3 dodge        ram 1500 …   4.7  2008     8 auto… 4         9    12 e     pick…
##  4 dodge        ram 1500 …   4.7  2008     8 manu… 4         9    12 e     pick…
##  5 honda        civic        1.8  2008     4 auto… f        25    36 r     subc…
##  6 honda        civic        1.8  2008     4 auto… f        24    36 c     subc…
##  7 jeep         grand che…   4.7  2008     8 auto… 4         9    12 e     suv  
##  8 toyota       corolla      1.8  2008     4 manu… f        28    37 r     comp…
##  9 volkswagen   jetta        1.9  1999     4 manu… f        33    44 d     comp…
## 10 volkswagen   new beetle   1.9  1999     4 manu… f        35    44 d     subc…
## 11 volkswagen   new beetle   1.9  1999     4 auto… f        29    41 d     subc…

There are 11 potential outliers according to the percentiles method. To reduce this number, you can set the percentiles to 1 and 99:

lower_bound <- quantile(dat$hwy, 0.01)
upper_bound <- quantile(dat$hwy, 0.99)

outlier_ind <- which(dat$hwy < lower_bound | dat$hwy > upper_bound)

dat[outlier_ind, ]

## # A tibble: 3 × 11
##   manufacturer model      displ  year   cyl trans  drv     cty   hwy fl    class
##   <chr>        <chr>      <dbl> <int> <int> <chr>  <chr> <int> <int> <chr> <chr>
## 1 volkswagen   jetta        1.9  1999     4 manua… f        33    44 d     comp…
## 2 volkswagen   new beetle   1.9  1999     4 manua… f        35    44 d     subc…
## 3 volkswagen   new beetle   1.9  1999     4 auto(… f        29    41 d     subc…

Setting the percentiles to 1 and 99 gives the same potential outliers as with the IQR criterion.

Z-scores

If your data come from a normal distribution, you can use the z-scores, defined as

\[z_i = \frac{x_i - \overline{X}}{s_X}\]

where $\overline{X}$ is the mean and $s_X$ is the standard deviation of the random variable $X$.

This is referred as scaling, which can be done with the scale() function in R.

According to this method, any z-score:

< -2 or > 2 is considered as rare
< -3 or > 3 is considered as extremely rare

Other authors also use a z-score below -3.29 or above 3.29 to detect outliers. This value of 3.29 comes from the fact that 1 observation out of 1000 is out of this interval if the data follow a normal distribution.

In our case:

dat$z_hwy <- scale(dat$hwy)

hist(dat$z_hwy)

summary(dat$z_hwy)

##        V1          
##  Min.   :-1.92122  
##  1st Qu.:-0.91360  
##  Median : 0.09402  
##  Mean   : 0.00000  
##  3rd Qu.: 0.59782  
##  Max.   : 3.45274

We see that there are some observations above 3.29, but none below -3.29.

To identify the line in the dataset of these observations:

which(dat$z_hwy > 3.29)

## [1] 213 222

We see that observations 213 and 222 can be considered as outliers according to this method.

Hampel filter

Another method, known as Hampel filter, consists of considering as outliers the values outside the interval ($I$) formed by the median, plus or minus 3 median absolute deviations ($MAD$):⁴

\[I = [median - 3 \cdot MAD; median + 3 \cdot MAD]\]

where $MAD$ is the median absolute deviation and is defined as the median of the absolute deviations from the data’s median $\tilde{X} = median(X)$:

\[MAD = median(|X_i - \tilde{X}|)\]

For this method we first set the interval limits thanks to the median() and mad() functions:⁵

lower_bound <- median(dat$hwy) - 3 * mad(dat$hwy, constant = 1)
lower_bound

## [1] 9

upper_bound <- median(dat$hwy) + 3 * mad(dat$hwy, constant = 1)
upper_bound

## [1] 39

According to this method, all observations below 9 and above 39 will be considered as potential outliers. The row numbers of the observations outside of the interval can then be extracted with the which() function:

outlier_ind <- which(dat$hwy < lower_bound | dat$hwy > upper_bound)
outlier_ind

## [1] 213 222 223

According to the Hampel filter, there are 3 outliers for the hwy variable.

Statistical tests

In this section, we present 3 hypothesis tests to detect outliers:

Grubbs’s test
Dixon’s test
Rosner’s test

These 3 statistical tests are part of more formal techniques of outliers detection as they all involve the computation of a test statistic that is compared to tabulated critical values (that are based on the sample size and the desired confidence level).

Note that the 3 tests are appropriate only when the data, without any outliers, are approximately normally distributed. It is recommended to check normality visually, with a QQ-plot, a histogram and/or a boxplot for instance. Although it can also be checked with a formal test for normality (such as the Shapiro-Wilk test for example), the presence of one or more outliers may cause the normality test to reject normality when it is in fact a reasonable assumption for applying one of the 3 outlier tests mentioned above.

We check the normality of our data thanks to a QQ-plot:

library(car)
qqPlot(dat$hwy)

## [1] 213 222

Too many points deviate from the Henry’s line, so based on the QQ-plot we conclude that our data do not follow a normal distribution. Therefore, we should not use one of the outlier test on these data.

For the sake of completeness, here is an example of data that could be used with the three outlier tests, together with the QQ-plot of these data:

dat_tests <- c(rnorm(50), 5)

qqPlot(dat_tests)

## [1] 51 18

hist(dat_tests)

For the illustration of the three outlier tests in the next sections, we thus use the simulated data (dat_tests) instead of the data used so far (dat$hwy).

Grubbs’s test

The Grubbs test allows to detect whether the highest or lowest value in a dataset is an outlier.

The Grubbs test detects one outlier at a time (highest or lowest value), so the null and alternative hypotheses are as follows:

$H_0$: The highest value is not an outlier
$H_1$: The highest value is an outlier

if we want to test the highest value, or:

$H_0$: The lowest value is not an outlier
$H_1$: The lowest value is an outlier

if we want to test the lowest value.

As for any statistical test, if the p-value is less than the chosen significance threshold (generally $\alpha = 0.05$) then the null hypothesis is rejected and we will conclude that the lowest/highest value is an outlier.

On the contrary, if the p-value is greater or equal than the significance level, the null hypothesis is not rejected, and we will conclude that, based on the data, we do not reject the hypothesis that the lowest/highest value is not an outlier.

Note that the Grubbs test is not appropriate for sample size of 6 or less ($n \le 6$).

To perform the Grubbs test in R, we use the grubbs.test() function from the {outliers} package:

# install.packages("outliers")
library(outliers)

# Grubbs test
test <- grubbs.test(dat_tests)
test

## 
## 	Grubbs test for one outlier
## 
## data:  dat_tests
## G = 3.68326, U = 0.72325, p-value = 0.001873
## alternative hypothesis: highest value 5 is an outlier

The p-value is 0.002. At the 5% significance level, we reject the hypothesis that the highest value 5 is not an outlier. In other words, based on this test, we conclude that the highest value 5 is an outlier.

By default, the test is performed on the highest value (as shown in the R output: alternative hypothesis: highest value 5 is an outlier). If you want to do the test for the lowest value, simply add the argument opposite = TRUE in the grubbs.test() function:

test <- grubbs.test(dat_tests, opposite = TRUE)
test

## 
## 	Grubbs test for one outlier
## 
## data:  dat_tests
## G = 2.02893, U = 0.91602, p-value = 0.9981
## alternative hypothesis: lowest value -2.65645542090478 is an outlier

The R output indicates that the test is now performed on the lowest value (see alternative hypothesis: lowest value -2.6564554 is an outlier).

The p-value is 0.998. At the 5% significance level, we do not reject the hypothesis that the lowest value -2.66 is not an outlier. In other words, we cannot conlude that the lowest value -2.66 is an outlier.

Dixon’s test

Similar to the Grubbs test, Dixon test is used to test whether a single low or high value is an outlier. So if more than one outliers is suspected, the test has to be performed on these suspected outliers individually.

Note that Dixon test is most useful for small sample size (usually $n \le 25$).

To perform the Dixon’s test in R, we use the dixon.test() function from the {outliers} package.

For this illustration, as the Dixon test can only be done on small samples, we take a subset of our simulated data which consists of the 20 first observations and the outlier. Note that R will throw an error and accepts only a dataset consisting of 3 to 30 observations for this test.

# subset of simulated data
subdat <- c(dat_tests[1:20], max(dat_tests))

# Dixon test
test <- dixon.test(subdat)
test

## 
## 	Dixon test for outliers
## 
## data:  subdat
## Q = 0.46668, p-value = 0.06429
## alternative hypothesis: highest value 5 is an outlier

Results of the Dixon test show that we cannot conclude that the highest value 5 is an outlier (p-value = 0.064).

To test for the lowest value, simply add the opposite = TRUE argument to the dixon.test() function:

test <- dixon.test(subdat,
  opposite = TRUE
)
test

## 
## 	Dixon test for outliers
## 
## data:  subdat
## Q = 0.27115, p-value = 0.6872
## alternative hypothesis: lowest value -2.65645542090478 is an outlier

Results of the test show that we cannot conclude that the lowest value -2.66 is an outlier (p-value = 0.687).

It is a good practice to always check the results of the statistical test for outliers against the boxplot to make sure we tested all potential outliers:

out <- boxplot.stats(subdat)$out

boxplot(subdat)
mtext(paste("Outlier: ", paste(out, collapse = ", ")))

From the boxplot, we see that the Dixon test has been applied to all potential outliers.

If you need to perform the test again without the highest or lowest value, this can be done by finding the row number of the maximum or minimum value, excluding this row number from the dataset and then finally apply the Dixon test on this new dataset. We illustrate the process as if we needed to perform the test again, but this time on the data excluding the highest value:

# find and exclude highest value
remove_ind <- which.max(subdat)
subsubdat <- subdat[-remove_ind]

# Dixon test on dataset without the maximum
test <- dixon.test(subsubdat)
test

## 
## 	Dixon test for outliers
## 
## data:  subsubdat
## Q = 0.30413, p-value = 0.5547
## alternative hypothesis: lowest value -2.65645542090478 is an outlier

Results show that we cannot conclude that the lowest value -2.66 is an outlier (p-value = 0.555).

Rosner’s test

Rosner’s test for outliers has the advantages that:

it is used to detect several outliers at once (unlike Grubbs and Dixon test which must be performed iteratively to screen for multiple outliers), and
it is designed to avoid the problem of masking, where an outlier that is close in value to another outlier can go undetected.

Unlike Dixon test, note that Rosner test is most appropriate when the sample size is large ($n \ge 20$). We therefore use again the similated dataset (i.e., dat_tests), which includes 51 observations.

To perform the Rosner test, we use the rosnerTest() function from the {EnvStats} package. This function requires at least 2 arguments: the data and the number of suspected outliers k (with k = 3 as the default number of suspected outliers).

For this example, we set the number of suspected outliers to be equal to 1, as suggested by the number of potential outliers outlined in the boxplot.⁶

library(EnvStats)

# Rosner test
test <- rosnerTest(dat_tests,
  k = 1
)
test

## 
## Results of Outlier Test
## -------------------------
## 
## Test Method:                     Rosner's Test for Outliers
## 
## Hypothesized Distribution:       Normal
## 
## Data:                            dat_tests
## 
## Sample Size:                     51
## 
## Test Statistic:                  R.1 = 3.683258
## 
## Test Statistic Parameter:        k = 1
## 
## Alternative Hypothesis:          Up to 1 observations are not
##                                  from the same Distribution.
## 
## Type I Error:                    5%
## 
## Number of Outliers Detected:     1
## 
##   i     Mean.i     SD.i Value Obs.Num    R.i+1 lambda.i+1 Outlier
## 1 0 0.06306688 1.340371     5      51 3.683258   3.136165    TRUE

The interesting results are provided in the $all.stats table:

test$all.stats

##   i     Mean.i     SD.i Value Obs.Num    R.i+1 lambda.i+1 Outlier
## 1 0 0.06306688 1.340371     5      51 3.683258   3.136165    TRUE

Based on the results of the Rosner test, we see that there is only one outlier (as the table contains 1 line), and that it is the observation 51 (see Obs.Num) with a value of 5 (see Value). This finding aligns with the Grubbs test presented above, which also detected the value 5 as an outlier.

Additional remarks

You will find many other methods to detect outliers:

in the {outliers} packages,
via the lofactor() function from the {DMwR} package: Local Outlier Factor (LOF) is an algorithm used to identify outliers by comparing the local density of a point with that of its neighbors,
the outlierTest() from the {car} package gives the most extreme observation based on the given model and allows to test whether it is an outlier,
in the {OutlierDetection} package, and
with the aq.plot() function from the {mvoutlier} package (Thanks KTR for the suggestion.).

We present these methods with the initial dataset mpg, using the cyl and hwy variables:

library(mvoutlier)

Y <- as.matrix(ggplot2::mpg[, c("cyl", "hwy")])
res <- aq.plot(Y)

Note also that some transformations may “naturally” eliminate outliers. The natural log or square root of a value reduces the variation caused by extreme values, so in some cases applying these transformations will help in making potential outliers less extreme.

Conclusion

Thanks for reading.

I hope this article helped you to detect outliers in R via several descriptive statistics (including minimum, maximum, histogram, boxplot and percentiles) or thanks to more formal techniques of outliers detection (including Hampel filter, Grubbs, Dixon and Rosner tests).

It is now your turn to try to detect outliers in your data, and decide how to treat them (i.e., keeping, removing or imputing them) before conducting your analyses.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

(Note that this article is available for download on my Gumroad page.)

References

Enderlein, G. 1987. “Hawkins, DM: Identification of Outliers. Chapman and Hall, London–New York 1980, 188 s.,£ 14, 50.” Biometrical Journal 29 (2): 198–98.

Thanks Felix Kluxen for the valuable suggestion.↩︎
Thanks to Victor for pointing out the range() function.↩︎
Thanks to Marlenildo for pointing out the identify_outliers() function.↩︎
The default is 3 (according to Pearson’s rule), but another value is also possible.↩︎
The constant in the mad() function is 1.4826 by default, so it has to be set to 1 to find the median absolute deviation. See help(mad) for more details. Thanks to Elisei for pointing this out to me.↩︎
In order to avoid flawed conclusions, it is important to pre-screen the data (graphically with a boxplot for example) to make the selection of the number of potential outliers as accurate as possible prior to running Rosner’s test.↩︎

Wilcoxon test in R: how to compare 2 groups under the non-normality assumption?

Sun, 07 Jun 2020 00:00:00 +0000

Introduction

In a previous article, we showed how to compare two groups under different scenarios using the Student’s t-test. The Student’s t-test requires that the distributions follow a normal distribution when in presence of small samples.¹

In this article, we show how to compare two groups when the normality assumption is violated, using the Wilcoxon test.

The Wilcoxon test is a non-parametric test, meaning that it does not rely on data belonging to any particular parametric family of probability distributions. Non-parametric tests have the same objective as their parametric counterparts. However, they have two advantages over parametric tests: they do not require the assumption of normality of distributions and they can deal with outliers.

A Student’s t-test for instance is only applicable if the data are Gaussian or if the sample size is large enough (usually $n \ge 30$, thanks to the central limit theorem). A non-parametric test should be used in other cases.

One may wonder why we would not always use a non-parametric test so we do not have to bother about testing for normality. The reason is that non-parametric tests are usually less powerful than corresponding parametric tests when the normality assumption holds.

Therefore, all else being equal, with a non-parametric test you are less likely to reject the null hypothesis when it is false if the data follows a normal distribution. It is thus preferred to use the parametric version of a statistical test when the assumptions are met.

In the remaining of the article, we present the two scenarios of the Wilcoxon test and how to perform them in R through two examples.

Two different scenarios

As for the Student’s t-test, the Wilcoxon test is used to compare two groups and see whether they are significantly different from each other in terms of the variable of interest.

The two groups to be compared are either:

independent, or
paired (i.e., dependent)

There are actually two versions of the Wilcoxon test:

The Mann-Whitney-Wilcoxon test (also referred as Wilcoxon rank sum test or Mann-Whitney U test) is performed when the samples are independent (so this test is the non-parametric equivalent to the Student’s t-test for independent samples).
The Wilcoxon signed-rank test (also sometimes referred as Wilcoxon test for paired samples) is performed when the samples are paired/dependent (so this test is the non-parametric equivalent to the Student’s t-test for paired samples).

Luckily, those two tests can be done in R with the same function: wilcox.test(). They are presented in the following sections.

Independent samples

For the Wilcoxon test with independent samples, suppose that we want to test whether grades at the statistics exam differ between female and male students.

We have collected grades for 24 students (12 girls and 12 boys):

dat <- data.frame(
  Sex = as.factor(c(rep("Girl", 12), rep("Boy", 12))),
  Grade = c(
    19, 18, 9, 17, 8, 7, 16, 19, 20, 9, 11, 18,
    16, 5, 15, 2, 14, 15, 4, 7, 15, 6, 7, 14
  )
)

dat

##     Sex Grade
## 1  Girl    19
## 2  Girl    18
## 3  Girl     9
## 4  Girl    17
## 5  Girl     8
## 6  Girl     7
## 7  Girl    16
## 8  Girl    19
## 9  Girl    20
## 10 Girl     9
## 11 Girl    11
## 12 Girl    18
## 13  Boy    16
## 14  Boy     5
## 15  Boy    15
## 16  Boy     2
## 17  Boy    14
## 18  Boy    15
## 19  Boy     4
## 20  Boy     7
## 21  Boy    15
## 22  Boy     6
## 23  Boy     7
## 24  Boy    14

Here are the distributions of the grades by sex (using {ggplot2}):

library(ggplot2)

ggplot(dat) +
  aes(x = Sex, y = Grade) +
  geom_boxplot(fill = "#0c4c8a") +
  theme_minimal()

We first check whether the 2 samples follow a normal distribution via a histogram and the Shapiro-Wilk test:

hist(subset(dat, Sex == "Girl")$Grade,
  main = "Grades for girls",
  xlab = "Grades"
)

hist(subset(dat, Sex == "Boy")$Grade,
  main = "Grades for boys",
  xlab = "Grades"
)

shapiro.test(subset(dat, Sex == "Girl")$Grade)

## 
## 	Shapiro-Wilk normality test
## 
## data:  subset(dat, Sex == "Girl")$Grade
## W = 0.84548, p-value = 0.0323

shapiro.test(subset(dat, Sex == "Boy")$Grade)

## 
## 	Shapiro-Wilk normality test
## 
## data:  subset(dat, Sex == "Boy")$Grade
## W = 0.84313, p-value = 0.03023

The histograms show that both distributions do not seem to follow a normal distribution and the p-values of the Shapiro-Wilk tests confirm it (since we reject the null hypothesis of normality for both distributions at the 5% significance level).

We just showed that normality assumption is violated for both groups so it is now time to see how to perform the Wilcoxon test in R.

Note that in order to use the Student’s t-test (the parametric version of the Wilcoxon test), it is required that both samples follow a normal distribution if samples are small.² Therefore, even if one sample follows a normal distribution (and the other does not follow a normal distribution), it is recommended to use the non-parametric test.

Remember that the null and alternative hypothesis of the Wilcoxon test are as follows:

$H_0$: the 2 groups are equal in terms of the variable of interest
$H_1$: the 2 groups are different in terms of the variable of interest

Applied to our research question, we have:

$H_0$: grades of girls and boys are equal
$H_1$: grades of girls and boys are different

test <- wilcox.test(dat$Grade ~ dat$Sex)
test

## 
## 	Wilcoxon rank sum test with continuity correction
## 
## data:  dat$Grade by dat$Sex
## W = 31.5, p-value = 0.02056
## alternative hypothesis: true location shift is not equal to 0

We obtain the test statistic, the p-value and a reminder of the hypothesis tested.³

The p-value is 0.021. Therefore, at the 5% significance level, we reject the null hypothesis and we conclude that grades are significantly different between girls and boys.

Given the boxplot presented above showing the grades by sex, one may see that girls seem to perform better than boys. This can be tested formally by adding the alternative = "less" argument to the wilcox.test() function:⁴

test <- wilcox.test(dat$Grade ~ dat$Sex,
  alternative = "less"
)
test

## 
## 	Wilcoxon rank sum test with continuity correction
## 
## data:  dat$Grade by dat$Sex
## W = 31.5, p-value = 0.01028
## alternative hypothesis: true location shift is less than 0

The p-value is 0.01. Therefore, at the 5% significance level, we reject the null hypothesis and we conclude that boys performed significantly worse than girls (which is equivalent than concluding that girls performed significantly better than boys).

Paired samples

For this second scenario, consider that we administered a math test in a class of 12 students at the beginning of a semester, and that we administered a similar test at the end of the semester to the exact same students. We have the following data:

dat2 <- data.frame(
  Beginning = c(16, 5, 15, 2, 14, 15, 4, 7, 15, 6, 7, 14),
  End = c(19, 18, 9, 17, 8, 7, 16, 19, 20, 9, 11, 18)
)

dat2

##    Beginning End
## 1         16  19
## 2          5  18
## 3         15   9
## 4          2  17
## 5         14   8
## 6         15   7
## 7          4  16
## 8          7  19
## 9         15  20
## 10         6   9
## 11         7  11
## 12        14  18

We transform the dataset to have it in a tidy format:

dat2 <- data.frame(
  Time = c(rep("Before", 12), rep("After", 12)),
  Grade = c(dat2$Beginning, dat2$End)
)
dat2

##      Time Grade
## 1  Before    16
## 2  Before     5
## 3  Before    15
## 4  Before     2
## 5  Before    14
## 6  Before    15
## 7  Before     4
## 8  Before     7
## 9  Before    15
## 10 Before     6
## 11 Before     7
## 12 Before    14
## 13  After    19
## 14  After    18
## 15  After     9
## 16  After    17
## 17  After     8
## 18  After     7
## 19  After    16
## 20  After    19
## 21  After    20
## 22  After     9
## 23  After    11
## 24  After    18

The distribution of the grades at the beginning and after the semester:

# Reordering dat2$Time
dat2$Time <- factor(dat2$Time,
  levels = c("Before", "After")
)

ggplot(dat2) +
  aes(x = Time, y = Grade) +
  geom_boxplot(fill = "#0c4c8a") +
  theme_minimal()

(See the {esquisse} and {questionr} addins to help you reorder levels of a factor variable and to easily draw plots with the {ggplot2} package.)

In this example, it is clear that the two samples are not independent since the same 12 students took the exam before and after the semester. Supposing also that the normality assumption is violated (and given the small sample size), we thus use the Wilcoxon test for paired samples, with the following hypotheses:⁵

$H_0$: grades before and after the semester are equal
$H_1$: grades before and after the semester are different

We add the paired = TRUE argument to the wilcox.test() function to take into consideration the dependency between the 2 samples:

before <- dat2$Grade[dat2$Time == "Before"]
after <- dat2$Grade[dat2$Time == "After"]
test <- wilcox.test(before, after, paired = TRUE)

test

## 
## 	Wilcoxon signed rank test with continuity correction
## 
## data:  before and after
## V = 21, p-value = 0.1692
## alternative hypothesis: true location shift is not equal to 0

We obtain the test statistic, the p-value and a reminder of the hypothesis tested.

The p-value is 0.169. Therefore, at the 5% significance level, we do not reject the null hypothesis that the grades are similar before and after the semester.

Combination of plot and statistical test

After having written this article, I discovered the {ggstatsplot} package which I believe is worth mentioning here, in particular the ggbetweenstats() and ggwithinstats() functions for independent and paired samples, respectively.

These two functions combine a boxplot—representing the distribution for each group—and the results of the statistical test displayed in the subtitle of the plot.

See examples below for independent and paired samples, using the same data than previously.

Independent samples

For independent samples, it is the ggbetweenstats() function which is used:

# load package
library(ggstatsplot)

# plot with statistical results
ggbetweenstats( # independent samples
  data = dat,
  x = Sex,
  y = Grade,
  plot.type = "box", # for boxplot
  type = "nonparametric", # for wilcoxon
  centrality.plotting = FALSE # remove median
)

The p-value (displayed after p = in the subtitle of the plot) indicates that we reject the null hypothesis, and we conclude that grades are significantly different between girls and boys (p-value = 0.02).

Paired samples

For paired samples, it is the ggwithinstats() function which is used:

# load package
library(ggstatsplot)

# plot with statistical results
ggwithinstats( # paired samples
  data = dat2,
  x = Time,
  y = Grade,
  type = "nonparametric", # for wilcoxon
  centrality.plotting = FALSE # remove median
)

The p-value (displayed after p = in the subtitle of the plot) indicates that we do not reject the null hypothesis, so we do not reject the hypothesis that grades are equal before and after the semester (p-value = 0.17).

The point of this section was to illustrate how to easily draw plots together with statistical results, which is exactly the aim of the {ggstatsplot} package. See more details and examples in this article.

Assumption of equal variances

As written at the beginning of the article, the Wilcoxon test does not require the assumption of normality in case of small samples.

Regarding the assumption of equal variances, this assumption may or may not be needed depending on your goal. If you only want to compare the two groups, you do not have to test the equality of variances because the two distributions do not have to have the same shape. However, if your goal is to compare medians of the two groups, then you will need to make sure that the two distributions have the same shape (and thus, the same variance).⁶

So results of your test of equality of variances will change your interpretation: differences in the “distributions” of two groups or differences in the “medians” of two groups.

In this article I do not wish to compare medians, I only want compare the groups by determining whether there are differences in the distributions of the two groups. This is the reason I do not test for equality of variances.

Note that this is equivalent when performing the Kruskal-Wallis test to compare three groups or more (i.e., the non-parametric version of the ANOVA): if you only want to test whether there are differences in the groups you do not need homoscedasticity, whereas if you want to compare the medians this assumption must be met.

Conclusion

Thanks for reading.

I hope this article helped you to compare two groups that do not follow a normal distribution in R using the Wilcoxon test. See also:

the one-sample Wilcoxon test if you have only one group and want to compare it to a default given value,
the Student’s t-test if you need to perform the parametric version of the two-sample Wilcoxon test,
and the ANOVA if you need to compare 3 groups or more.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Stevens, James P. 2013. Intermediate Statistics: A Modern Approach. Routledge.

Remember that the normality assumption can be tested via 3 complementary methods: (i) histogram, (ii) QQ-plot and (iii) normality tests (with the most common being the Shapiro-Wilk test). See how to determine if a distribution follows a normal distribution if you need a refresh.↩︎
In case of large samples, normality is not required (this is a common misconception!). By the central limit theorem, sample means of large samples are often well-approximated by a normal distribution even if the data are not normally distributed (Stevens 2013).↩︎
Note that the presence of equal elements (ties) prevents an exact p-value calculation. This can be tackled by computing the exact or asymptotic Wilcoxon-Mann-Whitney test with adjustment for ties, using the wilcox_test() function from the {coin} package: wilcox_test(dat$Grade ~ dat$Sex, distribution = exact()) or wilcox_test(dat$Grade ~ dat$Sex). In our case, conclusions remain unchanged.↩︎
We add alternative = "less" (and not alternative = "greater") because we want to test that grades for boys are less than grade for girls. Using "less" or "greater" can be deducted from the reference level in the dataset.↩︎
Note that for paired samples (when in presence of a small sample), normality must be checked on the differences between the two paired samples, and not individually on the two samples like it is done for independent samples.↩︎
See these three articles for a more detailed discussion on the assumption of equal variances in Wilcoxon test: 1, 2 & 3.↩︎

How to publish a Shiny app? An example with shinyapps.io

Fri, 29 May 2020 00:00:00 +0000

Introduction

The COVID-19 virus led many people to create interactive apps and dashboards. A reader recently asked me how to publish a Shiny app she just created. Similarly to a previous article where I show how to upload R code on GitHub, I thought it would be useful to some people to see how I publish my Shiny apps so they could do the same.

Before going through the different steps required to deploy your Shiny app online, you can check the final result with my apps here.

Note 1: The screenshots have been taken on MacOS and I have not tested it on Windows. Do not hesitate to let me know in the comments whether it is similar or not on other operating systems.

Note 2: There are other ways to publish your app (with Docker for example), but the method shown below is easy (in my opinion) and works well.

Prerequisite

I personally use the shinyapps.io platform to deploy my Shiny apps. So in order to follow this guide you will first need to create an account (if you do not already have one).

They offer a free plan, but you are limited to 5 active applications and a monthly usage of 25 active hours.

For your information, if you make your app available to a wide audience, expect to exceed the monthly cap of active hours quite quickly. To increase the monthly limit (or to publish more than 5 apps), you will need to upgrade your plan to a paying one.

Step-by-step guide

Below the steps to follow in pictures.

Step 1: Open RStudio and create a new Shiny app:

Step 2: Give it a name (without space), choose where to save it and click on the Create button:

Step 3: In the same way as when you open a new R Markdown document, the code for a basic Shiny app is created. Run the app by clicking on the Run App button to see the result:

Step 4: The basic app opens, publish it:

Step 5: If it is your first Shiny app, the box “Publish From Account” should be empty. Click on “Add New Account” to link the shinyapps.io account you just created:

Step 6: Click on the first alternative (ShinyApps.io):

Step 7: Click on the link to your ShinyApps account:

Step 8: Click on the Dashboard button to log in into your account:

Step 9: Click on your name and then on Tokens

Step 10: If this is your first app, there should be no token already created. Create one by clicking on the Add Token button. Then Click on the Show button:

Step 11: Click on the Show Secret button:

Step 12: Now the code is complete (nothing is hidden anymore). Click on the Copy to clipboard button:

Step 13: Copy the code and click on the OK button:

Step 14: Go back to RStudio, paste the code in the console and run it:

Your computer is now authorized to deploy applications to your shinyapps.io account.

Step 15: Go back to the window where you can publish your app, choose a title (without space) and click on the Publish button:

Step 16: After several seconds (depending on the weight of your app), the Shiny app should appear in your internet browser:

Step 17: You can now edit the app (or replace the entire code by another of your app), and run the app again by clicking on the Run App button. For this illustration, I just added a link for more information in the side panel:

Step 18: Check that the modifications have been taken into account (the link appears in the side panel as expected) and republish your app:

Step 19: Click on the Publish button:

Step 20: Your app is live! You can now share it and everyone with the link will be able to use it:

Additional notes

Settings of your app

If you need to change the settings of your Shiny app, go to your shinyapps.io dashboard and click on the app you just created to access the settings:

See the different settings in the tabs located at the top of the windows, and see the link to the app next to the URL field:

Publish your dataset

It is often the case that your Shiny app uses a dataset that is not loaded in R by default (it uses one of your dataset that is saved locally on your computer for instance).

To make the app work with your data, you will also need to publish the data when publishing your app (check the checkbox corresponding to your data):

Publish your dataset together with the app

If you would like to have more flexibility, another way to be able to use an external dataset in your app is to host it online:

if you have a website, you can easily publish it through your website
if you do not have a website, you can host the dataset via GitHub (if you do not have an account, it is a good time to create one!)

The last step is then to import the data via that new URL (put the data import in the code of your app).

Conclusion

Thanks for reading.

I hope this tutorial helped you to publish your first Shiny app.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Correlation coefficient and correlation test in R

Thu, 28 May 2020 00:00:00 +0000

Introduction

Correlations between variables play an important role in a descriptive analysis. A correlation measures the relationship between two variables, that is, how they are linked to each other. In this sense, a correlation allows to know which variables evolve in the same direction, which ones evolve in the opposite direction, and which ones are independent.

In this article, I show how to compute correlation coefficients, how to perform correlation tests and how to visualize relationships between variables in R.

Correlation is usually computed on two quantitative variables, but it can also be computed on two qualitative ordinal variables.¹ See the Chi-square test of independence if you need to study the relationship between two qualitative nominal variables.

If you need to quantify the relationship between two variables, I refer you to the article about linear regression.

Data

In this article, we use the mtcars dataset (loaded by default in R):

# display first 5 observations
head(mtcars, 5)

##                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2

The variables vs and am are categorical variables, so they are removed for this article:

# remove vs and am variables
library(tidyverse)
dat <- mtcars %>%
  select(-vs, -am)

# display 5 first obs. of new dataset
head(dat, 5)

##                    mpg cyl disp  hp drat    wt  qsec gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02    3    2

Correlation coefficient

Between two variables

The correlation between 2 variables is found with the cor() function.

Suppose we want to compute the correlation between horsepower (hp) and miles per gallon (mpg):

# Pearson correlation between 2 variables
cor(dat$hp, dat$mpg)

## [1] -0.7761684

Note that the correlation between variables X and Y is equal to the correlation between variables Y and X so the order of the variables in the cor() function does not matter.

The Pearson correlation is computed by default with the cor() function. If you want to compute the Spearman correlation, add the argument method = "spearman" to the cor() function:

# Spearman correlation between 2 variables
cor(dat$hp, dat$mpg,
  method = "spearman"
)

## [1] -0.8946646

The most common correlation methods (Run ?cor for more information about the different methods available in the cor() function) are:

Pearson correlation is often used for quantitative continuous variables that have a linear relationship
Spearman correlation (which is actually similar to Pearson but based on the ranked values for each variable rather than on the raw data) is often used to evaluate relationships involving at least one qualitative ordinal variable or two quantitative variables if the link is partially linear
Kendall’s tau-b which is computed from the number of concordant and discordant pairs is often used for qualitative ordinal variables

Note that there exists the point-biserial correlation (which can be used to measure the association between a continuous variable and a nominal variable of two levels), but this correlation is not covered here.

Correlation matrix: correlations for all variables

Suppose now that we want to compute correlations for several pairs of variables. We can easily do so for all possible pairs of variables in the dataset, again with the cor() function:

# correlation for all variables
round(cor(dat),
  digits = 2 # rounded to 2 decimals
)

##        mpg   cyl  disp    hp  drat    wt  qsec  gear  carb
## mpg   1.00 -0.85 -0.85 -0.78  0.68 -0.87  0.42  0.48 -0.55
## cyl  -0.85  1.00  0.90  0.83 -0.70  0.78 -0.59 -0.49  0.53
## disp -0.85  0.90  1.00  0.79 -0.71  0.89 -0.43 -0.56  0.39
## hp   -0.78  0.83  0.79  1.00 -0.45  0.66 -0.71 -0.13  0.75
## drat  0.68 -0.70 -0.71 -0.45  1.00 -0.71  0.09  0.70 -0.09
## wt   -0.87  0.78  0.89  0.66 -0.71  1.00 -0.17 -0.58  0.43
## qsec  0.42 -0.59 -0.43 -0.71  0.09 -0.17  1.00 -0.21 -0.66
## gear  0.48 -0.49 -0.56 -0.13  0.70 -0.58 -0.21  1.00  0.27
## carb -0.55  0.53  0.39  0.75 -0.09  0.43 -0.66  0.27  1.00

This correlation matrix gives an overview of the correlations for all combinations of two variables.

Interpretation of a correlation coefficient

First of all, correlation ranges from -1 to 1. It gives us an indication on two things:

The direction of the relationship between the 2 variables
The strength of the relationship between the 2 variables

Regarding the direction of the relationship: On the one hand, a negative correlation implies that the two variables under consideration vary in opposite directions, that is, if a variable increases the other decreases and vice versa. On the other hand, a positive correlation implies that the two variables under consideration vary in the same direction, i.e., if a variable increases the other one increases and if one decreases the other one decreases as well.

Regarding the strength of the relationship: The more extreme the correlation coefficient (the closer to -1 or 1), the stronger the relationship. This also means that a correlation close to 0 indicates that the two variables are independent, that is, as one variable increases, there is no tendency in the other variable to either decrease or increase.

As an illustration, the Pearson correlation between horsepower (hp) and miles per gallon (mpg) found above is -0.78, meaning that the 2 variables vary in opposite direction. This makes sense, cars with more horsepower tend to consume more fuel (and thus have a lower mileage per gallon). On the contrary, from the correlation matrix we see that the correlation between miles per gallon (mpg) and the time to drive 1/4 of a mile (qsec) is 0.42, meaning that fast cars (low qsec) tend to have a worse mileage per gallon (low mpg). This again makes sense as fast cars tend to consume more fuel.

Note that it is a good practice to visualize the type of the relationship between the two variables before interpreting the correlation coefficients. The reason is that the correlation coefficient could be biased due to an outlier or due to the type of link between the two variables.

For instance, see the two Pearson correlation coefficients (denoted by R in the following plots) when the outlier is excluded and included:

The Pearson correlation coefficient changes drastically due to a single point, and thus the interpretation. It goes from a negative correlation coefficient, indicating a negative relationship between the 2 variables, to a positive coefficient, indicating a positive relationship. We would have missed this insight if we had not visualized the data in a scatterplot (see how to draw a scatterplot in this section).

A correlation coefficient may also miss a non-linear link between two variables:

The Pearson correlation coefficient is equal to 0, indicating no relationship between the two variables, because it measures the linear relationship and it is clear from the plot that the link is non-linear.

So to recap, it is a good practice to visualize the data via a scatterplot before interpreting a correlation coefficient (it does not tell the whole story) and see how the correlation coefficient changes when using the parametric (Pearson) or nonparametric version (Spearman or Kendall’s tau-b).

Visualizations

The correlation matrix presented above is not easily interpretable, especially when the dataset is composed of many variables. In the following sections, we present some alternatives to the correlation matrix for better readability.

A scatterplot for 2 variables

A good way to visualize a correlation between 2 variables is to draw a scatterplot of the two variables of interest. Suppose we want to examine the relationship between horsepower (hp) and miles per gallon (mpg):

# scatterplot
library(ggplot2)

ggplot(dat) +
  aes(x = hp, y = mpg) +
  geom_point(colour = "#0c4c8a") +
  theme_minimal()

If you are unfamiliar with the {ggplot2} package, you can draw the scatterplot using the plot() function from R base graphics:

plot(dat$hp, dat$mpg)

or use the esquisse addin to easily draw plots using the {ggplot2} package.

Scatterplots for several pairs of variables

Suppose that instead of visualizing the relationship between only 2 variables, we want to visualize the relationship for several pairs of variables. This is possible thanks to the pair() function.

For this illustration, we focus only on miles per gallon (mpg), horsepower (hp) and weight (wt):

# multiple scatterplots
pairs(dat[, c("mpg", "hp", "wt")])

The figure indicates that weight (wt) and horsepower (hp) are positively correlated, whereas miles per gallon (mpg) seems to be negatively correlated with horsepower (hp) and weight (wt).

Another simple correlation matrix

This version of the correlation matrix presents the correlation coefficients in a slightly more readable way, i.e., by coloring the coefficients based on their sign. Applied to our dataset, we have:

# improved correlation matrix
library(corrplot)

corrplot(cor(dat),
  method = "number",
  type = "upper" # show only upper side
)

Correlation test

For 2 variables

Unlike a correlation matrix which indicates the correlation coefficients between some pairs of variables in the sample, a correlation test is used to test whether the correlation (denoted $\rho$) between 2 variables is significantly different from 0 or not in the population.

Actually, a correlation coefficient different from 0 in the sample does not mean that the correlation is significantly different from 0 in the population. This needs to be tested with a hypothesis test—and known as the correlation test.

The null and alternative hypothesis for the correlation test are as follows:

$H_0$: $\rho = 0$ (meaning that there is no linear relationship between the two variables)
$H_1$: $\rho \ne 0$ (meaning that there is a linear relationship between the two variables)

Via this correlation test, what we are actually testing is whether:

the sample contains sufficient evidence to reject the null hypothesis and conclude that the correlation coefficient does not equal 0, so the relationship exists in the population.
or on the contrary, the sample does not contain enough evidence that the correlation coefficient does not equal 0, so in this case we do not reject the null hypothesis of no relationship between the variables in the population.

Note that there are 2 assumptions for this test to be valid:

Independence of the data
For small sample sizes (usually $n < 30$), the two variables should follow a normal distribution

Suppose that we want to test whether the rear axle ratio (drat) is correlated with the time to drive a quarter of a mile (qsec):

# Pearson correlation test
test <- cor.test(dat$drat, dat$qsec)
test

## 
## 	Pearson's product-moment correlation
## 
## data:  dat$drat and dat$qsec
## t = 0.50164, df = 30, p-value = 0.6196
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.265947  0.426340
## sample estimates:
##        cor 
## 0.09120476

The p-value of the correlation test between these 2 variables is 0.62. At the 5% significance level, we do not reject the null hypothesis of no correlation. We therefore conclude that we do not reject the hypothesis that there is no linear relationship between the 2 variables.²

This test proves that even if the correlation coefficient is different from 0 (the correlation is 0.09 in the sample), it is actually not significantly different from 0 in the population.

Note that the p-value of a correlation test is based on the correlation coefficient and the sample size. The larger the sample size and the more extreme the correlation (closer to -1 or 1), the more likely the null hypothesis of no correlation will be rejected.

With a small sample size, it is thus possible to obtain a relatively large correlation in the sample (based on the correlation coefficient), but still find a correlation not significantly different from 0 in the population (based on the correlation test). For this reason, it is recommended to always perform a correlation test before interpreting a correlation coefficient to avoid flawed conclusions.

For several pairs of variables

Similar to the correlation matrix used to compute correlation for several pairs of variables, the rcorr() function (from the {Hmisc} package) allows to compute p-values of the correlation test for several pairs of variables at once. Applied to our dataset, we have:

# correlation tests for whole dataset
library(Hmisc)
res <- rcorr(as.matrix(dat)) # rcorr() accepts matrices only

# display p-values (rounded to 3 decimals)
round(res$P, 3)

##        mpg   cyl  disp    hp  drat    wt  qsec  gear  carb
## mpg     NA 0.000 0.000 0.000 0.000 0.000 0.017 0.005 0.001
## cyl  0.000    NA 0.000 0.000 0.000 0.000 0.000 0.004 0.002
## disp 0.000 0.000    NA 0.000 0.000 0.000 0.013 0.001 0.025
## hp   0.000 0.000 0.000    NA 0.010 0.000 0.000 0.493 0.000
## drat 0.000 0.000 0.000 0.010    NA 0.000 0.620 0.000 0.621
## wt   0.000 0.000 0.000 0.000 0.000    NA 0.339 0.000 0.015
## qsec 0.017 0.000 0.013 0.000 0.620 0.339    NA 0.243 0.000
## gear 0.005 0.004 0.001 0.493 0.000 0.000 0.243    NA 0.129
## carb 0.001 0.002 0.025 0.000 0.621 0.015 0.000 0.129    NA

Only correlations with p-values smaller than the significance level (usually $\alpha = 0.05$) should be interpreted.

Combination of correlation coefficients and correlation tests

Now that we covered the concepts of correlation coefficients and correlation tests, let see if we can combine the two concepts.

If you need to do this for a few pairs of variables, I recommend using the ggscatterstats() function from the {ggstatsplot} package. Let’s see it in practice with one pair of variables—wt and mpg:

## plot with statistical results
library(ggstatsplot)

ggscatterstats(
  data = dat,
  x = wt,
  y = mpg,
  bf.message = FALSE,
  marginal = FALSE # remove histograms
)

Based on the result of the test, we conclude that there is a negative correlation between the weight and the number of miles per gallon ($r = - 0.87$, $p$-value < 0.001).

If you need to do it for many pairs of variables, I recommend using the the correlation function from the easystats {correlation} package.

This function allows to combine correlation coefficients and correlation tests for several pairs of variables, all in a single table (thanks to krzysiektr for pointing it out to me):

library(correlation)

correlation::correlation(dat,
  include_factors = TRUE, method = "auto"
)

## # Correlation Matrix (auto-method)
## 
## Parameter1 | Parameter2 |     r |         95% CI | t(30) |         p
## --------------------------------------------------------------------
## mpg        |        cyl | -0.85 | [-0.93, -0.72] | -8.92 | < .001***
## mpg        |       disp | -0.85 | [-0.92, -0.71] | -8.75 | < .001***
## mpg        |         hp | -0.78 | [-0.89, -0.59] | -6.74 | < .001***
## mpg        |       drat |  0.68 | [ 0.44,  0.83] |  5.10 | < .001***
## mpg        |         wt | -0.87 | [-0.93, -0.74] | -9.56 | < .001***
## mpg        |       qsec |  0.42 | [ 0.08,  0.67] |  2.53 | 0.137    
## mpg        |       gear |  0.48 | [ 0.16,  0.71] |  3.00 | 0.065    
## mpg        |       carb | -0.55 | [-0.75, -0.25] | -3.62 | 0.016*   
## cyl        |       disp |  0.90 | [ 0.81,  0.95] | 11.45 | < .001***
## cyl        |         hp |  0.83 | [ 0.68,  0.92] |  8.23 | < .001***
## cyl        |       drat | -0.70 | [-0.84, -0.46] | -5.37 | < .001***
## cyl        |         wt |  0.78 | [ 0.60,  0.89] |  6.88 | < .001***
## cyl        |       qsec | -0.59 | [-0.78, -0.31] | -4.02 | 0.007**  
## cyl        |       gear | -0.49 | [-0.72, -0.17] | -3.10 | 0.054    
## cyl        |       carb |  0.53 | [ 0.22,  0.74] |  3.40 | 0.027*   
## disp       |         hp |  0.79 | [ 0.61,  0.89] |  7.08 | < .001***
## disp       |       drat | -0.71 | [-0.85, -0.48] | -5.53 | < .001***
## disp       |         wt |  0.89 | [ 0.78,  0.94] | 10.58 | < .001***
## disp       |       qsec | -0.43 | [-0.68, -0.10] | -2.64 | 0.131    
## disp       |       gear | -0.56 | [-0.76, -0.26] | -3.66 | 0.015*   
## disp       |       carb |  0.39 | [ 0.05,  0.65] |  2.35 | 0.177    
## hp         |       drat | -0.45 | [-0.69, -0.12] | -2.75 | 0.110    
## hp         |         wt |  0.66 | [ 0.40,  0.82] |  4.80 | < .001***
## hp         |       qsec | -0.71 | [-0.85, -0.48] | -5.49 | < .001***
## hp         |       gear | -0.13 | [-0.45,  0.23] | -0.69 | > .999   
## hp         |       carb |  0.75 | [ 0.54,  0.87] |  6.21 | < .001***
## drat       |         wt | -0.71 | [-0.85, -0.48] | -5.56 | < .001***
## drat       |       qsec |  0.09 | [-0.27,  0.43] |  0.50 | > .999   
## drat       |       gear |  0.70 | [ 0.46,  0.84] |  5.36 | < .001***
## drat       |       carb | -0.09 | [-0.43,  0.27] | -0.50 | > .999   
## wt         |       qsec | -0.17 | [-0.49,  0.19] | -0.97 | > .999   
## wt         |       gear | -0.58 | [-0.77, -0.29] | -3.93 | 0.008**  
## wt         |       carb |  0.43 | [ 0.09,  0.68] |  2.59 | 0.132    
## qsec       |       gear | -0.21 | [-0.52,  0.15] | -1.19 | > .999   
## qsec       |       carb | -0.66 | [-0.82, -0.40] | -4.76 | < .001***
## gear       |       carb |  0.27 | [-0.08,  0.57] |  1.56 | 0.774    
## 
## p-value adjustment method: Holm (1979)
## Observations: 32

As you can see, it gives, among other useful information, the correlation coefficients (column r) and the result of the correlation test (column 95% CI for the confidence interval or p for the $p$-value) for all pairs of variables.

Correlograms

The table above is very useful and informative, but let see if it is possible to combine the concepts of correlation coefficients and correlations test in one single visualization. A visualization that would be easy to read and interpret.

Ideally, we would like to have a concise overview of correlations between all possible pairs of variables present in a dataset, with a clear distinction for correlations that are significantly different from 0.

The figure below, known as a correlogram and adapted from the corrplot() function, does precisely this:

# do not edit
corrplot2 <- function(data,
                      method = "pearson",
                      sig.level = 0.05,
                      order = "original",
                      diag = FALSE,
                      type = "upper",
                      tl.srt = 90,
                      number.font = 1,
                      number.cex = 1,
                      mar = c(0, 0, 0, 0)) {
  library(corrplot)
  data_incomplete <- data
  data <- data[complete.cases(data), ]
  mat <- cor(data, method = method)
  cor.mtest <- function(mat, method) {
    mat <- as.matrix(mat)
    n <- ncol(mat)
    p.mat <- matrix(NA, n, n)
    diag(p.mat) <- 0
    for (i in 1:(n - 1)) {
      for (j in (i + 1):n) {
        tmp <- cor.test(mat[, i], mat[, j], method = method)
        p.mat[i, j] <- p.mat[j, i] <- tmp$p.value
      }
    }
    colnames(p.mat) <- rownames(p.mat) <- colnames(mat)
    p.mat
  }
  p.mat <- cor.mtest(data, method = method)
  col <- colorRampPalette(c("#BB4444", "#EE9988", "#FFFFFF", "#77AADD", "#4477AA"))
  corrplot(mat,
    method = "color", col = col(200), number.font = number.font,
    mar = mar, number.cex = number.cex,
    type = type, order = order,
    addCoef.col = "black", # add correlation coefficient
    tl.col = "black", tl.srt = tl.srt, # rotation of text labels
    # combine with significance level
    p.mat = p.mat, sig.level = sig.level, insig = "blank",
    # hide correlation coefficients on the diagonal
    diag = diag
  )
}

# edit from here
corrplot2(
  data = dat,
  method = "pearson",
  sig.level = 0.05,
  order = "original",
  diag = FALSE,
  type = "upper",
  tl.srt = 75
)

The correlogram shows correlation coefficients for all pairs of variables (with more intense colors for more extreme correlations), and correlations not significantly different from 0 are represented by a white box.

To learn more about this plot and the code used, I invite you to read the article entitled “Correlogram in R: how to highlight the most correlated variables in a dataset”.

For those of you who are still not completely satisfied, I recently found two alternatives—one with the ggpairs() function from the {GGally} package and one with the ggcormat() function from the {ggstatsplot} package.

The two functions are illustrated with the variables mpg, hp and wt:

library(GGally)

ggpairs(dat[, c("mpg", "hp", "wt")])

The plot above combines correlation coefficients, correlation tests (via the asterisks next to the coefficients³) and scatterplots for all possible pairs of variables present in a dataset.

library(ggstatsplot)

ggcorrmat(
  data = dat[, c("mpg", "hp", "wt")],
  type = "parametric", # parametric for Pearson, nonparametric for Spearman's correlation
  colors = c("darkred", "white", "steelblue") # change default colors
)

The plot above also shows the correlation coefficients and if any, the non-significant correlations (by default at the 5% significance level with the Holm adjustment method) are shown by a big cross on the correlation coefficients.

The advantage of these two alternatives compared to the first one is that it is directly available within a package, so you do not need to run the code of the function first in order to draw the correlogram.

Correlation does not imply causation

I am pretty sure you have already heard the statement “Correlation does not imply causation” in statistics. An article about correlation would not be complete without discussing about causation.

A non-zero correlation between two variables does not necessarily mean that there is a cause and effect relationship between these two variables!

Indeed, a significant correlation between two variables means that changes in one variable are associated (positively or negatively) with changes in the other variable. Nonetheless, a significant correlation does not indicate that variations in one variable cause the variations in the other variable.

A non-zero correlation between X and Y can appear in several cases:

X causes Y
Y causes X
a third variable causes X and Y
a combination of these three reasons

Sometimes it is quite clear that there is a causal relationship between two variables. Take for example the correlation between the price of a consumer product such as milk and its consumption. It is quite obvious that there is a causal link between the two: if the price of milk increases, it is expected that its consumption will decrease.

However, this causal link is not always present even if the correlation is significant. Maurage et al. (2013) showed that, although there is a positive and significant correlation between chocolate consumption and the number of Nobel laureates, this correlation comes from the fact that a third variable, Gross Domestic Product (GDP), causes chocolate consumption and the number of Nobel laureates. They found that countries with higher GDP tend to have a higher level of chocolate consumption and scientific research (leading to more Nobel laureates).

This example shows that one must be very cautious when interpreting correlations and avoid over-interpreting a correlation as a causal relationship.

Conclusion

Thanks for reading.

I hope this article helped you to compute correlation coefficients and perform correlation tests in R. If you would like to learn how to compute the coefficients by hand, see this step-by-step tutorial.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

(Note that this article is available for download on my Gumroad page.)

References

Maurage, Pierre, Alexandre Heeren, and Mauro Pesenti. 2013. “Does Chocolate Consumption Really Boost Nobel Award Chances? The Peril of over-Interpreting Correlations in Health Studies.” The Journal of Nutrition 143 (6): 931–33.

It is true that there is the point-biserial correlation which can be used with a nominal variable (consisting of two factors). Nonetheless, this type of correlation is much less known and usually not covered in introductory statistics classes; with one continuous and one nominal variable, it is much more frequent to learn about the Student’s t-test (for a nominal variable with 2 groups) or ANOVA (for a nominal variable with 3 or more groups). More information about choosing the most appropriate measure of association depending on the type of variable can be found in this article.↩︎
It is important to remember that we tested for a linear relationship between the two variables since we used the Pearson’s correlation. It may be the case that there is a relationship between the two variables in the population, but this relation may not be linear.↩︎
One asterisk means that the coefficient is significant at the 5% level, 2 is at the 1% significance level, and 3 is at the 0.1% significance level. This is usually the case in R; the more asterisks, the more it is significant.↩︎

How to upload your R code on GitHub? An example with an R script on MacOS

Sun, 24 May 2020 00:00:00 +0000

Introduction

A few days ago, a colleague asked me how to upload some R code on GitHub in order to make it accessible to everyone. Due to the lockdown, I could not just go into his office and show him on his computer. So I sent him several screenshots showing, step by step, how to do so.

Right before I deleted the screenshots I’d just taken, I thought that perhaps they would be useful for other persons, so I wrote this article.

Note 1: The screenshots have been taken on MacOS and I have not tested it on Windows. Do not hesitate to let me know in the comments whether it is similar or not on other operating systems.

Note 2: There must be other ways to do it, but the method shown below is (in my opinion) easy and works well.

Prerequisite

In order to follow this guide and upload your R code on GitHub, you will need at least:

a GitHub account
the GitHub Desktop application installed on your computer

Step-by-step guide

For this guide, I use a R script created to plot the evolution of the hospital admissions due to COVID-19 in Belgium. See the repository on GitHub in case you want to see the final result before proceeding further.

You often hear

“A picture is worth a thousand words”

So here is without further ado how to upload your R script on GitHub in images:

Step 1: Go to github.com/login and sign in

Step 2: Go to your GitHub profile and create a new repository

Step 3: Set the name, description, audience and README file for the new repository

Be careful that if you choose to make it public, it will be visible to everyone. If you do not want to share the code, but still want it to be uploaded on GitHub, choose the private option.

Initializing the repo with a README file is not mandatory, but I strongly recommend it so you will be able to add information for this repo.

Step 4: You now see your new repository with only the README file

Step 5: Open the GitHub Desktop application and sign in with your GitHub account

Step 6 a: Clone the repository you just created on github.com to your computer

Step 6 b: Clone the repository you just created on github.com to your computer

Step 6 c: Clone the repository you just created on github.com to your computer

Step 7: Your new repo appears on the GitHub Desktop application

Step 8: Open an R script and write your code

(See how to install R and RStudio if you are unfamiliar with it.)

Step 9 a: Save your R script in the corresponding folder

You can see the path to the folder in step 6 c, under “Local Path”.

Step 9 b: Save your R script in the corresponding folder

Step 10: Reopen GitHub Desktop, edit the commit title (1) and click on the commit button (2)

In step 10, make sure that the current repository (see top left) is the repo you are currently working on.

Step 11: Push the commit to github.com

Step 12: On your GitHub profile, click on the newly created repository

Step 13: You now see your script which has been added to the repository

Your R code is now available online via the new repository on your GitHub profile. If you need to share it, you can simply share the URL of the repo.

Additional notes

If you need to edit your code:

Find the folder of the repo on your computer by clicking on the button “Show in Finder”:

Find the folder of your repo on your computer

You will be redirected to the folder of your repo on your computer. Open the R script you want to work on, edit it and save it:

Open the R script, edit it and save it

After your script is saved, do not forget to commit the changes and push the commit (see step 10 above).

In case you need to see an old version of your code, you can see all commits by clicking on the commits tab accessible via the repo on github.com:

See all commits

Conclusion

Thanks for reading.

I hope this article helped you to create a GitHub repository, and upload your R scripts so that it is available to anyone.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

COVID-19 in Belgium: is it over yet?

Fri, 22 May 2020 00:00:00 +0000

Introduction

Note 1: The present article has been written on May 22, 2020 and has been updated infrequently. The current situation regarding COVID-19 in Belgium may therefore be different to what is presented below. See my Twitter profile for more frequent updates of the plots.

Note 2: This is a joint work with Prof. Niko Speybroeck, Prof. Catherine Linard, Prof. Simon Dellicour and Angel Rosas-Aguirre.

Belgium recently started to lift its lockdown measures initially imposed to contain the spread of the Covid-19. Following this decision taken by Belgian authorities, we analyze how the situation evolved so far.

Contrarily to a previous article in which I analyzed the outbreak of the Coronavirus in Belgium using the SIR model, in this article we focus on the evolution of the number of:

hospital admissions
patients in hospitals
patients in intensive care
new confirmed cases

at the province and national level.

Data is from Sciensano and all plots were created with the {ggplot2} package.

New hospital admissions

Overall

Belgian hospitalizations COVID-19

From the above figure, we see that the rate of hospitalizations continue with a decreasing trend in all provinces (and in Belgium as well).

Download the figure, or see the code.

Update of October 27, 2020:

COVID19 hospitalizations in Belgium

Download the figure, or see the code.

The detailed situation in Brabant:

Download the figure, or see the code.

By period

Daily COVID19 hospitalizations in Belgium from March to October 2020

Download the figure, or see the code.

Update of November 16, 2020:

Daily COVID19 hospitalizations in Belgium by period

Download the figure.

In the first wave, the province of Limburg recorded on average the highest number of COVID19 hospital admissions per million inhabitants. During the second wave, Liège and Hainaut struggled with the highest rates. With two exceptions (Antwerp and Limburg), last month was worse than in March-April. In three provinces (Hainaut, Namur and Liège), the number has more than doubled.

During the period from June 14 to July 15, 2020, the number of COVID19 hospital admissions in Belgium fell to very low relative levels, but we have failed to maintain them. Now that hospital admissions are no longer increasing, we hope that the colors will lighten up again a bit as the end of the year approaches.

Zooming in

Hospital admissions COVID-19 - Belgium

Download the figure or see the code.

Update of February 26, 2021:

There is some ongoing debate in Belgium on whether or not to ease restrictions. On February 26, 2021, Belgian authorities will meet, discuss, debate and decide. Current levels and trends of COVID-19 hospitalizations may guide them:

There is still no strong evidence that COVID-19 hospitalization curves decrease in Belgium. The comparison between the first (in gray - dates & curve) and second wave (in blue - dates & curve) needs to be done with care, but indicates that current hospitalization levels are not as low as some may like:

Zooming in provides some additional insights on the COVID-19 levels during the first and second waves at province level:

This shows that the second wave resulted in more hospitalizations than the first one in most Belgian provinces, despite the warning of a first deadly wave. It also illustrates the fact that daily hospitalizations in Belgium are currently still higher than what was observed at the end of the first wave.

Put simply, the bad news is that the combination of the number of contacts and the risk of transmission by contact seems (at the moment) not sufficiently low to result in a considerable decrease of hospitalizations. Yet (put simply), the good news today is that there is already some immunity in the population and that vaccinations may increase this immunity considerably. This can help in pushing curves down. Let’s not lose hope.

Download figures (1, 2 and 3) or see the code.

Update of May 10, 2021:

COVID19 hospitalizations - Wave 1 and 2

When looking at the above plot, bad news are that:

current levels correspond to levels of October 2020 and
current levels are still about double the target of 75 hospitalizations per day.

There are, however, three good news (compared to October 2020):

decreasing curve,
vaccination and
good weather.

Update of June 4, 2021

COVID-19 hospitalizations in Belgium below 75/day

The good news is that the number of COVID-19 hospitalizations in Belgium is now below the well-known threshold of 75 hospitalizations per day (which is a target defined by the Belgian government). This is the way to go, and we hope this trend will continue in the coming days/weeks.

Patients in hospitals

Below the evolution of the number of patients in hospitals in Belgium:

COVID19 patients in hospitals in Belgium

Download the figure or see the code.

We see that, as of October 28, 2020, the number of COVID19 patients in Belgian hospitals reached the peak of the first wave. So although patients stay shorter at the hospital during the second wave compared to the first wave, hospitals are still getting crowded.

Therefore, if the number of patients in hospitals follows the same path in the coming weeks, hospitals will quickly become too crowded and will not be able to accept new patients as their maximum capacity will soon be reached (if this is not already the case…).

Patients in intensive care

Below the evolution of COVID19 patients in intensive care in Belgium, with short-term projections and 99% confidence interval:

Evolution of COVID19 patients in intensive care in Belgium

Download the figure.

Short-term projections indicate what may have happened without the slow-down in transmission. This slow-down is positive news.

The maps show total intensive care patients by province if these would have had the Belgian population. Map at the top shows maximum levels in March-April and map at the bottom shows current levels. The maps indicate high intensive care use due to COVID19. In most Belgian provinces, numbers are still higher today than March-April peak numbers.

Observations are in line with other preliminary indications, such as trends of COVID19 hospitalizations (currently relatively volatile), indicating that transmission is slowing down:

Download the figure.

Confirmed cases

Note that the reported number of new confirmed cases is probably underestimated. This number does not take into account undiagnosed (without or with few symptoms) or untested cases. Therefore, figures with number of cases should be interpreted with extreme caution.

By province

New confirmed COVID-19 cases in Belgium

Download the figure or see the code.

By age group and sex

Static

Below another visualization of the number of cases by age group and sex in Belgium, for three different periods:

COVID-19 cases by age group and sex in Belgium

Download the figure or see the code.

This visualization shows the importance to report ages of cases and not just total number.

Moreover, we see that the distribution of cases per week by age group at the beginning of September is similar than during the summer holidays, but the number of cases per week is higher. The distribution of cases per week by age group at the beginning of September is however different from the “first wave” (period from March 1, 2020 to May 31, 2020). During the fist period, majority of cases were elderly, while at the beginning of September majority of cases are young people. It would be interesting to see how the distribution of cases by age group evolves during winter.

The figure above may be put in relation with the structure of the Belgian population:

Structure of Belgian population (2019)

Download the figure or see the code.

Dynamic

Additionally, these can be seen dynamically:

COVID-19 cases by age group and sex in Belgium - dynamic version

Download the figure or see the code.

With an update of the second wave:

Age and sex specific incidence per 100 000 of COVID19 cases in Belgium - dynamic version

Download the figure or see the code.

By age group, sex and province

Download the figure or see the code.

Conclusion

Thanks for reading.

We hope that these figures will evolve in the right direction. In the meantime, take care and stay safe!

If you would like to be further updated on the evolution of the COVID-19 epidemic, two options:

visit the blog from time to time, and
join Twitter and follow us: statsandr, NikoSpeybroeck & arosas_aguirre

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

One-proportion and chi-square goodness of fit test

Wed, 13 May 2020 00:00:00 +0000

Introduction

In a previous article, I presented the Chi-square test of independence in R which is used to test the independence between two categorical variables.

In this article, I show how to perform, first in R and then by hand, the:

one-proportion test (also referred as one-sample proportion test)
Chi-square goodness of fit test

The first test is used to compare an observed proportion to an expected proportion, when the qualitative variable has only two categories. The second test is used to compare multiple observed proportions to multiple expected proportions, in a situation where the qualitative variable has two or more categories.

Both tests allow to test the equality of proportions between the levels of the qualitative variable or to test the equality with given proportions. These given proportions could be determined arbitrarily or based on the theoretical probabilities of a known distribution.

In R

Data

For this section, we use the same dataset than in the article on descriptive statistics. It is the well-known iris dataset, to which we add the variable size. The variable size corresponds to small if the length of the petal is smaller than the median of all flowers, big otherwise:

# load iris dataset
dat <- iris

# create size variable
dat$size <- ifelse(dat$Sepal.Length < median(dat$Sepal.Length),
  "small", "big"
)

# show first 5 observations
head(dat, n = 5)

##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species  size
## 1          5.1         3.5          1.4         0.2  setosa small
## 2          4.9         3.0          1.4         0.2  setosa small
## 3          4.7         3.2          1.3         0.2  setosa small
## 4          4.6         3.1          1.5         0.2  setosa small
## 5          5.0         3.6          1.4         0.2  setosa small

One-proportion test

For this example, we have a sample of 150 flowers and we want to test whether the proportion of small flowers is different than the proportion of big flowers (measured by the variable size). Here are the number of flowers by size, and the corresponding proportions:

# barplot
library(ggplot2)
ggplot(dat) +
  aes(x = size) +
  geom_bar(fill = "#0c4c8a") +
  theme_minimal()

# counts by size
table(dat$size)

## 
##   big small 
##    77    73

# proportions by size, rounded to 2 decimals
round(prop.table(table(dat$size)), 2)

## 
##   big small 
##  0.51  0.49

Among the 150 flowers forming our sample, 51% and 49% are big and small, respectively. To test whether the proportions are different among both sizes, we use the prop.test() function which accepts the following arguments:

number of successes
number of observations/trials
expected probability (the one we want to test against)

The hypotheses in our example are:

$H_0$: proportions of big and small flowers are equal
$H_1$: proportions of big and small flowers are different

Considering (arbitrarily) that big is the success, we have:¹

# one-proportion test
test <- prop.test(
  x = 77, # number of successes
  n = 150, # total number of trials (77 + 73)
  p = 0.5 # we test for equal proportion so prob = 0.5 in each group
)

test

## 
##  1-sample proportions test with continuity correction
## 
## data:  77 out of 150, null probability 0.5
## X-squared = 0.06, df = 1, p-value = 0.8065
## alternative hypothesis: true p is not equal to 0.5
## 95 percent confidence interval:
##  0.4307558 0.5952176
## sample estimates:
##         p 
## 0.5133333

We obtain an output with:

the null probability (0.5),
the test statistic (X-squared = 0.06),
the degrees of freedom (df = 1),
the p-value (p-value = 0.8065),
the alternative hypothesis (true p is not equal to 0.5),
the 95% confidence interval (which can also be extracted with test$conf.int) and
the proportion in the sample (0.5133333).

The p-value is 0.806 so, at the 5% significance level, we do not reject the null hypothesis that the proportions of small and big flowers are the same.

An alternative is the ggpiestats() function from the {ggstatsplot} package:²

## plot with statistical results
library(ggstatsplot)
ggpiestats(
  data = dat,
  x = size,
  bf.message = FALSE
)

Note that the $p$-value (the value after p = in the subtitle of the plot) is slightly different because Yates’ continuity correction is not applied in ggpiestats() while it is applied by default in prop.test().

The conclusion remains however the same, that is, we do not reject the null hypothesis that proportions of big and small flowers are equal.

Assumption of `prop.test()` and `binom.test()`

Note that prop.test() uses a normal approximation to the binomial distribution. Therefore, one assumption of this test is that the sample size is large enough (usually, n > 30). If the sample size is small, it is recommended to use the exact binomial test.

The exact binomial test can be performed with the binom.test() function and accepts the same arguments as the prop.test() function.

For this example, suppose now that we have a sample of 12 big and 3 small flowers and we want to test whether the proportions are the same among both sizes:

# barplot
barplot(c(12, 3), # observed counts
  names.arg = c("big", "small"), # rename labels
  ylab = "Frequency", # y-axis label
  xlab = "Size" # x-axis label
)
abline(
  h = 15 / 2, # expected counts in each level
  lty = 2 # dashed line
)

# exact binomial test
test <- binom.test(
  x = 12, # counts of successes
  n = 15, # total counts (12 + 3)
  p = 0.5 # expected proportion
)

test

## 
##  Exact binomial test
## 
## data:  12 and 15
## number of successes = 12, number of trials = 15, p-value = 0.03516
## alternative hypothesis: true probability of success is not equal to 0.5
## 95 percent confidence interval:
##  0.5191089 0.9566880
## sample estimates:
## probability of success 
##                    0.8

The p-value is 0.035 so, at the 5% significance level, we reject the null hypothesis and we conclude that the proportions of small and big flowers are significantly different. This is equivalent than concluding that the proportion of big flowers is significantly different from 0.5 (since there are only two sizes).

If you want to test that the proportion of big flowers is greater than 50%, add the alternative = "greater" argument into the binom.test() function:³

test <- binom.test(
  x = 12, # counts of successes
  n = 15, # total counts (12 + 3)
  p = 0.5, # expected proportion
  alternative = "greater" # test that prop of big flowers is > 0.5
)

test

## 
##  Exact binomial test
## 
## data:  12 and 15
## number of successes = 12, number of trials = 15, p-value = 0.01758
## alternative hypothesis: true probability of success is greater than 0.5
## 95 percent confidence interval:
##  0.5602156 1.0000000
## sample estimates:
## probability of success 
##                    0.8

The p-value is 0.018 so, at the 5% significance level, we reject the null hypothesis and we conclude that the proportion of big flowers is significantly larger than 50%.

Chi-square goodness of fit test

Suppose now that the qualitative variable has more than two levels as it is the case for the variable Species:

# barplot
ggplot(dat) +
  aes(x = Species) +
  geom_bar(fill = "#0c4c8a") +
  theme_minimal()

# counts by Species
table(dat$Species)

## 
##     setosa versicolor  virginica 
##         50         50         50

The variable Species has 3 levels, with 50 observations in each level.

Suppose for this example that we want to test whether the 3 species are equally common. If they were equally common, they would be equally distributed and the expected proportions would be $\frac{1}{3}$ for each of the species.

The hypotheses are now:

$H_0$: proportions of each species are equal
$H_1$: there is at least one species with a different proportion⁴

This test can be done with the chisq.test() function, accepting the following arguments:

a numeric vector representing the observed proportions
a vector of probabilities (of the same length of the observed proportions) representing the expected proportions

Applied to our research question (i.e., are the 3 species equally common?), we have:

# chi-square goodness of fit test
test <- chisq.test(table(dat$Species), # observed proportions
  p = c(1 / 3, 1 / 3, 1 / 3) # expected proportions
)

test

## 
##  Chi-squared test for given probabilities
## 
## data:  table(dat$Species)
## X-squared = 0, df = 2, p-value = 1

The p-value is 1 so, at the 5% significance level, we do not reject the null hypothesis that the proportions are equal among all species.

This was quite obvious even before doing the statistical test given that there are exactly 50 flowers of each species, so it was easy to see that the species are equally common. We however still did the test to show how it works in practice.

Note that the alternative proposed by the {ggstatsplot} package can also be used for a Chi-square goodness of fit test:

## plot with statistical results
ggpiestats(
  data = dat,
  x = Species,
  bf.message = FALSE
)

Assumptions

One of the assumptions of the chi-square goodness of fit test is that the sample size is large enough in order for the chi-square approximation to be valid.

To be more precise, there must be at least 5 expected frequencies in each group of your categorical variable. This can be verified as follows:

chisq.test(table(dat$Species))$expected

##     setosa versicolor  virginica 
##         50         50         50

The assumption of sufficiently large sample size is met as all expected frequencies are above 5.

Does my distribution follow a given distribution?

In the previous section, we chose the proportions ourselves. The goodness of fit test is also particularly useful to compare observed proportions with expected proportions that are based on some known distribution.

Remember the hypotheses of the test:

$H_0$: there is no significant difference between the observed and the expected frequencies
$H_1$: there is a significant difference between the observed and the expected frequencies

For this example, suppose that we measured the number of girls in 100 families of 5 children. We want to test whether the (observed) distribution of number girls follows a binomial distribution.

Observed frequencies

Here is the distribution of the number of girls per family in our sample of 100 families of 5 children:

And the corresponding frequencies and relative frequencies (remember that the relative frequency is the frequency divided by the total sample size):

# counts
dat

##   Girls Frequency Relative_freq
## 1     0         5          0.05
## 2     1        12          0.12
## 3     2        28          0.28
## 4     3        33          0.33
## 5     4        17          0.17
## 6     5         5          0.05

Expected frequencies

In order to compare the observed frequencies to a binomial distribution and see if both distributions match, we first need to determine the expected frequencies that would be obtained in case of a binomial distribution.

The expected frequencies assuming a probability of 0.5 of having a girl (for each of the 5 children) are as follows:

# create expected frequencies for a binomial distribution
x <- 0:5
df <- data.frame(
  Girls = factor(x),
  Expected_relative_freq = dbinom(x, size = 5, prob = 0.5)
)
df$Expected_freq <- df$Expected_relative_freq * 100 # *100 since there are 100 families

# create barplot
p <- ggplot(df, aes(x = Girls, y = Expected_freq)) +
  geom_bar(stat = "identity", fill = "#F8766D") +
  xlab("Number of girls per family") +
  ylab("Expected frequency") +
  labs(title = "Binomial distribution Bi(x, n = 5, p = 0.5)") +
  theme_minimal()
p

# expected relative frequencies and (absolute) frequencies
df

##   Girls Expected_relative_freq Expected_freq
## 1     0                0.03125         3.125
## 2     1                0.15625        15.625
## 3     2                0.31250        31.250
## 4     3                0.31250        31.250
## 5     4                0.15625        15.625
## 6     5                0.03125         3.125

Observed vs. expected frequencies

We now compare the observed frequencies to the expected frequencies to see whether the two differ significantly. If the two differ significantly, we reject the hypothesis that the number of girls per family of 5 children follows a binomial distribution. On the other hand, if the observed and expected frequencies are similar, we do not reject the hypothesis that the number of girls per family follows a binomial distribution.

Visually we have:

# create data
data <- data.frame(
  num_girls = factor(rep(c(0:5), times = 2)),
  Freq = c(dat$Freq, df$Expected_freq),
  obs_exp = c(rep("observed", 6), rep("expected", 6))
)

# create plot
ggplot() +
  geom_bar(
    data = data, aes(
      x = num_girls, y = Freq,
      fill = obs_exp
    ),
    position = "dodge", # bar next to each other
    stat = "identity"
  ) +
  ylab("Frequency") +
  xlab("Number of girls per family") +
  theme_minimal() +
  theme(legend.title = element_blank()) # remove legend title

We see that the observed and expected frequencies are quite similar, so we expect that the number of girls in families of 5 children follows a binomial distribution. However, only the goodness of fit test will confirm our belief:

# chi-square goodness of fit test
test <- chisq.test(dat$Freq, # observed frequencies
  p = df$Expected_relative_freq # expected proportions
)

test

## 
##  Chi-squared test for given probabilities
## 
## data:  dat$Freq
## X-squared = 3.648, df = 5, p-value = 0.6011

The p-value is 0.601 so, at the 5% significance level, we do not reject the null hypothesis that the observed and expected frequencies are equal. This is equivalent than concluding that we cannot reject the hypothesis that the number of girls in families of 5 children follows a binomial distribution (since the expected frequencies were based on a binomial distribution).

Note that the chi-square goodness of fit test can of course be performed with other types of distribution than the binomial one. For instance, if you want to test whether an observed distribution follows a Poisson distribution, this test can be used to compare the observed frequencies with the expected proportions that would be obtained in case of a Poisson distribution.

By hand

Now that we showed how to perform the one-proportion and chi-square goodness of fit test in R, in this section we show how to do these tests by hand. We first illustrate the one-proportion test then the chi-square goodness of fit test.

One-proportion test

For this example, suppose that we tossed a coin 100 times and noted that it landed on heads 67 times. Following this, we want to test whether the coin is fair, that is, test whether the probability of landing on heads or tails is equal to 50%.

As for many hypothesis tests, we do it through 4 easy steps:

State the null and alternative hypotheses
Compute the test-statistic (also known as t-stat)
Find the rejection region
Conclude by comparing the test-statistic with the rejection region

Step 1.

In our example, the null and alternative hypotheses are:

$H_0$: $p_0 = 0.5$
$H_1$: $p_0 \ne 0.5$

where $p_0$ is the expected proportion of landing on heads.

Step 2.

The test statistic is:⁵

\[z_{obs} = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1 - p_0)}{n}}} = \frac{0.67 - 0.5}{\sqrt{\frac{0.5 \cdot (1 - 0.5)}{100}}} = 3.4\]

(See how to perform hypothesis tests in a Shiny app if you need more help in computing the test statistic.)

Step 3.

The rejection region is found via the normal distribution table. Assuming a significance level $\alpha = 0.05$, we have:

\[\pm z_{\alpha/2} = \pm z_{0.025} = \pm 1.96\]

Step 4.

We compare the test statistic (found in step 2) with the rejection region (found in step 3) and we conclude. Visually, we have:

The test statistic lies within the rejection region (i.e., the grey shaded areas). Therefore, at the 5% significance level, we reject the null hypothesis and we conclude that the proportion of heads (and thus tails) is significantly different than 50%. In other words, still at the 5% significance level, we conclude that the coin is unfair.

If you prefer to compute the p-value instead of comparing the t-stat and the rejection region, you can use this Shiny app to easily compute p-values for different probability distributions. After having opened the app, set the t-stat, the corresponding alternative and you will find the p-value at the top of the page.

Verification in R

Just for the sake of illustration, here is the verification of the above example in R:

# one-proportion test
test <- prop.test(
  x = 67, # number of heads
  n = 100, # number of trials
  p = 0.5 # expected probability of heads
)

test

## 
##  1-sample proportions test with continuity correction
## 
## data:  67 out of 100, null probability 0.5
## X-squared = 10.89, df = 1, p-value = 0.0009668
## alternative hypothesis: true p is not equal to 0.5
## 95 percent confidence interval:
##  0.5679099 0.7588442
## sample estimates:
##    p 
## 0.67

The p-value is 0.001 so, at the 5% significance level, we reject the null hypothesis that the proportions of heads and tails are equal, and we conclude that the coin is biased. This is the same conclusion than the one found by hand.

Goodness of fit test

We now illustrate the chi-square goodness of fit test by hand with the following example.

Suppose that we toss a dice 100 times, we note how many times it lands on each face (1 to 6) and we test whether the dice is fair. Here are the observed counts by dice face:

## dice_face
##  1  2  3  4  5  6 
## 15 24 10 19 19 13

With a fair dice, we would expect it to land $\frac{100}{6} \approx 16.67$ times on each face (this expected value is represented by the dashed line in the above plot). Although the observed frequencies are different than the expected value of 16.67:

##   dice_face observed_freq expected_freq
## 1         1            15         16.67
## 2         2            24         16.67
## 3         3            10         16.67
## 4         4            19         16.67
## 5         5            19         16.67
## 6         6            13         16.67

we need to test whether they are significantly different. For this, we perform the appropriate hypothesis test following the 4 easy steps mentioned above:

State the null and alternative hypotheses
Compute the test-statistic (also known as t-stat)
Find the rejection region
Conclude by comparing the test-statistic with the rejection region

Step 1.

The null and alternative hypotheses of the chi-square goodness of fit test are:

$H_0$: there is no significant difference between the observed and the expected frequencies
$H_1$: there is a significant difference between the observed and the expected frequencies

Applied to our example, we have:

$H_0$: all faces occur in the same proportion
$H_1$: at least one proportion is not equal to 1/6

Step 2.

The test statistic is:

\[\chi^2 = \sum_{i = 1}^k \frac{(O_i - E_i)^2}{E_i}\]

where $O_i$ is the observed frequency, $E_i$ is the expected frequency and $k$ is the number of categories (in our case, there are 6 categories, representing the 6 dice faces).

This $\chi^2$ statistic is obtained by calculating the difference between the observed number of cases and the expected number of cases in each category. This difference is squared (to avoid negative and positive differences being compensated) and divided by the expected number of cases in that category. These values are then summed for all categories, and the total is referred to as the $\chi^2$ statistic. Large values of this test statistic lead to the rejection of the null hypothesis, small values mean that the null hypothesis cannot be rejected.⁶

Given our data, we have:

\[\chi^2 = \frac{(15 - 16.67)^2}{16.67} + \frac{(24 - 16.67)^2}{16.67} + \\ \frac{(10 - 16.67)^2}{16.67} +\frac{(19 - 16.67)^2}{16.67} + \\ \frac{(19 - 16.67)^2}{16.67} + \frac{(13 - 16.67)^2}{16.67} = 7.52\]

Step 3.

Whether the $\chi^2$ test statistic is small or large depends on the rejection region. The rejection region is found via the $\chi^2$ distribution table. With a degrees of freedom equals to $k - 1$ (where $k$ is the number of categories) and assuming a significance level $\alpha = 0.05$, we have:

\[\chi^2_{\alpha; k-1} = \chi^2_{0.05; 5} = 11.0705\]

Step 4.

We compare the test statistic (found in step 2) with the rejection region (found in step 3) and we conclude. Visually, we have:

The test statistic does not lie within the rejection region (i.e., the grey shaded area). Therefore, at the 5% significance level, we do not reject the null hypothesis that there is no significant difference between the observed and the expected frequencies. In other words, still at the 5% significance level, we cannot reject the hypothesis that the dice is fair.

Again, you can use the Shiny app to easily compute the p-value given the test statistic if you prefer this method over the comparison between the t-stat and the rejection region.

Verification in R

Just for the sake of illustration, here is the verification of the above example in R:

# chi-square goodness of fit test
test <- chisq.test(dat$observed_freq, # observed frequencies for each dice face
  p = rep(1 / 6, 6) # expected probabilities for each dice face
)

test

## 
##  Chi-squared test for given probabilities
## 
## data:  dat$observed_freq
## X-squared = 7.52, df = 5, p-value = 0.1847

The test statistic and degrees of freedom are exactly the same than the ones found by hand. The p-value is 0.185 which, still at the 5% significance level, leads to the same conclusion than by hand (i.e., failing to reject the null hypothesis).

Conclusion

Thanks for reading.

I hope this article helped you to understand and perform the one-proportion and chi-square goodness of fit test in R and by hand. Learn more about the Chi-square test of independence in R and by hand if you want to analyze two categorical variables instead of one.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Choosing big or small as the success event gives the exact same conclusion.↩︎
Note that if possible, it is best to avoid pie charts and use bar charts instead. Unfortunately, the ggbarstats() function works only for the independence Chi-square test.↩︎
Similarly, this argument can also be added to the prop.test() function to test whether the observed proportion is larger than the expected proportion. Use alternative = "less" if you want to test whether the observed proportion is smaller than the expected one.↩︎
Be careful that the alternative hypothesis is not that all proportions are different. One different from the others is sufficient to reject the null hypothesis. It is, in some sense, similar to the alternative hypothesis of the ANOVA which says that at least one mean is different than another.↩︎
One assumption of this test is that $n \cdot p \ge 5$ and $n \cdot (1 - p) \ge 5$. The assumption is met so we can use the normal approximation to the binomial distribution.↩︎
Source: http://uregina.ca/~gingrich/ch10.pdf.↩︎

A package to download free Springer books during Covid-19 quarantine

Sun, 26 Apr 2020 00:00:00 +0000

Update

The promotion has ended so it is not possible to download the books through R. If you did not download the books in time, you can still have access to them via this link.¹

Introduction

You probably already have seen that Springer released about 500 books for free following the COVID-19 pandemic. According to Springer, these textbooks will be available free of charge until at least the end of July.

Following this announcement, I already downloaded a couple of statistics and R programming textbooks from their website and I will probably download a few more in the coming weeks.

In this article, I present a package that saved me a lot of time and which may be of interest to many of us: the {springerQuarantineBooksR} package, developed by Renan Xavier Cortes.

This package allows you to easily download all (or a selection of) Springer books made available free of charge during the COVID-19 quarantine.

With this large collection of high quality resources and my collection of top R resources about the Coronavirus, we do not have any excuse to not read and learn during this quarantine.

In this article, I show:

how to download all available textbooks at once and
how to download a subset of books, given a specific title, author or subject

Without further ado, here is how the package works in practice.

Installation

After having installed the {devtools} package, you can install the {springerQuarantineBooksR} package from GitHub and load it with:

# install.packages("devtools")
devtools::install_github("renanxcortes/springerQuarantineBooksR", force = TRUE)
library(springerQuarantineBooksR)

Download all books at once

First, set the path where you would like to save all books with the setwd() function then download all of them at once with the download_springer_book_files() function. Note that it takes several minutes (depending on the speed of your internet connection) since all books combined amount for almost 8GB.

setwd("path_of_your_choice") # where you want to save the books
download_springer_book_files() # download all of them at once

You will find all downloaded books (in PDF format) in a folder named “springer_quarantine_books”, organized by category.²

If you want to download the EPUB version (or both the PDF and EPUB versions), add the filetype argument to the function:

# for EPUB version:
download_springer_book_files(filetype = "epub")

# for both PDF and EPUB versions:
download_springer_book_files(filetype = "both")

By default, it downloads only the English books. However, it is also possible to download all German books by adding the argument lan = 'ger':

download_springer_book_files(lan = "ger")

Note that in total, there are 407 unique titles in English and 52 in German.

Create a table of Springer books

Like me, if you do not know which books are offered by Springer and you do not want to download all of them, you probably may want to have an overview or a list of the released books before downloading any.

For this, you can load a table containing all the titles made available by Springer into an R session with the download_springer_table() function:

springer_table <- springerQuarantineBooksR::download_springer_table()

This table can then be improved with the {DT} package to:

keep only a minimum of information,
allow searching a book by its title, author, classification or year,
allow downloading the list of available books, and
make the Springer links clickable for instance

# install.packages("DT")
library(DT)

springer_table$open_url <- paste0(
  '<a target="_blank" href="', # opening HTML tag
  springer_table$open_url, # href link
  '">SpringerLink</a>' # closing HTML tag
)

springer_table <- springer_table[, c(1:3, 19, 20)] # keep only relevant information

datatable(springer_table,
  rownames = FALSE, # remove row numbers
  filter = "top", # add filter on top of columns
  extensions = "Buttons", # add download buttons
  options = list(
    autoWidth = TRUE,
    dom = "Blfrtip", # location of the download buttons
    buttons = c("copy", "csv", "excel", "pdf", "print"), # download buttons
    pageLength = 5, # show first 5 entries, default is 10
    order = list(0, "asc") # order the title column by ascending order
  ),
  escape = FALSE # make URLs clickable
)

This table allows you to see which textbooks Springer offers (together with some information) and allows you to find the ones that you are most likely to be interested in.

Note that you can create a similar table for German books with the download_springer_table(lan = "ger") function.

Download only specific books

Now that you have a better idea about the books you are interested in, you can download them by their title, author or subject.

By title

Say that you are interested in downloading only one specific book and you know its title. For instance, suppose you want to download the book entitled “All of Statistics”:

download_springer_book_files(springer_books_titles = "All of Statistics")

If you are interested to download more than one book, run the following command:

download_springer_book_files(
  springer_books_titles = c(
    "All of Statistics",
    "A Modern Introduction to Probability and Statistics"
  )
)

Alternatively, if you do not have a specific title in mind but you are interested in downloading all books with the word “Statistics” in the title, you can run:

springer_table <- download_springer_table()

library(dplyr)
specific_titles_list <- springer_table %>%
  filter(str_detect(
    book_title, # look for a pattern in the book_title column
    "Statistics" # specify the word
  )) %>%
  pull(book_title)

download_springer_book_files(springer_books_titles = specific_titles_list)

Tip: If you would like to download all books with the word “Statistics” or “Data Science” in the title, replace "Statistics" in the above code by "Statistics|Data Science".

By author

If you want to download all books from a specific author, you can run:

springer_table <- download_springer_table()

# library(dplyr)
specific_titles_list <- springer_table %>%
  filter(str_detect(
    author, # look for a pattern in the author column
    "John Hunt" # specify the author
  )) %>%
  pull(book_title)

download_springer_book_files(springer_books_titles = specific_titles_list)

By subject

You can also download all textbooks covering a specific subject (see all subjects in the subject_classification column in the summary table). For instance, here is how to download all books categorized in the Statistics subject:

springer_table <- download_springer_table()

# library(dplyr)
specific_titles_list <- springer_table %>%
  filter(str_detect(
    subject_classification, # look for a pattern in the subject_classification column
    "Statistics" # specify the subject
  )) %>%
  pull(book_title)

download_springer_book_files(springer_books_titles = specific_titles_list)

Improvements

Below a list of features that can potentially be implemented in order to improve the package:

Add the possibility to download all editions of a book. Currently, only the latest edition can be downloaded.
Add the possibility to resume downloading if it stopped. Currently, if the code is executed again, the downloads start from scratch.
Add the possibility of downloading books by topic. Currently, it is only possible by title, author or subject.

Feel free to open a pull request on GitHub if you have another improvement in mind.

Acknowledgments

I would like to thank:

Renan Xavier Cortes (and all contributors) for providing this package
The springer_free_books Python project which was used as inspiration to the {springerQuarantineBooksR} package
And last but not least, Springer who offers many of their excellent books for free!

Conclusion

Thanks for reading.

I hope this article will help you to download and read more high quality materials made available by Springer during this Covid-19 quarantine.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Note that I am not the author nor the maintainer of this shared folder. Therefore, I do not know how long the books will be available through this link, and I am not responsible if some (or all) books are removed.↩︎
Note that you can change the folder name by specifying the argument destination_folder = "name_of_your_choice".↩︎

COVID-19 in Belgium

Tue, 31 Mar 2020 00:00:00 +0000

Introduction

The Novel COVID-19 Coronavirus is still spreading quickly in several countries and it does not seem like it is going to stop anytime soon as the peak has not yet been reached in many countries.

Since the beginning of its expansion, a large number of scientists across the world have been analyzing this Coronavirus from different perspectives and with different technologies with the hope of coming up with a cure in order to stop its expansion and limit its impact on citizens.

Top R resources on Coronavirus

In the meantime, epidemiologists, statisticians and data scientists are working towards a better understanding of the spread of the virus in order to help governments and health agencies in taking the most optimal decisions. This led to the publication of a great deal of online resources about the virus, which I collected and organized in an article covering the top R resources on Coronavirus. This article is a collection of the best resources I’ve had the chance to discover, with a brief summary for each of them. It includes Shiny apps, dashboards, R packages, blog posts and datasets.

Publishing this collection led many readers to submit their piece of work, which made the article even more complete and more insightful for anyone interested in analyzing the virus from a quantitative perspective. Thanks to everyone who contributed and who helped me in collecting and summarizing these R resources about COVID-19!

Given my field of expertise, I am not able to help in this fight against the virus from a medical point of view. However, I still wanted to contribute as much as I could. From understanding better the disease to bringing scientists and doctors together to build something bigger and more impactful, I truly hope that this collection will, to a small extent, help to fight the pandemic.

Coronavirus dashboard for your own country

Besides receiving analyses, blog posts, R code and Shiny apps from people across the world, I realized that many people were trying to create a dashboard tracking the spread of the Coronavirus for their own country. So in addition to the collection of top R resources, I also published an article detailing the steps to follow to create a dashboard specific to a country. See how to create such dashboard in this article and an example with Belgium.

The code has been made available on GitHub and is open source so everyone can copy it and adapt it to their own country. The dashboard was intentionally kept simple so anyone with a minimum knowledge in R could easily replicate it, and advanced users could enhance it according to their needs.

Motivations, limitations and structure of the article

By seeing and organizing many R resources about COVID-19, I am fortunate enough to have read a lot of excellent analyses on the disease outbreak, the impact of different health measures, forecasts of the number of cases, projections about the length of the pandemic, hospitals capacity, etc.

Furthermore, I must admit that some countries such as China, South Korea, Italy, Spain, UK and Germany received a lot of attention as shown by the number of analyses done on these countries. However, to my knowledge and at the date of publication of this article, I am not aware of any analysis of the spread of the Coronavirus specifically for Belgium.¹ The present article aims at filling that gap.

Throughout my PhD thesis in statistics, my main research interest is about survival analysis applied to cancer patients (more information in the research section of my personal website). I am not an epidemiologist and I have no extensive knowledge in modelling disease outbreaks via epidemiological models.

I usually write articles only about things I consider myself familiar with, mainly statistics and its applications in R. At the time of writing this article, I was however curious where Belgium stands regarding the spread of this virus, I wanted to play with this kind of data in R (which is new to me) and see what comes out.

In order to satisfy my curiosity while not being an expert, in this article I am going to replicate analyses done by more knowledgeable people and apply them to my country, that is, Belgium. From all the analyses I have read so far, I decided to replicate the analyses done by Tim Churches and Prof. Dr. Holger K. von Jouanne-Diedrich. This article is based on a mix of their articles which can be found here and here. They both present a very informative analysis on how to model the outbreak of the Coronavirus and show how contagious it is. Their articles also allowed me to gain an understanding of the topic and in particular an understanding of the most common epidemiological model. I strongly advise interested readers to also read their more recent articles for more advanced analyses and for an even deeper understanding of the spread of the COVID-19 pandemic.

Other more complex analyses are possible and even preferable, but I leave this to experts in this field. Note also that the following analyses take into account only the data until the date of publication of this article, so the results should not be viewed, by default, as current findings.

In the remaining of the article, we first introduce the model which will be used to analyze the Coronavirus outbreak in Belgium. We also briefly discuss and show how to compute an important epidemiological measure, the reproduction number. We then use our model to analyze the outbreak of the disease in the case where there would be no public health intervention. We conclude the article by summarizing more advanced tools and techniques that could be used to further model COVID-19 in Belgium.

Analysis of Coronavirus in Belgium

A classic epidemiological model: the SIR model

Before diving into the real-life application, we first introduce the model that will be used.

There are many epidemiological models but we will use one of the most common one, the SIR model. The SIR model can be complexified to incorporate more specificities of the virus outbreak, but in this article we keep its simplest version. Tim Churches’ explanation of this model and how to fit it using R is so nice, I will reproduce it here with a few minor changes.

The basic idea behind the SIR model (Susceptible - Infectious - Recovered) of communicable disease outbreaks is that there are three groups (also called compartments) of individuals:

S: those who are healthy but susceptible to the disease (i.e., at risk of being contaminated). At the start of the pandemic, S is the entire population since no one is immune to the virus.
I: the infectious (and thus, infected) people
R: individuals who were contaminated but who have either recovered or died. They are not infectious anymore.

These groups evolve over time as the virus progresses in the population:

S decreases when individuals are contaminated and move to the infectious group I
As people recover or die, they go from the infected group I to the recovered group R

To model the dynamics of the outbreak we need three differential equations to describe the rates of change in each group, parameterised by:

$\beta$, the infection rate, which controls the transition between S and I
$\gamma$, the removal or recovery rate, which controls the transition between I and R

Formally, this gives:

\[\frac{dS}{dt} = - \frac{\beta IS}{N} \text{ (Eq. 1)}\]

\[\frac{dI}{dt} = \frac{\beta IS}{N} - \gamma I \text{ (Eq. 2)}\]

\[\frac{dR}{dt} = \gamma I \text{ (Eq. 3)}\]

The first equation (Eq. 1) states that the number of susceptible individuals (S) decreases with the number of newly infected individuals, where new infected cases are the result of the infection rate ($\beta$) multiplied by the number of susceptible individuals (S) who had a contact with infectious individuals (I).

The second equation (Eq. 2) states that the number of infectious individuals (I) increases with the newly infected individuals ($\beta I S$), minus the previously infected people who recovered (i.e., $\gamma I$ which is the removal rate $\gamma$ multiplied by the infectious individuals I).

Finally, the last equation (Eq. 3) states that the recovered group (R) increases with the number of individuals who were infectious and who either recovered or died ($\gamma I$).

An epidemic develops as follows:

Before the start of the disease outbreak, S equals the entire population as no one has anti-bodies.
At the beginning of the outbreak, as soon as the first individual is infected, S decreases by 1 and I increases by 1 as well.
This first infectious individual contaminates (before recovering or dying) other individuals who were susceptible.
The dynamic continues, with recently contaminated individuals who in turn infect other susceptible people before they recover.

Visually, we have:

SIR model. Source: Kai Sasaki.

Before fitting the SIR model to the data, the first step is to express these differential equations as an R function, with respect to time t.

SIR <- function(time, state, parameters) {
  par <- as.list(c(state, parameters))
  with(par, {
    dS <- -beta * I * S / N
    dI <- beta * I * S / N - gamma * I
    dR <- gamma * I
    list(c(dS, dI, dR))
  })
}

Fitting a SIR model to the Belgium data

To fit the model to the data we need two things:

a solver for these differential equations
an optimiser to find the optimal values for our two unknown parameters, $\beta$ and $\gamma$

The function ode() (for ordinary differential equations) from the {deSolve} R package makes solving the system of equations easy, and to find the optimal values for the parameters we wish to estimate, we can just use the optim() function built into base R.

Specifically, what we need to do is minimise the sum of the squared differences between $I(t)$, which is the number of people in the infectious compartment $I$ at time $t$, and the corresponding number of cases as predicted by our model $\hat{I}(t)$. This quantity is known as the residual sum of squares (RSS):

\[RSS(\beta, \gamma) = \sum_t \big(I(t) - \hat{I}(t) \big)^2\]

In order to fit a model to the incidence data for Belgium, we need a value N for the initial uninfected population. The population of Belgium in November 2019 was 11,515,793 people, according to Wikipedia. We will thus use N = 11515793 as the initial uninfected population.

Next, we need to create a vector with the daily cumulative incidence for Belgium, from February 4 (when our daily incidence data starts), through to March 30 (last available date at the time of publication of this article). We will then compare the predicted incidence from the SIR model fitted to these data with the actual incidence since February 4. We also need to initialise the values for N, S, I and R. Note that the daily cumulative incidence for Belgium is extracted from the {coronavirus} R package developed by Rami Krispin.

# devtools::install_github("RamiKrispin/coronavirus")
library(coronavirus)
data(coronavirus)

`%>%` <- magrittr::`%>%`

# extract the cumulative incidence
df <- coronavirus %>%
  dplyr::filter(country == "Belgium") %>%
  dplyr::group_by(date, type) %>%
  dplyr::summarise(total = sum(cases, na.rm = TRUE)) %>%
  tidyr::pivot_wider(
    names_from = type,
    values_from = total
  ) %>%
  dplyr::arrange(date) %>%
  dplyr::ungroup() %>%
  dplyr::mutate(active = confirmed - death - recovered) %>%
  dplyr::mutate(
    confirmed_cum = cumsum(confirmed),
    death_cum = cumsum(death),
    recovered_cum = cumsum(recovered),
    active_cum = cumsum(active)
  )

# put the daily cumulative incidence numbers for Belgium from
# Feb 4 to March 30 into a vector called Infected
library(lubridate)

sir_start_date <- "2020-02-04"
sir_end_date <- "2020-03-30"

Infected <- subset(df, date >= ymd(sir_start_date) & date <= ymd(sir_end_date))$active_cum

# Create an incrementing Day vector the same length as our
# cases vector
Day <- 1:(length(Infected))

# now specify initial values for N, S, I and R
N <- 11515793
init <- c(
  S = N - Infected[1],
  I = Infected[1],
  R = 0
)

Note that what is needed are currently infected persons (cumulative infected minus the removed, i.e. recovered or dead). However, numbers of recovered persons are hard to obtain and probably underestimated due to underreporting bias. I thus consider the cumulative number of infected people, which is probably not an issue here since the number of recovered cases is negligible at the time of the analysis.

Then we need to define a function to calculate the RSS, given a set of values for $\beta$ and $\gamma$.

# define a function to calculate the residual sum of squares
# (RSS), passing in parameters beta and gamma that are to be
# optimised for the best fit to the incidence data
RSS <- function(parameters) {
  names(parameters) <- c("beta", "gamma")
  out <- ode(y = init, times = Day, func = SIR, parms = parameters)
  fit <- out[, 3]
  sum((Infected - fit)^2)
}

Finally, we can fit the SIR model to our data by finding the values for $\beta$ and $\gamma$ that minimise the residual sum of squares between the observed cumulative incidence (observed in Belgium) and the predicted cumulative incidence (predicted by our model). We also need to check that our model has converged, as indicated by the message shown below:

# now find the values of beta and gamma that give the
# smallest RSS, which represents the best fit to the data.
# Start with values of 0.5 for each, and constrain them to
# the interval 0 to 1.0

# install.packages("deSolve")
library(deSolve)

Opt <- optim(c(0.5, 0.5),
  RSS,
  method = "L-BFGS-B",
  lower = c(0, 0),
  upper = c(1, 1)
)

# check for convergence
Opt$message

## [1] "CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH"

Convergence is confirmed. Note that you may find different estimates for different choices of initial values or constraints. This proves that the fitting process is not stable. Here is a potential solution for a better fitting process.

Now we can examine the fitted values for $\beta$ and $\gamma$:

Opt_par <- setNames(Opt$par, c("beta", "gamma"))
Opt_par

##      beta     gamma 
## 0.5841185 0.4158816

Remember that $\beta$ controls the transition between S and I (i.e., susceptible and infectious) and $\gamma$ controls the transition between I and R (i.e., infectious and recovered). However, those values do not mean a lot but we use them to get the fitted numbers of people in each compartment of our SIR model for the dates up to March 30 that were used to fit the model, and compare those fitted values with the observed (real) data.

# time in days for predictions
t <- 1:as.integer(ymd(sir_end_date) + 1 - ymd(sir_start_date))

# get the fitted values from our SIR model
fitted_cumulative_incidence <- data.frame(ode(
  y = init, times = t,
  func = SIR, parms = Opt_par
))

# add a Date column and the observed incidence data
library(dplyr)
fitted_cumulative_incidence <- fitted_cumulative_incidence %>%
  mutate(
    Date = ymd(sir_start_date) + days(t - 1),
    Country = "Belgium",
    cumulative_incident_cases = Infected
  )

# plot the data
library(ggplot2)
fitted_cumulative_incidence %>%
  ggplot(aes(x = Date)) +
  geom_line(aes(y = I), colour = "red") +
  geom_point(aes(y = cumulative_incident_cases), colour = "blue") +
  labs(
    y = "Cumulative incidence",
    title = "COVID-19 fitted vs observed cumulative incidence, Belgium",
    subtitle = "(Red = fitted from SIR model, blue = observed)"
  ) +
  theme_minimal()

From the above graph we see that the number of observed confirmed cases follows (unfortunately) the number of confirmed cases expected by our model. The fact that both trends are overlapping indicates that the pandemic is clearly in an exponential phase in Belgium. More data would be needed to see whether this trend is confirmed in the long term.

The following graph is similar than the previous one, except that the y-axis is measured on a log scale. This kind of plot is called a semi-log plot or more precisely a log-linear plot because only the y-axis is transformed with a logarithm scale. Transforming the scale in log has the advantage that it is more easily readable in terms of difference between the observed and expected number of confirmed cases and it also shows how the number of observed confirmed cases differs from an exponential trend.

fitted_cumulative_incidence %>%
  ggplot(aes(x = Date)) +
  geom_line(aes(y = I), colour = "red") +
  geom_point(aes(y = cumulative_incident_cases), colour = "blue") +
  labs(
    y = "Cumulative incidence",
    title = "COVID-19 fitted vs observed cumulative incidence, Belgium",
    subtitle = "(Red = fitted from SIR model, blue = observed)"
  ) +
  theme_minimal() +
  scale_y_log10(labels = scales::comma)

The plot indicates that, at the beginning of the pandemic and until March 12, the number of confirmed cases stayed below what would be expected in an exponential phase. In particular, the number of confirmed cases stayed constant at 1 case from February 4 to February 29. From March 13 and until March 30, the number of confirmed cases kept increasing at a rate close to an exponential rate.

We also notice a small jump between March 12 and March 13, which may potentially indicate an error in the data collection, or a change in the testing/screening methods.

Reproduction number $R_0$

Our SIR model looks like a good fit to the observed cumulative incidence data in Belgium, so we can now use our fitted model to calculate the basic reproduction number $R_0$, also referred as basic reproduction ratio, and which is closely linked to $\beta$ and $\gamma$.²

The basic reproduction number $R_0$ gives the average number of susceptible people who are infected by each infectious person where all individuals are susceptible to infection. In other words, the reproduction number refers to the number of healthy people that get infected per number of sick people. When $R_0 > 1$ the disease starts spreading in a population, but not if $R_0 < 1$. Usually, the larger the value of $R_0$, the harder it is to control the epidemic and the higher the probability of a pandemic.

Formally, we have:

\[R_0 = \frac{\beta}{\gamma}\]

We can compute it in R:

Opt_par

##      beta     gamma 
## 0.5841185 0.4158816

R0 <- as.numeric(Opt_par[1] / Opt_par[2])
R0

## [1] 1.404531

An $R_0$ of 1.4 is below values found by others for COVID-19 and the $R_0$ for SARS and MERS, which are similar diseases also caused by coronavirus. Furthermore, in the literature, it has been estimated that the reproduction number for COVID-19 is approximately 2.7 (with $\beta$ close to 0.54 and $\gamma$ close to 0.2). Our reproduction number being lower is mainly due to the fact that the number of confirmed cases stayed constant and equal to 1 at the beginning of the pandemic.

A $R_0$ of 1.4 means that, on average in Belgium, 1.4 persons are infected for each infected person.

For simple models, the proportion of the population that needs to be effectively immunized to prevent sustained spread of the disease, known as the “herd immunity threshold”, has to be larger than $1 - \frac{1}{R_0}$ (Fine, Eames, and Heymann 2011).

The reproduction number of 1.4 we just calculated suggests that, given the formula 1 - (1 / 1.4), 28.8% of the population should be immunized to stop the spread of the infection. With a population in Belgium of approximately 11.5 million, this translates into roughly 3.3 million people.

Using our model to analyze the outbreak if there was no intervention

It is instructive to use our model fitted to the first 56 days of available data on confirmed cases in Belgium, to see what would happen if the outbreak were left to run its course, without public health intervention.

# time in days for predictions
t <- 1:120

# get the fitted values from our SIR model
fitted_cumulative_incidence <- data.frame(ode(
  y = init, times = t,
  func = SIR, parms = Opt_par
))

# add a Date column and join the observed incidence data
fitted_cumulative_incidence <- fitted_cumulative_incidence %>%
  mutate(
    Date = ymd(sir_start_date) + days(t - 1),
    Country = "Belgium",
    cumulative_incident_cases = c(Infected, rep(NA, length(t) - length(Infected)))
  )

# plot the data
fitted_cumulative_incidence %>%
  ggplot(aes(x = Date)) +
  geom_line(aes(y = I), colour = "red") +
  geom_line(aes(y = S), colour = "black") +
  geom_line(aes(y = R), colour = "green") +
  geom_point(aes(y = cumulative_incident_cases),
    colour = "blue"
  ) +
  scale_y_continuous(labels = scales::comma) +
  labs(y = "Persons", title = "COVID-19 fitted vs observed cumulative incidence, Belgium") +
  scale_colour_manual(name = "", values = c(
    red = "red", black = "black",
    green = "green", blue = "blue"
  ), labels = c(
    "Susceptible",
    "Recovered", "Observed", "Infectious"
  )) +
  theme_minimal()

The same graph in log scale for the y-axis and with a legend for better readability:

# plot the data
fitted_cumulative_incidence %>%
  ggplot(aes(x = Date)) +
  geom_line(aes(y = I, colour = "red")) +
  geom_line(aes(y = S, colour = "black")) +
  geom_line(aes(y = R, colour = "green")) +
  geom_point(aes(y = cumulative_incident_cases, colour = "blue")) +
  scale_y_log10(labels = scales::comma) +
  labs(
    y = "Persons",
    title = "COVID-19 fitted vs observed cumulative incidence, Belgium"
  ) +
  scale_colour_manual(
    name = "",
    values = c(red = "red", black = "black", green = "green", blue = "blue"),
    labels = c("Susceptible", "Observed", "Recovered", "Infectious")
  ) +
  theme_minimal()

More summary statistics

Other interesting statistics can be computed from the fit of our model. For example:

the peak of the pandemic
the number of severe cases
the number of people in need of intensive care
the number of deaths

fit <- fitted_cumulative_incidence

# peak of pandemic
fit[fit$I == max(fit$I), c("Date", "I")]

##          Date        I
## 89 2020-05-02 531000.4

# severe cases
max_infected <- max(fit$I)
max_infected * 0.2

## [1] 106200.1

# cases with need for intensive care
max_infected * 0.06

## [1] 31860.03

# deaths with supposed 4.5% fatality rate
max_infected * 0.045

## [1] 23895.02

Given these predictions, with the exact same settings and no intervention at all to limit the spread of the pandemic, the peak in Belgium is expected to be reached by the beginning of May. About 530,000 people would be infected by then, which translates to about 106,000 severe cases, about 32,000 persons in need of intensive care (given that there are about 2000 intensive care units in Belgium, the health sector would be completely overwhelmed) and up to 24,000 deaths (assuming a 4.5% fatality rate, as suggested by this source).

At this point, we understand why such strict containment measures and regulations are taken in Belgium!

Note that those predictions should be taken with a lot of caution. On the one hand, as mentioned above, they are based on rather unrealistic assumptions (for example, no public health interventions, fixed reproduction number $R_0$, etc.). More advanced projections are possible with the {projections} package, among others (see this section for more information on this matter). On the other hand, we still have to be careful and strictly follow public health interventions because previous pandemics such as the Spanish and swine flu have shown that incredibly high numbers are not impossible!

The purpose of this article was to give an illustration of how such analyses are done in R with a simple epidemiological model. Those are the numbers our simple model produces and we hope they are wrong because the cost in terms of lives would be enormous.

Additional considerations

As previously mentioned, the SIR model and the analyses done above are rather simplistic and may not give a true representation of the reality. In the following sections, we highlight five improvements that could be done to enhance theses analyses and lead to a better overview of the spread of the Coronavirus in Belgium.

Ascertainment rates

In the previous analyses and graphs, it is assumed that the number of confirmed cases represent all the cases that are infectious. This is far from reality as only a proportion of all cases are screened, detected and counted in the official figures. This proportion is known as the ascertainment rate.

The ascertainment rate is likely to vary during the course of an outbreak, in particular if testing and screening efforts are increased, or if detections methods are changed. Such changing ascertainment rates can be easily incorporated into the model by using a weighting function for the incidence cases.

In his first article, Tim Churches demonstrates that a fixed ascertainment rates of 20% makes little difference to the modelled outbreak with no intervention, except that it all happens a bit more quickly.

More sophisticated models

More sophisticated models could also be used to better reflect real-life transmission processes. For instance, another classical model in disease outbreak is the SEIR model. This extended model is similar to the SIR model, where S stands for Susceptible and R stands for Recovered, but the infected people are divided into two compartments:

E for the Exposed/infected but asymptomatic
I for the Infected and symptomatic

These models belong to the continuous-time dynamic models that assume fixed transition rates. There are other stochastic models that allow for varying transition rates depending on attributes of individuals, social networking, etc.

Modelling the epidemic trajectory using log-linear models

As noted above, the initial exponential phase of an outbreak, when shown in a log-linear plot (the y-axis on a log scale and the x-axis without transformation), appears (somewhat) linear. This suggests that we can model epidemic growth, and decay, using a simple log-linear model of the form:

\[log(y)=rt+b\]

where y is the incidence, r is the growth rate, t is the number of days since a specific point in time (typically the start of the outbreak), and b is the intercept. In this context, two log-linear models:

one to the growth phase (before the peak), and
one to the decay phase (after the peak)

are fitted to the epidemic (incidence cases) curve.

The doubling and halving time estimates which you very often hear in the news can be estimated from these log-linear models. Furthermore, these log-linear models can also be used on the epidemic trajectory to estimate the reproduction number $R_0$ in the growth and decay phases of the epidemic.

The {incidence} package in R, part of the R Epidemics Consortium (RECON) suite of packages for epidemic modelling and control, makes the fitting of this kind of models very convenient.

Estimating changes in the effective reproduction number $R_e$

In our model, we set a reproduction number $R_0$ and kept it constant. It would nonetheless be useful to estimate the current effective reproduction number $R_e$ on a day-by-day basis so as to track the effectiveness of public health interventions, and possibly predict when an incidence curve will start to decrease.

The {EpiEstim} package in R can be used to estimate $R_e$ and allow to take into consideration human travel from other geographical regions in addition to local transmission (Cori et al. 2013; Thompson et al. 2019).

More sophisticated projections

In addition to naïve predictions based on a simple SIR model, more advanced and complex projections are also possible, notably, with the {projections} package. This packages uses data on daily incidence, the serial interval and the reproduction number to simulate plausible epidemic trajectories and project future incidence.

Conclusion

This article started with (i) a description of a couple of R resources on the Coronavirus pandemic (i.e., a collection and a dashboard) that can be used as background materials and (ii) the motivations behind this article. We then detailed the most common epidemiological model, i.e. the SIR model, before actually applying it on Belgium incidence data.

This resulted in a visual comparison of the fitted and observed cumulative incidence in Belgium. It showed that the COVID-19 pandemic is clearly in an exponential phase in Belgium in terms of number of confirmed cases.

We then explained what is the reproduction number and how to compute it in R. Finally, our model was used to analyze the outbreak of the Coronavirus if there was no public health intervention at all.

Under this (probably too) simplistic scenario, the peak of the COVID-19 in Belgium is expected to be reached by the beginning of May, 2020, with around 530,000 infected people and about 24,000 deaths. These very alarmist naïve predictions highlight the importance of restrictive public health actions taken by governments, and the urgency for citizens to follow these health actions in order to mitigate the spread of the virus in Belgium (or at least slow it enough to allow health care systems to cope with it).

We concluded this article by describing five improvements that could be implemented to further analyze the disease outbreak.

Note that this article has been subject to a talk at UCLouvain.

Thanks for reading. I hope this article gave you a good understanding of the spread of the COVID-19 Coronavirus in Belgium. Feel free to use this article as a starting point for analyzing the outbreak of this disease in your own country.

For the interested readers, see also:

the evolution of hospital admissions and number of confirmed cases in Belgium
a collection of top R resources on Coronavirus to gain even further knowledge

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Fine, Paul, Ken Eames, and David L Heymann. 2011. “"Herd Immunity": A Rough Guide.” Clinical Infectious Diseases 52 (7): 911–16.

Thompson, RN, JE Stockwin, RD van Gaalen, JA Polonsky, ZN Kamvar, PA Demarsh, E Dahlqwist, et al. 2019. “Improved Inference of Time-Varying Reproduction Numbers During Infectious Disease Outbreaks.” Epidemics 29: 100356.

Feel free to let me know in the comments or by contacting me if you performed some analyses specifically for Belgium and which I could include in my article covering the top R resources on the Coronavirus.↩︎
See a more detailed note on the reproduction number by James Holland Jones if you need a deeper understanding.↩︎

How to create a simple Coronavirus dashboard specific to your country in R?

Mon, 23 Mar 2020 00:00:00 +0000

Coronavirus dashboard: the case of Belgium

Introduction

The Novel COVID-19 Coronavirus is the hottest topic right now. Every day, the media and newspapers share the number of new cases and deaths in several countries, try to measure the impacts of the virus on citizens and remind us to stay home in order to stay safe. The Coronavirus is on everyone’s lips.

In addition to governments, media and companies discussing about it, data scientists and data professionals in general are putting their knowledge and time at the service of the virus. This leads to a proliferation of applications, dashboards, blog posts, videos, datasets and code analyzing, in one way or another, the expansion of the COVID-19 and how it spreads in the population.

Top R resources on Coronavirus

As a data lover myself, I discovered a multitude of great resources about the Coronavirus. However, these resources were spread all over the internet and were often hidden by the mass of information of another type (e.g., alarming headlines, names of infected celebrities, companies showing off how they helped health care agencies, etc.). To tackle this issue, I collected and then shared in a previous article the best R resources on the Coronavirus I came across.

Note that only resources on R are shared in this article as R is my favorite statistical program and the one I am most familiar with. The fact that I use this program almost daily makes it easier for me to realize the complexity and time put behind a resource, and appreciate its quality and its potential.

I am sure that there are other very interesting resources online (see for example the probably most popular dashboard by the Johns Hopkins Coronavirus Resource Center).¹ Nonetheless, a lot of people are in a better position than I am to judge the quality of resources made with programming languages for which I am not competent.

This article made me discover so many great resources about the Coronavirus and I keep receiving data visualizations and data analyses from scientists all over the world so that I include them in the collection. Thanks for that, it continuously improves the quality and completeness of the collection.

In addition to receiving R resources, a question often asked by readers was “How can I create a dashboard myself?” or “How can I build a dashboard specific to my country?”. I therefore thought it would serve some people if I created a dashboard specific to my country (Belgium) and detailed the steps on how to build it.

Questions on how to develop such dashboards came mostly from R beginners as advanced R users most probably know how to do one, or at least can easily use the resources I collected here as sources of inspiration for their own work. Furthermore, in response to the craze about the Coronavirus, interested users were quite in a hurry and wanted to have their own dashboard running as quickly as possible.

These questions led me to the idea of creating a simple (yet powerful and visually appealing) dashboard, as opposed to a Shiny app. Shiny apps have the advantage that they are interactive in the sense that users can edit the outputs and the visualizations by simply changing some inputs in a user-friendly way, while dashboards are static and cannot be modified by the final user. On the other hand, the advantage of a dashboard over a Shiny app is that it is much easier to code, especially if you are already proficient in R Markdown.

Coronavirus dashboard: the case of Belgium

Amongst all the visualizations I have seen so far, one is standing out by its simplicity and at the same time, by its completeness and by the quality of its visualizations. I thus decided to create a Coronavirus dashboard based on this already existing dashboard by Rami Krispin (which comes with a license that allows to be freely adapted and shared) and adapt it so that it is specific to Belgium. Note that I also removed some visualizations and tables from the initial dashboard to keep it really simple and straight to the point.

Before reading further, here is my Coronavirus dashboard adapted to Belgium and previews of the main sections below:

The dashboard is segmented into several sections that can be selected at the top:

The summary section provides key measures about the Coronavirus (total cases, active cases and deaths) and a plot displaying the cumulative number of active cases and deaths from January 22, 2020 to the latest available date.
The comparison section presents a comparison of the number of daily new cases (left panel) and the distribution of cases by type (right panel) with other European countries (you can also change these countries by replacing them in the code).
The map section shows a world map of the confirmed cases and deaths. You can uncheck one or several types of cases (top right corner) and zoom in or out (top left corner) to adapt the map to your needs.
The about section gives more information about the data, the dashboard in general and how often it is updated.

I believe this simple dashboard is easy enough to be adapted to any country (and by anyone from beginner to expert), and still communicate key measures about the virus throughout some visualizations.

A little extra which is worth mentioning is the fact that all plots are generated with the {plotly} package. This package allows to enhance plots by displaying additional relevant information when hovering over them (try by yourself!).

How to create your own Coronavirus dashboard

If you want to build your own dashboard specific to a country, follow these steps:

Open the dashboard here
See the entire code via the button “Source code” located in the top right corner of the dashboard, or see the code on GitHub. Copy that code.
Open a new R Markdown file (.Rmd), type any title and author (they will be replaced in the next step anyway), select HTML as the output format and click on OK:

Remove all the template code already present and paste the code you copied in step 1.
Make sure that the required packages are installed:

install.packages(c("devtools", "flexdashboard", "leaflet", "leafpop"))
devtools::install_github("RamiKrispin/coronavirus", force = TRUE)

If R asks you which package you would like to update, there should be no need to update them: type 3 for “None”.

In the code, replace Belgium with your country. Here is the list of all available countries in the dataset:

Canada, United Kingdom, China, Netherlands, Australia, New Zealand, Denmark, France, Afghanistan, Albania, Algeria, Andorra, Angola, Antarctica, Antigua and Barbuda, Argentina, Armenia, Austria, Azerbaijan, Bahamas, Bahrain, Bangladesh, Barbados, Belarus, Belgium, Belize, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Botswana, Brazil, Brunei, Bulgaria, Burkina Faso, Burma, Burundi, Cabo Verde, Cambodia, Cameroon, Central African Republic, Chad, Chile, Colombia, Comoros, Congo (Brazzaville), Congo (Kinshasa), Costa Rica, Cote d’Ivoire, Croatia, Cuba, Cyprus, Czechia, Diamond Princess, Djibouti, Dominica, Dominican Republic, Ecuador, Egypt, El Salvador, Equatorial Guinea, Eritrea, Estonia, Eswatini, Ethiopia, Fiji, Finland, Gabon, Gambia, Georgia, Germany, Ghana, Greece, Grenada, Guatemala, Guinea, Guinea-Bissau, Guyana, Haiti, Holy See, Honduras, Hungary, Iceland, India, Indonesia, Iran, Iraq, Ireland, Israel, Italy, Jamaica, Japan, Jordan, Kazakhstan, Kenya, Kiribati, Korea, North, Korea, South, Kosovo, Kuwait, Kyrgyzstan, Laos, Latvia, Lebanon, Lesotho, Liberia, Libya, Liechtenstein, Lithuania, Luxembourg, Madagascar, Malawi, Malaysia, Maldives, Mali, Malta, Marshall Islands, Mauritania, Mauritius, Mexico, Micronesia, Moldova, Monaco, Mongolia, Montenegro, Morocco, Mozambique, MS Zaandam, Namibia, Nepal, Nicaragua, Niger, Nigeria, North Macedonia, Norway, Oman, Pakistan, Palau, Panama, Papua New Guinea, Paraguay, Peru, Philippines, Poland, Portugal, Qatar, Romania, Russia, Rwanda, Saint Kitts and Nevis, Saint Lucia, Saint Vincent and the Grenadines, Samoa, San Marino, Sao Tome and Principe, Saudi Arabia, Senegal, Serbia, Seychelles, Sierra Leone, Singapore, Slovakia, Slovenia, Solomon Islands, Somalia, South Africa, South Sudan, Spain, Sri Lanka, Sudan, Summer Olympics 2020, Suriname, Sweden, Switzerland, Syria, _Taiwan*_, Tajikistan, Tanzania, Thailand, Timor-Leste, Togo, Tonga, Trinidad and Tobago, Tunisia, Turkey, Uganda, Ukraine, United Arab Emirates, Uruguay, US, Uzbekistan, Vanuatu, Venezuela, Vietnam, West Bank and Gaza, Winter Olympics 2022, Yemen, Zambia and Zimbabwe

Note that if your country is spelled in two words or more, you will need to surround it by a backtick (but only at one specific line in the code, see an example with United Kingdom):

#----------------------------------------
# Plotting the data

daily_confirmed %>%
  plotly::plot_ly() %>%
  plotly::add_trace(
    x = ~date,
    y = ~`United Kingdom`,
    type = "scatter",
    mode = "lines+markers",
    name = "United Kingdom"
  ) %>%

Do not add backticks in the rest of the code as everywhere else the country’s name is surrounded by double quotes "".

Do not forget to also change the title and the author at the top of the document, and edit the about section at the bottom of the document. Last but not least, as you can see on the plot in the summary section, the arrows point to different (sad) “milestones” in Belgium (i.e., first case, first death and new containment measures).

You will need to adapt these milestones for your country (or remove them if you do not want to have any milestone displayed on the plot). Change this in the code following the plotly::add_annotations() functions.

Knit the document (see this article if you are unfamiliar with R Markdown). Your dashboard should appear in HTML format.

Following these 7 steps, you should already have a simple dashboard specific to your country. I have intentionally kept it simple so that everyone could copy it and have their own dashboard in a limited amount of time.

If you are familiar with the {flexdashboard}, {plotly} and {leaflet} packages for the dashboard interface and the visualizations, and the {dplyr} and {tidyr} packages for the data manipulation, feel free to edit the code according to your needs and improve your dashboard.

Additional notes

Data

The input data for this dashboard is the dataset available from the {coronavirus} R package. Make sure to download the development version of the package to have the latest data:

install.packages("devtools")
devtools::install_github("RamiKrispin/coronavirus")

To update your dashboard with the latest data, you have to manually update the data by reinstalling the {coronavirus} package with devtools::install_github("RamiKrispin/coronavirus", force = TRUE). Again, if R asks you whether you would like to update other packages, type 3 for “None”.

This question was often raised so I repeat, your dashboard will not update by itself every day, you need to manually update it. After updating the data, you may also need to restart your R session in order to have the last available data.

The raw data is pulled from the Johns Hopkins University Center for Systems Science and Engineering (JHU CCSE) Coronavirus repository.

Open source

This dashboard and the code available on GitHub are open source so feel free to copy it, adapt it and share it as much as you want.

Accuracy

Please note that this dashboard has been built mainly for educational purposes. I update the dashboard as often as possible to keep it accurate. However, there is some uncertainty concerning the COVID-19 case numbers and the testing methods vary between countries so the figures on this dashboard may be slightly different compared to other sources. Currently, the maintainer of the dataset updates it on a daily basis, but updates may become less frequent in the future.

Publish your dashboard

If you want to share your dashboard, you can either:

Upload it on your website if you have one (and I strongly suggest you to create one if you do not already have one)
Publish it through RPubs (it is free and easy to publish your work directly from RStudio)

Conclusion

Thanks for reading.

I hope this article helped you to build your first Coronavirus dashboard in R. See these top R resources on Coronavirus if you need inspiration to enhance further your dashboard.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

I would be glad to mention a collection of Python resources if someone is willing to create a collection of resources about the Coronavirus made with this programming language. Feel free to contact me if this is the case.↩︎

How to do a t-test or ANOVA for more than one variable at once in R?

Thu, 19 Mar 2020 00:00:00 +0000

Introduction

As part of my teaching assistant position in a Belgian university, students often ask me for some help in their statistical analyses for their master’s thesis.

A frequent question is how to compare groups of patients in terms of several quantitative continuous variables. Most of us know that:

To compare two groups, a Student’s t-test should be used¹
To compare three groups or more, an ANOVA should be performed

These two tests are quite basic and have been extensively documented online and in statistical textbooks so the difficulty is not in how to perform these tests.

In the past, I used to do the analyses by following these 3 steps:

Draw boxplots illustrating the distributions by group (with the boxplot() function or thanks to the {esquisse} R Studio addin if I wanted to use the {ggplot2} package)
Perform a t-test or an ANOVA depending on the number of groups to compare (with the t.test() and oneway.test() functions for t-test and ANOVA, respectively)
Repeat steps 1 and 2 for each variable

This was feasible as long as there were only a couple of variables to test. Nonetheless, most students came to me asking to perform these kind of tests not on one or two variables, but on multiples variables. So when there were more than one variable to test, I quickly realized that I was wasting my time and that there must be a more efficient way to do the job.

Note: you must be very careful with the issue of multiple testing (also referred as multiplicity) which can arise when you perform multiple tests. In short, when a large number of statistical tests are performed, some will have $p$-values less than 0.05 purely by chance, even if all null hypotheses are in fact really true. This is known as multiplicity or multiple testing. You can tackle this problem by using the Bonferroni correction, among others. The Bonferroni correction is a simple method that allows many t-tests to be made while still assuring an overall confidence level is maintained. For this, instead of using the standard threshold of $\alpha = 5$% for the significance level, you can use $\alpha = \frac{0.05}{m}$ where $m$ is the number of t-tests. For example, if you perform 20 t-tests with a desired $\alpha = 0.05$, the Bonferroni correction implies that you would reject the null hypothesis for each individual test when the $p$-value is smaller than $\alpha = \frac{0.05}{20} = 0.0025$.

Note also that there is no universally accepted approach for dealing with the problem of multiple comparisons. Usually, you should choose a p-value adjustment measure familiar to your audience or in your field of study. The Bonferroni correction is easy to implement. It is however not appropriate if you have a very large number of tests to perform (imagine you want to do 10,000 t-tests, a p-value would have to be less than $\frac{0.05}{10000} = 0.000005$ to be significant). A more powerful method is also to adjust the false discovery rate using the Benjamini-Hochberg or Holm procedure (McDonald 2014).

Another option is to use a multivariate ANOVA (MANOVA), if your independent variable has more than two levels. This is particularly useful when your dependent variables are correlated. Correlation between the dependent variables provides MANOVA the following advantages:

Identify patterns between several dependent variables: The independent variables can influence the relationship between dependent variables instead of influencing a single dependent variable.
Address the issue of multiple testing: with MANOVA, the error rate equals the significance level (with no p-value adjustment method needed).
Greater statistical power: When the dependent variables are correlated, MANOVA can identify effects that are too small for the ANOVA to detect.

Note that MANOVA is used if your independent variable has more than two levels. If your independent variable has only two levels, the multivariate equivalent of the t-test is Hotelling’s $T^2$.

This article aims at presenting a way to perform multiple t-tests and ANOVA from a technical point of view (how to implement it in R). Discussion on which adjustment method to use or whether there is a more appropriate model to fit the data is beyond the scope of this article (so be sure to understand the implications of using the code below for your own analyses). Make sure also to test the assumptions of the ANOVA before interpreting results.

Perform multiple tests at once

I thus wrote a piece of code that automated the process, by drawing boxplots and performing the tests on several variables at once. Below is the code I used, illustrating the process with the iris dataset. The Species variable has 3 levels, so let’s remove one, and then draw a boxplot and apply a t-test on all 4 continuous variables at once. Note that the continuous variables that we would like to test are variables 1 to 4 in the iris dataset.

dat <- iris

# remove one level to have only two groups
dat <- subset(dat, Species != "setosa")
dat$Species <- factor(dat$Species)

# boxplots and t-tests for the 4 variables at once
for (i in 1:4) { # variables to compare are variables 1 to 4
  boxplot(dat[, i] ~ dat$Species, # draw boxplots by group
    ylab = names(dat[i]), # rename y-axis with variable's name
    xlab = "Species"
  )
  print(t.test(dat[, i] ~ dat$Species)) # print results of t-test
}

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -5.6292, df = 94.025, p-value = 1.866e-07
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.8819731 -0.4220269
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    5.936                    6.588

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -3.2058, df = 97.927, p-value = 0.001819
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.33028364 -0.07771636
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    2.770                    2.974

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -12.604, df = 95.57, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -1.49549 -1.08851
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    4.260                    5.552

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -14.625, df = 89.043, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.7951002 -0.6048998
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    1.326                    2.026

As you can see, the above piece of code draws a boxplot and then prints results of the test for each continuous variable, all at once.

At some point in the past, I even wrote code to:

draw a boxplot
test for the equality of variances (thanks to the Levene’s test)
depending on whether the variances were equal or unequal, the appropriate test was applied: the Welch test if the variances were unequal and the Student’s t-test in the case the variances were equal (see more details about the different versions of the t-test for two samples)
apply steps 1 to 3 for all continuous variables at once

I had a similar code for ANOVA in case I needed to compare more than two groups.

The code was doing the job relatively well. Indeed, thanks to this code I was able to test several variables in an automated way in the sense that it compared groups for all variables at once.

The only thing I had to change from one project to another is that I needed to modify the name of the grouping variable and the numbering of the continuous variables to test (Species and 1:4 in the above code).

Concise and easily interpretable results

T-test

Although it was working quite well and applicable to different projects with only minor changes, I was still unsatisfied with another point.

Someone who is proficient in statistics and R can read and interpret the output of a t-test without any difficulty. However, as you may have noticed with your own statistical projects, most people do not know what to look for in the results and are sometimes a bit confused when they see so many graphs, code, output, results and numeric values in a document. They are quite easily overwhelmed by this mass of information and unable to extract the key message.

With my old R routine, the time I was saving by automating the process of t-tests and ANOVA was (partially) lost when I had to explain R outputs to my students so that they could interpret the results correctly. Although most of the time it simply boiled down to pointing out what to look for in the outputs (i.e., p-values), I was still losing quite a lot of time because these outputs were, in my opinion, too detailed for most real-life applications and for students in introductory classes. In other words, too much information seemed to be confusing for many people so I was still not convinced that it was the most optimal way to share statistical results to nonscientists.

Of course, they came to me for statistical advices, so they expected to have these results and I needed to give them answers to their questions and hypotheses. Nonetheless, I wanted to find a better way to communicate these results to this type of audience, with the minimum of information required to arrive at a conclusion. No more and no less than that.

After a long time spent online trying to figure out a way to present results in a more concise and readable way, I discovered the {ggpubr} package. This package allows to indicate the test used and the p-value of the test directly on a ggplot2-based graph. It also facilitates the creation of publication-ready plots for non-advanced statistical audiences.

After many refinements and modifications of the initial code (available in this article), I finally came up with a rather stable and robust process to perform t-tests and ANOVA for more than one variable at once, and more importantly, make the results concise and easily readable by anyone (statisticians or not).

A graph is worth a thousand words, so here are the exact same tests than in the previous section, but this time with my new R routine:

library(ggpubr)

# Edit from here #
x <- which(names(dat) == "Species") # name of grouping variable
y <- which(names(dat) == "Sepal.Length" # names of variables to test
| names(dat) == "Sepal.Width" |
  names(dat) == "Petal.Length" |
  names(dat) == "Petal.Width")
method <- "t.test" # one of "wilcox.test" or "t.test"
paired <- FALSE # if paired make sure that in the dataframe you have first all individuals at T1, then all individuals again at T2
# Edit until here


# Edit at your own risk
for (i in y) {
  for (j in x) {
    if (paired == TRUE) {
      p <- ggpaired(dat,
        x = colnames(dat[j]), y = colnames(dat[i]),
        color = colnames(dat[j]), line.color = "gray", line.size = 0.4,
        palette = "npg",
        legend = "none",
        xlab = colnames(dat[j]),
        ylab = colnames(dat[i]),
        add = "jitter"
      )
    } else {
      p <- ggboxplot(dat,
        x = colnames(dat[j]), y = colnames(dat[i]),
        color = colnames(dat[j]),
        palette = "npg",
        legend = "none",
        add = "jitter"
      )
    }
    #  Add p-value
    print(p + stat_compare_means(aes(label = paste0(after_stat(method), ", p-value = ", after_stat(p.format))),
      method = method,
      paired = paired,
      # group.by = NULL,
      ref.group = NULL
    ))
  }
}

As you can see from the graphs above, only the most important information is presented for each variable:

a visual comparison of the groups thanks to boxplots
the name of the statistical test
the p-value of the test

Of course, experts may be interested in more advanced results. However, this simple yet complete graph, which includes the name of the test and the p-value, gives all the necessary information to answer the question: “Are the groups different?”.

In my experience, I have noticed that students and professionals (especially those from a less scientific background) understand way better these results than the ones presented in the previous section.

The only lines of code that need to be modified for your own project is the name of the grouping variable (Species in the above code), the names of the variables you want to test (Sepal.Length, Sepal.Width, etc.),² whether you want to apply a t-test (t.test) or Wilcoxon test (wilcox.test) and whether the samples are paired or not (FALSE if samples are independent, TRUE if they are paired).

Based on these graphs, it is easy, even for non-experts, to interpret the results and conclude that the versicolor and virginica species are significantly different in terms of all 4 variables (since all p-values $< \frac{0.05}{4} = 0.0125$ (remind that the Bonferroni correction is applied to avoid the issue of multiple testing, so we divide the usual $\alpha$ level by 4 because there are 4 t-tests)).

Additional p-value adjustment methods

If you would like to use another p-value adjustment method, you can use the p.adjust() function. Below are the raw p-values found above, together with p-values derived from the main adjustment methods (presented in a dataframe):

raw_pvalue <- numeric(length = length(1:4))
for (i in (1:4)) {
  raw_pvalue[i] <- t.test(dat[, i] ~ dat$Species,
    alternative = "two.sided"
  )$p.value
}

df <- data.frame(
  Variable = names(dat[, 1:4]),
  raw_pvalue = round(raw_pvalue, 3)
)

df$Bonferroni <-
  p.adjust(df$raw_pvalue,
    method = "bonferroni"
  )
df$BH <-
  p.adjust(df$raw_pvalue,
    method = "BH"
  )
df$Holm <-
  p.adjust(df$raw_pvalue,
    method = "holm"
  )
df$Hochberg <-
  p.adjust(df$raw_pvalue,
    method = "hochberg"
  )
df$Hommel <-
  p.adjust(df$raw_pvalue,
    method = "hommel"
  )
df$BY <-
  round(p.adjust(df$raw_pvalue,
    method = "BY"
  ), 3)
df

##       Variable raw_pvalue Bonferroni    BH  Holm Hochberg Hommel    BY
## 1 Sepal.Length      0.000      0.000 0.000 0.000    0.000  0.000 0.000
## 2  Sepal.Width      0.002      0.008 0.002 0.002    0.002  0.002 0.004
## 3 Petal.Length      0.000      0.000 0.000 0.000    0.000  0.000 0.000
## 4  Petal.Width      0.000      0.000 0.000 0.000    0.000  0.000 0.000

Regardless of the p-value adjustment method, the two species are different for all 4 variables. Note that the adjustment method should be chosen before looking at the results to avoid choosing the method based on the results.

Below another function that allows to perform multiple Student’s t-tests or Wilcoxon tests at once and choose the p-value adjustment method. The function also allows to specify whether samples are paired or unpaired and whether the variances are assumed to be equal or not. (The code has been adapted from Mark White’s article.)

t_table <- function(data, dvs, iv,
                    var_equal = TRUE,
                    p_adj = "none",
                    alpha = 0.05,
                    paired = FALSE,
                    wilcoxon = FALSE) {
  if (!inherits(data, "data.frame")) {
    stop("data must be a data.frame")
  }

  if (!all(c(dvs, iv) %in% names(data))) {
    stop("at least one column given in dvs and iv are not in the data")
  }

  if (!all(sapply(data[, dvs], is.numeric))) {
    stop("all dvs must be numeric")
  }

  if (length(unique(na.omit(data[[iv]]))) != 2) {
    stop("independent variable must only have two unique values")
  }

  out <- lapply(dvs, function(x) {
    if (paired == FALSE & wilcoxon == FALSE) {
      tres <- t.test(data[[x]] ~ data[[iv]], var.equal = var_equal)
    } else if (paired == FALSE & wilcoxon == TRUE) {
      tres <- wilcox.test(data[[x]] ~ data[[iv]])
    } else if (paired == TRUE & wilcoxon == FALSE) {
      tres <- t.test(data[[x]] ~ data[[iv]],
        var.equal = var_equal,
        paired = TRUE
      )
    } else {
      tres <- wilcox.test(data[[x]] ~ data[[iv]],
        paired = TRUE
      )
    }

    c(
      p_value = tres$p.value
    )
  })

  out <- as.data.frame(do.call(rbind, out))
  out <- cbind(variable = dvs, out)
  names(out) <- gsub("[^0-9A-Za-z_]", "", names(out))

  out$p_value <- p.adjust(out$p_value, p_adj)
  out$conclusion <- ifelse(out$p_value < alpha,
    paste0("Reject H0 at ", alpha * 100, "%"),
    paste0("Do not reject H0 at ", alpha * 100, "%")
  )
  out$p_value <- ifelse(out$p_value < 0.001,
    "<0.001",
    round(out$p_value, 3)
  )

  return(out)
}

Applied to our dataset, with no adjustment method for the p-values:

result <- t_table(
  data = dat,
  c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width"),
  "Species"
)

result

##       variable p_value      conclusion
## 1 Sepal.Length  <0.001 Reject H0 at 5%
## 2  Sepal.Width   0.002 Reject H0 at 5%
## 3 Petal.Length  <0.001 Reject H0 at 5%
## 4  Petal.Width  <0.001 Reject H0 at 5%

And with the Holm (1979) adjustment method:

result <- t_table(
  data = dat,
  c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width"),
  "Species",
  p_adj = "holm"
)

result

##       variable p_value      conclusion
## 1 Sepal.Length  <0.001 Reject H0 at 5%
## 2  Sepal.Width   0.002 Reject H0 at 5%
## 3 Petal.Length  <0.001 Reject H0 at 5%
## 4  Petal.Width  <0.001 Reject H0 at 5%

Again, with the Holm’s adjustment method, we conclude that, at the 5% significance level, the two species are significantly different from each other in terms of all 4 variables.

ANOVA

Below the same process with an ANOVA. Note that we reload the dataset iris to include all three Species this time:

dat <- iris

# Edit from here
x <- which(names(dat) == "Species") # name of grouping variable
y <- which(names(dat) == "Sepal.Length" # names of variables to test
| names(dat) == "Sepal.Width" |
  names(dat) == "Petal.Length" |
  names(dat) == "Petal.Width")
method1 <- "anova" # one of "anova" or "kruskal.test"
method2 <- "t.test" # one of "wilcox.test" or "t.test"
my_comparisons <- list(c("setosa", "versicolor"), c("setosa", "virginica"), c("versicolor", "virginica")) # comparisons for post-hoc tests
# Edit until here


# Edit at your own risk
for (i in y) {
  for (j in x) {
    p <- ggboxplot(dat,
      x = colnames(dat[j]), y = colnames(dat[i]),
      color = colnames(dat[j]),
      legend = "none",
      palette = "npg",
      add = "jitter"
    )
    print(
      p + stat_compare_means(aes(label = paste0(after_stat(method), ", p-value = ", after_stat(p.format))),
        method = method1, label.y = max(dat[, i], na.rm = TRUE)
      )
      + stat_compare_means(comparisons = my_comparisons, method = method2, label = "p.format") # remove if p-value of ANOVA or Kruskal-Wallis test >= alpha
    )
  }
}

Like the improved routine for the t-test, I have noticed that students and non-expert professionals understand ANOVA results presented this way much more easily compared to the default R outputs.

With one graph for each variable, it is easy to see that all species are different from each other in terms of all 4 variables.³

If you want to apply the same automated process to your data, you will need to modify the name of the grouping variable (Species), the names of the variables you want to test (Sepal.Length, etc.), whether you want to perform an ANOVA (anova) or Kruskal-Wallis test (kruskal.test) and finally specify the comparisons for the post-hoc tests.⁴

To go even further

As we have seen, these two improved R routines allow to:

Perform t-tests and ANOVA on a small or large number of variables with only minor changes to the code. I basically only have to replace the variable names and the name of the test I want to use. It takes almost the same time to test one or several variables so it is quite an improvement compared to testing one variable at a time.
Share test results in a much proper and cleaner way. This is possible thanks to a graph showing the observations by group and the p-value of the appropriate test included directly on the graph. This is particularly important when communicating results to a wider audience or to people from diverse backgrounds.

However, like most of my R routines, these two pieces of code are still a work in progress. Below are some additional features I have been thinking of and which could be added in the future to make the process of comparing two or more groups even more optimal:

Add the possibility to select variables by their numbering in the dataframe. For the moment it is only possible to do it via their names. This will allow to automate the process even further because instead of typing all variable names one by one, we could simply type 4:25 (to test variables 4 to 25 for instance).
Add the possibility to choose a p-value adjustment method. Currently, raw p-values are displayed in the graphs and I manually adjust them afterwards or adjust the $\alpha$.
When comparing more than two groups, it is only possible to apply an ANOVA or Kruskal-Wallis test at the moment. A major improvement would be to add the possibility to perform a repeated measures ANOVA (i.e., an ANOVA when the samples are dependent). It is currently already possible to do a t-test with two paired samples, but it is not yet possible to do the same with more than two groups.
Another less important (yet still nice) feature when comparing more than 2 groups would be to automatically apply post-hoc tests only in the case where the null hypothesis of the ANOVA or Kruskal-Wallis test is rejected (so when there is at least one group different from the others, because if the null hypothesis of equal groups is not rejected we do not apply a post-hoc test). At the present time, I manually add or remove the code that displays the p-values of post-hoc tests depending on the global p-value of the ANOVA or Kruskal-Wallis test.

I will try to add these features in the future, or I would be glad to help if the author of the {ggpubr} package needs help in including these features (I hope he will see this article!).

Last but not least, the following packages may be of interest to some readers:

If you want to report statistical results on a graph, I advise you to check the {ggstatsplot} package and in particular the ggbetweenstats() and ggwithinstats() functions. These functions allow to compare a continuous variable across multiple groups or conditions (for both independent and paired samples). Two advantages of the functions is that:
- it is very easy to switch from parametric to nonparametric tests and
- it automatically runs an ANOVA or t-test depending on the number of groups to compare

Note that many different statistical results are displayed on the graph, not only the name of the test and the p-value so a bit of simplicity and clarity is lost for more precision. However, it is still very convenient to be able to include tests results on a graph in order to combine the advantages of a visualization and a sound statistical analysis. Something that I still need to figure out is how to run the code on several variables at once.

The {compareGroups} package also provides a nice way to compare groups. It comes with a really complete Shiny app, available with:

# install.packages("compareGroups")
library(compareGroups)
cGroupsWUI()

Update with the `{ggstatsplot}` package

Several months after having written this article, I finally found a way to plot and run analyses on several variables at once with the package {ggstatsplot} (Patil 2021). This was the main feature I was missing and which prevented me from using it more often.

Although I still find that too much statistical details are displayed (in particular for non experts), I still believe the ggbetweenstats() and ggwithinstats() functions are worth mentioning in this article. I actually now use those two functions almost as often as my previous routines because:

I do not have to care about the number of groups to compare, the functions automatically choose the appropriate test according to the number of groups (ANOVA for 3 groups or more, and t-test for 2 groups)
I can select variables based on their column numbering, and not based on their names anymore (which prevents me from writing those variable names manually)
When comparing 3 or more groups (so for ANOVA, Kruskal-Wallis, repeated measure ANOVA or Friedman), $p$-values of the post-hoc tests within each dependent variable are by default the adjusted $p$-values (Holm is the default but many adjustment methods are available)
It is possible to compare both independent and paired samples, no matter the number of groups (remember that with the ggpubr package I could only do paired samples for two samples, not for 3 samples)
They allow to easily switch between the parametric and nonparametric version
All this in a more concise manner using the {purrr} package

For those of you who are interested, below my updated R routine which include these functions and applied this time on the penguins dataset.

library(palmerpenguins)

dat <- penguins
str(dat)

## tibble [344 × 8] (S3: tbl_df/tbl/data.frame)
##  $ species          : Factor w/ 3 levels "Adelie","Chinstrap",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ island           : Factor w/ 3 levels "Biscoe","Dream",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ bill_length_mm   : num [1:344] 39.1 39.5 40.3 NA 36.7 39.3 38.9 39.2 34.1 42 ...
##  $ bill_depth_mm    : num [1:344] 18.7 17.4 18 NA 19.3 20.6 17.8 19.6 18.1 20.2 ...
##  $ flipper_length_mm: int [1:344] 181 186 195 NA 193 190 181 195 193 190 ...
##  $ body_mass_g      : int [1:344] 3750 3800 3250 NA 3450 3650 3625 4675 3475 4250 ...
##  $ sex              : Factor w/ 2 levels "female","male": 2 1 1 NA 1 2 1 2 NA NA ...
##  $ year             : int [1:344] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...

We illustrate the routine for two groups with the variables sex (two factors) as independent variable, and the 4 quantitative continuous variables bill_length_mm, bill_depth_mm, flipper_length_mm and body_mass_g as dependent variables:

library(ggstatsplot)
library(tibble)

# Comparison between sexes

# edit from here
x <- "sex"
cols <- 3:6 # the 4 continuous dependent variables
type <- "parametric" # given the large number of observations, we use the parametric version
paired <- FALSE # FALSE for independent samples, TRUE for paired samples
# edit until here

# edit at your own risk
plotlist <-
  purrr::pmap(
    .l = list(
      data = list(as_tibble(dat)),
      x = x,
      y = as.list(colnames(dat)[cols]),
      plot.type = "box", # for boxplot
      type = type, # parametric or nonparametric
      pairwise.comparisons = TRUE, # to run post-hoc tests if more than 2 groups
      pairwise.display = "significant", # show only significant differences
      bf.message = FALSE, # remove message about Bayes Factor
      centrality.plotting = FALSE # remove central measure
    ),
    .f = ifelse(paired, # automatically use ggwithinstats if paired samples, ggbetweenstats otherwise
      ggstatsplot::ggwithinstats,
      ggstatsplot::ggbetweenstats
    ),
    violin.args = list(width = 0, linewidth = 0) # remove violin plots and keep only boxplots
  )

# print all plots together with statistical results
for (i in 1:length(plotlist)) {
  print(plotlist[[i]])
}

We now illustrate the routine for 3 groups or more with the variable species (three factors) as independent variable, and the 4 same dependent variables:

# Comparison between species

# edit from here
x <- "species"
cols <- 3:6 # the 4 continuous dependent variables
type <- "parametric" # given the large number of observations, we use the parametric version
paired <- FALSE # FALSE for independent samples, TRUE for paired samples
# edit until here

# edit at your own risk
plotlist <-
  purrr::pmap(
    .l = list(
      data = list(as_tibble(dat)),
      x = x,
      y = as.list(colnames(dat)[cols]),
      plot.type = "box", # for boxplot
      type = type, # parametric or nonparametric
      pairwise.comparisons = TRUE, # to run post-hoc tests if more than 2 groups
      pairwise.display = "significant", # show only significant differences
      bf.message = FALSE, # remove message about Bayes Factor
      centrality.plotting = FALSE # remove central measure
    ),
    .f = ifelse(paired, # automatically use ggwithinstats if paired samples, ggbetweenstats otherwise
      ggstatsplot::ggwithinstats,
      ggstatsplot::ggbetweenstats
    ),
    violin.args = list(width = 0, linewidth = 0) # remove violin plots and keep only boxplots
  )

# print all plots together with statistical results
for (i in 1:length(plotlist)) {
  print(plotlist[[i]])
}

As you can see, I only have to specify:

the name of the grouping variable (sex and species),
the number of the dependent variables (variables 3 to 6 in the dataset),
whether I want to use the parametric or nonparametric version and
whether samples are independent (paired = FALSE) or paired (paired = TRUE).

Everything else is automated—the outputs show a graphical representation of what we are comparing, together with the details of the statistical analyses in the subtitle of the plot (the $p$-value among others).

Note that the code shown above is actually the same if I want to compare 2 groups or more than 2 groups. I wrote twice the same code (once for 2 groups and once again for 3 groups) for illustrative purposes only, but they are the same and should be treated as one for your projects.

I must admit I am quite satisfied with this routine, now that:

I can automate it on many variables at once and I do not need to write the variable names manually anymore,
at the same time, I can choose the appropriate test among all the available ones (depending on the number of groups, whether they are paired or not, and whether I want to use the parametric or nonparametric version).

Nonetheless, I must also admit that I am still not satisfied with the level of details of the statistical results. As already mentioned, many students get confused and get lost in front of so much information (except the $p$-value and the number of observations, most of the details are rather obscure to them because they are not covered in introductory statistic classes).

I saved time thanks to all improvements in comparison to my previous routine, but I definitely lose time when I have to point out to them what they should look for. After discussing with other professors, I noticed that they have the same problem.

For the moment, you can only print all results or none. I have opened an issue kindly requesting to add the possibility to display only a summary (with the $p$-value and the name of the test for instance).⁵ I will update again this article if the maintainer of the package includes this feature in the future. So stay tuned!

Conclusion

Thanks for reading.

I hope this article will help you to perform t-tests and ANOVA for multiple variables at once and make the results more easily readable and interpretable by non-scientists. Learn more about the t-test to compare two groups, or the ANOVA to compare 3 groups or more.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Holm, Sture. 1979. “A Simple Sequentially Rejective Multiple Test Procedure.” Scandinavian Journal of Statistics, 65–70.

McDonald, JH. 2014. “Multiple Tests.” Handbook of Biological Statistics. 3rd Ed Baltimore, Maryland: Sparky House Publishing, 233–36.

Patil, Indrajeet. 2021. “Visualizations with statistical details: The ’ggstatsplot’ approach.” Journal of Open Source Software 6 (61): 3167. https://doi.org/10.21105/joss.03167.

In theory, an ANOVA can also be used to compare two groups as it will give the same results compared to a Student’s t-test, but in practice we use the Student’s t-test to compare two groups and the ANOVA to compare three groups or more.↩︎
Do not forget to separate the variables you want to test with |.↩︎
Do not forget to adjust the $p$-values or the significance level $\alpha$. If you use the Bonferroni correction, the adjusted $\alpha$ is simply the desired $\alpha$ level divided by the number of comparisons.↩︎
Post-hoc test is only the name used to refer to a specific type of statistical tests. Post-hoc test includes, among others, the Tukey HSD test, the Bonferroni correction, Dunnett’s test. Even if an ANOVA or a Kruskal-Wallis test can determine whether there is at least one group that is different from the others, it does not allow us to conclude which are different from each other. For this purpose, there are post-hoc tests that compare all groups two by two to determine which ones are different, after adjusting for multiple comparisons. Concretely, post-hoc tests are performed to each possible pair of groups after an ANOVA or a Kruskal-Wallis test has shown that there is at least one group which is different (hence “post” in the name of this type of test). The null and alternative hypotheses and the interpretations of these tests are similar to a Student’s t-test for two samples.↩︎
I am open to contribute to the package if I can help!↩︎

Top 100 R resources on COVID-19 Coronavirus

Thu, 12 Mar 2020 00:00:00 +0000

Warning: Some links or resources may have been moved or deleted, and are thus not accessible anymore. If you are the author and would like to update the URL, feel free to contact me so I can update the link.

The Coronavirus is a serious concern around the globe. With its expansion, there are also more and more online resources about it. This article presents a selection of the best R resources on the COVID-19 virus.

This list is by no means exhaustive. I am not aware of all R resources available online about the Coronavirus, so please feel free to let me know in the comments or by contacting me if you believe that another resource (R package, Shiny app, R code, blog posts, datasets, etc.) deserves to be on this list.

R Shiny apps and dashboards

Coronavirus tracker

Developed by John Coene, this Shiny app tracks the spread of the Coronavirus, based on three data sources (John Hopkins, Weixin and DXY Data). The Shiny app, built with shinyMobile (which makes it responsive on different screen sizes), presents in a really nice way the number of deaths, confirmed, suspected and recovered cases by time and region.

The code is available on GitHub.

Coronavirus dashboard from the `{coronavirus} package`

Developed by the author of the {coronavirus} package, this dashboard provides an overview of the 2019 Novel Coronavirus COVID-19 (2019-nCoV) epidemic. The data and dashboard are refreshed on a daily basis.

The code is available on GitHub.

From this dashboard, I created another dashboard specific to Belgium. Feel free to use the code available on GitHub to build one specific to your country. See more details in this article.

Visualization of Covid-19 Cases

Developed by Nico Hahn, this Shiny app uses leaflet, plotly and the data from Johns Hopkins University to visualize the outbreak of the novel Coronavirus and shows data for the entire world or singular countries.

The code is available on GitHub.

Modeling COVID-19 Spread vs Healthcare Capacity

Developed by Dr. Alison Hill, this Shiny app uses an epidemiological model based on the classic SEIR model to describe the spread and clinical progression of COVID-19. It includes different clinical trajectories of infection, interventions to reduce transmission, and comparisons to healthcare capacity.

The code is available on GitHub.

COVID-19 Data Visualization Platform

Developed by Shubhram Pandey, this Shiny app provides a clear visualization of Covid19 impact all over the world and it also provides a sentiment analysis using natural language processing from Twitter.

The code is available on GitHub.

Coronavirus 10-day forecast

Developed by the Spatial Ecology and Evolution Lab, this Shiny app gives a ten-day forecast, by country, on likely numbers of Coronavirus cases and gives citizens a sense of how fast this epidemic is progressing.

See a detailed explanation of the app and how to read it in this blog post. The code is available on GitHub.

Coronavirus (COVID-19) across the world

Developed by Anisa Dhana in collaboration with datascience+, this Shiny app monitors the spread of COVID-19 across the world via a map visualization of the confirmed cases and some graphs on the growth of the virus.

The dataset used is from Johns Hopkins CSSE and part of the code is available in this blog post.

COVID-19 outbreak

Developed by Dr. Thibaut Fabacher in collaboration with the department of Public Health of the Strasbourg University Hospital and the Laboratory of Biostatistics and Medical Informatics of the Strasbourg Medicine Faculty, this Shiny app shows an interactive map for global monitoring of the infection. It focuses on the evolution of the number of cases per country and for a given period in terms of incidence and prevalence.

The code is available on GitHub and this blog post discusses it in more detail.

Flatten the Curve

Developed by Tinu Schneider, this Shiny app illustrates, in an interactive way, the different scenarios behind the #FlattenTheCurve message.

The app has been built upon Michael Höhle’s article and the code is available on GitHub.

Explore the spread of Covid-19

Developed by Joachim Gassen, this Shiny app allows you to visualize confirmed, recovered cases and reported deaths for several countries via one summary graph.

The Shiny app is based on data from:

Johns Hopkins University CSSE team on the spread of the SARS-CoV-2 virus
ACAPS governmental measures database
World Bank

This blog post explains the Shiny app in further details and in particular the {tidycovid19} R package behind it.

Governments and COVID-19

Developed by Sebastian Engel-Wolf, this Shiny app presents in a elegant way the following measurements:

Maximum time of exponential growth in a row
Days to double infections
Exponential growth today
Confirmed cases
Deaths
Population
Confirmed cases on 100,000 inhabitants
Mortality rate

The code is available on GitHub and this article explains it in further details.

Simulating COVID-19 Epidemic in Togo - West Africa

Developed by Dr. Kankoé Sallah, this Shiny app uses SEIR metapopulation model with mobility between catchment areas to describe country level spread of COVID-19 and the impact of interventions in Togo, West Africa.

Covid-19 Prediction

Developed by Manuel Oviedo and Manuel Febrero (Modestya research group of the University of Santiago de Compostela), this Shiny app predicts the growth rate at 5-day horizon using the evolution during the last 15 days of growth rate. Three functional regression models are fitted and re-estimated when new data is available. The app also shows an interactive plot and table for the expected number of accumulated cases and new daily cases to each horizon (for confirmed and deaths responses) by country (from Johns Hopkins CSSE) and Spanish region (from ISCII).

See an explanation of the methodology in the About tab.

Covid-19 Dashboard

Developed by Philippe De Brouwer, this dashboard displays several key measures regarding the outbreak of the virus (by country or for all countries combined), together with some forecasts, a world map and other interactive plots.

Healthcare worker deaths from novel Coronavirus (COVID-19) in the US

Developed by Jonathan Gross, this Shiny app visualizes healthcare worker deaths from Coronavirus (COVID-19) in the US reported in the news. It is updated daily and the code is available on GitHub.

Covid-19 Hospitalizations in Belgium

Developed by Jean-Michel Bodart, this dashboard provides an overview of the evolution of Covid-19-related hospitalizations in Belgium, by region and province.

The code is available on GitHub.

COVIDMINDER: Where you live matters!

Developed by The Rensselaer Institute for Data Exploration and Applications, this Shiny app reveals the regional disparities in outcomes, determinants and medications (e.g., mortality rates, test cases, diabetes, and hospital beds) across United States, with a special focus on New York.

This blog post explains the Shiny app in more detail and the code is available on GitHub.

COVID-19 Canada Data Explorer Tool

Developed by Petr Baranovskiy from Data Enthusiast’s Blog, this Shiny app processes the official dataset available from the Government of Canada and shows several indicators related to the SARS-CoV-2 epidemic in Canada.

This blog post details the application in further detail.

Philippine COVID-19 Case Forecasting

Developed by Jamal Kay Rogers and Yvonne Grace Arandela, this Shiny app provides a 5-day forecast of confirmed positive, deaths, and recoveries of COVID-19 cases in The Philippines.

The app also delivers graphical plots of a 10-day forecast and the daily and cumulated cases of COVID-19 in The Philippines. The data source is Johns Hopkins University Center for Systems Science and Engineering (JHU CSSE).

COVID-19 Case & Death Report Number Corrector

Developed by Matt Maciejewski, this Shiny app focuses on the correction of underreported Covid-19 case and death counts using a reference country based on Lachmann et al. (2020), and via a multiplicative estimator for total deaths and cases. The estimator will be turned into a posterior prediction once data becomes available.

The app is explained in more detail in this article and the code of the Shiny app can be found on GitHub.

Covid-19: the SPQEIR model

Developed by several researchers from the Luxembourg Centre for Systems Biomedicine (LCSB) of the University of Luxembourg, the KU Leuven and the UGent, this Shiny app uses the new SPQEIR model to simulate the impact of various suppression strategies (social distancing, lockdown, protection, etc.) on the development of COVID-19.

The Belgian Covid Cases Tracker

Developed by Patrick Sciortino, this Shiny app aims at estimating the curve of true Covid-19 cases based on the idea that a hospitalized case in time t informs us about an infection that took place a few days earlier.

COVID-19 monitor

Developed by Trafford Data Lab, this Shiny app visualizes daily confirmed Coronavirus cases and deaths in the UK.

It uses the following data sources:

The code can be found on GitHub.

COVID-19 Bulletin Board

Developed by Wei Su, this dashboard shows real-time visualization of the COVID-19 epidemic in Japan. It mainly shows various indicators including, but not limited to, PCR test, positive confirmed, hospital discharge and death, as well as trends in each prefecture in Japan. There are also a variety of charts such as cluster network, new confirmed cases in log scale for users’ reference.

The dashboard is based on this Japanese version (developed by the same author). The code can be found on GitHub.

Covid-19 Statistics Displayer

Created by Carl Sansfaçon, this Wordpress plugin associates R {ggplot2} graphics with ARIMA forecast and PHP coding to show evolution of the confirmed, death and recovered cases in different countries, states/provinces and US cities.

It uses data from the COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University. The plugin can be installed as a Wordpress plugin.

CoronaMapper

Supported by OxyLabs, created by Paolo Montemurro and Peter, this visualization displays the four days average growth indicator, which clearly shows how a certain statistic of the virus is evolving over time, filtering out the noise.

The website receives data each hour from several official data-sources, and visualizes the historical evolution of COVID19 in an intuitive & interactive way.

CoronaDash

Developed by Peter Laurinec, this Shiny app provides various data mining and visualization techniques for comparing countries’ COVID-19 data statistics as:

extrapolating total confirmed cases by exponential smoothing model,
trajectories of cases/deaths spread,
multidimensional clustering of countries’ data/ statistics - with dendrogram and table of clusters averages,
aggregated views for the whole world,
hierarchical clustering of countries’ trajectories based on DTW distance and preprocessing by SMA (+ normalization), for fast comparison of a large number of countries’ COVID-19 magnitudes and trends.

This blog post explained in further detail the last point of the above list. The code of the app is available on GitHub.

Covidfrance

Developed by Guillaume Pressiat, this Shiny app illustrates the evolution of number of hospitalizations, intensive care units, recoveries and deaths in France (by department).

This blog post presents the application and the code of the app is available on GitHub.

The Impact of COVID-19 on Mobility

Developed by Dimiter Toshkov, this Shiny app shows relative changes in mobility (visits and length of stay) for a specific category of places within a country compared to a baseline. The baseline is computed as the median for the day of the week during the 5-week period between January 3 and February 6, 2020. Hence, the plot shows how mobility has changed relative to the situation in the same country in the beginning of the year.

The author also developed a version for the US states.

Data is from Google Community Mobility Reports.

Coronavirus Analysis Platform

Developed by Khaled M Alqahtani, this Shiny app offers a powerful analysis tools, including:

Descriptive analysis
Growth rate and curve flatting
Cumulative forecast
Daily cases forecasting containing 16 different models
Newspaper analysis
TV analysis

Data is from Johns Hopkins CSSE and some GDELT APIs.

COVID-19 Tracker

Developed by Dr. Magda Bucholc from Ulster University, this dashboard reports cases at the local government district in Northern Ireland and county level across the island of Ireland, providing gender and age breakdowns of reported cases, growth rates, and statistics per 100,000 of the population; it also has daily mobility data from Google and Apple.

More information about this dashboard can be found here.

COVID-19 Overview

Developed by Fabian Dablander, Alexandra Rusu, Marcel Schreiner, and Aleksandar Tomasevic as part of the Science versus Corona project, this dashboard provides an overview of confirmed cases, deaths, and measures that countries have taken to curb the spread of the virus.

For a more information about this dashboard, see this blog post.

COVID-19 Exit Strategies

Developed as part of the Science versus Corona project, this Shiny app compares several alternative exit strategies that either aim to keep the number of infections as low as possible (e.g., contact tracing), or that aim to develop herd immunity without exceeding health care capacity.

The Shiny app is based on the stochastic individual-based SEIR model developed by de Vlas and Coffeng (2020). This post explains the Shiny app in more detail.

Corona Virus Statistics : COVID19

Developed by Dr. Mohammed N. Alenezi, this dashboard displays the latest information about Coronavirus for Kuwait, GCC, and the world. The dashboard contains a collection of different plots and models presented under 8 tabs.

Covid19 Data

Developed by Dhafer Malouche, this dashboard presents statistics for more than 200 countries and regions, with the estimation of the reproduction number $R(t)$ in the previous 60 days and a Covid19 country classification.

WHO COVID-19 Explorer

Developed by the World Health Organization (WHO), this Shiny app aims to provide frequently updated data visualizations about confirmed cases and deaths at the global, regional and country level.

COVID-19 Scenario Analysis Tool

Developed by the MRC Centre for Global Infectious Disease Analysis (Imperial College London), this dashboard illustrates the epidemic trajectory, the healthcare demand and the $R_t$ & $R_{eff}$ measures for many countries over time in interactive plots.

The dashboard uses the squire R package, among others.

Covid-19 track

Developed by Redzuan, this Shiny dashboard analyses, combines and harmonizes the latest Covid-19 data into progress timeline, comparative analysis on various areas, mapping, latest news, forecasting using various models, prediction on Herd Immunity target, summary table, downloadable fact sheet and other features.

R packages

`{nCov2019}`

The {nCov2019} package gives you access to epidemiological data on the Coronavirus outbreak.¹ The package gives real-time statistics, includes historical data and a Shiny app. The vignette explains the main functions and possibilities of the package.

Furthermore, the authors of the package also developed a website with interactive plots and time-series forecasts, which could be useful in informing the public and studying how the virus spread in populous countries.

`{coronavirus}`

Developed by Rami Krispin, the {coronavirus} package provides a tidy format dataset of the 2019 Novel Coronavirus COVID-19 (2019-nCoV) epidemic. Pulled from the dataset of John Hopkins, the R package gives a daily summary of the Coronavirus cases by state/province. The data set contains various variables such as confirmed cases, death, and recovered across different countries and states.

More details are available here, a csv format of the package dataset is available here and a summary dashboard is available here.

`{tidycovid19}`

Developed by Joachim Gassen, the {tidycovid19} package allows you to download, tidy and visualize Covid-19 related data (including data on governmental measures) directly from authoritative sources. It also provides a flexible function and an accompanying Shiny app to visualize the spreading of the virus.

The package is available on GitHub and these blog posts here and here explain it in more detail.

R packages from R Epidemics Consortium

These R packages from R Epidemics Consortium allows you to find the most advanced tools used by professional epidemiologists and experts in the domain of analyzing disease outbreaks.

`{covdata}`

Published by Prof. Kieran Healy, the {covdata} package is a R package providing COVID-19 case data from multiple sources:

National level data from the European Centers for Disease Control
State-level data for the United States from the COVID Tracking Project
State-level and county-level data for the United States from the New York Times
Data from the US Centers for Disease Control’s Coronavirus Disease 2019 (COVID-19)-Associated Hospitalization Surveillance Network (COVID-NET)
Data from Apple on relative trends in mobility in cities and countries since mid-January of 2020, based on usage of their Maps application
Data from Google on relative trends in mobility in regions and countries since mid-January of 2020, based on location and activity information

The code is available on GitHub.

`{covid19italy}`

The covid19italy R package provides a tidy format dataset of the 2019 Novel Coronavirus COVID-19 (2019-nCoV) pandemic outbreak in Italy. The package includes the following three datasets:

italy_total: daily summary of the outbreak on the national level
italy_region: daily summary of the outbreak on the region level
italy_province: daily summary of the outbreak on the province level

More information about the package datasets available in this vignette, this blog post, and this supporting dashboard.

Data source: Italy Department of Civil Protection

`{COVID19}`

The goal of COVID-19 Data Hub is to provide the research community with a unified data hub by collecting worldwide fine-grained case data, merged with exogenous variables helpful for a better understanding of COVID-19. Featured by the University of Milano and funded by the Institute for Data Valorization IVADO, Canada.

The package collects COVID-19 data across governmental sources, includes policy measures from Oxford COVID-19 Government Response Tracker, and extends the dataset via an interface to World Bank Open Data, Google Mobility Reports and Apple Mobility Reports.

The package is available on CRAN, it is 100% open source and external contributors are welcomed to join.

`{COVOID}`

COVOID (for COVID-19 Open-source Infection Dynamics project) is a R package for modelling COVID-19 and other infectious diseases using deterministic compartmental models (DCMs).

It contains a built-in Shiny app enabling easy use and demonstration of key concepts to those without R programming backgrounds, along with an expanding API for simulating and estimating homogeneous and age-structured SIR, SEIR and extended models. In particular COVOID allows the simultaneous simulation of age specific (e.g. school closures) and general interventions over varying time intervals.

The code is available on GitHub.

`{cdccovidview}`

Published by Bob Rudis, the {cdccovidview} package can be used to work with the U.S. CDC’s New COVID-19 Trackers: COVIDView and COVID-NET.

`{babsim.hospital}`

Published by several researchers from TH Köln, the {babsim.hospital} package implements a discrete-event simulation model for a hospital resource planning problem. It can be used by health departments to forecast demand for intensive care beds, ventilators, and staff resources.

The team also developed a Shiny app which predicts COVID-19 ICU bed resources in hospitals. The app is available in English and German.

`{EpiLPS}`

Published by several researchers working in the field of epidemiology, the package shows how to smooth epidemic curves and estimate the time-varying reproduction number in a flexible way.

More information can be found in:

the research paper,
the accompanying website, and
this summary written by one of the authors.

R code and blog posts

Analyzing COVID-19 outbreak data with R

Written by Tim Churches, these two articles (part 1 and part 2) explore the R tools and packages that might be used to analyze the COVID-19 data. In particular, the author considers when the pandemic will subside in China, and then turns the analysis on Japan, South Korea, Italy and Iran. He also shows improvements on the cumulative incidence plots that are so common. Moreover, he presents R code to analyze how contagious is the Coronavirus thanks to the classic SIR (Susceptible-Infectious-Recovered) compartmental model of communicable disease outbreaks.²

The code is available on GitHub (part 1 and part 2).

Part 1 is actually based on another shorter blog post by Prof. Dr. Holger K. von Jouanne-Diedrich from Learning Machines. Read his article for a more concise analysis on how to model the outbreak of the Coronavirus and discover how contagious it is. Note that I have personally written an article analyzing COVID-19 in Belgium based on articles from these two authors.

More recently, Tim Churches published a series of other interesting articles:

Modelling the effect of various public health interventions on the local epidemic spread of COVID-19 infection using stochastic individual compartmental models (ICMs) implemented by the {EpiModel} package for R:
- Part 1 (code here)
- Part 2 (code here)
The use of simulations to explore the effects of various interventions on COVID-19 spread
COVID-19 epidemiology with R: in this blog post, the author, using relatively early and partial US data, separates out inbound from community cases, and predicts the next few weeks of incident numbers. He also highlights several R functions to analyze a disease outbreak.
We can “shrink” the COVID-19 curve, rather than just flatten it (in collaboration with Louisa Jorm)

COVID-19 Data Analysis with `{tidyverse}` and `{ggplot2}`

Dr. Yanchang Zhao from RDataMining published a data analysis around the Coronavirus with the {tidyverse} and {ggplot2} packages, for China and world wide.

Both documents are a mix of data cleaning, data processing and visualizations of the confirmed/cured cases and death rates across countries or regions.

COVID-19 cumulative observed case fatality rate over time

Written by Peter Ellis, this article focuses on how the observed case fatality rate of COVID-19 has evolved over time across 7 countries and comments on why the rates vary (low testing rates, age of the population, overwhelmed hospitals, etc.).

The code is available at the end of the article. The data is from John Hopkins and it uses the {coronavirus} package.

More recently, the author published a series of other articles:

Impact of a country’s age breakdown on COVID-19 case fatality rate: it looks at estimated fatalities in different countries according to the age distributions in those countries (based on Italy’s data). The data is from The Istituto Superiore di Sanità (Roma) and all the code is shown in the post.
How to make that crazy Fox News y axis chart with ggplot2 and scales: less about COVID19 than about how a bizarre Fox News graph can be re-created with the correct transformations needed to make its scale appropriate.
Test positivity rates and actual incidence and growth of diseases: this blog post looks at several different ways of accounting for the information given to us by high positive testing rates for COVID-19 and looks at the impact on estimates of effective reproduction number at a point in time.
Incidence of COVID-19 in Texas after adjusting for test positivity: the author examines the trends in COVID-19 cases in Texas, with and without being adjusted by a multiplier of the square root of the test positivity rate.

Covid 19 Tracking

Written by Prof. Kieran Healy, this article discusses how to get an overview of best-available counts of deaths, using the COVID-19 Data from the European Centers for Disease Control.

Code can be found in the article and on GitHub.

More recently, the author published three other articles:

A COVID Small Multiple: this article discusses how to create a small-multiple plot of cases by country, showing the trajectory of the outbreak for a large number of countries, with the background of each small-multiple panel also showing (in grey) the trajectory of every other country for comparison.

Apple’s COVID Mobility Data: this article uses Apple’s time series mobility data for several cities and countries (via the {covdata} package) to plot three modes of getting around: driving, public transit, and walking. The series begins on January 13th indexed to 100 at the beginning of the series, so trends are relative to that baseline.

New Orleans and Normalization: this article responds to a thoughtful post by Dr. Drang regarding an improvement in normalization of the data.

Infectious diseases and nonlinear differential equations

Published by Fabian Dablander, this math intensive blog post explains what SIR and SIRS models take into account and how they calculate their results.

From a pandemic perspective, the author writes “The SIRS model extends the SIR model, allowing the recovered population to become susceptible again (hence the extra ‘S’). It assumes that the susceptible population increases proportional to the recovered population”.

More recently, the author, in collaboration with other researchers, published another blog post outlining a number of excellent visualizations of the COVID19 pandemic, as well as presenting their own dashboard.

Epidemic modelling of COVID-19 in the UK using an SIR model

Published by Thomas Wilding, this blog post applies the SIR model to UK data.

As further extensions to the model, the author suggests:

Using an SEIR model (adding an Exposed compartment for people who are infected but not yet infectious)
Adding a “Q” layer since a lot of people are being Quarantined or isolated
Considering the “hidden”” population that is infected but is denied being tested due to shortage of tests
Feasibility of a second wave / outbreak of the epidemic later in the year (as seen in previous outbreaks, such as Swine Flu)

Data sources:

Modeling Pandemics

Published by Arthur Charpentier, this series of 3 blog post (part 1, part 2, part 3) walks through the SIR model and its parameters, how ODEquations solves it, and generating the reproductive rate. It also gives a mathematical explanation of a model for how quickly a pandemic will return, albeit with diminishing intensity. Last, it explains a model that is more sophisticated than SIR, the SEIR model, and illustrates it with Ebola data.

More recently, the author published another article which examines what proportion of the population in various U.S. states have been tested for the novel Coronavirus and tries to answer the following two questions:

How many people are tested on a daily basis?
What are we actually testing for?

Last but not least, this post reproduces one of his scientific paper entitled “COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability”.

COVID-19: The Case of Germany

Published by Prof. Dr. Holger K. von Jouanne-Diedrich from Learning Machines, this blog post uses the SIR model and German data to estimate the duration and severity of the pandemic.

Download the data from Morgenpost.

More recently, the author published other articles:

COVID-19 in the US: Back-of-the-Envelope Calculation of Actual Infections and Future Deaths: Working back from reported deaths from Covid19, the post shows how to estimate infections at a prior date, based on several assumptions about fatality rates and infected periods (and acknowledging many unknowns and data problems).
How to analyze mobility trends with R using anonymized and aggregated Apple’s mobility data available to the public. The article presents a R function to return the data in a well-structured format for countries and major cities, and to visualize the drop in vehicular and pedestrian movement caused by the pandemic.

COVID-19: False Positive Alarm, demonstrating the importance of infection rates on the likelihood that someone testing positive for the Coronavirus is actually positive.

Flatten the COVID-19 Curve

Published by Michael Höhle from Theory meets practice, this blog post discusses why the message of flattening the COVID-19 curve is right, but why some of the visualizations used to show the effect are wrong: Reducing the basic reproduction number does not just stretch the outbreak, it also reduces the final size of the outbreak.

From a pandemic point of view, the author writes “Because of limited health capacities, stretching out the outbreak over a longer time period will ensure, that a larger proportion of those in need of hospital treatment will actually get it. Other advantages of this approach are to win time in order to find better treatment forms and, possibly, to eventually develop a vaccine”.

A Shiny app has also been built upon this article to investigate different scenarios.

In a second article entitled “Effective reproduction number estimation”, Michael Höhle estimates with the {R0} package the time-varying effective reproduction number during an infectious disease outbreak such as COVID-19. Using a single simulated outbreak he compares the performance of three different estimation methods.

More recently, in this article the author looks at “superspreading” in infectious disease transmission from a statistical point of view. He characterises heterogeneity in the offspring distribution [who becomes infected by the superspreader person] by the Gini coefficient instead of the usual dispersion parameter of the negative binomial distribution. This allows us to consider more flexible offspring distributions.

Flattening vs shrinking: the math of #FlattenTheCurve

Published by Ben Bolker and Jonathan Dushoff, this blog post gives a clear explanation of physical distancing and explains how physical distancing makes several beneficial outcomes possible.

The code is available on GitHub.

explainCovid19 challenge

Published by Przemyslaw Biecek, this blog post gives an overview of a model that uses gradient boosting to predict survival based on age, country, and gender. It also shows how older people are more at risk and it lets you play with the model yourself with a modelStudio interactive dashboard.

Data sources:

Google sheet (with most recent data at the end of February)
Kaggle dataset

An R Package to explore the Novel Coronavirus

Published by Patrick Tung via Towards Data Science, this blog post translates into English an R package originally written in Chinese.

Data is collected from Tencent, at https://news.qq.com/zt2020/page/feiyan.htm, which contains one of the most up-to-date public information of the Coronavirus.

Coronavirus model using R – Colombia

Published by Daniel Pena Chavez, this blog post uses the code from Prof. Dr. Holger K. von Jouanne-Diedrich to model height of pandemic in Colombia and projected deaths. The author also points out that a huge number of other variables need to be considered, such as density, climate and government response.

Data is from Rami Krispin’s GitHub.

More recently, the author published another article comparing China and Italy’s rates on log scales.

COVID-19: The Case of Spain

Written by Jose from Diarium - Statistics and R software, this blog post, using data for Spain, applies the SIR model, and then a cubic polynomial regression model to predict infections, hospitalizations, deaths and peak date.

Tidying the new Johns Hopkins Covid-19 time-series datasets

Written by Joachim Gassen, this blog post provides functions and code to deal with different country names and changes on the Johns Hopkins site.

More recently, the author published a series of other interesting articles:

Merge Covid-19 Data with Governmental Interventions Data: this article analyzes five kinds of intervention on the spread of COVID-19.

Scraping Google Covid-19 community movement data from PDF figures: this article explains how to scrape data from a Google site that tracks movements of people. The author uses the {tidycovid19} R package and prepares an analysis of Germany and then across countries.

Covid-19: Explore Your Visualizer Degrees of Freedom: in this article, the author uses COVID-19 data to demonstrate how graphs can communicate very differently, and be manipulated. He shows that getting a ‘neutral’ message to the reader is far from trivial and that visualizations without guidance can be particularly misleading.
{tidycovid19} New data and documentation: A recent update to the {tidycovid19} package brings data on testing, alternative case data, some regional data and proper data documentation. Using all this, you can use the package to explore the associations of (the lifting of) governmental measures, citizen behavior and the Covid-19 spread.
Exploring and Benchmarking Oxford Government Response Data: A post assessing the impact of non-pharmaceutical interventions on the spread of Covid-19 based on the Assessment Capacities Project (ACAPS) and the Oxford Covid-19 Government Response Tracker.

COVID-19 in Belgium

Based on Tim Churches’ article, I published an analysis of the COVID-19 specifically for Belgium. In this article, I also use the most common epidemiological model, the SIR model (to its simplest form), to analyze the outbreak of the disease in the case where there would be no public health intervention. I also show how to compute the reproduction number and I present some additional improvements that can be made to further analyze the epidemic.

The code is available on GitHub, so feel free to use it as starting point for an analysis of the virus outbreak in your own country.

Facts About Coronavirus Disease 2019 (COVID-19) in 5 Charts created with R and ggplot2

Written by Gregory Kanevsky, this blog post compiles some useful facts about COVID-19 into 5 charts, including gauge charts, and discusses R and {ggplot2} techniques used to create them.

Contagiousness of COVID-19 Part I: Improvements of Mathematical Fitting

Written by Martijn Weterings on Learning Machines, this guest post describes the fitting of Covid-19 data with the SIR model and explains tricky parts of the fitting methodology and how we can mitigate some of the problems (e.g., early stopping of the algorithm or an ill-conditioned problem). It provides a very clear explanation of some tweaks to the standard model.

The code is available here.

Coronavirus : spatially smoothed decease in France and decease animation map

Published by Michael Ires, this blog post shows R code on how to use kernel weighted smoothing with arbitrary bounding areas to display a map of deaths from Covid-19 in France.

The author also published two other articles on how to build an animated map of deaths from Covid-19 in France and in Europe.

More recently, the author published an article where he uses the {geogrid} package to show the incidence of Covid19 by French departments as well as spread.

Another “flatten the COVID-19 curve” simulation… in R

Written by Javier Fernandez-Lopez, this blog post shows R code to create static plots and then simulations to demonstrate how social distancing could help to “flat the curve” of COVID-19 infections.

Tracking Covid19 Cases Throughout NJ with R

Written by Kevin Zolea, this blog post shows how to use the {gganimate} package to create an animated time series map showing how Covid19 spread throughout the U.S. state of New Jersey’s counties.

It’s fun to look at the YACM (Yet Another COVID Model)

Written by Adrian Barnett from Median Watch, all the models in this blog post are based on the excellent ordinary differential equation models by Alison Hill. They are microsimulations of those models that make heavy use of the {MicSim} package for running microsimulations in R.

Is COVID-19 as bad as all that? Yes it probably is

Written by Dr. Francis Smart from Econometrics By Simulation, this article explains clearly some of the factors that determine the infections and deaths from COVID19, with different scenarios of seriousness.

Potential Long-Term Intervention Strategies for COVID-19

On this website, several professors and members of Stanford University (Marissa Childs, Morgan Kain, Devin Kirk, Mallory Harris, Jacob Ritchie, Lisa Couper, Isabel Delwel, Nicole Nova, Erin Mordecai) developed a compartmental model of COVID-19 to evaluate possible outcomes of non-pharmaceutical interventions such as social distancing.

The website presents an introduction to the problem, the possibility to play around with the model to predict the effects of COVID intervention strategies (thanks to a Shiny app), the model details, and predictions for Santa Clara County, California.

The code is available on GitHub.

Animations in the time of Coronavirus

Written by Martin Henze from the Heads or Tails blog, this article describes how to extract and prepare the necessary data to animate the spread of the virus over time in Germany, using {gganimate} and {sf} R packages to create animated map visuals.

The author posted the dataset associated with the Germany maps to Kaggle, where he is maintaining it on a daily basis. In addition, he posted a version of the JHU US county level dataset where he added some key demographic info from the US census. This dataset is also updated daily.

COVID-19 Data and Prediction for Michigan

Written by Nagdev Amruthnath, this blog post builds and tests an exponential regression model based on (not much) State of Michigan data.

Data Visualization of COVID-19 in the US

Written by Daniel Reiff, this article examines COVID-19 growth dynamics using exponential and logistic curves.

The spread of COVID-19 across countries visualization with R

Written by Sergey Bryl, this article examines the speed and spreading of the virus across countries. One animated visualization and two stationary charts show:

how long and intensive were previous phases and
compare the effectiveness against COVID-19 for different countries

The code can be found at the end of the article.

Covid-19 and Rural Areas in the U.S

Written by Elliot Meador from Deltanomics, this blog post focuses on cases of Covid-19 in rural areas of the U.S, including whether in the South any particular state appears to be an outlier.

Covid Death Rates: Is the data correct?

Written by Sam Weiss, this post raises questions about the accuracy of reports of case numbers, in that they may fail to backfill for corrected data.

More recently, the author published two articles (part 1 and part 2) in which he finds and visualizes an association between number of people that tested positive for COVID-19 in a country and imports from China. In addition he finds that there are particular industries that are particularly correlated with COVID-19 rates.

COVID-19 Risk Heat Maps with Location Data, Apache Arrow, Markov Chain Modeling, and R Shiny

Written by Filip Stachura, this post describes Appsilon’s solution (CoronaRank) submitted to the recent Pandemic Response Hackathon. Inspired by Google’s PageRank, it uses geolocation data in the Apache Parquet format from Veraset for effective exposure risk assessment using Markov Chain modeling.

COVID-19 Tracker Indonesia

Written by Dio Ariadi from DataWizArt, this post shows some very nice plots on regional variation in COVID-19 cases and deaths in Indonesia, with very neatly integrated R code for each plot.

COVID-19 Projections Using Machine Learning

Developed by Youyang Gu, this website presents an intuitive model that builds machine learning techniques on top of a classic infectious disease model to make COVID-19 infections and deaths projections for the US, all 50 US states, and more than 60 countries.

The code can be found on GitHub.

COVID-19 in Belgium: is it over yet?

Written by myself in collaboration with Prof. Niko Speybroeck and Angel Rosas-Aguirre, this article shows the evolution of the number of hospital admissions and the number of confirmed cases in Belgium (by province and at the national level).

Code of the plots is available in the article.

COVID-19 Cases by Ethnicity

Written by Tommi Suvitaival, this article investigates Covid19 deaths as a function of the percentage of a county’s population that comes from various ethnic backgrounds.

Tennessee COVID-19 Update

Written by Prof. James M. Luther, this document presents a summary of the daily data for the state of Tennessee. The author uses an interactive map, a seven-day rolling average of various metrics, and facet charts.

Simulating Coronavirus Outbreak in Cities with Origin-Destination Matrix and SEIR Model

Written by Yihui Fan, this blog post lays out a step-by-step guide on simulating and visualising the spread of Coronavirus in the Greater Tokyo Area based on Origin-Destination Matrix and SEIR Model.

Another post from the same author focuses on the effectiveness of reducing population movement in managing Coronavirus outbreak.

COVID-19 Population Mobility - How has human mobility changed under the COVID-19 Pandemic?

Written by Kelsey E. Gonzalez, this visualization aims to understand population behavior during the COVID-19 pandemic.

Data is from Cuebiq and the code is available on GitHub.

How to Build COVID-19 Data-Driven Shiny Apps in 5 minutes

Written by Emanuele Guidotti, this tutorial shows how to build a simple yet complete Shiny application using the R Package COVID19: R Interface to COVID-19 Data Hub.

In another post, the author explores the R package {COVID19} in further detail.

Analyzing data from COVID19 R package

Written by Pablo Cánovas, this blog post explores whether deaths from COVID19 are being reported accurately.

It uses data from The Human Mortality Database and the COVID-19 Data Hub.

Body Mass and Risk from COVID-19 and Influenza

Written by Prof. Radford Neal, this blog post looks at data from flu-like illnesses and some preliminary Covid19 data. The author concludes that being underweight and being seriously obese are both risk factors for serious respiratory illness.

More recently, the author published a series of other articles:

The Puzzling Linearity of COVID-19 discussing the fact that for many countries, the linear plots of total cases or total deaths go up exponentially at first, and then approach a straight line that is not horizontal
Seasonality of COVID-19, Other Coronaviruses, and Influenza: this post looks at the evidence for seasonality in influenza and the common cold Coronaviruses, and to what extent one might expect COVID-19 to also be seasonal
Critique of “Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period”: this post analyzes and criticizes an earlier paper by Kissler et al. (2020). See also part 2 and part 3

HMD – Weekly Data

Written by Ronald Richman, this blog post explores the highly improbable level of deaths currently being reported using the Human Mortality Database and its recently begun special time series of weekly death data across 13 countries.

Guest posts on Chris Muir’s blog

Written by Skylar Hara, this blog post examines whether the incidence of Covid19 changed after the Governor of Hawaii issued a stay-at-home order.

Written by Ronja Steinbach, this blog post gathers data on Trump or Clinton states, percentages of minorities, and median household income to answer the following questions:

Does party affiliation of a state in the 2016 election have a significant impact on the incidence rate of the virus in that state?
Does the proportion of the population that is minority and median household income affect the incident rate of the virus across states?

Written by Akemi Santiago, this post shows a statistically different mortality rate between African-Americans in the United States and white Americans.

Written by Masha Rutenberg, this post focuses on the correlation between the population of a country and the number of confirmed infections in the country.

All authors are students of Prof. Chris Muir.

An R View into Epidemiology

Written by Joseph Rickert in R Views, this blog post tracks down R packages that help epidemiology research and shows the number of downloads in recent months for the five most popular packages.

Articles by Rob J Hyndman

Rob J. Hyndman, Professor of Statistics and Head of the Department of Econometrics and Business Statistics at Monash University (Australia), published a series of Covid19 related articles:

Forecasting COVID-19: this blog post does not use R, although Prof. Hyndman is an expert with it, but it does explain some of the problems with time series forecasting or other methods of forecasting
Why log ratios are useful for tracking COVID-19: this post presents the benefits of reporting log-scale graphics
Excess deaths for 2020: the reported COVID19 deaths in each country are often underestimated. One way to explore the true mortality effect of the pandemic is to look at “excess deaths” — the difference between death rates this year and the same time in previous years
Seasonal mortality rates: this post shows how the weekly mortality data published by the Human Mortality Database can be used to explore seasonality in mortality rates. Mortality rates are known to be seasonal due to temperatures and other weather-related effects

Turkey vs. Germany: COVID-19

Written by Selcuk Disci from DataGeeek, this article compares efforts by Turkey and Germany to control the pandemic, and tests several regression models.

Hands-on: How to build an interactive map in R-Shiny: An example for the COVID-19 Dashboard

Written by Sangmeng, this post explains how to build an interactive dashboard with Shiny with an example for the COVID-19 Dashboard.

Modelling COVID-19 in Morocco

Written by Zakariah Gassasse, this blog post presents data on cases of Covid19 in Morocco and applies the SIR model to the data.

More recently, the author published two other articles. The first article provides a short-term forecast of COVID-19 cases and deaths in Morocco by using simple but effective time-series analyses. The second article maps the outbreak to see which regions are suffering the most.

SIR models with Kermack and McKendrick

Written by Pierre Jacob, this blog post is mostly a retrospective look at the origins of the much-used SIR model.

Johns Hopkins Covid-19 Data and R

Written by Steve Miller, these blog posts (part 1 & part 2) showcase the handling of daily data of cases/deaths from Covid-19 in the U.S. published by Johns Hopkins University, and visualize moving averages of cases and deaths.

The author also published a blog post putting unemployment claims in the U.S. in perspective.

Estimating COVID-19’s $R_t$ in Real-Time

Written by Ramnath Vaidyanathan, this tutorial shows how to estimate $R_t$, the measure known as effective reproduction number, which is the number of people who become infected per infectious person at time $t$.

From static to animated time series: the tidyverse way

Written by Giulia Ruggeri, this post goes through the steps necessary to create an animated COVID-19 time series plot.

Sneak peek: new Summit data tool helps clients visualize US areas that are most heavily impacted by the COVID-19 virus

Written by Colby Ziegler, this blog post describes a tool (built primarily in R using the {leaflet}, {tidyverse}, and {tigris} packages) tracking and displaying the total number of confirmed COVID-19 cases and deaths by U.S. county, overlaid with the locations of Certified Community Development Financial Institutions (CDFIs).

How to Reproduce Financial Times Style COVID19 Daily Reporting?

Written by Peyman Kor, this blog post shows how to reproduce in R the Financial Times facet plot by country.

Visualizing COVID cases in Belgium

In this blog post, Koen Hufkens plots cases in Belgium and addresses a challenge in geo-spatial plotting.

A spatio-temporal analysis of the environmental correlates of COVID-19 incidence in Spain

Written by Antonio Paez and several co-authors, this blog post looks at weather, humidity and other factors in Spain to create a SUR model.

Another post by Antonio Paez investigates the incidence of COVID-19 in the United States using Google Community Mobility Reports.

Covid-19 Analysis

Written by Rizami Annuar, this post compares data on cases and deaths in Malaysia to other countries, including correlations.

Australian governments can choose to slow the spread of coronavirus, but they would need to act immediately

Written by Matt Cowgill, this blog post shows that Australia early on had relatively few cases, but the post argues that the country needed to act urgently. Data is from the {gtrendsR} package.

In a more recent post, using Google trends data for key words associated with unemployment, the authors trace the effects of COVID19 in Australia in the early months.

In addition to these posts, a series of other posts have been published by Stephen Duckett and Brendan Coates:

Does Covid raise everyone’s relative risk of dying by a similar amount? More evidence

Written by Prof. David Spiegelhalter (author of the great book “The Art of Statistics”), this post looks at relative mortality rates by age and gender, using data from the U.K.’s Office for National Statistics.

Tracking Coronavirus: Building Parameterized Reports to Analyze Changing Data Sources

In this post, using Johns Hopkins data, Tyler Sanders builds a virus dashboard that can be updated each day with just the click of a button as an example of how to build parameterized reports with R Markdown.

Mapping NZ cases of COVID-19

In this post, using his own choropleth package ({DHBins}) and its hexagonal bins, Thomas Lumley maps cases of COVID19 by Health Boards in New Zealand.

Visualize the Pandemic with R #COVID-19

In this post, Xinhan Qian carries out a variety of explorations with Covid19 data, including the precipitous declines in U.S. movie box office revenue and restaurant reservations.

Exploring the Temporal Evolution of COVID-19 Cases in the United States

Written by Robert Winkelman and Colin Waltz, this post shows clearly how to create animated plots of the spread of COVID19 infections in the United States.

R Data Analysis: COVID-19

Published on the blog of Sharp Sight, this series of blog posts (part 1, 2, 3, 4, 5 and 6) explain how to rename and reorder columns, standardize dates (with the {lubridate} package), merge datasets, take other preparatory steps, plot with the {ggplot2} package and finally, reproduce in R a plot that shows the relative progress of 16 countries in coping with the pandemic.

All posts use the Johns Hopkins data and data from the company’s own collection.

GA COVID-19 Reports

Based on data from the GA Department of Public Health’s report, Andrew Benesh posts a daily analysis of the U.S. state Georgia’s cases, deaths, ICU usage etc.

All his reports are posted on Medium and the code is available here.

Corona in Belgium

Published by bnosac, this post covers the early exploration of the exponential spread of Covid19, with a focus on Belgium and the Netherlands.

The Coronavirus in Italy from the Twitter’s Point of View

Written by the Kode team, these two posts (here and here) use text-mining techniques to analyze tweets in Italy early in the pandemic and on speeches by the Prime Minister of Italy regarding Covid19.

They also published an interactive dashboard (in Italian) allowing to explore the data released daily by the Civil Protection.

Use R and Tidycensus to Look at COVID-19 Risk Factors

Written by René F. Najera, this post uses US census data to identify “overcrowded” areas and considers them in terms of Covid19 risk.

Animating U.S. COVID-19 hotspots over time

Written by Nathan Chaney, this post presents an animated map of the 7-day rolling average of new COVID-19 cases in US. Code for the animated map is available directly at the end of the post.

This post is an extension of his previous post on visualizing COVID-19 in Arkansas.

Understanding COVID19 in Connecticut. It takes a town

Based on these two posts by Nathan Chaney, Chuck Powell shows in his post how to create an animated map of 7-day rolling average of new COVID19 cases per 100,000 in Connecticut (by town).

Data

2019 Novel Coronavirus COVID-19 (2019-nCoV) Data Repository by Johns Hopkins CSSE: this dataset is used by many resources mentioned in this article and has become the gold standard for COVID-19 modeling
World Health Organization (WHO). See also their accompanying Shiny app
COVID-19 Open Research Dataset Challenge (CORD-19) (via Kaggle)
Novel Corona Virus 2019 Dataset: Day level information on Covid-19 affected cases (via Kaggle)
COVID-19 Data from the European Centers for Disease Control
The COVID Tracking Project
Tidying the John Hopkins Covid-19 data to long format and merging some World Bank data
A Short Review of COVID-19 Data Sources
COVID-19 datasets by esri
beoutbreakprepared/nCoV2019: one of the very few non-aggregated dataset available online. Such a dataset of individual-level information on patients with confirmed COVID-19, (including their travel history, location, symptoms, reported onset and confirmation dates and basic demographics) is important to understand, among others, transmissibility, risk of geographic spread, routes of transmission and risk factors for infection
Fighting the Covid-19: All the datasets and data efforts in one place: this post gathers many relevant datasets and data efforts
A Google Sheet which helps with tracking of the local lockdowns in the UK

Other lists or collections of resources

With so many great resources about the Coronavirus, other people also collected and organized similar lists.³ Below some collections I have been fortunate enough to discover:

COVID-19 Blog Post Directory: developed by Connor Rothschild and Rees Morrison, this Shiny app lets users interactively search a collection of over 400 posts by primary topic, post title, date, and whether the post uses a particular mathematical technique or data source. See also the accompanying blog post.
COVID-19 Modelling Resources, Data and Challenges by IDEA (not only R)
GitHub repo covid19-r by Mine Cetinkaya-Rundel (only R)
Amplifying people I trust on COVID-19: written by Jeff Leek from Simply Statistics, this article is a collection of trustworthy people and experts who share good information about the COVID-19 pandemic
Some Select COVID-19 Modeling Resources and More Select COVID-19 Resources by Joseph Rickert, assembling dashboards, Shiny apps, blog posts, packages, datasets, videos and conference proceedings that pertain to the Covid19 pandemic
CovidR Contest: launched by the European R users meeting (eRum), this contest is an open-source contest and pre-conference event, featuring any work done with R around the topic of the COVID-19 pandemic
Turning COVID-19 into a data visualization exercise for your students: written by Daniela Duca, this blog post presents a variety of methods to visualize data, drawing on several blog posts, Shiny apps and dashboards
Outbreak.info is an open source tool built by Scripps Research that standardizes and aggregates COVID-19 data. The interface aggregates journal articles, preprints, datasets, clinical trials, protocols, and other resources in one place, standardizing metadata and applying NLP to make these sources searchable and more accessible. The interactive data dashboards allow for quick comparison of countries, states, counties, and metro areas, and includes an API and R Package for researchers who want to access all of the raw data

I hope that, in addition to my collection, these rich lists done by others will give you enough background materials to analyze the outbreak of COVID-19 on your own (or at least some ideas)!

Non-english resources

This section may be of interested to only a limited number or people, but still, there are great resources in languages other than English. See a collection of them below listed by language:

Japanese:
- Coronavirus infection bulletin: original version of this dashboard
German:
- Developed by Prof. Dr. Helmut Küchenhoff, this CoronaMaps presents the situation of the Coronavirus in the world, in Europe and in Germany via a map and a table
Spanish:
- GitHub repository with official government data and R code used to extract it, see here and here
- COVID-19 en Chile: this Shiny app shows the accumulated confirmed contagion cases and provides an estimate of the growth rate for each municipality
- Developed by Que Oferton, this Shiny app provides an overview of the 2019 Novel Coronavirus COVID-19 epidemic in Central America, including statistics, forecast, SIR and SEIR models. The data and dashboard are refreshed on a daily basis
French:
- Written by Arthur Charpentier, this blog post shows how to quantify excess mortality using french mortality data

Conclusion

Thanks for reading.

I hope you will find these R resources on the COVID-19 Coronavirus useful. Feel free to let me know in the comments if I missed one.

A special thanks to Rees Morrison for his tremendous work on collecting and organizing several articles, which greatly helped in improving the section about blog posts. Read his articles (part 1 and 2) presenting a descriptive analysis of all the posts collected.

Although I have carefully read all resources, inclusion on the list does not mean that I endorse the findings. Moreover, some of the analyses, code, dashboards, packages or datasets might be out of date, so these should not be viewed, by default, as current findings. If you are the author of one of these resources, do not hesitate to contact me if you see any inconsistency or if you would like to remove it from this article.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

de Vlas, Sake J, and Luc E Coffeng. 2020. “A Phased Lift of Control: A Practical Strategy to Achieve Herd Immunity Against Covid-19 at the Country Level.” medRxiv.

Kissler, Stephen M, Christine Tedijanto, Edward Goldstein, Yonatan H Grad, and Marc Lipsitch. 2020. “Projecting the Transmission Dynamics of SARS-CoV-2 Through the Postpandemic Period.” Science 368 (6493): 860–68.

The package has also been the subject of a preprint.↩︎
See more information about this epidemiological model in this post by Marc Choisy.↩︎
Note that unlike my list, collections by others may include resources on COVID-19 using other tools than R.↩︎

R on Stats and R

McNemar's test in R

Introduction

Data

McNemar’s test

Aim and hypotheses

Assumptions

In R

Interpretations

Summary

Bayesian Neural Networks in {tidymodels} with {kindling}

What Are Bayesian Neural Networks?

From point estimates to distributions

The reparameterisation trick

Is it possible?

Setup

Loading namespace

The train_nnsnip() Interface

Classification: Iris

The drawbacks

Updating the model

References

You can do more for neural networks in R with {kindling}

Why this post matters

What problem {kindling} solves (and what it does not)

Setup

Three levels of interaction

1) Generate model code (*_generator())

2) Direct Training

3) ML Framework Integration: {tidymodels} with mlp_kindling() / rnn_kindling()

Example 1: Iris classification with reproducible good practice

Data Preprocessing

MLP with 2 Hidden Layers

Why these choices?

Example 2: A more realistic tabular benchmark (Sonar)

About early stopping and callbacks

Supported architectures (current scope)

Variable importance (FFNN)

When {kindling} is a good fit

Limitations to keep in mind

Takeaways

nycOpenData: A unified R interface to NYC Open Data APIs

nycOpenData: A unified R interface to NYC Open Data APIs

Why nycOpenData?

What does the package do?

Mini analysis

Designed for reproducible workflows

Who is it for?

Availability

Acknowledgements

AssociationExplorer: A user-friendly shiny application for exploring associations and visual patterns

Why AssociationExplorer?

What does the app do?

A paper published in SoftwareX

The R package is also on CRAN

Who is it for, and how can it be used?

Acknowledgements

Useful links

References

Nonparametric serial interval estimation

Motivation

Simulated data

Real data

References

EpiLPS for estimation of incubation times

Motivation

Coarse data

Simulated example

Real data example

References

Binary logistic regression in R

Introduction

Linear versus logistic regression

Univariable versus multivariable logistic regression

Data

Binary logistic regression in R

Univariable binary logistic regression

Quantitative independent variable

Qualitative independent variable

Multivariable binary logistic regression

The `train_nnsnip()` Interface

What problem `{kindling}` solves (and what it does not)

1) Generate model code (`*_generator()`)

3) ML Framework Integration: `{tidymodels}` with `mlp_kindling()` / `rnn_kindling()`

When `{kindling}` is a good fit