Probability on Stats and R

What is the probability that two persons have the same initials?

Wed, 06 Dec 2023 00:00:00 +0000

Introduction

Last week, I joined a team to work on a collaborative project. The team was already established for a few months, with several scientists working together on the project. For simplicity, they used to sign documents, mention colleagues in emails, etc. with their initials (the first letter of their first name followed by the first letter of their last name).

A couple of days after joining the project, when I needed to sign my first document with my initials, we realized that another person in the team had the exact same initials than me.

This was not really an issue, as we decided that I would write my initials backward, that is, “SA” instead of “AS”, and the other person would keep signing with “AS” as usual.

It could have stopped here. However, the idea to write a post about this rather trivial anecdote came to me when the team leader claimed, in the middle of a meeting: “That’s very unfortunate that you two have the same initials! What are the chances of this happening to us?!”.

We spent a couple of minutes trying to estimate this probability, which in the end were mostly based on our intuitions rather than on a formal calculation. This piqued my curiosity.

Given that the project we are working on requires the use of simulations, I decided to focus on answering this question via simulations in R. That being said, as for most simulations, it is a good practice to verify these results. This is done using probability theory. This comparison will allow to assess the truthfulness of results obtained through simulations.

Furthermore, I thought that it would be a nice way to illustrate methods not often presented in my posts: for loops, replications and writing functions in R.

How likely is it?

Before answering the question raised by the team leader, there are three things to note:

Although the team leader was curious to know the probability that exactly two persons have the same initials, we are actually more interested in the probability that at least two persons have the same initials (as the problem also occurs if more than two persons within a team have the same initials).
The team consists of 8 people.
We restrict ourselves to two-letters initials (the first letter being the first letter of the first name, the second letter being the first letter of the last name). This means that middle names are not taken into account, and only the first letter is considered for compound names.

In this post, we will show how to compute this probability:

in our context, that is, for a team of 8 persons, and
for completeness, for teams of all sizes from 2 to 100 persons.

As stated in the introduction, we will compute these probabilities first through simulations and then through probability theory.

For our team

We start by creating a vector of size 8, corresponding to the initials of a team of 8 persons randomly sampled among all 26 letters of the Latin alphabet:

# number of persons
n_persons <- 8

# create vector of initials
initials <- replicate(
  n = n_persons, # number of replications
  paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)) # sample letters
)

# display initials
initials

## [1] "UJ" "MN" "XD" "CY" "BB" "ZB" "CU" "HQ"

# are there duplicates?
any(duplicated(initials))

## [1] FALSE

As we can see, everyone has different initials in this simulated team of 8 persons, but this will not always be the case.

To estimate, via simulations, how likely is that at least two persons have the same initials among the team, we need to replicate this vector of 8 sampled initials a large number of times (say 1,000 replications):¹

# number of replications
reps <- 1000

# create and save replications
dat <- replicate(
  n = reps, # number of replications
  replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
)

# dimensions
dim(dat)

## [1]    8 1000

# display first 4 simulated teams
dat[, 1:4]

##      [,1] [,2] [,3] [,4]
## [1,] "VA" "BU" "LU" "PT"
## [2,] "JG" "SM" "HM" "OL"
## [3,] "BY" "NA" "VJ" "OT"
## [4,] "RT" "CM" "WT" "YT"
## [5,] "PS" "CT" "NB" "QJ"
## [6,] "MG" "KR" "SV" "US"
## [7,] "PL" "SN" "PN" "XW"
## [8,] "NJ" "BR" "DD" "ZC"

The result is a matrix of 8 rows and 1000 columns, where:

each rows corresponds to the sampled initials of a person, and
each column corresponds to one simulated team of 8 people.

For better readability, we rename:

the row names as M1 to M8, corresponding to persons 1 to 8, and
the column names as T1 to T1000, corresponding to teams 1 to 1000.

# rename rows
rownames(dat) <- paste0("M", 1:n_persons)

# rename columns
colnames(dat) <- paste0("T", 1:reps)

# display first 4 simulated teams
dat[, 1:4]

##    T1   T2   T3   T4  
## M1 "VA" "BU" "LU" "PT"
## M2 "JG" "SM" "HM" "OL"
## M3 "BY" "NA" "VJ" "OT"
## M4 "RT" "CM" "WT" "YT"
## M5 "PS" "CT" "NB" "QJ"
## M6 "MG" "KR" "SV" "US"
## M7 "PL" "SN" "PN" "XW"
## M8 "NJ" "BR" "DD" "ZC"

We now need to compute, among the 1000 teams simulated, how many of them have at least two persons with the same initials:

# transform to data frame
dat <- as.data.frame(dat)

# save which teams have duplicates
duplicates <- rep(NA, reps) # create empty vector
for (i in 1:reps) { # for loop over i from 1 to 1,000
  duplicates[i] <- any(duplicated(dat[, i])) # save results TRUE/FALSE in duplicates vector
}

# count how many teams have duplicates
sum(duplicates)

## [1] 41

Here, for each column of our data frame dat (from the first to the 1000th column), we ask whether there are duplicates or not. This is done repeatedly over all columns thanks to a for loop. For each column, the result is TRUE if there are duplicates, otherwise it is FALSE. The result of each iteration is saved in the duplicates vector. As TRUE = 1 and FALSE = 0 in R, we can then count how many columns (and thus teams) have duplicates by summing the number of TRUE in the duplicates vector.

As we can see from the output above, among the 1000 simulated teams, 41 of them have duplicates, that is, 41 of them have at least two persons with the same initials.

Therefore, based on the simulations, we can expect the probability that at least two persons with the same initials in a team of 8 persons to be close to 4.1%.

This is a good starting point. Notice, however, that I wrote close to 4.1% because this probability will vary each time it is computed via simulations.

For instance, if we repeat the exact same process a second time:

# create and save replications
dat <- replicate(
  n = reps, # number of replications
  replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
)

# transform to data frame
dat <- as.data.frame(dat)

# save which teams have duplicates
duplicates <- rep(NA, reps) # create empty vector
for (i in 1:reps) { # for loop over i from 1 to 1,000
  duplicates[i] <- any(duplicated(dat[, i])) # save results in the duplicates vector (as TRUE/FALSE)
}

# count how many teams have duplicates
sum(duplicates)

## [1] 44

We now find a probability of 4.4%. This is not an error, but it is due to randomness when sampling initials.

Luckily, we can make the computation of this probability more robust thanks to replications. Intuitively, it works as follows. We repeat the same computation multiple times, giving us a range of possible probabilities. This allows us to assess the uncertainty of our result, and understand how the probability might vary due to taking different random samples of initials.

So the goal is to compute our probability multiple times (say 100 times), and see its distribution.

To repeat the same computation multiple times, it is best to write a function in order to avoid copy pasting the same code over and over. So we first write a function (called initials) which computes the probability that at least two persons share the same initials among a team of \(n\) people:

initials <- function(n_persons, reps = 1000) {
  # simulate data
  dat <- as.data.frame(replicate(
    reps,
    replicate(n_persons, paste0(sample(LETTERS, size = 1), sample(LETTERS, size = 1)))
  ))

  # save which teams have duplicates
  duplicates <- rep(NA, reps)
  for (i in 1:reps) {
    duplicates[i] <- any(duplicated(dat[, i]))
  }

  # proportion of teams with duplicates
  return(mean(duplicates))
}

A function in R requires to include:

the parameters inside (), and
the computation inside {}.

We can then use our function to compute the probability that at least two persons share the same initials among a team of 8 people. And we combine it with the replicate() function to compute this probability 100 times.

# compute and save probabilities
probs <- replicate(100, initials(n_persons = 8))

# display probabilities
probs

##   [1] 0.032 0.037 0.040 0.043 0.033 0.042 0.039 0.047 0.045 0.038 0.052 0.042
##  [13] 0.042 0.040 0.023 0.044 0.041 0.039 0.036 0.048 0.041 0.037 0.027 0.030
##  [25] 0.052 0.038 0.043 0.035 0.038 0.045 0.047 0.044 0.030 0.036 0.036 0.048
##  [37] 0.038 0.045 0.044 0.034 0.031 0.043 0.045 0.034 0.049 0.047 0.051 0.036
##  [49] 0.051 0.040 0.043 0.044 0.038 0.049 0.043 0.050 0.035 0.043 0.048 0.038
##  [61] 0.041 0.044 0.039 0.045 0.033 0.057 0.036 0.043 0.041 0.041 0.041 0.041
##  [73] 0.038 0.044 0.031 0.034 0.049 0.041 0.040 0.034 0.032 0.036 0.049 0.047
##  [85] 0.048 0.038 0.038 0.037 0.036 0.037 0.043 0.040 0.026 0.049 0.046 0.044
##  [97] 0.048 0.038 0.026 0.029

Finally, we visualize the distribution of these 100 probabilities thanks to a histogram and a boxplot (with the {ggplot2} package):

# visualize distribution of the computed probabilities
# build and save plots
library(ggplot2)

p1 <- ggplot(mapping = aes(x = probs)) +
  geom_histogram(color = "black", fill = "steelblue", bins = 8) +
  labs(
    x = "Probabilities",
    y = "Frequencies"
  ) +
  scale_x_continuous(labels = scales::percent) # format x-axis in %

p2 <- ggplot(mapping = aes(x = probs)) +
  geom_boxplot(color = "black", fill = "steelblue") +
  labs(x = "Probabilities") +
  theme(
    axis.text.y = element_blank(),
    axis.ticks.y = element_blank()
  ) +
  scale_x_continuous(labels = scales::percent) # format x-axis in %

# combine plots
library(patchwork)

p1 + p2

These two plots show that the probability that at least two persons share the same initials among a team of 8 people is most likely between 3.5% and 4.5%.

For the record, during the meeting at the root of all this thinking, most of us thought that it was much less likely. Indeed, I believe we were tempted to compute the probability that someone who joins the team has “AS” as initials. This is indeed much less likely, as the probability is only \(\frac{1}{26} \times \frac{1}{26} \simeq 0.15\%\).

However, this does not take into account the fact:

that the newcomer can have the same initials as any other person, and
that it is not only the newcomer who can have the same initials as another person (2 people already working in the team when the newcomer arrives could have the same initials as well).

If you are puzzled by this finding, I recommend reading about the birthday’s paradox. The birthday’s paradox states that the probability of two people sharing the same birthday becomes surprisingly high with a relatively small group of individuals. In practice, in a group of just 23 people, there is a greater than 50% chance that at least two individuals share the same birthday, illustrating our counterintuitive intuitions about the likelihood of such coincidences. This phenomenon arises due to the multitude of possible birthday pairs within the group, similar to the multitude of possible pairs if initials within a team.

For teams of different sizes

We are now interested in computing this probability not just for a team of 8 persons, but for teams of different sizes. We can do this with the help of our function defined earlier.

For the illustration, let’s compute the probability that at least two persons have the same initials, for teams of 2 and up to 100 persons:

# set lower and upper bounds of number of persons
min_persons <- 2
max_persons <- 100

# create empty vector of probabilities
probs <- rep(NA, length(min_persons:max_persons))

# compute and save probabilities for teams of size 2 to 100
for (i in min_persons:max_persons) {
  probs[i] <- initials(n_persons = i)
}

# display probabilities
probs

##   [1]    NA 0.001 0.005 0.013 0.012 0.019 0.036 0.040 0.047 0.057 0.074 0.083
##  [13] 0.103 0.128 0.158 0.166 0.178 0.215 0.232 0.260 0.275 0.296 0.300 0.329
##  [25] 0.357 0.392 0.405 0.405 0.439 0.478 0.495 0.536 0.535 0.563 0.578 0.599
##  [37] 0.653 0.656 0.686 0.693 0.715 0.711 0.767 0.760 0.786 0.784 0.814 0.817
##  [49] 0.825 0.826 0.842 0.845 0.867 0.893 0.901 0.920 0.919 0.911 0.917 0.942
##  [61] 0.950 0.951 0.946 0.947 0.969 0.959 0.965 0.964 0.977 0.984 0.977 0.977
##  [73] 0.985 0.978 0.986 0.981 0.989 0.991 0.989 0.988 0.992 0.993 0.994 0.996
##  [85] 0.997 0.995 0.994 0.999 0.999 0.999 1.000 0.999 0.997 1.000 0.999 0.999
##  [97] 0.999 1.000 1.000 0.999

We are left with storing these probabilities together with the number of persons in the team in a data frame:

# create data frame with saved probabilities and number of persons
dat_plot_sim <- data.frame(
  n_persons = (min_persons - 1):max_persons,
  prob = probs
)

# display first 6 rows
head(dat_plot_sim)

##   n_persons  prob
## 1         1    NA
## 2         2 0.001
## 3         3 0.005
## 4         4 0.013
## 5         5 0.012
## 6         6 0.019

Of course, two people having the same initials in a team of 1 (if we can call this a team…) is impossible.

An event which is impossible has a probability equal to 0. We thus impute this probability in our data frame, in the first row:

# set proba = 0 when n_person = 1
dat_plot_sim[1, 2] <- 0

# display first 6 rows
head(dat_plot_sim)

##   n_persons  prob
## 1         1 0.000
## 2         2 0.001
## 3         3 0.005
## 4         4 0.013
## 5         5 0.012
## 6         6 0.019

Finally, we visualize these probabilities in function of the number of persons in the team:

# visualize probabilities
ggplot(dat_plot_sim) +
  aes(x = n_persons, y = probs) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

From the plot above, we see that the probability that at least two persons have the same initials reaches 50% when the team exceeds around 30 people.

Moreover, notice that this probability becomes close to 100% when the team reaches around 75 people.

Verification

For the sake of completeness, we now compare results obtained through simulations with results obtained from probability theory.²

We first define the function that will be used to compare results found above:

# define function
have_same <- function(s, n) {
  sample_space <- s
  probability <- 1
  for (i in 0:(n - 1)) {
    probability <- probability * (sample_space - i) / sample_space
  }
  1 - probability
}

For our team

# number of possible two-letter initials
n_initials <- 26^2

# apply function
have_same(n_initials, n_persons)

## [1] 0.0407218

The probability that at least two persons have the same initials in a team of 8 is 4.07%. This is close to the probability found with simulations, and within the range of 3.5%–4.5%.

For teams of different sizes

We now compute the probability for teams between 1 and 100 persons:

# compute and save probabilities for teams between 1 and 100 persons
probs <- vector(length = max_persons)
for (i in 1:max_persons) {
  probs[i] <- have_same(n_initials, i)
}

# create data frame with saved probabilities and number of persons
dat_plot_theory <- data.frame(
  n_persons = (min_persons - 1):max_persons,
  prob = probs
)

# display first 6 rows
head(dat_plot_theory)

##   n_persons        prob
## 1         1 0.000000000
## 2         2 0.001479290
## 3         3 0.004433493
## 4         4 0.008851688
## 5         5 0.014716471
## 6         6 0.022004071

Finally, we visualize these probabilities in function of the number of persons in the team:

# visualize probabilities
ggplot(dat_plot_theory) +
  aes(x = n_persons, y = probs) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

For an easier comparison, we plot probabilities found thanks to simulations and thanks to probability theory on the same plot:

# combine the two data frames into one and add the method as variable
dat_plot_sim$Method <- "Simulations"
dat_plot_theory$Method <- "Theory"
dat_plot_all <- rbind(dat_plot_sim, dat_plot_theory)

# visualize probabilities on same plot
ggplot(dat_plot_all) +
  aes(x = n_persons, y = prob, color = Method) +
  geom_line(linewidth = 1) +
  labs(
    x = "# of persons in the team",
    y = "Probability",
    title = "What is the probability that at least 2 persons have the same initials?"
  ) +
  scale_y_continuous(labels = scales::percent) # format y-axis in %

The plot above shows that results using probability theory are relatively similar to results obtained through simulations, indicating that the simulations are trustworthy.

Conclusion

The initial question, raised during a meeting, was “What is the probability that, among our team consisting of 8 persons, two have the same initials?”.

In this post, we first showed how to compute this probability through simulations in R. Secondly, we verified the veracity of the simulations thanks to probability theory. Furthermore, we illustrated how for loops, replications and writing a function can be used in R to answer a probability problem.

As a side note, it is important to keep in mind that in this post, we assumed the following:

All letters of the alphabet had the same probability of occurring, meaning that all pairs of initials were equally probable. This is probably not the case in reality, as a first and last name starting both with X is not as probable as a first and last name starting respectively with M and K. This bias could be limited by specifying different weights when sampling initials.
We restricted ourselves to two-letters initials. Therefore, for compound first or last names, only the first letter is considered. Middle names are also not taken into account.

Last but not least, note that you will find slightly different results than mine, even if you use the exact same code. This is due to randomness. To replicate results as shown in this post, use set.seed(6).

Thanks for reading.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

You can always use a larger number of replications, but in our case the final result is similar with more replications, and the aim of the post is more to show the development than the final answer.↩︎
I thank Richard for writing the first version of the code used for the verifications.↩︎

Koh-Lanta 2022: the ambassadors probability problem

Mon, 16 May 2022 00:00:00 +0000

Introduction

There is a popular TV show broadcasted in France and the french-speaking part of Belgium called “Koh-Lanta”.

In this show, several adventurers are dropped off on a desert island with almost no food nor equipment (just a personal backpack with their clothes and a small portion of rice). They must learn to survive on the hostile island by building their hut, finding water and food, etc.

Each season, adventurers are divided into teams (called tribes), and the teams compete against each other in games involving ability, strength, thinking and endurance. Every other game, the winning tribe receives some food or survival equipment (something to fish or something to make a fire, for instance). The losing tribe receives nothing. For the other half of the games, each member of the losing tribe has to elect an adventurer. The adventurer with the most votes leaves the show definitely. The winning tribe goes back to its island with all of its members.

At some point during the show, the two competing tribes are grouped together into one single tribe, and it continues this time with each adventurer competing against each other (so they play individually). The winner of the show is the last one to “survive”.

The 2022 season started with 24 adventurers. Just before being grouped together, each tribe has to select an adventurer in the opposing tribe. The two selected adventurers become ambassadors of their tribe. The two ambassadors must then go to another island in order to choose the adventurer who is going to leaves the show definitely. The adventurer selected by the two ambassadors will not be part of the reunification of the two tribes and her adventure stops there.

Of course, both ambassadors want to eliminate a member of the opposing tribe (to arrive at the reunification with the most allies). If the two ambassadors cannot agree, they have to play a game that will determine which of the 2 ambassadors must leave the show. This game is entirely based on luck. For the rest of the article, we call this game the ambassadors’ game.

Before 2022

Here were the rules of the ambassadors’ game before the 2022 season:

There are two identical urns, one in front of each ambassador. Each urn contains exactly one black ball and one white ball. Each ambassador has to draw a ball among the 2 from his urn. Both urns are of course closed, so no one sees which ball is picked (nor which one is not picked). The winner of the game (remember that the winner stays in the show, the loser has to leave definitely) is the one who picks a white ball while the other ambassador draws a black ball. If both ambassadors draw the same ball (both black or both white), the balls are put back in the urns and the game start over (with the exact same conditions) until the two ambassadors draw a ball of different color.

For each draw, there are thus four possible results:

Ambassador from tribe A draws a white ball and ambassador from tribe B draws a black ball: ambassador from tribe A wins.
Ambassador from tribe A draws a black ball and ambassador from tribe B draws a white ball: ambassador from tribe B wins.
Both ambassadors draw a black ball: the game start over.
Both ambassadors draw a white ball: the game start over.

Let’s compute the probability for each result to occur. Since the two events are independent (the fact that ambassador A draws a white ball does not change the probability for ambassador B to draw a white ball makes the two events independent), we can multiply the probabilities to compute the joint probability of the two events.

With \(P_A\) (\(P_B\)) denoting the probability that ambassador from tribe A (B) draws a white ball, we have:

\(P_A \cdot (1 - P_B) = 0.5 \cdot 0.5 = 0.25\)
\((1 - P_A) \cdot P_B = 0.5 \cdot 0.5 = 0.25\)
\((1 - P_A) \cdot (1 - P_B) = 0.5 \cdot 0.5 = 0.25\)
\(P_A \cdot P_B = 0.5 \cdot 0.5 = 0.25\)

The sum of the 4 probabilities gives 1 (i.e., 100%), which makes sense since it covers all possible outcomes.

This means that, on the first draw, each ambassador has a probability of 25% to win the game (outcome 1 for ambassador A, outcome 2 for ambassador B).

Of course, since the game is repeated until there is a winner, probabilities for outcomes 3 and 4 tend, in the long run, to decrease until it becomes null (0%). If this statement is not straightforward to you, think about it like this: if you play that game with your friend up to 100 times, what is the probability that there is still no winner, meaning that you and your friend drew the same ball (never a different color) 100 times in a row. You conceive that it is highly unlikely.

In this context, since both urns are identical and the game is played indefinitely until there is a winner, ambassadors have exactly the same probability of winning that game. It is indeed a 50-50 chance for each of them.

In 2022

Of course, I would not write an article about that ambassadors’ game if it is was this easy and straightforward.

The 2022 season differs from the previous ones in the sense that for each game, the losing tribe receives an additional punishment (called a curse in the show). As a consequence, this year, the two urns at the ambassadors’ game were not identical.

To give you some context, the red tribe won against the yellow tribe in the last game before the ambassadors’ negotiation. The punishment for the yellow tribe was the following:

The yellow tribe (with its ambassador Colin) had an urn with 2 black balls and 1 white ball.
The red tribe (with its ambassador Louana) had an urn with 1 black ball and 1 white ball.

This is indeed a punishment for the yellow tribe since the game is not fair anymore: the ambassador of the red tribe clearly has a higher chance of winning that game compared to the ambassador of the yellow tribe.

For the curious among you, here is how it happened:

The two ambassadors were informed by the presenter Denis Brogniart about the composition of the urns for each tribe.
Knowing that the odds were not in his favour, Colin (ambassador of the yellow tribe), chose not to play the game.¹
They finally agreed on the name of the adventurer who would leave the show (an adventurer of the yellow tribe).

You guessed it by now, the reason for writing this article is of course not to explain you what actually happened—the news sites do it better and much faster than me. The reason is that I wanted to compute the chance of winning the game for each ambassador if they had not agreed on an adventurer to eliminate.²

Moreover, I wanted to compute these probabilities in R through simulation. And why not, reuse the code in case organizers of Koh-Lanta decide to change the rules again in the future.

Even if you do not watch Koh-Lanta (because it is not broadcasted in your country, or you do not like the show), it could be of interest to those of you who want to see:

how a real life example can be transferred into R,
and how a for loop and a function can be used to answer the initial question.

Probabilities computation in R

For the remaining of this article, we denote:

\(p_c\), the probability that Colin (ambassador of the yellow tribe) draws a white ball,
\(p_l\), the probability that Louana (ambassador of the red tribe) draws a white ball,
\(q_c\), the probability that Colin draws a black ball,
\(q_l\), the probability that Louana draws a black ball.

Based on the composition of the urns given by Denis Brogniart, we have:

p_c <- 1 / 3
p_l <- 1 / 2
q_c <- (1 - p_c)
q_l <- (1 - p_l)

First draw

To start easy, let’s first compute the winning probabilities for each ambassador on the first draw only. Remember that to have a winner, balls must be of different colors.

Louana wins if and only if Colin draws a black ball and Louana draws a white ball:

# Louana winning on first draw
l_win <- q_c * p_l
l_win

## [1] 0.3333333

Louana has a 33.33% chance of winning on the first draw.

On the other hand, Colin wins if and only if Louana draws a black ball and Colin draws a white ball:

# Colin winning on first draw
c_win <- q_l * p_c
c_win

## [1] 0.1666667

Colin has a 16.67% chance of winning on the first draw.

You can already see that the game is in favour of Louana, as expected. Let’s see now how it evolves when playing several times.

Second draw

To win exactly on the second draw, it must be a tie on the first draw.

# Louana winning on second draw
tie <- (p_c * p_l) + (q_c * q_l)

l_win2 <- tie * l_win
l_win2

## [1] 0.1666667

# Colin winning on second draw
c_win2 <- tie * c_win
c_win2

## [1] 0.08333333

Louana has a 16.67% chance of winning on the second draw.
Colin has a 8.33% chance of winning on the second draw.

Third draw

To win exactly on the third draw, it must be a tie on the first two draws.

# Louana winning on third draw
l_win3 <- (tie^2) * l_win
l_win3

## [1] 0.08333333

# Colin winning on third draw
c_win3 <- (tie^2) * c_win
c_win3

## [1] 0.04166667

Louana has a 8.33% chance of winning on the third draw.
Colin has a 4.17% chance of winning on the third draw.

A pattern seems to emerge in the code.

Game limited to 3 draws

We can already compute the probabilities of winning for each ambassador as if the game was limited to three draws, by summing the probabilities for each of the first three draws:

# Louana winning on draw 1, 2 or 3
l_win_tot <- l_win + l_win2 + l_win3
l_win_tot

## [1] 0.5833333

# Colin winning on draw 1, 2 or 3
c_win_tot <- c_win + c_win2 + c_win3
c_win_tot

## [1] 0.2916667

Louana has a 58.33% chance of winning if the game is limited to three draws.
Colin has a 29.17% chance of winning if the game is limited to three draws.

Game limited to 5 draws

Now if we compute the probabilities as if the game was limited to 5 draws and generalize the computation to see the pattern even more clearly, we have:

# Louana winning
l_win_tot <- ((tie^0) * l_win) +
  ((tie^1) * l_win) +
  ((tie^2) * l_win) +
  ((tie^3) * l_win) +
  ((tie^4) * l_win)

l_win_tot

## [1] 0.6458333

# Colin winning
c_win_tot <- ((tie^0) * c_win) +
  ((tie^1) * c_win) +
  ((tie^2) * c_win) +
  ((tie^3) * c_win) +
  ((tie^4) * c_win)

c_win_tot

## [1] 0.3229167

Louana has a 64.58% chance of winning if the game is limited to 5 draws.
Colin has a 32.29% chance of winning if the game is limited to 5 draws.

Game limited to 100 draws

We could continue like this for a long time, but for now let’s compute it for up to 100 draws, using a for loop. Using a for loop is necessary here in order to avoid to copy-paste our computations a hundred times.

For the ease of illustration, we compute only the probability of winning for Louana. We will show later on how the probability for Colin can easily be computed.

n_draws <- 100 # number of draws
l_win_tot <- c() # set empty vector

for (i in 1:n_draws) {
  l_win_tot[i] <- ((tie^(i - 1)) * l_win) # prob of Louana winning up to n_draws
  print(paste0("Draw ", i, ": ", sum(l_win_tot))) # print sum of winning up to n_draws
}

## [1] "Draw 1: 0.333333333333333"
## [1] "Draw 2: 0.5"
## [1] "Draw 3: 0.583333333333333"
## [1] "Draw 4: 0.625"
## [1] "Draw 5: 0.645833333333333"
## [1] "Draw 6: 0.65625"
## [1] "Draw 7: 0.661458333333333"
## [1] "Draw 8: 0.6640625"
## [1] "Draw 9: 0.665364583333333"
## [1] "Draw 10: 0.666015625"
## [1] "Draw 11: 0.666341145833333"
## [1] "Draw 12: 0.66650390625"
## [1] "Draw 13: 0.666585286458333"
## [1] "Draw 14: 0.6666259765625"
## [1] "Draw 15: 0.666646321614583"
## [1] "Draw 16: 0.666656494140625"
## [1] "Draw 17: 0.666661580403646"
## [1] "Draw 18: 0.666664123535156"
## [1] "Draw 19: 0.666665395100911"
## [1] "Draw 20: 0.666666030883789"
## [1] "Draw 21: 0.666666348775228"
## [1] "Draw 22: 0.666666507720947"
## [1] "Draw 23: 0.666666587193807"
## [1] "Draw 24: 0.666666626930237"
## [1] "Draw 25: 0.666666646798452"
## [1] "Draw 26: 0.666666656732559"
## [1] "Draw 27: 0.666666661699613"
## [1] "Draw 28: 0.66666666418314"
## [1] "Draw 29: 0.666666665424903"
## [1] "Draw 30: 0.666666666045785"
## [1] "Draw 31: 0.666666666356226"
## [1] "Draw 32: 0.666666666511446"
## [1] "Draw 33: 0.666666666589056"
## [1] "Draw 34: 0.666666666627862"
## [1] "Draw 35: 0.666666666647264"
## [1] "Draw 36: 0.666666666656965"
## [1] "Draw 37: 0.666666666661816"
## [1] "Draw 38: 0.666666666664241"
## [1] "Draw 39: 0.666666666665454"
## [1] "Draw 40: 0.66666666666606"
## [1] "Draw 41: 0.666666666666364"
## [1] "Draw 42: 0.666666666666515"
## [1] "Draw 43: 0.666666666666591"
## [1] "Draw 44: 0.666666666666629"
## [1] "Draw 45: 0.666666666666648"
## [1] "Draw 46: 0.666666666666657"
## [1] "Draw 47: 0.666666666666662"
## [1] "Draw 48: 0.666666666666664"
## [1] "Draw 49: 0.666666666666666"
## [1] "Draw 50: 0.666666666666666"
## [1] "Draw 51: 0.666666666666666"
## [1] "Draw 52: 0.666666666666667"
## [1] "Draw 53: 0.666666666666667"
## [1] "Draw 54: 0.666666666666667"
## [1] "Draw 55: 0.666666666666667"
## [1] "Draw 56: 0.666666666666667"
## [1] "Draw 57: 0.666666666666667"
## [1] "Draw 58: 0.666666666666667"
## [1] "Draw 59: 0.666666666666667"
## [1] "Draw 60: 0.666666666666667"
## [1] "Draw 61: 0.666666666666667"
## [1] "Draw 62: 0.666666666666667"
## [1] "Draw 63: 0.666666666666667"
## [1] "Draw 64: 0.666666666666667"
## [1] "Draw 65: 0.666666666666667"
## [1] "Draw 66: 0.666666666666667"
## [1] "Draw 67: 0.666666666666667"
## [1] "Draw 68: 0.666666666666667"
## [1] "Draw 69: 0.666666666666667"
## [1] "Draw 70: 0.666666666666667"
## [1] "Draw 71: 0.666666666666667"
## [1] "Draw 72: 0.666666666666667"
## [1] "Draw 73: 0.666666666666667"
## [1] "Draw 74: 0.666666666666667"
## [1] "Draw 75: 0.666666666666667"
## [1] "Draw 76: 0.666666666666667"
## [1] "Draw 77: 0.666666666666667"
## [1] "Draw 78: 0.666666666666667"
## [1] "Draw 79: 0.666666666666667"
## [1] "Draw 80: 0.666666666666667"
## [1] "Draw 81: 0.666666666666667"
## [1] "Draw 82: 0.666666666666667"
## [1] "Draw 83: 0.666666666666667"
## [1] "Draw 84: 0.666666666666667"
## [1] "Draw 85: 0.666666666666667"
## [1] "Draw 86: 0.666666666666667"
## [1] "Draw 87: 0.666666666666667"
## [1] "Draw 88: 0.666666666666667"
## [1] "Draw 89: 0.666666666666667"
## [1] "Draw 90: 0.666666666666667"
## [1] "Draw 91: 0.666666666666667"
## [1] "Draw 92: 0.666666666666667"
## [1] "Draw 93: 0.666666666666667"
## [1] "Draw 94: 0.666666666666667"
## [1] "Draw 95: 0.666666666666667"
## [1] "Draw 96: 0.666666666666667"
## [1] "Draw 97: 0.666666666666667"
## [1] "Draw 98: 0.666666666666667"
## [1] "Draw 99: 0.666666666666667"
## [1] "Draw 100: 0.666666666666667"

The probabilities found up to draw 1, 3 and 5 are coherent with what we found in the previous sections. Moreover, we see that from draw 52 onwards, the probability of Louana winning remains constant at 66.67%. This is referred as the limit.

Game limited to the number of necessary draws

Without printing the results of the above for loop, we do not know how many draws are necessary to reach the limit.

Let’s now try to include the information about the number of necessary draws in order to avoid computing unnecessary draws:

n_draws <- 9999 # initial number of draws, set intentionally to a high number
p_tie <- c() # set empty vector for prob of ties
l_win_tot <- c() # set empty vector for prob of Louana winning

# find number of necessary draws:
for (i in 1:n_draws) {
  p_tie[i] <- tie^i # prob of tie for each draw
  limit_ndraws <- sum(p_tie > 2.2e-16) # number of necessary draws
}

# compute Louana winning probabilities with the smallest number of necessary draws
for (i in 1:limit_ndraws) {
  l_win_tot[i] <- ((tie^(i - 1)) * l_win) # prob of Louana winning up to limited number of draws
  print(paste0("Draw ", i, ": ", sum(l_win_tot))) # sum of winning up to limited number of draws
}

## [1] "Draw 1: 0.333333333333333"
## [1] "Draw 2: 0.5"
## [1] "Draw 3: 0.583333333333333"
## [1] "Draw 4: 0.625"
## [1] "Draw 5: 0.645833333333333"
## [1] "Draw 6: 0.65625"
## [1] "Draw 7: 0.661458333333333"
## [1] "Draw 8: 0.6640625"
## [1] "Draw 9: 0.665364583333333"
## [1] "Draw 10: 0.666015625"
## [1] "Draw 11: 0.666341145833333"
## [1] "Draw 12: 0.66650390625"
## [1] "Draw 13: 0.666585286458333"
## [1] "Draw 14: 0.6666259765625"
## [1] "Draw 15: 0.666646321614583"
## [1] "Draw 16: 0.666656494140625"
## [1] "Draw 17: 0.666661580403646"
## [1] "Draw 18: 0.666664123535156"
## [1] "Draw 19: 0.666665395100911"
## [1] "Draw 20: 0.666666030883789"
## [1] "Draw 21: 0.666666348775228"
## [1] "Draw 22: 0.666666507720947"
## [1] "Draw 23: 0.666666587193807"
## [1] "Draw 24: 0.666666626930237"
## [1] "Draw 25: 0.666666646798452"
## [1] "Draw 26: 0.666666656732559"
## [1] "Draw 27: 0.666666661699613"
## [1] "Draw 28: 0.66666666418314"
## [1] "Draw 29: 0.666666665424903"
## [1] "Draw 30: 0.666666666045785"
## [1] "Draw 31: 0.666666666356226"
## [1] "Draw 32: 0.666666666511446"
## [1] "Draw 33: 0.666666666589056"
## [1] "Draw 34: 0.666666666627862"
## [1] "Draw 35: 0.666666666647264"
## [1] "Draw 36: 0.666666666656965"
## [1] "Draw 37: 0.666666666661816"
## [1] "Draw 38: 0.666666666664241"
## [1] "Draw 39: 0.666666666665454"
## [1] "Draw 40: 0.66666666666606"
## [1] "Draw 41: 0.666666666666364"
## [1] "Draw 42: 0.666666666666515"
## [1] "Draw 43: 0.666666666666591"
## [1] "Draw 44: 0.666666666666629"
## [1] "Draw 45: 0.666666666666648"
## [1] "Draw 46: 0.666666666666657"
## [1] "Draw 47: 0.666666666666662"
## [1] "Draw 48: 0.666666666666664"
## [1] "Draw 49: 0.666666666666666"
## [1] "Draw 50: 0.666666666666666"
## [1] "Draw 51: 0.666666666666666"
## [1] "Draw 52: 0.666666666666667"

Final winning probabilities

Remember that the initial question was:

What is the probability of winning the game for each ambassador?

The probability that Louana wins the game can easily be extracted as follows:

sum(l_win_tot)

## [1] 0.6666667

And since the game only stops when there is a winner, the sum of the winning probabilities for Louana and Colin must be equal to 1, so the probability that Colin wins the game is:

c_win_tot <- 1 - sum(l_win_tot)
c_win_tot

## [1] 0.3333333

To summarize:

Louana has 66.67% chance to win the ambassadors’ game.
Colin has 33.33% chance to win the ambassadors’ game.

Visual representations

To visualize the probabilities for each ambassador, we miss the probabilities of winning for Colin so let’s compute them first:

c_win_tot <- c() # set empty vector for prob of Colin winning

# compute Colin winning probabilities with the smallest number of necessary draws
for (i in 1:limit_ndraws) {
  c_win_tot[i] <- ((tie^(i - 1)) * c_win) # prob of Colin winning
  print(paste0("Draw ", i, ": ", sum(c_win_tot))) # print sum of winning
}

## [1] "Draw 1: 0.166666666666667"
## [1] "Draw 2: 0.25"
## [1] "Draw 3: 0.291666666666667"
## [1] "Draw 4: 0.3125"
## [1] "Draw 5: 0.322916666666667"
## [1] "Draw 6: 0.328125"
## [1] "Draw 7: 0.330729166666667"
## [1] "Draw 8: 0.33203125"
## [1] "Draw 9: 0.332682291666667"
## [1] "Draw 10: 0.3330078125"
## [1] "Draw 11: 0.333170572916667"
## [1] "Draw 12: 0.333251953125"
## [1] "Draw 13: 0.333292643229167"
## [1] "Draw 14: 0.33331298828125"
## [1] "Draw 15: 0.333323160807292"
## [1] "Draw 16: 0.333328247070312"
## [1] "Draw 17: 0.333330790201823"
## [1] "Draw 18: 0.333332061767578"
## [1] "Draw 19: 0.333332697550456"
## [1] "Draw 20: 0.333333015441895"
## [1] "Draw 21: 0.333333174387614"
## [1] "Draw 22: 0.333333253860474"
## [1] "Draw 23: 0.333333293596904"
## [1] "Draw 24: 0.333333313465118"
## [1] "Draw 25: 0.333333323399226"
## [1] "Draw 26: 0.33333332836628"
## [1] "Draw 27: 0.333333330849806"
## [1] "Draw 28: 0.33333333209157"
## [1] "Draw 29: 0.333333332712452"
## [1] "Draw 30: 0.333333333022892"
## [1] "Draw 31: 0.333333333178113"
## [1] "Draw 32: 0.333333333255723"
## [1] "Draw 33: 0.333333333294528"
## [1] "Draw 34: 0.333333333313931"
## [1] "Draw 35: 0.333333333323632"
## [1] "Draw 36: 0.333333333328483"
## [1] "Draw 37: 0.333333333330908"
## [1] "Draw 38: 0.333333333332121"
## [1] "Draw 39: 0.333333333332727"
## [1] "Draw 40: 0.33333333333303"
## [1] "Draw 41: 0.333333333333182"
## [1] "Draw 42: 0.333333333333258"
## [1] "Draw 43: 0.333333333333295"
## [1] "Draw 44: 0.333333333333314"
## [1] "Draw 45: 0.333333333333324"
## [1] "Draw 46: 0.333333333333329"
## [1] "Draw 47: 0.333333333333331"
## [1] "Draw 48: 0.333333333333332"
## [1] "Draw 49: 0.333333333333333"
## [1] "Draw 50: 0.333333333333333"
## [1] "Draw 51: 0.333333333333333"
## [1] "Draw 52: 0.333333333333333"

We create a dataset with the probabilities for both ambassadors:

dat <- data.frame(
  Draw = rep(1:limit_ndraws, 2),
  Probability = c(cumsum(l_win_tot), cumsum(c_win_tot)),
  Ambassador = c(rep("Louana", limit_ndraws), rep("Colin", limit_ndraws))
)

We can now visualize these probabilities up to the number of necessary draws:

# load package
library(ggplot2)

# plot
ggplot(dat) +
  aes(x = Draw, y = Probability, colour = Ambassador) +
  geom_line(linewidth = 2L) +
  labs(
    y = "Probability of winning",
    caption = "Source: Koh-Lanta 2022"
  ) +
  scale_y_continuous(labels = scales::percent_format(accuracy = 1), limits = c(0, 1)) +
  theme_minimal()

But I believe the most appropriate plot to answer the initial question is with a barplot:

# create dataset
dat_barplot <- data.frame(
  Ambassador = c("Louana", "Colin"),
  Probability = c(sum(l_win_tot), sum(c_win_tot))
)

# plot
ggplot(data = dat_barplot, aes(x = Ambassador, y = Probability)) +
  geom_bar(stat = "identity", fill = "steelblue") +
  labs(
    y = "Probability of winning",
    caption = "Source: Koh-Lanta 2022"
  ) +
  scale_y_continuous(labels = scales::percent_format(accuracy = 1), limits = c(0, 1)) +
  theme_minimal()

Coded into a function

Let’s try to implement this problem in a function to be able to reuse it with other initial probabilities.

With:

p_a and p_b denoting, respectively, the probability that ambassador A and ambassador B draw a white ball,
n_draws denoting the maximum number of draws that is allowed (default = 9999),

we have the following function:

ambassadors_game <- function(p_a, p_b, n_draws = 9999) {
  q_a <- (1 - p_a)
  q_b <- (1 - p_b)

  a_win <- q_b * p_a
  b_win <- q_a * p_b

  tie <- (p_a * p_b) + (q_a * q_b)

  p_tie <- c() # set empty vector for prob of ties
  a_win_tot <- c() # set empty vector for prob of A winning
  b_win_tot <- c() # set empty vector for prob of B winning

  # find number of necessary draws:
  for (i in 1:n_draws) {
    p_tie[i] <- tie^i # prob of tie for each draw
    limit_ndraws <- sum(p_tie > 2.2e-16) # number of necessary draws
  }

  # compute A and B winning probabilities with the smallest number of necessary draws:
  for (i in 1:limit_ndraws) {
    a_win_tot[i] <- ((tie^(i - 1)) * a_win) # prob of A winning up to limited number of draws
    b_win_tot[i] <- ((tie^(i - 1)) * b_win) # prob of B winning up to limited number of draws
  }

  # save P(A), P(B) and number of necessary draws:
  res <- list(
    "p_a" = sum(a_win_tot),
    "p_b" = sum(b_win_tot),
    "ndraws" = limit_ndraws
  )
  # print results:
  return(res)
}

We test the function to see if it matches results found above.

First, if both ambassadors have identical urns as it was the case before the 2022 season:

ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 2
)

## $p_a
## [1] 0.5
## 
## $p_b
## [1] 0.5
## 
## $ndraws
## [1] 52

The game is indeed fair, with a 50% chance of winning for each ambassador.

Second, with the urns presented to Louana and Colin, but for the first draw only (setting arbitrarily that Louana is ambassador A and Colin is ambassador B):

# first draw only
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 1
)

## $p_a
## [1] 0.3333333
## 
## $p_b
## [1] 0.1666667
## 
## $ndraws
## [1] 1

Third, still with the urns presented to Louana and Colin, but for a game limited to exactly 3 and 5 draws:

# up to 3 draws
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 3
)

## $p_a
## [1] 0.5833333
## 
## $p_b
## [1] 0.2916667
## 
## $ndraws
## [1] 3

# up to 5 draws
ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3,
  n_draws = 5
)

## $p_a
## [1] 0.6458333
## 
## $p_b
## [1] 0.3229167
## 
## $ndraws
## [1] 5

And now, the final verification with the real situation of Louana and Colin in Koh-Lanta 2022:

# Koh-Lanta 2022 situation
out <- ambassadors_game(
  p_a = 1 / 2,
  p_b = 1 / 3
)

out

## $p_a
## [1] 0.6666667
## 
## $p_b
## [1] 0.3333333
## 
## $ndraws
## [1] 52

All results match the ones presented above.

(Note that the probabilities for each ambassador can be extracted as follows:)

# prob ambassador A
out$p_a

## [1] 0.6666667

# prob ambassador B
out$p_b

## [1] 0.3333333

Conclusion

The initial question, coming from the television show Koh-Lanta, was “what is the probability of winning for each participant if they play the ambassadors’ game?”.

In this article, we have shown how to compute these probabilities. Moreover, we have illustrated the process of how a simple probability problem could be generalized to suit many real life situations, and how a for loop and a function could be used to implement a real life situation into R.

Last but not least, I would like to focus on something Denis Brogniart (the well-known presenter of the show) said just after the two ambassadors came back to the island to announce their choice to the other adventurers. He mentioned that, due to the punishment afflicted to the yellow tribe, there was a difference of chances of 16% between Louana and Colin.

This comes naturally from the following two situations:

First situation:

Louana has a 50% chance of picking a white ball.
Colin has a 33.33% chance of picking a white ball.
The difference is 50 - 33.33 = 16.67%, truncated to 16%.

Or, if the game is limited to the first draw only:

Second situation:

Louana has 33.33% chance of winning the game.
Colin has a 16.67% chance of winning the game.
The difference is 33.33 - 16.67 = 16.66%, truncated to 16%.

It is true that, seeing the problem from those two angles, the difference is ~16%. Denis Brogniart is right in mentioning this 16% difference.

However, the difference of chances between the two ambassadors is larger if we see the game from a broader perspective. The rules of the game say that it stops only when there is a winner. From that point of view, as demonstrated above:

Louana has 66.67% chance of winning the game.
Colin has 33.33% chance of winning the game.

In this situation, the difference in probability of winning between the 2 ambassadors is 66.67 - 33.33 = 33.34%!

I must admit that I was not expecting such a large difference between the two ambassadors. Seeing the ambassadors’ game from that perspective looks different to what we were told, or to what we thought before actually computing the probabilities. (Denis Brogniart, if you happen to read this, feel free to let me know whether this relatively large difference was intended or not.)

For those of you who do not watch the show: upon the return of the ambassadors on the island with all adventurers, Colin has been heavily criticized by the members of his tribe. His decision not to play the ambassadors’ game (and the decision to eliminate one member of his tribe with the aim of saving himself) was seen as a betrayal. As a consequence of this, he was eliminated by the reunited tribe directly after that episode.

I am going to conclude this article with the following question:

If you were an ambassador in the 2022 Koh-Lanta season and given that now you know the exact probabilites of winning for each tribe, what would you have done?

Thanks for reading.

As always, if you have any question related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

We will never know if he would have played the game if the chances of winning were equal for both ambassadors. Since I started to watch this television show, I have never seen any ambassadors’ negotiation leading to the ambassadors’ game, they all ended with the designation of an adventurer.↩︎
And to be honest, I wanted to compute these probabilities because while I was watching the show with my girlfriend, she looked at me and asked “what are the probabilities for each of them?”. I am now able to give her a precise answer.↩︎

The 9 concepts and formulas in probability that every data scientist should know

Tue, 03 Mar 2020 00:00:00 +0000

What is probability?

A probability is a number that reflects the chance that a particular event will occur. In other words, it quantifies (on a scale from 0 to 1, or from 0% to 100%) how likely an event is to occur.

Probability is a branch of mathematics that provides models to describe random processes. These mathematical tools allow to establish theoretical models for random phenomena and to use them to make predictions. Like every model, the probabilistic model is a simplification of the world. However, the model is useful as soon as it captures the essential features.

In this article, we present 9 fundamental formulas and concepts in probability that every data scientist should understand and master in order to appropriately handle any project involving probabilities.

1. A probability is always between 0 and 1

The probability of an event is always between 0 and 1 (or 0% and 100%). If we denote the probability that an event A (which could be any event) occurs by \(P(A)\), we have:

\[0 \le P(A) \le 1\]

If an event is impossible: \(P(A) = 0\)
If an event is certain: \(P(A) = 1\)

For example, throwing a 7 with a standard six-sided dice (with faces ranging from 1 to 6) is impossible so its probability is equal to 0. Throwing head or tail with a coin is certain, so its probability is equal to 1.

2. Compute a probability

If the elements of a sample space (the set of all possible results of a randomized experiment) are equiprobable (= all elements have the same probability), then the probability of an event occurring is equal to the number of favourable cases (number of ways it can happen) divided by the number of possible cases (total number of outcomes):

\[P(A) = \frac{\text{number of favourable cases}}{\text{number of possible cases}}\]

For example, all numbers of a six-sided dice are equiprobable since they all have the same probability of occurring. The probability of rolling a 3 with a dice is thus

\[P(3) = \frac{\text{number of favourable cases}}{\text{number of possible cases}} = \frac{1}{6}\]

because there is only one favourable case (there is only one face with a 3 on it), and there are 6 possible cases (because there are 6 faces altogether).

3. Complement of an event

The probability of the complement (or opposite) of an event is:

\[P(\text{not A}) = P(\bar{A}) = 1 - P(A)\]

For instance, the probability of not throwing a 3 with a dice is:

\[P(\bar{A}) = 1 - P(A) = 1 - \frac{1}{6} = \frac{5}{6}\]

4. Union of two events

The probability of the union of two events is the probability of either occurring:

\[\begin{align} P(\text{A or B)} &= P(A \cup B) \\ &= P(A) + P(B) - P(A \cap B) \end{align}\]

Suppose that the probability of a fire breaking out in two houses in a given year is:

in house A: 60%, so \(P(A) = 0.6\)
in house B: 45%, so \(P(B) = 0.45\)
in at least one of the two houses: 80%, so \(P(A \cup B) = 0.8\)

Graphically we have

The probability of a fire breaking out in house A or house B is

\[P(A \cup B) = P(A) + P(B) - P(A \cap B)\] \[= 0.6 + 0.45 - 0.25 = 0.8\]

By summing \(P(A)\) and \(P(B)\), the intersection of A and B, i.e. \(P(A \cap B)\), is counted twice. This is the reason we subtract it to count it only once.

If two events are mutually exclusive (i.e., two events that cannot occur simultaneously), the probability of both events occurring \(P(A \cap B)\) is equal to 0, so the above formula becomes

\[P(A \cup B) = P(A) + P(B)\]

For example, the event “rolling a 3” and the event “rolling a 6” on a six-sided dice are two mutually exclusive events since they cannot both occur at the same time. Since their joint probability is equal to 0, the probability of rolling a 3 or 6 on a six-sided dice is

\[P(3 \cup 6) = P(3) + P(6) = \frac{1}{6} + \frac{1}{6} = \frac{1}{3}\]

5. Intersection of two events

If two events are independent, the probability of the intersection of the two events (i.e., the joint probability) is the probability of the two events occurring:

\[P(\text{A and B)} = P(A \cap B) = P(A) \cdot P(B)\]

For instance, if two coins are flipped, the probability of both coins being tails is

\[P(T_1 \cap T_2) = P(T_1) \cdot P(T_2) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}\]

where \(T_1\) (\(T_2\)) denotes the event that the first (second) coin is tail.

Note that \(P(A \cap B) = P(B \cap A)\).

If two events are mutually exclusive, their joint probability is equal to 0:

\[P(A \cap B) = 0\]

6. Independence of two events

Another important concept in probability is the independence of two events. Formally, the events A and B are independent if and only if

\[P(A \cap B) = P(A) \cdot P(B)\]

If the equality holds, the two events are said to be independent, otherwise the two events are said to be dependent.

In the example of the two coins:

\[P(T_1 \cap T_2) = \frac{1}{4}\]

and

\[P(T_1) \cdot P(T_2) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}\]

so the following equality holds

\[P(T_1 \cap T_2) = P(T_1) \cdot P(T_2) = \frac{1}{4}\]

The two events are thus independent, denoted \(T_1{\perp\!\!\!\perp}T_2\).

In the example of the fire breaking out in two houses (see section 4):

\[P(A \cap B) = 0.25\]

and

\[P(A) \cdot P(B) = 0.6 \cdot 0.45 = 0.27\]

so the following equality does not hold

\[P(A \cap B) \ne P(A) \cdot P(B)\]

The two events are thus dependent (or not independent), denoted \(A \not\!\perp\!\!\!\perp B\).

7. Conditional probability

Suppose two events A and B and \(P(B) > 0\). The conditional probability of A given (knowing) B is the likelihood of event A occurring given that event B has occurred:

\[P(A | B) = \frac{P(A \cap B)}{P(B)}\] \[= \frac{P(B \cap A)}{P(B)} \text{ (since } P(A \cap B) = P(B \cap A))\]

Note that, in general, the probability of A given B is not equal to the probability of B given A, that is, \(P(A | B) \ne P(B | A)\).

From the formula of the conditional probability, we can derive the multiplicative law:

\[P(A | B) = \frac{P(A \cap B)}{P(B)} \text{ (Eq. 1)}\] \[P(A | B) \cdot P(B) = \frac{P(A \cap B)}{P(B)} \cdot P(B)\] \[P(A | B) \cdot P(B) = P(A \cap B) \text{ (multiplicative law)}\]

If two events are independent, \(P(A \cap B) = P(A) \cdot P(B)\), and:

\(P(B) > 0\), the conditional probability becomes

\[P(A | B) = \frac{P(A \cap B)}{P(B)}\] \[P(A | B) = \frac{P(A) \cdot P(B)}{P(B)}\] \[P(A | B) = P(A) \text{ (Eq. 2)}\]

\(P(A) > 0\), the conditional probability becomes

\[P(B | A) = \frac{P(B \cap A)}{P(A)}\] \[P(B | A) = \frac{P(B) \cdot P(A)}{P(A)}\] \[P(B | A) = P(B) \text{ (Eq. 3)}\] Equations 2 and 3 mean that knowing that one event occurred does not influence the probability of the outcome of the other event. This is in fact the definition of the independence: if knowing that one event occurred does not help to predict (does not influence) the outcome of the other event, the two events are by essence independent.

Bayes’ theorem

From the formulas of the conditional probability and the multiplicative law, we can derive the Bayes’ theorem:

\[\begin{align} P(B | A) &= \frac{P(B \cap A)}{P(A)} \\ & \text{(from conditional probability)} \\ &= \frac{P(A \cap B)}{P(A)} \\ & \text{(since } P(A \cap B) = P(B \cap A)) \\ &= \frac{P(A | B) \cdot P(B)}{P(A)} \\ & \text{ (from multiplicative law)} \end{align}\]

which is equivalent to

\[\begin{align} P(B | A) &= \frac{P(B | A) \cdot P(A)}{P(B)} \\ & \text{(Bayes' theorem)} \end{align}\]

Example

In order to illustrate the conditional probability and the Bayes’ theorem, suppose the following problem:

In order to determine the presence of a disease in a person, a blood test is performed. When a person has the disease, the test can reveal the disease in 80% of cases. When the disease is not present, the test is negative in 90% of cases. Experience has shown that the probability of the disease being present is 10%. A researcher would like to know the probability that an individual has the disease given that the result of the test is positive.

To answer this question, the following events are defined:

P: the test result is positive
D: the person has the disease

Moreover, we use a tree diagram to illustrate the statement:

(The sum of all 4 scenarios must be equal to 1 since these 4 scenarios cover all possible cases.)

We are looking for the probability that an individual has the disease given that the result of the test is positive, \(P(D | P)\).

Following the formula of the conditional probability (Eq. 1) we have:

\[P(A | B) = \frac{P(A \cap B)}{P(B)}\]

In terms of our problem:

\[P(D | P) = \frac{P(D \cap P)}{P(P)}\] \[P(D | P) = \frac{0.08}{P(P)} \text{ (Eq. 4)}\]

From the tree diagram, we can see that a positive test result is possible under two scenarios:

when a person has the disease, or
when the person does not actually have the disease (because the test is not always correct).

In order to find the probability of a positive test result, \(P(P)\), we need to sum up those two scenarios:

\[\begin{align} P(P) &= P(D \cap P) + P(\bar{D} \cap P) \\ &= 0.08 + 0.09 \\ &= 0.17 \end{align}\]

Eq. 4 then becomes

\[P(D | P) = \frac{0.08}{0.17} = 0.4706\]

The probability of having the disease given that the result of the test is positive is only 47.06%. This means that in this specific case (with the same percentages), an individual with a positive test has less than 1 chance out of 2 of having the disease!

This relatively small percentage is due to the facts that the disease is quite rare (only 10% of the population is affected) and that the test is not always correct (sometimes it detects the disease although it is not present, and sometimes it does not detect it although it is present).

As a consequence, a higher percentage of healthy people have a positive result (9%) compared to the percentage of people who have a positive result and who actually have the disease (8%). This explains why several diagnostic tests are often performed before announcing the diagnosis, especially for rare diseases.

8. Accuracy measures

Based on the example of the disease and the diagnostic test presented above, we explain the most common accuracy measures:

False negative (number and rate)
False positive (number and rate)
Sensitivity
Specificity
Positive predictive value
Negative predictive value

Before diving into the details of these accuracy measures, here is an overview of the measures and the tree diagram with the labels added for each of the 4 scenarios:

Adapted from Wikipedia

False negatives

The false negatives (FN) are the number of people incorrectly labeled as not having the disease or the condition, when in reality it is present. It is like telling a women who is 7 months pregnant that she is not pregnant.

From the tree diagram, we have:

\[FN = P(D \cap \bar{P}) = 0.02\]

Moreover, the false negative rate (FNR) is defined as

\[\begin{align} FNR &= \frac{FN}{FN + TP} \\ &= P(\bar{P} | D) \\ &= \frac{P(\bar{P} \cap D)}{P(D)} \\ &= \frac{0.02}{0.08 + 0.02} \\ &= 0.2 \end{align}\]

False positives

The false positives (FP) are the number of people incorrectly labeled as having the disease or the condition, when in reality it is not present. It is like telling a man he is pregnant.

From the tree diagram, we have:

\[FP = P(\bar{D} \cap P) = 0.09\]

Moreover, the false positive rate (FPR) is defined as

\[\begin{align} FPR &= \frac{FP}{FP + TN} \\ &= P(P | \bar{D}) \\ &= \frac{P(P \cap \bar{D})}{P(\bar{D})} \\ &= \frac{0.09}{0.09 + 0.81} \\ &= 0.1 \end{align}\]

Sensitivity

The sensitivity of a test, also referred as the recall, measures the ability of a test to detect the condition when the condition is present (the percentage of sick people who are correctly identified as having the disease):

\[ Sensitivity = \frac{TP}{TP + FN}\]

where TP is the true positives.

From the tree diagram, we have:

\[\begin{align} Sensitivity &= \frac{TP}{TP + FN} \\ &= P(P|D) \\ &= 0.8 \end{align}\]

Note also that \(1 - sensitivity = FNR\) (false negative rate).

Specificity

The specificity of a test measures the ability of a test to correctly exclude the condition when the condition is absent (the percentage of healthy people who are correctly identified as not having the disease):

\[Specificity = \frac{TN}{TN + FP}\]

where TN is the true negatives.

From the tree diagram, we have:

\[\begin{align} Specificity &= \frac{TN}{TN + FP} \\ &= P(\bar{P} | \bar{D}) \\ &= 0.9 \end{align}\]

Note also that \(1 - specificity = FPR\) (false positive rate).

Positive predictive value

The positive predictive value, also referred as the precision, is the proportion of positives that correspond to the presence of the condition, so the proportions of positive results that are true positive results:

\[PPV = \frac{TP}{TP+FP}\]

From the tree diagram, we have:

\[\begin{align} PPV &= \frac{TP}{TP+FP} \\ &= P(D | P) \\ &= \frac{P(D \cap P)}{P(P)} \\ &= \frac{0.08}{0.08+0.09} \\ &= 0.4706 \end{align}\]

Negative predictive value

The negative predictive value is the proportion of negatives that correspond to the absence of the condition, so the proportions of negative results that are true negative results:

\[NPV = \frac{TN}{TN + FN}\]

From the tree diagram, we have:

\[\begin{align} NPV &= \frac{TN}{TN + FN} \\ &= P(\bar{D} | \bar{P}) \\ &= \frac{P(\bar{D} \cap \bar{P})}{P(\bar{P})} \\ &= \frac{0.81}{0.81+0.02} \\ &= 0.9759 \end{align}\]

9. Counting techniques

In order to use the formula in section 2, one must know how to count the number of possible elements (both for favorable and possible cases).

There are 3 main counting techniques in probability:

Multiplication
Permutation
Combination

See below how to count the number of possible elements in case of equiprobable results.

Multiplication

The multiplication rule is as follows:

\[\#(A \times B) = (\#A) \times (\#B)\]

where \(\#\) is the number of elements.

Example

In a restaurant, a customer has to choose a starter, a main course and a dessert. The restaurant offers 2 starters, 3 main courses and 2 desserts. How many different choices are possible?

There are \(2 \cdot 3 \cdot 2 = 12\) different possible choices.

Permutation

The number of permutations is as follows:

\[\begin{align} P^r_n &= n \times (n - 1) \times \cdots \times (n - r + 1) \\ &= \frac{n !}{(n - r)!} \end{align}\]

with \(r\) the length, \(n\) the number of elements and \(r \le n\). Note that \(0! = 1\) and \(k! = k \times (k - 1) \times (k - 2) \times \cdots \times 2 \times 1\) if \(k = 1, 2, \dots\)

The order is important in permutations!

Example

Count the permutations of length 2 of the set \(A = \{a, b, c, d\}\), without a letter being repeated. How many permutations do you find?

By hand

\[P^4_2 = \frac{4!}{(4-2)!} = \frac{4\cdot3\cdot2\cdot1}{2\cdot1} = 12\]

In R

library(gtools)

x <- c("a", "b", "c", "d")

# See all different permutations
perms <- permutations(
  n = 4, r = 2, v = x,
  repeats.allowed = FALSE
)
perms

##       [,1] [,2]
##  [1,] "a"  "b" 
##  [2,] "a"  "c" 
##  [3,] "a"  "d" 
##  [4,] "b"  "a" 
##  [5,] "b"  "c" 
##  [6,] "b"  "d" 
##  [7,] "c"  "a" 
##  [8,] "c"  "b" 
##  [9,] "c"  "d" 
## [10,] "d"  "a" 
## [11,] "d"  "b" 
## [12,] "d"  "c"

# Count the number of permutations
nrow(perms)

## [1] 12

Combination

The number of combinations is as follows:

\[\begin{align} C^r_n &= \frac{P^r_n}{r!} \\ &= \frac{n !}{r!(n - r)!} \\ &= {n \choose r} \\ &= \frac{n}{r} \times \frac{n - 1}{r - 1} \times \dots \times \frac{n - r + 1}{1} \end{align}\]

with \(r\) the length, \(n\) the number of elements and \(r \le n\).

The order is not important in combinations!

Example

In a family of 5 children, what is the probability that there are 3 girls and 2 boys? Assume that the probabilities of giving birth to a girl and a boy are equal.

By hand

Count of 3 girls and 2 boys (favourable cases): \(C^3_5 = {5 \choose 3} = \frac{5!}{3!(5-3)!} = 10\)
Count of possible cases: \(2^5 = 32\)

\(\Rightarrow P(3 \text{ girls and 2 boys}) = \frac{\text{# of favourable cases}}{\text{# of possible cases}}\) \[= \frac{10}{32} = 0.3125\]

In R

Count of 3 girls and 2 boys:

choose(n = 5, k = 3)

## [1] 10

Count of possible cases:

2^5

## [1] 32

Probability of 3 girls and 2 boys:

choose(n = 5, k = 3) / 2^5

## [1] 0.3125

Conclusion

Thanks for reading.

I hope this article helped you to understand the most important formulas and concepts from probability theory.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.