Tips on Stats and R

10 potential career options with a degree in statistics

Fri, 24 Mar 2023 00:00:00 +0000

This post has been written in collaboration with Daniel Williams.

Introduction

A reader recently contacted me because he was hesitating between starting his studies in statistics or economics. I studied economics and I am now doing a PhD in statistics, so he reached out to me to know what I thought about these two fields, and in particular, what were my feelings about the career opportunities available with these two degrees.

Before considering pursuing a graduate degree, it is important to have a clear understanding of the skills you will develop and the career opportunities that will become available to you.

I leave it to those who have studied economics and work in this field to discuss career opportunities with a degree in economics. In this post, I will focus on the career options that are available for people with a degree in statistics.

Bear in mind that there are many professional opportunities with a degree in statistics. Indeed, statistical expertise is highly sought after in a wide range of computing and data analysis job roles; as soon as there are data, statistical expertise is required. Moreover, there are almost as many different jobs as people. Therefore, the list of jobs below is non-exhaustive and you may get a job that is not mentioned in this list. However, I hope it will still give you an overview of what you can expect after your studies.

If you believe I missed one—if you studied statistics and your job is not mentioned here for example, feel free to leave a comment at the end of the post.

What types of jobs are available?

If you are skilled in statistics and R programming, there are a plethora of job opportunities available to you.

Some of the roles you can pursue include:

Statistician: Collect, analyze and interpret data in a wide range of fields, including economics, finance, marketing, social and medical sciences.
Data scientist, data/business analyst, data engineer or machine learning engineer: Collect, analyze and interpret large and complex data sets using statistical and machine learning techniques to gain insights and inform decision-making to business and organizations. Mastering these techniques is key to the role, and in industry, turning statistical models into production-ready solutions often requires collaboration with a professional machine learning development company.
Actuary or actuarial analyst: Use statistical modeling to assess risk and uncertainty, and to create financial projections for insurance and investment companies.
Financial (risk) analyst, investment analyst, financial trader, financial manager or quantitative analyst: Use statistical and mathematical modeling to analyze financial and market data, identify and evaluate risks for investment and trading decisions.
Business intelligence analyst: Use data analysis and visualization tools to create reports and dashboards that help businesses make informed decisions.
Operational researcher or quality control analyst: Use mathematical and statistical techniques to monitor and optimize the quality of products and processes in manufacturing.
Market or survey researcher: Conduct research and surveys, gather and analyze data to help businesses develop and improve their marketing strategies.
Economist or econometrician: Use statistical methods to analyze (socio)economic data and develop economic models to inform policy and decision-making.
Teacher: Facilitate learning and provide guidance to students in order to help them acquire knowledge, skills, and values that will prepare them for their future.
(Freelance) consultant: Provide statistical consulting services to clients, helping them solve problems using a combination of statistical methods and business expertise.

In the following sections, we delve deeper into the aforementioned job roles and outline the qualifications expected of prospective candidates.

Statistician

As a graduate in statistics, you have the potential to land a coveted statistician’s job in a reputable private company or public agency. These types of roles typically involve analyzing and assessing data, and utilizing various tools and software to manage data effectively.

With data proliferation and more and more companies/organizations that rely heavily on data analysis, your skills will be valuable in a broad range of fields and industries. Being a statistician means that you could work for instance as an:

environmental statistician: analyze data related to environmental issues, such as climate change and pollution, to inform policy and decision-making,
sports statistician: analyze data related to sports performance, such as player statistics and game outcomes, to inform coaching and strategy decisions,
government statistician: collect, analyze and interpret data for government agencies and public departments to guide policymakers in their decisions,
biostatistician: use statistical methods and data analysis techniques to study and interpret medical and (public) health-related data,
etc.

Statistical expertise is required wherever there is data, so feel free to specialize in the field or industry you care about most!

Data scientist, data/business analyst, data engineer or machine learning engineer

As a relatively new job role in various organizations, data scientists and data analysts are experts in manipulating, analyzing and interpreting data. These roles typically require the analysis of large datasets using statistical methods to assist organizations in making more informed business decisions. These professionals dive into unstructured information to uncover valuable insights for businesses, ultimately increasing their revenue.

To pursue this career, one would usually require a graduate degree in science, statistics, or mathematics, as well as training in data mining. It is also expected that you are familiar with one or several programming languages such as R, Python or SAS.¹

Statistics graduates can also pursue their career as machine learning engineers. Job roles in this field typically involve conducting experiments and implementing machine learning algorithms. Consequently, many organizations hire these professionals to create a wide range of AI products. To do so, they must possess strong programming and statistical skills. Data science and software engineering knowledge are also beneficial.

Actuary or actuarial analyst

Actuaries usually apply statistical modeling tools on data related to retirement, financial/insurance products, accidents or mortality. As an actuary, one would be responsible for analyzing financial costs and risks. It means examining uncertainties associated with investments and other financial products and making predictions about the risks involved in a particular venture.

Actuaries are also in charge of writing insurance proposals, determining policy terms, and calculating premiums for different products, among other responsibilities. You may also play a vital role in deciding whether to accept or reject insurance applications by evaluating their risks. You will draw upon your statistical, actuarial, and background information to accomplish these tasks.

Typically, actuaries have a range of financial companies as clients, and they must identify potential risks and recommend compensation strategies accordingly. It means that their work is closely related to insurance products, as they must assess the likelihood of certain events occurring and the resulting financial impact.

Typically, those who pursue this role hold a degree in actuarial science, mathematics or statistics, and are trained in statistical analysis tools and software. Note that each country has its own specific accreditation related to this profession.

Financial (risk) analyst, investment analyst, financial trader, financial manager or quantitative analyst

As a financial analyst, one is required to gather data from various sources, organize and analyze historical results, and make projections and forecasts. Furthermore, in addition to analyzing data, their role often involves reporting financial results to the board of directors/management to help them in setting the overall strategy and direction of the company.

Investment analysts are experts in evaluating different financial assets such as stocks, securities, and bonds. They do not stop there, though. They also conduct research and make crucial decisions about purchasing business financials.

As a financial trader, you will have a keen understanding of financial markets and the ability to execute trades (i.e., buy or sell shares, bonds, and other assets) on behalf of clients. Note that there are also various sub-roles to explore within this job segment.

Financial managers combine their passion for finance with their love of numbers and statistics. Financial managers are experts at creating and interpreting complex financial reports, providing invaluable insights into a company’s financial health. They can also advise management on investment strategies and assist with vital financial decisions. With their keen analytical skills, financial managers keep a watchful eye on daily financial activities and help ensure a company’s long-term success.

These skills are in high demand in financial organizations, such as banks, actuarial firms, insurance companies (commercial insurance, reinsurance, general insurance, life insurance, car insurance, etc.), and other similar establishments.

The educational background of these professionals is diverse. Graduates in finance, mathematics, and statistics all bring unique perspectives to the table.

Business intelligence analyst

A business intelligence (BI) analyst is responsible for collecting, organizing, analyzing, and presenting large amounts of data from various sources such as databases, spreadsheets, and software applications. They use various tools and techniques to identify trends, patterns, and relationships in the data to create reports, dashboards, and visualizations to communicate their findings to stakeholders. They may also be responsible for monitoring business performance metrics and KPIs, identifying areas for improvement, and making recommendations to optimize business operations.

Additionally, BI analysts may also be involved in the development and implementation of data-driven software applications, such as business intelligence platforms and data warehouses, to improve data accessibility, accuracy, and efficiency. Overall, the role of a BI analyst is crucial in filling the gap between business analysts and the IT team.

The most common software tools and platforms used by BI analysts are, at the time of writing this post, R Shiny, Microsoft Power BI, Tableau, QlikView and SAS BI.

Operational researcher or quality control analyst

Operational researchers are responsible for using mathematical and analytical methods to solve complex problems and optimize business operations. They collect and analyze data, develop models and algorithms, test and validate them, and make recommendations for improvements.

On the other hand, quality control analysts are responsible for ensuring that products or services meet the required standards of quality. They monitor and analyze product or service performance, identify areas for improvement, and develop strategies for maintaining or improving quality. They may also work with production teams to develop and implement quality control processes and procedures.

While operational researchers focus on optimizing processes and systems, quality control analysts focus on ensuring that the end product or service meets the required quality standards. Both roles require analytical skills and the ability to work with data, but operational researchers focus more on mathematical modeling and optimization, while quality control analysts focus more on monitoring and improving quality.

Individuals who pursue job roles of this nature typically possess strong mathematical abilities, often having graduated with degrees in statistics or mathematics. They are trained to analyze vast amounts of data and are well-versed in various analytical tools, such as simulation, mathematical modeling, and data science.

Market or survey researcher

Market or survey researchers are professionals who specialize in conducting research to gather information about consumer behavior and preferences, market trends, and competitive landscapes. They use various research methods and techniques to collect and analyze data, and provide insights and recommendations to businesses and organizations.

Market research professionals typically work with marketing agencies on various projects for clients across different sectors. In addition to a degree in statistics, knowledge in marketing and familiarity with the industry is beneficial.

Economist or econometrician

Economists and econometricians study and analyze economic systems, markets, and policies using quantitative methods and models. They collect and analyze data, develop economic models, conduct economic analysis, provide recommendations to policymakers and organizations, and communicate their findings to a variety of audiences. Their role is to inform economic policies and decisions in both public and private sectors.

If you have a background in statistics in addition to knowledge in economics, there is a whole world of vocational opportunities in economics just waiting for you. Indeed, armed with the ability to appropriately analyze data (thanks to your background in statistics) and the ability to understand (socio)economic issues and financial data, you will be a hot commodity as an advisor to governments and businesses on all economic decisions.

(Freelance) consultant

From a general point of view, a consultant (or freelance consultant) is a professional who provides expert advice and guidance to organizations or individuals on a contractual basis. They analyze problems, develop solutions, and provide recommendations to clients in a specific field or industry.

A consultant specializing in statistics provides expert advice and guidance to organizations or individuals on statistical analysis, data management, and modeling. They use statistical tools and techniques to analyze and interpret complex data, and provide insights and recommendations based on their analyses. The role of a statistical consultant may include identifying research questions, designing studies, collecting and managing data, analyzing data using appropriate statistical methods, interpreting results, and presenting findings to clients. They may also assist clients with the implementation of statistical methods and tools, provide training on statistical software and techniques, and develop custom statistical models for specific applications.

As a side note, note that many consultants in statistics provide their service as data visualization consultants. These consultants are responsible for helping clients to effectively communicate complex data through the creation of clear and visually appealing graphics, charts, and other forms of visual representation that can aid in decision-making and enhance understanding.

You can either be a consultant in a consulting firm, or a freelance consultant. If you choose to be a freelance consultant, you can more easily select the projects you want to work on and the industry you want to focus on, but you will also need to take care of all the administrative tasks such as finding and building relationships with clients, do your bookkeeping, etc.

If you are not ready to be a full-time freelance consultant but would like to experience a taste of it, you can always keep your primary job and accept a couple of side projects. This is known as a side hustle. This way, you will gradually build your portfolio and create relationships with clients, all that with the advantage of keeping a safe source of income. This is what I am doing with datanalyze.be, and I recommend it to anyone who is considering the option of being a freelance in the future.

Teacher

If you care about education, you like to transmit your knowledge and explain complex things in a simple manner, you may be interested in being a teacher in high school or a university professor. With a degree in statistics, you may be eligible to teach different subjects such as mathematics, statistics, physics, or science in general. Your ultimate goal is to educate and inspire students to develop an understanding and appreciation of these subjects (which are often not well appreciated by students), as well as to develop critical thinking skills, problem-solving abilities, and scientific literacy.

Like actuaries, note that each country has its own specific accreditation related to this profession.

To know whether this job suits you, you can start by being a private tutor. This way, you will experience what the job is like and you will be more able to tell whether it is the direction you want to take.

To become a university professor, note that a PhD (and even sometimes a postdoc) is usually required. This brings me to the last career option I would like to mention: a PhD.

I do not include a PhD in the list of the 10 potential career options because it is more seen as an additional degree rather than a job, and even for those who see it as a job, it is a temporary one. However, I would still like to mention it because it is worth considering with a degree in statistics (I am of course biased, but my biased opinion may be of interest to some readers).

PhD student

For those who enjoy academic research, pursuing a career as a researcher after taking a statistics degree may be a worthwhile consideration. Typically, pursuing a PhD in statistics involves working in close collaboration with one or two university professors (your supervisors) on a specific topic.

I cannot speak for all PhD students, but I can speak from my experience. It involves conducting original research in statistical theory and methodology, developing new statistical models and methods, and applying statistical techniques to solve real-world problems or to advance knowledge in a specific field. My research focuses on applying biostatistical procedures to cancer patients, so it is a rather applied PhD, but many of my colleagues work on a more theoretical subject.

We also sometimes participate in academic conferences, present our research findings, and collaborate with other researchers in our field of research.

After completing our PhD, we can continue to evolve in academia (and become a university professor for instance), or decide to work in a private or public research organization. Because professionals with a PhD in statistics become experts in statistical theory and methodology, commercial and public organizations often seek such profiles.

The length of a PhD depends on the country and the type of contract. If you are interested to know more, feel free to contact me, I might be able to help you (at least with how it goes in Belgium). Otherwise, the program director at your university will definitely be able to answer the questions you may have.

Conclusion

As you have seen, with a degree in statistics and knowledge in programming, you will be able to pursue positions at various companies or organizations. You can also always enhance your qualifications with additional courses if you feel that you miss some skills that are required for the job you want to apply to.

Statisticians are in demand in government departments across many regions and sectors. Actuarial firms, banks, investment firms, and market research organizations are also great potential employers. The demand for statistical and analytical skills is on the rise across various fields. Job seekers with these skills are particularly needed. However, it is becoming increasingly common for these positions to require specialized education and training. For example, if you have a degree in statistics, pursuing a finance specialization could open doors to opportunities in banking, investment, accountancy, or insurance firms. Last but not least, those passionate about solving complex problems may find roles as researchers (within or outside academia) appealing.

Thanks for reading.

I hope this article helped you to get an overview of the career options you have with a degree in statistics.

If you are looking for a job, I wish you the best of luck in finding your dream job! Remember that you will find a variety of job openings in academic, research-oriented, and analytical fields on online and offline job boards. Moreover, do not underestimate the power of friends, colleagues and family: make sure to network with people in your field to increase the chance of getting a job.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

There are many programming languages for data analysis, and it will certainly continue to evolve in the future. The most common ones at the time of writing this post are: R, Python, SAS, Jamovi, SPSS, JMP, Stata and Matlab. Depending on the role and the organization, more or less programming skills will be required.↩︎

Top 10 errors in R and how to fix them

Tue, 07 Feb 2023 00:00:00 +0000

Introduction

If you are just starting with R, you will often encounter errors in your code which prevent it to run. I remember when I was just starting to use R, errors in my code were so frequent that I almost gave up learning this programming language. I even recall that I went back to Excel a few times to finish my analyses because I could not find what was causing the issue.

Fortunately, I forced myself to continue despite the difficulties of the beginning. And today, even if I still encounter errors almost every time I write R code, with experience and practice, it takes less and less time to fix them. If you are also struggling at the beginning, rest assured, it is normal: everyone experiences some frustration when learning a new programming language (and this is the case not only with R).

In this post, I highlight the 10 most common errors in R and how to fix them. Of course, errors depend on your code and your analyses, so it is impossible to cover all of them (and Google does it way better than me). However, I would like to focus on some common syntax mistakes that are frequent when learning R, and which can sometimes take a long time to be fixed before realizing that the solution is right in front of our eyes.

This collection is based on my personal experience and the errors encountered by my students when I teach R. This list being non-exhaustive, feel free to comment (at the end of the post) with errors you often face when using R.

For each error, I provide examples and solutions to fix them. I also mention a couple of warnings (which are, strictly speaking, not errors) at the end of the post.

1. Unmatched parentheses, curly braces, square brackets or quotes

One rather trivial but still quite frequent error is a missing parenthesis, curly brace, square bracket or quotation mark.

This type of error is applicable to many programming languages. In R, for instance:

## missing a closing parenthesis:
mean(c(1, 7, 13)
     
## missing a curly brace before "else":
x <- 7 
if(x > 10) {
  print("x > 10")
 else {
  print("x <= 10")
 }
  
## missing a square bracket:
summary(ggplot2::diamonds[, "price")

## missing a closing quote in 2nd element:
grp <- c("Group 1", "Group 2) 
grp

These errors are easy to detect when the code is basic, but can become much harder to spot with a more complex code, for instance:¹

for (i in y) {
  for (j in x) {
    p <- ggboxplot(dat,
      x = colnames(dat[j]), y = colnames(dat[i]),
      color = colnames(dat[j]),
      legend = "none",
      palette = "npg",
      add = "jitter"
    )
    print(
      p + stat_compare_means(aes(label = paste0(..method.., ", p-value = ", ..p.format..),
        method = method1, label.y = max(dat[, i], na.rm = TRUE)
      )
      + stat_compare_means(comparisons = my_comparisons, method = method2, label = "p.format")
    )
  }

Thankfully, if you use RStudio,² a closing parenthesis, curly brace, square bracket or quotation mark will automatically be written when you open one.

Bear in mind that when installing a package, you must use (single or double) quotation marks around the package’s name:

## NOT correct:
install.packages(ggplot2)

## Error in install.packages : object 'ggplot2' not found

Instead, write one of the two following options:

# install.packages("ggplot2")

# install.packages('ggplot2')

Solution

The solution of course is to simply match all opening parentheses, curly braces, square brackets and quotation marks with their closing counterparts:

mean(c(1, 7, 13))

## [1] 7

x <- 7
if (x > 10) {
  print("x > 10")
} else {
  print("x <= 10")
}

## [1] "x <= 10"

summary(ggplot2::diamonds[, "price"])

##      price      
##  Min.   :  326  
##  1st Qu.:  950  
##  Median : 2401  
##  Mean   : 3933  
##  3rd Qu.: 5324  
##  Max.   :18823

grp <- c("Group 1", "Group 2")
grp

## [1] "Group 1" "Group 2"

Also, make sure:

to correctly place commas:

## NOT correct (misplaced comma):
c("Group 1," "Group 2")

## Error: unexpected string constant in "c("Group 1," "Group 2""

## also NOT correct (missing comma):
c("Group 1" "Group 2")

## Error: unexpected string constant in "c("Group 1" "Group 2""

## correct:
c("Group 1", "Group 2")

you do not mix single and double quotation marks for the same element:

## NOT correct:
c("Group 1')

## correct:
c("Group 1")

## also correct:
c('Group 1')

Note that c('Group 1', "Group 2") does not throw an error but for consistency, it is not recommended to mix single and double quotes within the same vector.

2. Using a function that is not installed or loaded

If you encounter the following error: “Error in … : could not find function ‘…’”, for example:

it means you are trying to use a function belonging to a package which is not yet installed or loaded.

Solution

To solve this error, you have to install the package (if it is not installed yet) and load it with the install.packages() and library() functions, respectively:

## install package:
install.packages("ggplot2")

## load package:
library(ggplot2)

If you are not sure about the usage of these two functions, see more details about installing and loading a package in R.

3. Typos in function, variable, dataset, object or package names

Another common mistake is to misspell a function, a variable, a dataset, an object or a package’s name, for example:

## typo in function name:
maen(c(1, 7, 13))

## Error in maen(c(1, 7, 13)) : could not find function "maen"

## typo in variable name:
summary(ggplot2::diamonds[, "detph"])

## Error: Column `detph` doesn't exist

## typo in dataset name:
data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)
summary(dta[, 2])

## Error in summary(dta[, 2]) : object 'dta' not found

## typo in object name:
test <- c(1, 7, 13)
mean(tset)

## Error in mean(tset) : object 'tset' not found

## typo in package name:
library("tydiverse")

## Error in library("tydiverse") : there is no package called ‘tydiverse’

Solution

Make sure that you correctly spell all your functions, variables, datasets, objects and packages:

Note that R is case sensitive; mean() is considered different than Mean() for R!

mean(c(1, 7, 13))

## [1] 7

summary(ggplot2::diamonds[, "depth"])

##      depth      
##  Min.   :43.00  
##  1st Qu.:61.00  
##  Median :61.80  
##  Mean   :61.75  
##  3rd Qu.:62.50  
##  Max.   :79.00

data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)
data[, 2]

##  [1]  1.3048697  2.2866454 -1.3888607 -0.2787888 -0.1333213  0.6359504
##  [7] -0.2842529 -2.6564554 -2.4404669  1.3201133

test <- c(1, 7, 13)
mean(test)

## [1] 7

library(tidyverse)

## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.3     ✔ readr     2.1.4
## ✔ forcats   1.0.0     ✔ stringr   1.5.0
## ✔ ggplot2   3.4.3     ✔ tibble    3.2.1
## ✔ lubridate 1.9.2     ✔ tidyr     1.3.0
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

If you are sure that you correctly spelled an object, a function or a dataset but you still have an error stating that “object ‘…’ is not found”, make sure that you defined your object/function/dataset before calling it!

It often happens that a student asks me to come to his/her computer because he/she runs the exact same code than me, but cannot make it work. Most of the time, if his/her code is indeed exactly the same than mine, he/she simply has not executed a object/function/dataset before running the code which includes that object/function/dataset. In other words, he/she simply tries to use an undefined object or variable.

Remember that writing code in a R script (contrarily to the console) does not mean it is compiled. You actually have to run it (by clicking on the Run button or using the keyboard shortcut) in order the code to be executed and used later. If you are still struggling with this, see the basics of R and RStudio.

4. Missing, incorrect or misspelled arguments in functions

Most R functions require arguments. For example, the rnorm() function requires at least the number of observations, specified via the argument n.

Your code will not run if you do not specify compulsory arguments, or if incorrectly specify an argument. Moreover, the result might not be what you expect if you misspell an argument:

## missing compulsory argument:
rnorm()

## Error in rnorm() : argument "n" is missing, with no default

## incorrect argument:
rnorm(n = 3, var = 1)

## Error in rnorm(n = 3, var = 1) : unused argument (var = 1)

## misspelled argument:
mean(c(1, 7, 13, NA), narm = TRUE)

## [1] NA

The last piece of code does not throw an error, but the result is not what we want.

Solution

To solve these errors, make sure to specify at least all compulsory arguments of the function, and the correct ones:

In rnorm(), it is the standard deviation, sd, which can be specified in addition to the number of observations n (instead of the variance var).
Removing NA is done with na.rm (instead of narm).

rnorm(n = 3, sd = 1)

## [1] -0.3066386 -1.7813084 -0.1719174

mean(c(1, 7, 13, NA), na.rm = TRUE)

## [1] 7

If you do not know the arguments of a function by heart, you can always check the documentation with ?function_name or help(function_name), for example:

?rnorm()

## or:
help(rnorm)

5. Wrong, inappropriate or inconsistent data types

There are several data types in R, the main ones being:

Numeric
Character
Factor
Logical

You know that some operations and analyses are possible and appropriate only with some specific types of data.

For example, it is not appropriate to compute the mean of a factor or character variable:

gender <- factor(c("female", "female", "male", "female", "male"))

mean(gender)

## Warning in mean.default(gender): argument is not numeric or logical: returning
## NA

## [1] NA

Likewise, although it is technically possible, it makes little sense to draw a barplot of a quantitative continuous variable because in most cases, the frequency will be 1 for each value:

barplot(table(rnorm(10)))

(By the way, if your data is not already displayed in the form of a table, do not forget to add table() inside the barplot() function.)

Solution

Make sure to use the appropriate operation and type of analysis depending on the variable(s) of interest.

For example:

for factor variables, it is more appropriate to compute frequencies and/or relative frequencies, and draw barplots
for quantitative continuous variables, it is more appropriate to compute the mean, median, etc. and draw histograms, boxplots, etc.
for logical variables, the mean,³ a frequency table and a barplot are appropriate
for character variables, word clouds are the most appropriate (unless the variable can be considered as a factor variable because there are not too many different levels)

We now illustrate the examples in R:⁴

## factor:
table(gender)

## gender
## female   male 
##      3      2

prop.table(table(gender))

## gender
## female   male 
##    0.6    0.4

barplot(table(gender))

## quantitative continuous:
x <- rnorm(100)

summary(x)

##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
## -2.99309 -0.74143  0.01809 -0.08570  0.58937  2.70189

par(mfrow = c(1, 2)) ## combine two plots
hist(x)
boxplot(x)

## logical:
x <- c(TRUE, FALSE, FALSE, TRUE, TRUE)

mean(x)

## [1] 0.6

table(x)

## x
## FALSE  TRUE 
##     2     3

barplot(table(x))

For the interested reader, see the most common descriptive statistics in R for different types of data.

Note that, as for descriptive statistics, the choice of the statistical test depends on the variable’s type. See this flowchart to help you in selecting the most appropriate statistical test depending on the number of variables and their types.

An error linked to the one mentioned above is inconsistent data type. See it in practice with the following example:

x <- c(2.4, 3.7, 5.1, 9.8)
class(x)

## [1] "numeric"

y <- c(2.4, 3.7, 5.1, "9.8")
class(y)

## [1] "character"

As you can see, vector x is numerical, whereas vector y is in the form of character. This is due to the fact that the last element of y is surrounded with quotation marks (and thus considered as a string instead of a numerical value), so the entire vector takes the character form.

This can happen when you import a dataset into R and one or several elements of a variable are not encoded correctly. This leads to the entire variable to be considered as a character variable by R.

To avoid this, it is a good practice to check the structure of your dataset (with str()) after importing it to make sure all your variables have the desired format. If not, you can either correct the values in the initial file or change the format in R (with as.numeric()).

6. Forgetting the + sign in ggplot2

If you just learned to use the ggplot2 package for your visualizations (and I highly recommend it!), a common mistake is to forget the + sign.

You know that a visualization made with ggplot2 is constructed by adding several layers:

## load package:
library(ggplot2)

## first layer, the dataset:
ggplot(data = diamonds) +
  ## second layer, the aesthetics:
  aes(x = cut, y = price) +
  ## third layer, the type of plot:
  geom_boxplot() +
  ## add more layers:
  theme_minimal()

Solution

For all your graphics with ggplot2, do not forget to add a + sign after each layer except the last one.

7. Misunderstanding between = and ==

Assignment in R can be done in three ways, from the most to the least common:

<-
=
assign()

The second method, that is =, should not be confused with ==.

Indeed, assigning an object (with any of the three above methods) is used to save something in R. For example, if we want to save the vector (1, 3, 7) and rename that vector x, we can write:

x <- c(1, 3, 7)

## or:
x = c(1, 3, 7)

## or:
assign("x", c(1, 3, 7))

When executing this piece of code, you will see that the vector x of size 3 appears in the tab “Environment” (the top right panel if you use the default view of RStudio):

From now on, we can use that vector simply by calling it by its name:

## [1] 1 3 7

By no means, you can assign an object with ==:

## NOT correct if we want to assign c(1, 3, 7) to x:
x == c(1, 3, 7)

So you are wondering, when would we need to use ==? Actually, it is used when you want to use an equal sign.

I understand that it may be abstract and confusing at the moment, so let’s suppose the following two scenarios as examples (which are the two most common cases when we use ==):

we want to check whether an assigned object or variable respects some conditions, and
we want to subset a dataframe based on one or several conditions.

For these examples, suppose a sample of 5 children:

## create dataframe:
dat <- data.frame(
  Name = c("Mary", "Linda", "James", "John", "Patricia"),
  Age = c(7, 10, 3, 9, 7),
  Gender = c("Girl", "Girl", "Boy", "Boy", "Girl")
)

## print dataframe:
dat

##       Name Age Gender
## 1     Mary   7   Girl
## 2    Linda  10   Girl
## 3    James   3    Boy
## 4     John   9    Boy
## 5 Patricia   7   Girl

Let’s now write different pieces of code for these two scenarios to illustrate them:

We want to check whether the variable Age is equal to the vector (1, 2, 3, 4, 5):

dat$Age == 1:5

## [1] FALSE FALSE  TRUE FALSE FALSE

With this code, we ask whether the first element of the variable Age is equal to 1, the second element of the variable Age is equal to 2, and so on. The answer is of course FALSE, FALSE, TRUE, FALSE and FALSE since only the third child has an age equal to 3 years.

We want to know which of our 5 sampled children are girls:

dat$Gender == "Girl"

## [1]  TRUE  TRUE FALSE FALSE  TRUE

The results show that the first, second and fifth children are girls, while the third and fourth children are not girls.

If you write any of these two lines:

## this overwrites Age and Gender:
dat$Age = 1:5
dat$Gender = "Girl"

You actually overwrite the Age and Gender variables, such that our 5 children will have an age from 1 to 5 (1 year for the first child, up to 5 years for the fifth child) and all of them will be girls.

Now suppose we want to subset our dataframe based on a condition, namely, we want to extract only the children who are 7 years old:

subset(dat, Age == 7)

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

If you do not want to use the subset function, you can also use square brackets:

dat[dat$Age == 7, ]

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

As you can see in the previous examples, we do not want to assign anything. Instead, we are asking “is this variable or vector equal to something else?”. For that specific need, we use ==.

So to sum up, for technical reasons and in order to distinguish between the two concepts, R uses = for assignments, and == for the equality sign. Make sure to understand the difference between the two to avoid any errors.

8. Undefined columns selected

If you are used to subset dataframes with square brackets, [], instead of the subset() or filter() functions, you may have faced the error “Error in [.data.frame(…) : undefined columns selected”.

This occurs when R does not understand the column you want to use while subsetting the dataset.

Considering the same sample of 5 children introduced earlier, the following code will throw an error:

dat[dat$Age == 7]

## Error in `[.data.frame`(dat, dat$Age == 7) : undefined columns selected

because it does not specify the column dimension.

Solution

Remember that dataframes in R have two dimensions:

the rows (one for each experimental unit), and
the columns (one for each variable)

and in that particular order (so row first, then column)!

Since dataframes have two dimensions, R expects two dimensions when you call dat[].

In particular, it expects the first and then the second dimension, separated by a comma:

dat[dat$Age == 7, ]

##       Name Age Gender
## 1     Mary   7   Girl
## 5 Patricia   7   Girl

This code means that we are extracting all rows where Age is equal to 7 (first dimension, i.e. before the comma), for all variables of the dataset (since we did not specify any column after the comma).

For the interested reader, see more ways to subset and manipulate data in R.

9. Problem when importing or using the wrong data file

Importing a dataset in R can be quite challenging for beginners, mainly due to the misunderstanding about the working directory.

When importing a file, R will not search for the file in all your folders of your computer. Instead, it will look only in one specific folder. If your dataset is not inside that folder, it will result in an error such as “cannot open file ‘…’: No such file or directory”:

To fix this, you must specify the path to the folder where your dataset is located. In other words, you need to tell R in which folder you want it to work, hence the name working directory.

Setting the working directory can be done with the setwd() function or via the “Files” tab in the lower right panel of RStudio:

Alternatively, you can move the dataset in the folder where R is currently working (this can be found with getwd()). See more details on importing a file into R and about the working directory.

Another related problem is to use the wrong file. This error is different than the previous ones in the sense that you will not encounter an error but your analyses will still be wrong.

It may sound trivial, but make sure to import and use the correct data file! This is particularly the case if you have files for different points in time and which have a common structure (for example weekly or monthly data files with the exact same variables). It happened to me that I reported results for the wrong week (fortunately, without much consequence).

Also, make sure that you actually use all the rows you want to include in your analyses. It happened to me that, in order to test a model (and avoid long computing times), I extracted a random sample of the original dataset, and almost forgot about this sampling when running my final analyses.

It is thus a good practice to remind you to remove sampling and filters after you have tested your code (and before interpreting the final results).

10. Problem when using the $ operator

For the last error of this top 10, I would like to focus on two related errors:

“$ operator is invalid for atomic vectors”, and
“object of type ‘closure’ is not subsettable”.

I gather them in one single section because they are linked to each other in the sense that they both involve the $ operator.

$ operator is invalid for atomic vectors

To understand this error, we first must recall that an atomic vector is a one-dimensional object (usually created with c()). This is different than dataframes or matrices which are two-dimensional (i.e., rows form the first dimension and columns correspond to the second dimension).

The error “$ operator is invalid for atomic vectors” occurs when we try to access an element of an atomic vector using the dollar operator ($):

## define atomic vector:
x <- c(1, 3, 7)

## set names:
names(x) <- LETTERS[1:3]

## print vector:
x

## A B C 
## 1 3 7

## access value of element C:
x$C

## Error in x$C : $ operator is invalid for atomic vectors

Solution

The $ operator cannot be used to extract elements in atomic vectors. Instead, we must use double brackets [[]] notation:

x[["C"]]

## [1] 7

Remember that the $ operator can be used with dataframes, so we can also fix this error by first converting the atomic vector to a dataframe,⁵ and then access an element by its name with the $ operator:

## convert atomic vector to dataframe:
x <- as.data.frame(t(x))

## print x:
x

##   A B C
## 1 1 3 7

## access value of element C:
x$C

## [1] 7

object of type ‘closure’ is not subsettable

Another error (which I must admit is quite obscure and confusing when learning R) is the following: “object of type ‘closure’ is not subsettable”.

This error occurs when we try to subset or access some elements of a function. An example with the well-known mean() function:

mean[1:3]

## Error in mean[1:3] : object of type 'closure' is not subsettable

In R, we can subset lists, vectors, matrices, dataframes, but not functions. So it throws an error because it is impossible to subset an object of type “closure”, and a function is of that type:

typeof(mean)

## [1] "closure"

Most of the times, you will not encounter this error when using a basic function such as the mean() function (because it is unlikely that your goal is really to subset a function…).

Indeed, you will most likely face this error when trying to subset a dataset named data, but this dataset is not defined in the environment (because it has not been imported or created properly for instance).

To understand the concept, see the following examples:

## create dataset:
data <- data.frame(
  x = rnorm(10),
  y = rnorm(10)
)

## print variable x:
data$x

##  [1]  1.12288964  1.43985574 -1.09711377 -0.11731956  1.20149840 -0.46972958
##  [7] -0.05246948 -0.08610730 -0.88767902 -0.44468400

So far so good. Now suppose we made a mistake when creating the dataset:

## create dataset (with mistake):
data <- data.frame(x = rnorm(10)
                   y = rnorm(10))

You will notice that a comma is missing between variables x and y. As a result, the dataset named data is not created and thus not defined.

Therefore, if we now try to access the variable x from that dataset data, R will actually try to subset the function named data instead of the dataset named data!

data$x

## Error in data$x : object of type 'closure' is not subsettable

This happens because, I repeat, the dataset data does not exist, so R looks for an object named data and find a function with that name:

class(data)

## [1] "function"

Warnings

Warnings are different than errors in the sense that they alert you about something, but it does not prevent you from running the code. It is a good practice to read these warnings as they may give you valuable information.

There are too many warnings to mention them all, but I would like to focus on two common ones:

“NAs introduced by coercion”, and
“Removed … rows containing non-finite values (stat_bin())”.

NAs introduced by coercion

This warning occurs when you try to convert a vector which includes at least one non-numerical value to a numeric vector:

x <- c(1, 3, 7, "Emma")

as.numeric(x)

## Warning: NAs introduced by coercion

## [1]  1  3  7 NA

You do not need to fix it since it is only a warning and not an error. R is simply informing you that at least one element in the initial vector was converted to NA because it could not be converted to a numeric value.

Removed … rows containing non-finite values (stat_bin())

This warning occurs when you use ggplot2 to draw plots. For instance:

ggplot(airquality) +
  aes(x = Ozone) +
  geom_histogram()

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

## Warning: Removed 37 rows containing non-finite values (`stat_bin()`).

Again, as it is a warning you do not need to fix it. It is simply informing you that there are some missing values (NA) in the variable of interest and that these missing values are removed to construct the plot.

Conclusion

Thanks for reading.

I hope that this collection of errors prevented you from making some coding mistakes, or that it helped you in debugging your code.

If you still cannot fix your error, I would recommend to read the documentation of the function (if you struggle with a function in particular), or look online for the solution. Bear in mind that if you encounter an error, it is very likely that someone else posted the answer online (Stack Overflow is usually a good resource).

R has a steep learning curve, in particular if you are not familiar with another programming language. Nonetheless, with practice and time, you will make less and less coding errors, but more importantly, you will be more and more proficient in typing the right keywords in search engines, resulting in less time spent looking for the solution.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

There are 2 mistakes in that piece of code, feel free to try to fix them as an exercise.↩︎
And I strongly recommend using RStudio and not just R. See the differences here.↩︎
Note that mean() applied to a logical variable gives the proportion of TRUE.↩︎
par(mfrow = c(1, 2)) is used to put two plots next to each other.↩︎
Note that we also need to take the transpose of the vector x in order to have it as 1 row, 3 columns.↩︎

How to keep yourself updated with the latest R news?

Thu, 13 Oct 2022 00:00:00 +0000

Introduction

At the end of one of the training sessions I gave on R, a student asked me the following question:

How do you keep yourself updated with the latest R news?

It is true that R, being open source (meaning that everyone can contribute), is evolving rapidly. This means that even if I am using R for several years and on a daily basis, I like to stay informed in order to stay up to date with the program and the latest coding practices.

In fact, I learn about new packages, new functions and new features almost everyday. Most of them are not particularly useful for my research or my teaching tasks, but sometimes I discover such a nice package or function that I replace my code with new one.¹

The training was an advanced one, so the student had a good knowledge of R and was not looking for more tutorials or courses. She was interested in knowing where to look for updates about current and new R packages and functions.

After sharing my sources with all students following the training, I thought it would be useful to others. In this article, I share my sources—from where I get the latest R updates and news.

The sources are divided into two main categories: Twitter and newsletters.

How do I keep track?

Twitter

To be honest, I mostly use Twitter to keep up to date with R news.

Twitter allows me to follow discussions about statistical methods or approaches, and to keep me informed about publications of new blog posts.

What I particularly like with Twitter is that there is a mix between:

short messages about new functions or packages (most of the time with an illustration or an example), and
announcements of new blog posts that cover specific subjects in details.

For this, I follow people (researchers, professors, bloggers, statisticians, data scientists, etc.) that are working in my domains of interest. For instance, I am mostly interested in the application of statistics in R, data science, biostatistics and data visualization. I thus follow accounts which regularly post about these topics. I also avoid following accounts that cover topics I am not interested in, so that my Twitter feed really shows information I am most likely to be interested in.

Below, you will see a list of some of the accounts I follow, classified by themes. Of course, this is a non-exhaustive list! There are plenty of very inspiring and intelligent people that are not in the list, simply because I cannot afford to put them all.

If you follow people that post regularly about the themes covered below, feel free to add them in the comments. I am always looking for new inspiring accounts to follow.

Note that the accounts are displayed in alphabetical order and the table is searchable.

You will also find many news when exploring #rstats on Twitter.

Newsletters

Besides Twitter, I also read new blog posts and stay informed through these newsletters:

The 2 first newsletters are actually blog aggregators, so collections of many many blogs. If you have your own blog, don’t hesitate to submit it to make it accessible to more people.

You can always subscribe to a blog you like, but if it is related to R, it will most likely be shared via R-Bloggers or R Weekly.

Conclusion

Thanks for reading.

I hope this article will help you to keep track of great blogs, tutorials and other resources about R. Feel free to follow me on Twitter (@statsandr), where I tweet my new articles and retweet everything I find interesting or worth mentioning.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

See for example this article. The package I discovered was so useful and interesting (to me), that I added a new section to the post and now use it instead of the older package.↩︎

How to do a t-test or ANOVA for more than one variable at once in R?

Thu, 19 Mar 2020 00:00:00 +0000

Introduction

As part of my teaching assistant position in a Belgian university, students often ask me for some help in their statistical analyses for their master’s thesis.

A frequent question is how to compare groups of patients in terms of several quantitative continuous variables. Most of us know that:

To compare two groups, a Student’s t-test should be used¹
To compare three groups or more, an ANOVA should be performed

These two tests are quite basic and have been extensively documented online and in statistical textbooks so the difficulty is not in how to perform these tests.

In the past, I used to do the analyses by following these 3 steps:

Draw boxplots illustrating the distributions by group (with the boxplot() function or thanks to the {esquisse} R Studio addin if I wanted to use the {ggplot2} package)
Perform a t-test or an ANOVA depending on the number of groups to compare (with the t.test() and oneway.test() functions for t-test and ANOVA, respectively)
Repeat steps 1 and 2 for each variable

This was feasible as long as there were only a couple of variables to test. Nonetheless, most students came to me asking to perform these kind of tests not on one or two variables, but on multiples variables. So when there were more than one variable to test, I quickly realized that I was wasting my time and that there must be a more efficient way to do the job.

Note: you must be very careful with the issue of multiple testing (also referred as multiplicity) which can arise when you perform multiple tests. In short, when a large number of statistical tests are performed, some will have $p$-values less than 0.05 purely by chance, even if all null hypotheses are in fact really true. This is known as multiplicity or multiple testing. You can tackle this problem by using the Bonferroni correction, among others. The Bonferroni correction is a simple method that allows many t-tests to be made while still assuring an overall confidence level is maintained. For this, instead of using the standard threshold of $\alpha = 5$% for the significance level, you can use $\alpha = \frac{0.05}{m}$ where $m$ is the number of t-tests. For example, if you perform 20 t-tests with a desired $\alpha = 0.05$, the Bonferroni correction implies that you would reject the null hypothesis for each individual test when the $p$-value is smaller than $\alpha = \frac{0.05}{20} = 0.0025$.

Note also that there is no universally accepted approach for dealing with the problem of multiple comparisons. Usually, you should choose a p-value adjustment measure familiar to your audience or in your field of study. The Bonferroni correction is easy to implement. It is however not appropriate if you have a very large number of tests to perform (imagine you want to do 10,000 t-tests, a p-value would have to be less than $\frac{0.05}{10000} = 0.000005$ to be significant). A more powerful method is also to adjust the false discovery rate using the Benjamini-Hochberg or Holm procedure (McDonald 2014).

Another option is to use a multivariate ANOVA (MANOVA), if your independent variable has more than two levels. This is particularly useful when your dependent variables are correlated. Correlation between the dependent variables provides MANOVA the following advantages:

Identify patterns between several dependent variables: The independent variables can influence the relationship between dependent variables instead of influencing a single dependent variable.
Address the issue of multiple testing: with MANOVA, the error rate equals the significance level (with no p-value adjustment method needed).
Greater statistical power: When the dependent variables are correlated, MANOVA can identify effects that are too small for the ANOVA to detect.

Note that MANOVA is used if your independent variable has more than two levels. If your independent variable has only two levels, the multivariate equivalent of the t-test is Hotelling’s $T^2$.

This article aims at presenting a way to perform multiple t-tests and ANOVA from a technical point of view (how to implement it in R). Discussion on which adjustment method to use or whether there is a more appropriate model to fit the data is beyond the scope of this article (so be sure to understand the implications of using the code below for your own analyses). Make sure also to test the assumptions of the ANOVA before interpreting results.

Perform multiple tests at once

I thus wrote a piece of code that automated the process, by drawing boxplots and performing the tests on several variables at once. Below is the code I used, illustrating the process with the iris dataset. The Species variable has 3 levels, so let’s remove one, and then draw a boxplot and apply a t-test on all 4 continuous variables at once. Note that the continuous variables that we would like to test are variables 1 to 4 in the iris dataset.

dat <- iris

# remove one level to have only two groups
dat <- subset(dat, Species != "setosa")
dat$Species <- factor(dat$Species)

# boxplots and t-tests for the 4 variables at once
for (i in 1:4) { # variables to compare are variables 1 to 4
  boxplot(dat[, i] ~ dat$Species, # draw boxplots by group
    ylab = names(dat[i]), # rename y-axis with variable's name
    xlab = "Species"
  )
  print(t.test(dat[, i] ~ dat$Species)) # print results of t-test
}

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -5.6292, df = 94.025, p-value = 1.866e-07
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.8819731 -0.4220269
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    5.936                    6.588

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -3.2058, df = 97.927, p-value = 0.001819
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.33028364 -0.07771636
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    2.770                    2.974

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -12.604, df = 95.57, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -1.49549 -1.08851
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    4.260                    5.552

## 
## 	Welch Two Sample t-test
## 
## data:  dat[, i] by dat$Species
## t = -14.625, df = 89.043, p-value < 2.2e-16
## alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
## 95 percent confidence interval:
##  -0.7951002 -0.6048998
## sample estimates:
## mean in group versicolor  mean in group virginica 
##                    1.326                    2.026

As you can see, the above piece of code draws a boxplot and then prints results of the test for each continuous variable, all at once.

At some point in the past, I even wrote code to:

draw a boxplot
test for the equality of variances (thanks to the Levene’s test)
depending on whether the variances were equal or unequal, the appropriate test was applied: the Welch test if the variances were unequal and the Student’s t-test in the case the variances were equal (see more details about the different versions of the t-test for two samples)
apply steps 1 to 3 for all continuous variables at once

I had a similar code for ANOVA in case I needed to compare more than two groups.

The code was doing the job relatively well. Indeed, thanks to this code I was able to test several variables in an automated way in the sense that it compared groups for all variables at once.

The only thing I had to change from one project to another is that I needed to modify the name of the grouping variable and the numbering of the continuous variables to test (Species and 1:4 in the above code).

Concise and easily interpretable results

T-test

Although it was working quite well and applicable to different projects with only minor changes, I was still unsatisfied with another point.

Someone who is proficient in statistics and R can read and interpret the output of a t-test without any difficulty. However, as you may have noticed with your own statistical projects, most people do not know what to look for in the results and are sometimes a bit confused when they see so many graphs, code, output, results and numeric values in a document. They are quite easily overwhelmed by this mass of information and unable to extract the key message.

With my old R routine, the time I was saving by automating the process of t-tests and ANOVA was (partially) lost when I had to explain R outputs to my students so that they could interpret the results correctly. Although most of the time it simply boiled down to pointing out what to look for in the outputs (i.e., p-values), I was still losing quite a lot of time because these outputs were, in my opinion, too detailed for most real-life applications and for students in introductory classes. In other words, too much information seemed to be confusing for many people so I was still not convinced that it was the most optimal way to share statistical results to nonscientists.

Of course, they came to me for statistical advices, so they expected to have these results and I needed to give them answers to their questions and hypotheses. Nonetheless, I wanted to find a better way to communicate these results to this type of audience, with the minimum of information required to arrive at a conclusion. No more and no less than that.

After a long time spent online trying to figure out a way to present results in a more concise and readable way, I discovered the {ggpubr} package. This package allows to indicate the test used and the p-value of the test directly on a ggplot2-based graph. It also facilitates the creation of publication-ready plots for non-advanced statistical audiences.

After many refinements and modifications of the initial code (available in this article), I finally came up with a rather stable and robust process to perform t-tests and ANOVA for more than one variable at once, and more importantly, make the results concise and easily readable by anyone (statisticians or not).

A graph is worth a thousand words, so here are the exact same tests than in the previous section, but this time with my new R routine:

library(ggpubr)

# Edit from here #
x <- which(names(dat) == "Species") # name of grouping variable
y <- which(names(dat) == "Sepal.Length" # names of variables to test
| names(dat) == "Sepal.Width" |
  names(dat) == "Petal.Length" |
  names(dat) == "Petal.Width")
method <- "t.test" # one of "wilcox.test" or "t.test"
paired <- FALSE # if paired make sure that in the dataframe you have first all individuals at T1, then all individuals again at T2
# Edit until here


# Edit at your own risk
for (i in y) {
  for (j in x) {
    if (paired == TRUE) {
      p <- ggpaired(dat,
        x = colnames(dat[j]), y = colnames(dat[i]),
        color = colnames(dat[j]), line.color = "gray", line.size = 0.4,
        palette = "npg",
        legend = "none",
        xlab = colnames(dat[j]),
        ylab = colnames(dat[i]),
        add = "jitter"
      )
    } else {
      p <- ggboxplot(dat,
        x = colnames(dat[j]), y = colnames(dat[i]),
        color = colnames(dat[j]),
        palette = "npg",
        legend = "none",
        add = "jitter"
      )
    }
    #  Add p-value
    print(p + stat_compare_means(aes(label = paste0(after_stat(method), ", p-value = ", after_stat(p.format))),
      method = method,
      paired = paired,
      # group.by = NULL,
      ref.group = NULL
    ))
  }
}

As you can see from the graphs above, only the most important information is presented for each variable:

a visual comparison of the groups thanks to boxplots
the name of the statistical test
the p-value of the test

Of course, experts may be interested in more advanced results. However, this simple yet complete graph, which includes the name of the test and the p-value, gives all the necessary information to answer the question: “Are the groups different?”.

In my experience, I have noticed that students and professionals (especially those from a less scientific background) understand way better these results than the ones presented in the previous section.

The only lines of code that need to be modified for your own project is the name of the grouping variable (Species in the above code), the names of the variables you want to test (Sepal.Length, Sepal.Width, etc.),² whether you want to apply a t-test (t.test) or Wilcoxon test (wilcox.test) and whether the samples are paired or not (FALSE if samples are independent, TRUE if they are paired).

Based on these graphs, it is easy, even for non-experts, to interpret the results and conclude that the versicolor and virginica species are significantly different in terms of all 4 variables (since all p-values $< \frac{0.05}{4} = 0.0125$ (remind that the Bonferroni correction is applied to avoid the issue of multiple testing, so we divide the usual $\alpha$ level by 4 because there are 4 t-tests)).

Additional p-value adjustment methods

If you would like to use another p-value adjustment method, you can use the p.adjust() function. Below are the raw p-values found above, together with p-values derived from the main adjustment methods (presented in a dataframe):

raw_pvalue <- numeric(length = length(1:4))
for (i in (1:4)) {
  raw_pvalue[i] <- t.test(dat[, i] ~ dat$Species,
    alternative = "two.sided"
  )$p.value
}

df <- data.frame(
  Variable = names(dat[, 1:4]),
  raw_pvalue = round(raw_pvalue, 3)
)

df$Bonferroni <-
  p.adjust(df$raw_pvalue,
    method = "bonferroni"
  )
df$BH <-
  p.adjust(df$raw_pvalue,
    method = "BH"
  )
df$Holm <-
  p.adjust(df$raw_pvalue,
    method = "holm"
  )
df$Hochberg <-
  p.adjust(df$raw_pvalue,
    method = "hochberg"
  )
df$Hommel <-
  p.adjust(df$raw_pvalue,
    method = "hommel"
  )
df$BY <-
  round(p.adjust(df$raw_pvalue,
    method = "BY"
  ), 3)
df

##       Variable raw_pvalue Bonferroni    BH  Holm Hochberg Hommel    BY
## 1 Sepal.Length      0.000      0.000 0.000 0.000    0.000  0.000 0.000
## 2  Sepal.Width      0.002      0.008 0.002 0.002    0.002  0.002 0.004
## 3 Petal.Length      0.000      0.000 0.000 0.000    0.000  0.000 0.000
## 4  Petal.Width      0.000      0.000 0.000 0.000    0.000  0.000 0.000

Regardless of the p-value adjustment method, the two species are different for all 4 variables. Note that the adjustment method should be chosen before looking at the results to avoid choosing the method based on the results.

Below another function that allows to perform multiple Student’s t-tests or Wilcoxon tests at once and choose the p-value adjustment method. The function also allows to specify whether samples are paired or unpaired and whether the variances are assumed to be equal or not. (The code has been adapted from Mark White’s article.)

t_table <- function(data, dvs, iv,
                    var_equal = TRUE,
                    p_adj = "none",
                    alpha = 0.05,
                    paired = FALSE,
                    wilcoxon = FALSE) {
  if (!inherits(data, "data.frame")) {
    stop("data must be a data.frame")
  }

  if (!all(c(dvs, iv) %in% names(data))) {
    stop("at least one column given in dvs and iv are not in the data")
  }

  if (!all(sapply(data[, dvs], is.numeric))) {
    stop("all dvs must be numeric")
  }

  if (length(unique(na.omit(data[[iv]]))) != 2) {
    stop("independent variable must only have two unique values")
  }

  out <- lapply(dvs, function(x) {
    if (paired == FALSE & wilcoxon == FALSE) {
      tres <- t.test(data[[x]] ~ data[[iv]], var.equal = var_equal)
    } else if (paired == FALSE & wilcoxon == TRUE) {
      tres <- wilcox.test(data[[x]] ~ data[[iv]])
    } else if (paired == TRUE & wilcoxon == FALSE) {
      tres <- t.test(data[[x]] ~ data[[iv]],
        var.equal = var_equal,
        paired = TRUE
      )
    } else {
      tres <- wilcox.test(data[[x]] ~ data[[iv]],
        paired = TRUE
      )
    }

    c(
      p_value = tres$p.value
    )
  })

  out <- as.data.frame(do.call(rbind, out))
  out <- cbind(variable = dvs, out)
  names(out) <- gsub("[^0-9A-Za-z_]", "", names(out))

  out$p_value <- p.adjust(out$p_value, p_adj)
  out$conclusion <- ifelse(out$p_value < alpha,
    paste0("Reject H0 at ", alpha * 100, "%"),
    paste0("Do not reject H0 at ", alpha * 100, "%")
  )
  out$p_value <- ifelse(out$p_value < 0.001,
    "<0.001",
    round(out$p_value, 3)
  )

  return(out)
}

Applied to our dataset, with no adjustment method for the p-values:

result <- t_table(
  data = dat,
  c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width"),
  "Species"
)

result

##       variable p_value      conclusion
## 1 Sepal.Length  <0.001 Reject H0 at 5%
## 2  Sepal.Width   0.002 Reject H0 at 5%
## 3 Petal.Length  <0.001 Reject H0 at 5%
## 4  Petal.Width  <0.001 Reject H0 at 5%

And with the Holm (1979) adjustment method:

result <- t_table(
  data = dat,
  c("Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width"),
  "Species",
  p_adj = "holm"
)

result

##       variable p_value      conclusion
## 1 Sepal.Length  <0.001 Reject H0 at 5%
## 2  Sepal.Width   0.002 Reject H0 at 5%
## 3 Petal.Length  <0.001 Reject H0 at 5%
## 4  Petal.Width  <0.001 Reject H0 at 5%

Again, with the Holm’s adjustment method, we conclude that, at the 5% significance level, the two species are significantly different from each other in terms of all 4 variables.

ANOVA

Below the same process with an ANOVA. Note that we reload the dataset iris to include all three Species this time:

dat <- iris

# Edit from here
x <- which(names(dat) == "Species") # name of grouping variable
y <- which(names(dat) == "Sepal.Length" # names of variables to test
| names(dat) == "Sepal.Width" |
  names(dat) == "Petal.Length" |
  names(dat) == "Petal.Width")
method1 <- "anova" # one of "anova" or "kruskal.test"
method2 <- "t.test" # one of "wilcox.test" or "t.test"
my_comparisons <- list(c("setosa", "versicolor"), c("setosa", "virginica"), c("versicolor", "virginica")) # comparisons for post-hoc tests
# Edit until here


# Edit at your own risk
for (i in y) {
  for (j in x) {
    p <- ggboxplot(dat,
      x = colnames(dat[j]), y = colnames(dat[i]),
      color = colnames(dat[j]),
      legend = "none",
      palette = "npg",
      add = "jitter"
    )
    print(
      p + stat_compare_means(aes(label = paste0(after_stat(method), ", p-value = ", after_stat(p.format))),
        method = method1, label.y = max(dat[, i], na.rm = TRUE)
      )
      + stat_compare_means(comparisons = my_comparisons, method = method2, label = "p.format") # remove if p-value of ANOVA or Kruskal-Wallis test >= alpha
    )
  }
}

Like the improved routine for the t-test, I have noticed that students and non-expert professionals understand ANOVA results presented this way much more easily compared to the default R outputs.

With one graph for each variable, it is easy to see that all species are different from each other in terms of all 4 variables.³

If you want to apply the same automated process to your data, you will need to modify the name of the grouping variable (Species), the names of the variables you want to test (Sepal.Length, etc.), whether you want to perform an ANOVA (anova) or Kruskal-Wallis test (kruskal.test) and finally specify the comparisons for the post-hoc tests.⁴

To go even further

As we have seen, these two improved R routines allow to:

Perform t-tests and ANOVA on a small or large number of variables with only minor changes to the code. I basically only have to replace the variable names and the name of the test I want to use. It takes almost the same time to test one or several variables so it is quite an improvement compared to testing one variable at a time.
Share test results in a much proper and cleaner way. This is possible thanks to a graph showing the observations by group and the p-value of the appropriate test included directly on the graph. This is particularly important when communicating results to a wider audience or to people from diverse backgrounds.

However, like most of my R routines, these two pieces of code are still a work in progress. Below are some additional features I have been thinking of and which could be added in the future to make the process of comparing two or more groups even more optimal:

Add the possibility to select variables by their numbering in the dataframe. For the moment it is only possible to do it via their names. This will allow to automate the process even further because instead of typing all variable names one by one, we could simply type 4:25 (to test variables 4 to 25 for instance).
Add the possibility to choose a p-value adjustment method. Currently, raw p-values are displayed in the graphs and I manually adjust them afterwards or adjust the $\alpha$.
When comparing more than two groups, it is only possible to apply an ANOVA or Kruskal-Wallis test at the moment. A major improvement would be to add the possibility to perform a repeated measures ANOVA (i.e., an ANOVA when the samples are dependent). It is currently already possible to do a t-test with two paired samples, but it is not yet possible to do the same with more than two groups.
Another less important (yet still nice) feature when comparing more than 2 groups would be to automatically apply post-hoc tests only in the case where the null hypothesis of the ANOVA or Kruskal-Wallis test is rejected (so when there is at least one group different from the others, because if the null hypothesis of equal groups is not rejected we do not apply a post-hoc test). At the present time, I manually add or remove the code that displays the p-values of post-hoc tests depending on the global p-value of the ANOVA or Kruskal-Wallis test.

I will try to add these features in the future, or I would be glad to help if the author of the {ggpubr} package needs help in including these features (I hope he will see this article!).

Last but not least, the following packages may be of interest to some readers:

If you want to report statistical results on a graph, I advise you to check the {ggstatsplot} package and in particular the ggbetweenstats() and ggwithinstats() functions. These functions allow to compare a continuous variable across multiple groups or conditions (for both independent and paired samples). Two advantages of the functions is that:
- it is very easy to switch from parametric to nonparametric tests and
- it automatically runs an ANOVA or t-test depending on the number of groups to compare

Note that many different statistical results are displayed on the graph, not only the name of the test and the p-value so a bit of simplicity and clarity is lost for more precision. However, it is still very convenient to be able to include tests results on a graph in order to combine the advantages of a visualization and a sound statistical analysis. Something that I still need to figure out is how to run the code on several variables at once.

The {compareGroups} package also provides a nice way to compare groups. It comes with a really complete Shiny app, available with:

# install.packages("compareGroups")
library(compareGroups)
cGroupsWUI()

Update with the `{ggstatsplot}` package

Several months after having written this article, I finally found a way to plot and run analyses on several variables at once with the package {ggstatsplot} (Patil 2021). This was the main feature I was missing and which prevented me from using it more often.

Although I still find that too much statistical details are displayed (in particular for non experts), I still believe the ggbetweenstats() and ggwithinstats() functions are worth mentioning in this article. I actually now use those two functions almost as often as my previous routines because:

I do not have to care about the number of groups to compare, the functions automatically choose the appropriate test according to the number of groups (ANOVA for 3 groups or more, and t-test for 2 groups)
I can select variables based on their column numbering, and not based on their names anymore (which prevents me from writing those variable names manually)
When comparing 3 or more groups (so for ANOVA, Kruskal-Wallis, repeated measure ANOVA or Friedman), $p$-values of the post-hoc tests within each dependent variable are by default the adjusted $p$-values (Holm is the default but many adjustment methods are available)
It is possible to compare both independent and paired samples, no matter the number of groups (remember that with the ggpubr package I could only do paired samples for two samples, not for 3 samples)
They allow to easily switch between the parametric and nonparametric version
All this in a more concise manner using the {purrr} package

For those of you who are interested, below my updated R routine which include these functions and applied this time on the penguins dataset.

library(palmerpenguins)

dat <- penguins
str(dat)

## tibble [344 × 8] (S3: tbl_df/tbl/data.frame)
##  $ species          : Factor w/ 3 levels "Adelie","Chinstrap",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ island           : Factor w/ 3 levels "Biscoe","Dream",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ bill_length_mm   : num [1:344] 39.1 39.5 40.3 NA 36.7 39.3 38.9 39.2 34.1 42 ...
##  $ bill_depth_mm    : num [1:344] 18.7 17.4 18 NA 19.3 20.6 17.8 19.6 18.1 20.2 ...
##  $ flipper_length_mm: int [1:344] 181 186 195 NA 193 190 181 195 193 190 ...
##  $ body_mass_g      : int [1:344] 3750 3800 3250 NA 3450 3650 3625 4675 3475 4250 ...
##  $ sex              : Factor w/ 2 levels "female","male": 2 1 1 NA 1 2 1 2 NA NA ...
##  $ year             : int [1:344] 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...

We illustrate the routine for two groups with the variables sex (two factors) as independent variable, and the 4 quantitative continuous variables bill_length_mm, bill_depth_mm, flipper_length_mm and body_mass_g as dependent variables:

library(ggstatsplot)
library(tibble)

# Comparison between sexes

# edit from here
x <- "sex"
cols <- 3:6 # the 4 continuous dependent variables
type <- "parametric" # given the large number of observations, we use the parametric version
paired <- FALSE # FALSE for independent samples, TRUE for paired samples
# edit until here

# edit at your own risk
plotlist <-
  purrr::pmap(
    .l = list(
      data = list(as_tibble(dat)),
      x = x,
      y = as.list(colnames(dat)[cols]),
      plot.type = "box", # for boxplot
      type = type, # parametric or nonparametric
      pairwise.comparisons = TRUE, # to run post-hoc tests if more than 2 groups
      pairwise.display = "significant", # show only significant differences
      bf.message = FALSE, # remove message about Bayes Factor
      centrality.plotting = FALSE # remove central measure
    ),
    .f = ifelse(paired, # automatically use ggwithinstats if paired samples, ggbetweenstats otherwise
      ggstatsplot::ggwithinstats,
      ggstatsplot::ggbetweenstats
    ),
    violin.args = list(width = 0, linewidth = 0) # remove violin plots and keep only boxplots
  )

# print all plots together with statistical results
for (i in 1:length(plotlist)) {
  print(plotlist[[i]])
}

We now illustrate the routine for 3 groups or more with the variable species (three factors) as independent variable, and the 4 same dependent variables:

# Comparison between species

# edit from here
x <- "species"
cols <- 3:6 # the 4 continuous dependent variables
type <- "parametric" # given the large number of observations, we use the parametric version
paired <- FALSE # FALSE for independent samples, TRUE for paired samples
# edit until here

# edit at your own risk
plotlist <-
  purrr::pmap(
    .l = list(
      data = list(as_tibble(dat)),
      x = x,
      y = as.list(colnames(dat)[cols]),
      plot.type = "box", # for boxplot
      type = type, # parametric or nonparametric
      pairwise.comparisons = TRUE, # to run post-hoc tests if more than 2 groups
      pairwise.display = "significant", # show only significant differences
      bf.message = FALSE, # remove message about Bayes Factor
      centrality.plotting = FALSE # remove central measure
    ),
    .f = ifelse(paired, # automatically use ggwithinstats if paired samples, ggbetweenstats otherwise
      ggstatsplot::ggwithinstats,
      ggstatsplot::ggbetweenstats
    ),
    violin.args = list(width = 0, linewidth = 0) # remove violin plots and keep only boxplots
  )

# print all plots together with statistical results
for (i in 1:length(plotlist)) {
  print(plotlist[[i]])
}

As you can see, I only have to specify:

the name of the grouping variable (sex and species),
the number of the dependent variables (variables 3 to 6 in the dataset),
whether I want to use the parametric or nonparametric version and
whether samples are independent (paired = FALSE) or paired (paired = TRUE).

Everything else is automated—the outputs show a graphical representation of what we are comparing, together with the details of the statistical analyses in the subtitle of the plot (the $p$-value among others).

Note that the code shown above is actually the same if I want to compare 2 groups or more than 2 groups. I wrote twice the same code (once for 2 groups and once again for 3 groups) for illustrative purposes only, but they are the same and should be treated as one for your projects.

I must admit I am quite satisfied with this routine, now that:

I can automate it on many variables at once and I do not need to write the variable names manually anymore,
at the same time, I can choose the appropriate test among all the available ones (depending on the number of groups, whether they are paired or not, and whether I want to use the parametric or nonparametric version).

Nonetheless, I must also admit that I am still not satisfied with the level of details of the statistical results. As already mentioned, many students get confused and get lost in front of so much information (except the $p$-value and the number of observations, most of the details are rather obscure to them because they are not covered in introductory statistic classes).

I saved time thanks to all improvements in comparison to my previous routine, but I definitely lose time when I have to point out to them what they should look for. After discussing with other professors, I noticed that they have the same problem.

For the moment, you can only print all results or none. I have opened an issue kindly requesting to add the possibility to display only a summary (with the $p$-value and the name of the test for instance).⁵ I will update again this article if the maintainer of the package includes this feature in the future. So stay tuned!

Conclusion

Thanks for reading.

I hope this article will help you to perform t-tests and ANOVA for multiple variables at once and make the results more easily readable and interpretable by non-scientists. Learn more about the t-test to compare two groups, or the ANOVA to compare 3 groups or more.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

References

Holm, Sture. 1979. “A Simple Sequentially Rejective Multiple Test Procedure.” Scandinavian Journal of Statistics, 65–70.

McDonald, JH. 2014. “Multiple Tests.” Handbook of Biological Statistics. 3rd Ed Baltimore, Maryland: Sparky House Publishing, 233–36.

Patil, Indrajeet. 2021. “Visualizations with statistical details: The ’ggstatsplot’ approach.” Journal of Open Source Software 6 (61): 3167. https://doi.org/10.21105/joss.03167.

In theory, an ANOVA can also be used to compare two groups as it will give the same results compared to a Student’s t-test, but in practice we use the Student’s t-test to compare two groups and the ANOVA to compare three groups or more.↩︎
Do not forget to separate the variables you want to test with |.↩︎
Do not forget to adjust the $p$-values or the significance level $\alpha$. If you use the Bonferroni correction, the adjusted $\alpha$ is simply the desired $\alpha$ level divided by the number of comparisons.↩︎
Post-hoc test is only the name used to refer to a specific type of statistical tests. Post-hoc test includes, among others, the Tukey HSD test, the Bonferroni correction, Dunnett’s test. Even if an ANOVA or a Kruskal-Wallis test can determine whether there is at least one group that is different from the others, it does not allow us to conclude which are different from each other. For this purpose, there are post-hoc tests that compare all groups two by two to determine which ones are different, after adjusting for multiple comparisons. Concretely, post-hoc tests are performed to each possible pair of groups after an ANOVA or a Kruskal-Wallis test has shown that there is at least one group which is different (hence “post” in the name of this type of test). The null and alternative hypotheses and the interpretations of these tests are similar to a Student’s t-test for two samples.↩︎
I am open to contribute to the package if I can help!↩︎

An efficient way to install and load R packages

Fri, 31 Jan 2020 00:00:00 +0000

What is an R package and how to use it?

Unlike other programs, only fundamental functionalities come by default with R. You will thus often need to install some “extensions” to perform the analyses you want. These extensions which are collections of functions and datasets developed and published by R users are called packages.

Packages extend existing base R functionalities by adding new ones. R is open source so everyone can write code and publish it as a package, and everyone can install a package and start using the functions or datasets built inside the package, all this for free.

In order to use a package, it needs to be installed on your computer by running install.packages("name_of_package") (do not forget "" around the name of the package, otherwise R will look for an object saved under that name!). Once the package is installed, you must load the package and only after it has been loaded you can use all the functions and datasets it contains. To load a package, run library(name_of_package) (this time "" around the name of the package are optional, but can still be used if you wish).

Note that packages must be installed only once (until you update your R, then you have to install them again), whereas packages must be loaded every time you open R.¹

Inefficient way to install and load R packages

Depending on how long you have been using R, you may use a limited amount of packages or, on the contrary, a large amount of them. As you use more and more packages you will soon start to have (too) many lines of code just for installing and loading them.

Here is a preview of the code from my PhD thesis showing how the installation and loading of R packages looked like when I started working on R (only a fraction of them are displayed to shorten the code):

# Installation of required packages
install.packages("tidyverse")
install.packages("ggplot2")
install.packages("readxl")
install.packages("dplyr")
install.packages("tidyr")
install.packages("ggfortify")
install.packages("DT")
install.packages("reshape2")
install.packages("knitr")
install.packages("lubridate")

# Load packages
library("tidyverse")
library("ggplot2")
library("readxl")
library("dplyr")
library("tidyr")
library("ggfortify")
library("DT")
library("reshape2")
library("knitr")
library("lubridate")

As you can guess the code became longer and longer as I needed more and more packages for my analyses. Moreover, I tended to reinstall all packages as I was working on 4 different computers and I could not remember which packages were already installed on which machine. Reinstalling all packages every time I opened my script or R Markdown document was a waste of time.

More efficient way

Then one day, a colleague of mine shared some of his code with me. I am glad he did as he introduced me to a much more efficient way to install and load R packages. He gave me the permission to share the tip, so here is the code I now use to perform the task of installing and loading R packages:

# Package names
packages <- c("ggplot2", "readxl", "dplyr", "tidyr", "ggfortify", "DT", "reshape2", "knitr", "lubridate", "pwr", "psy", "car", "doBy", "imputeMissings", "RcmdrMisc", "questionr", "vcd", "multcomp", "KappaGUI", "rcompanion", "FactoMineR", "factoextra", "corrplot", "ltm", "goeveg", "corrplot", "FSA", "MASS", "scales", "nlme", "psych", "ordinal", "lmtest", "ggpubr", "dslabs", "stringr", "assist", "ggstatsplot", "forcats", "styler", "remedy", "snakecaser", "addinslist", "esquisse", "here", "summarytools", "magrittr", "tidyverse", "funModeling", "pander", "cluster", "abind")

# Install packages not yet installed
installed_packages <- packages %in% rownames(installed.packages())
if (any(installed_packages == FALSE)) {
  install.packages(packages[!installed_packages])
}

# Packages loading
invisible(lapply(packages, library, character.only = TRUE))

This code for installing and loading R packages is more efficient in several ways:

The function install.packages() accepts a vector as argument, so one line of code for each package in the past is now one line including all packages
In the second part of the code, it checks whether a package is already installed or not, and then install only the missing ones
Regarding the packages loading (the last part of the code), the lapply() function is used to call the library() function on all packages at once, which makes the code more condense.
The output when loading a package is rarely useful. The invisible() function removes this output.

From that day on, every time I need to use a new package, I simply add it to the vector packages at the top of the code, which is located at the top of my scripts and R Markdown documents. No matter on which computer I am working on, running the entire code will install only the missing packages and will load all of them. This greatly reduced the running time for the installation and loading of my R packages.

Most efficient way

`{pacman}` package

After this article was published, a reader informed me about the {pacman} package. After having read the documentation and try it out myself, I learned that the function p_load() from {pacman} checks to see if a package is installed, if not it attempts to install the package and then loads it. It can also be applied to several packages at once, all this in a very condensed way:

install.packages("pacman")

pacman::p_load(ggplot2, tidyr, dplyr)

Find more about this package on CRAN.

`{librarian}` package

Like {pacman}, the shelf() function from the {librarian} package automatically installs, updates, and loads R packages that are not yet installed in a single function. The function accepts packages from CRAN, GitHub, and Bioconductor (only if Bioconductor’s Biobase package is installed). The function also accepts multiple package entries, provided as a comma-separated list of unquoted names (so no "" around package names).

Last but not least, the {librarian} package allows to load packages automatically at the start of every R session (thanks to the lib_startup() function) and search for new packages on CRAN by keywords or regular expressions (thanks to the browse_cran() function).

Here is an example of how to install missing packages and load them with the shelf() function:

# From CRAN:
install.packages("librarian")

librarian::shelf(ggplot2, DesiQuintans / desiderata, pander)

For CRAN packages, provide the package name as normal without "" and for GitHub packages, provide the username and package name separated by / (i.e., UserName/RepoName as shown for the desiderata package).

Find more about this package on CRAN.

Conclusion

Thanks for reading.

I hope the article helped you to install and load R packages in a more efficient way.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

A special thanks Danilo and James for informing me about the {pacman} and {librarian} packages.

As suggested by Patrick, it is a good practice to comment (with a # in front of the line of code) the installation of your packages after they have been installed on your computer. This avoids installing packages on someone else’s computer when you share your code. If they want to install them before running your code, they will need to do it by themselves.↩︎

RStudio addins, or how to make your coding life easier?

Sun, 26 Jan 2020 00:00:00 +0000

What are RStudio addins?

Although I have been using RStudio for several years, I only recently discovered RStudio addins. Since then, I am using these addins almost every time I use RStudio.

What are RStudio addins? RStudio addins are extensions which provide a simple mechanism for executing advanced R functions from within RStudio. In simpler words, when executing an addin (by clicking a button in the Addins menu), the corresponding code is executed without you having to write the code. If it is still not clear, remember that for importing a dataset in RStudio, you have two options:

import it by writing the code (thanks to the read.csv() function for instance)
or you can import it by clicking on the “Import Dataset” button in the Environment pane, set the importing settings, then click on “Import”

An RStudio addin is exactly like the Import Dataset button but for other common functionalities. So you could write code as you can import a dataset by writing code, but thanks to RStudio addins you can execute code without actually writing the necessary code. By using the RStudio addins, RStudio will run the required code for you. RStudio addins can be as simple as a function that inserts a commonly used snippet of code, and as complex as a Shiny application that accepts input from the user to draw a plot. RStudio addins have the advantage that they allow you to execute complex and advanced code much more easily than if you would have to write it yourself.

I believe addins are worth trying for all R users. Beginners will have the possibility to use functions that they would not have used otherwise because the code is too complex, whereas advanced users may find them useful to speed up the writing of their code in some circumstances. For other tips in R, see the article “Tips and tricks in RStudio and R Markdown”.

Installation

RStudio Addins are distributed as R packages. So before being able to use them, you need to install them.

You can install an addin exactly the same way you install a package: install.packages("name_of_addin"). Once you have installed the R package that contains the addin, it will immediately become available within RStudio, via the Addins menu located at the top.

RStudio addins toolbar

See this guide if you want to install a personal package as addin. In short, you have to create an R package, put your functions in a specific file and RStudio will automatically discover and register these addins when your package is installed.

Addins

If you are still not convinced, see below a list of the addins I find most useful together with concrete examples in the following sections.

Note that this list is not exhaustive and you are likely to find others useful too depending on what type of analyses you do on RStudio. Feel free to comment at the end of the article to let me know (and other readers) addins you found worth using.

Esquisse

{esquisse} is an addin developed by a French company called dreamRs. Here is how they define it:

This addin allows you to interactively explore your data by visualizing it with the ggplot2 package. It allows you to draw bar plots, curves, scatter plots, histograms, boxplot and sf objects, then export the graph or retrieve the code to reproduce the graph.

With this addin you can easily create beautiful graphs from the {ggplot2} package and the best part according to me is that you can retrieve the code to reproduce the graph. Compared to the default {graphics} package, it is true that graphs from the {ggplot2} package look usually better but the code is also longer and more complex. With this addin, you can draw graphs from the {ggplot2} package by dragging and dropping variables of interest in an user-friendly and interactive window, and then use the generated code in your script.

For the sake of illustration, let’s say we want to create a scatter plot of the variables Sepal.Length and Petal.Length of the dataset iris and color the points by the variable Species. For this, follow these steps:

load the dataset and rename it:¹

dat <- iris

install the package {esquisse}. This must be done only once

install.packages("esquisse")

open the ‘ggplot2’ builder from the RStudio Addins menu:

Step 3: Open ‘ggplot2’ builder from the RStudio addins menu

Select the dataset you want to work on (in this case dat) and click on “Validate imported data” after checking that the number of observations and variables are correct (green box):

Step 4: Select dataset and validate the imported data

Drag and drop the variables of interest in the corresponding areas. In this case, we would like to draw a scatter plot of the variables Sepal.Length and Petal.Length and color points based on the variable Species:

Step 5: Drag and drop the variables in the corresponding areas

Click on “</> Export & code” at the bottom right of the window. You can either copy the code and paste it where you want to place it in your script, or you can click on “Insert code in script” to place the code where your cursor is located in your script:

Step 6: Retrieve the code to use it in your script

If you chose the second option, the following code should appear where your cursor was located:

ggplot(dat) +
  aes(x = Sepal.Length, y = Petal.Length, colour = Species) +
  geom_point(shape = "circle", size = 1.5) +
  scale_color_hue(direction = 1) +
  theme_minimal()

Many different options and customizations are possible (e.g., axis labels, colors, legend position, theme, data filtering, etc.). For this, use the buttons located at the bottom of the window (“Labels & title”, “Plot options” and “Data”). You can see the changes instantly in the window, and when the plot corresponds to your needs, export the code into your script.

I will not go into more details regarding the different types of plots and the customizations, but make sure to try other types of plots by moving variables and customize it to see what is possible.

ggThemeAssist

Note that there is another addin called {ggThemeAssist} which helps you to edit the theme() layer of a {ggplot2} plot.

This layer allows you to modify the appearance of the background, grids, axes, labels, legend, (sub)title, caption, etc.

To use this addin:

Write the code of the plot in {ggplot2} (it does not work for a plot written in base R)
Highlight the code
Select the {ggThemeAssist} addin in the addins menu
Customize your plot according to your needs
Click on the button Done and your code will be edited with your changes:

ggplot(dat) +
  aes(x = Sepal.Length, y = Petal.Length, colour = Species) +
  geom_point(shape = "circle", size = 1.5) +
  scale_color_hue(direction = 1) +
  theme_minimal() +
  theme(
    panel.grid.major = element_line(linetype = "blank"),
    panel.grid.minor = element_line(linetype = "blank"),
    legend.position = c(0.88, 0.22)
  ) +
  theme(plot.caption = element_text(face = "italic")) +
  labs(
    title = "Sepal & petal length by species",
    x = "Sepal length", y = "Petal length",
    caption = "Data: iris"
  )

Find more information about the theme() layer in this cheatsheet.

Questionr

The {questionr} addin is useful for survey analysis and when dealing with factor variables. With this addin, you can easily reorder and recode factor variables. The addin also allows to easily transform a numeric variable into factors (i.e., categorize a continuous variable) thanks to the cut() function. Like any other addin, after having installed the {questionr} package, you should see it appearing in the addins menu at the top. From the addins dropdown menu, choose whether you want to reorder or recode your factor variable, or categorize your numeric variable.

Recoding factors

Instead of writing the recode() function from the {dplyr} package, we can use the {questionr} addin.

For this example, let’s say we want to recode the Species variable to shorten the length of the factors, and then store this new variable as Species_rec:

Step 1: Select the variable to be recoded and the recoding settings

Step 2: Specify the names of the new factors

Step 3: Check the results thanks to the contingency table and use the code at the top

Reordering factors

Similar to recoding, we can reorder factors thanks to the {questionr} addin. Let’s say we want to reorder the 3 factors of the Species variable such that the order is versicolor then virginica and finally setosa. This can be done as follows:

Step 1: Select the variable to reorder and the new variable name

Step 2: Specify the order you want

Step 3: Use the code at the top in your script

Categorize a numeric variable

The {questionr} addin also allows to transform a numeric variable into a categorical variable. This is often done for the age, when age is transformed into age groups for instance. For this example, let’s say we want to create 3 categories of the variable Sepal.Length:

Step 1: Choose the variable to be transformed and the new variable name

Step 2: Set the number of breaks equal to 3

(Try by yourself the other cutting methods and see the results directly in the window.)

Step 3: Check the result thanks to the barplot at the bottom

Step 4: Use the code in your script

Remedy

If you often write in R Markdown, the {remedy} addins will greatly facilitate your work. The addins allow you to add bold, creating lists, urls, italics, title (H1 to H6), footnote, etc. in an efficient way. I reckon that some of these tasks can be done faster by using the code directly rather than by going through the addins menu and then selecting the transformations you want. However, I personally cannot remember the code for all transformations, and it is quicker to apply it through the menu than by searching for the answer on Google or on your Markdown cheat sheet.

Styler

The {styler} addin allows to reformat your code in a more readable format by running styler::tidyverse_style(). It works both for R scripts and R Markdown documents. You can either reformat the selected code, the active file or the active package. I find this addin particularly useful before sharing or publishing my code, so that it respects the most common styling guidelines for code.

For instance, a piece of code like this:

1+1
#this is a comment
  for(i in 1:10){if(!i%%2){next}
print(i)
 }

becomes much more neat and readable:

1 + 1
# this is a comment
for (i in 1:10) {
  if (!i %% 2) {
    next
  }
  print(i)
}

Snakecaser

The {snakecaser} addins converts a character string to the snake case styling. Snake case styling is the practice of writing character strings of several words separated by space into character strings with words separated with an underscore (_). Moreover, it replaces capital letters with lowercases. For instance, the following string:

This is the Test 1

will be transformed to:

this_is_the_test_1

The snake case styling is particularly useful (and even recommended by many R users) for variables, functions and file names, etc.

ViewPipeSteps

Thanks to a reader of this article, I discovered the ViewPipeSteps addin. This addin allows to print or view the output of your pipe chain after each step.

For instance, here is a chain with the dataset diamonds:

library(tidyverse)

diamonds %>%
  select(carat, cut, color, clarity, price) %>%
  group_by(color) %>%
  summarise(n = n(), price = mean(price)) %>%
  arrange(desc(color))

## # A tibble: 7 × 3
##   color     n price
##   <ord> <int> <dbl>
## 1 J      2808 5324.
## 2 I      5422 5092.
## 3 H      8304 4487.
## 4 G     11292 3999.
## 5 F      9542 3725.
## 6 E      9797 3077.
## 7 D      6775 3170.

If you are unsure about your pipe chain or want to debug it, you can view the output after each step by highlighting your entire chain then clicking on “View Pipe Chain Steps” from the addins menu:

From the addins menu, you can choose to either print the result to the console, or view the result in a new pane (as if you called the function View() after each step of the pipe). Clicking on View Pipe Chain Steps will open a new window with the output of each step:

Note that you have to use the following command to install the ViewPipeSteps addin:

devtools::install_github("daranzolin/ViewPipeSteps")
library(ViewPipeSteps)

Now, you have no more excuses to use this pipe operator!

Ymlthis

ymlthis addin to easily write YAML header

The ymlthis addin makes it easy to write YAML front matter for R Markdown and related documents. The addin will create YAML for you and put it in a file, such as an .Rmd file, or on your clipboard.

This addin is particularly useful if you want to write (more complicated) YAML header.

Reprex

If you often ask the R community for help, this addin may be very useful!

It is important to remember that when you ask someone for help, you have to help them help you by giving a precise and clear overview of your problem. This helps the community to quickly understand your issue and thus reduces the response time.

In most cases, this involves providing a reproducible example, that is to say a piece of code (as small and as readable as possible) reproducing the problem encountered.

The {reprex} addin allows you to transform your reproducible example so that it can easily be shared on platforms such as GitHub, Stack Overflow, RStudio community, etc. The layout of your reproducible example will be adapted to the platform, and you can even include information about your R session.²

Here is how to use it steps by steps:

First create your reproducible example in R (remember to keep it as short and readable as possible to save time to potential helpers):

Minimal reproducible example

Select the {reprex} addin in the list of addins:

Select the {reprex} addin

Select the platform on which you will post your issue (and check “Append session info” if you want to display your session info at the end of your reproducible example):

Set options in the {reprex} addin

You can now see the output of your reproducible example (right panel), but more importantly, it has been copied in your clipboard and it is now ready to be pasted on the platform of your choice:

Your reprex is copied in your clipboard

Paste your reprex on the platform of your choice (here, it is posted as an issue on GitHub):

Paste your reprex

Check the final result of your reproducible example:

Final result

(Visit this GitHub issue to see the final result.)

Blogdown

I put this addin at the end of the list because it will be of interest for only a limited number of RStudio users: people maintaining a blog written in R like this one (see why I recommend everyone to start a technical blog).

The most useful functionalities in this addin are the following:

New post: create a new post with blogdown::new_post(). It can be used to create new pages as well, not only posts
Insert image: insert an external image into a blog post
Update metadata: update the title, author, date, categories and tags of the current blog post
Serve site: run blogdown::serve_site() to live preview your website locally

Conclusion

Thanks for reading.

I hope that you will find these addins useful for your future R-related projects. See other tips and tricks in RStudio and R Markdown.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

You actually do not need to rename it. However, I often rename the datasets I work on with the same generic name dat so when I reuse codes from a previous project for a new project, I do not have to change the name of the dataset in the code.↩︎
Thanks to Josep for the suggestion.↩︎

Tips and tricks in RStudio and R Markdown

Tue, 21 Jan 2020 00:00:00 +0000

If you have the chance to work with an experienced programmer, you may be amazed by how fast she can write code. In this article, I share some tips and shortcuts you can use in RStudio and R Markdown to speed up the writing of your code.

Run code

You most probably already know this shortcut but I still mention it for new R users. From your script you can run a chunk of code with:

command + Enter on Mac
Ctrl + Enter on Windows

Insert a comment in R and R Markdown

To insert a comment:

command + Shift + C on Mac
Ctrl + Shift + C on Windows

This shortcut can be used both for:

R code when you want to comment your code. It will add a # at the beginning of the line
for text in R Markdown. It will add  around the text

Note that if you want to comment more than one line, select all the lines you want to comment then use the shortcut. If you want to uncomment a comment, apply the same shortcut.

Knit a R Markdown document

You can knit R Markdown documents by using this shortcut:

command + Shift + K on Mac
Ctrl + Shift + K on Windows

Code snippets

Code snippets is usually a few characters long and is used as a shortcut to insert a common piece of code. You simply type a few characters then press Tab and it will complete your code with a larger code. Tab is then used again to navigate through the code where customization is required. For instance, if you type fun then press Tab, it will auto-complete the code with the required code to create a function:

name <- function(variables) {
  
}

Pressing Tab again will jump through the placeholders for you to edit it. So you can first edit the name of the function, then the variables and finally the code inside the function (try by yourself!).

There are many code snippets by default in RStudio. Here are the code snippets I use most often:

lib to call library()

library(package)

mat to create a matrix

matrix(data, nrow = rows, ncol = cols)

if, el, and ei to create conditional expressions such as if() {}, else {} and else if () {}

if (condition) {
  
}

else {
  
}

else if (condition) {
  
}

fun to create a function

name <- function(variables) {

}

for to create for loops

for (variable in vector) {

}

ts to insert a comment with the current date and time (useful if you have very long code and share it with others so they see when it has been edited)

# Tue Jan 21 20:20:14 2020 ------------------------------

shinyapp every time I create a new shiny app

library(shiny)

ui <- fluidPage()

server <- function(input, output, session) {

}

shinyApp(ui, server)

You can see all default code snippets and add yours by clicking on Tools > Global Options… > Code (left sidebar) > Edit Snippets…

Ordered list in R Markdown

In R Markdown, when creating an ordered list such as this one:

Item 1
Item 2
Item 3

Instead of bothering with the numbers and typing

1. Item 1
2. Item 2
3. Item 3

you can simply type

1. Item 1
1. Item 2
1. Item 3

for the exact same result (try it yourself or check the code of this article!). This way you do not need to bother which number is next when creating a new item.

To go even further, any numeric will actually render the same result as long as the first item is the number you want to start from. For example, you could type:

1. Item 1
7. Item 2
3. Item 3

which renders

Item 1
Item 2
Item 3

However, I suggest always using the number you want to start from for all items because if you move one item at the top, the list will start with this new number. For instance, if we move 7. Item 2 from the previous list at the top, the list becomes:

7. Item 2
1. Item 1
3. Item 3

which incorrectly renders

Item 2
Item 1
Item 3

New code chunk in R Markdown

When editing R Markdown documents, you will need to insert a new R code chunk many times. The following shortcuts will make your life easier:

command + option + I on Mac (or command + alt + I depending on your keyboard)
Ctrl + ALT + I on Windows

New R code chunk in R Markdown

Reformat code

A clear and readable code is always easier and faster to read (and look more professional when sharing it to collaborators). To automatically apply the most common coding guidelines such as whitespaces, indents, etc., use:

cmd + Shift + A on Mac
Ctrl + Shift + A on Windows

So for example the following code which does not respect the guidelines (and which is not easy to read):

1+1
  for(i in 1:10){if(!i%%2){next}
print(i)
 }

becomes much more neat and readable:

1 + 1
for (i in 1:10) {
  if (!i %% 2) {
    next
  }
  print(i)
}

RStudio addins

RStudio addins are extensions which provide a simple mechanism for executing advanced R functions from within RStudio. In simpler words, when executing an addin (by clicking a button in the Addins menu), the corresponding code is executed without you having to write the code. RStudio addins have the advantage that they allow you to execute complex and advanced code much more easily than if you would have to write it yourself.

The addin I use most often is probably the {esquisse} addin, which allows to draw plots with the {ggplot2} package in a user-friendly and interactive way, and without having to write the code myself.

RStudio addins are quite diverse and require a more detailed explanation, so I wrote an article focusing on these addins. See the article here.

`{pander}` and `{report}` for aesthetics

The pander() function from the {pander} package is very useful for R Markdown documents and reporting. It is not actually a shortcut but it greatly improves the aesthetics of R outputs.

For instance, see below the difference between the default output of a Chi-square test of independence and the output from the same test with the pander() function (using the diamonds dataset from the {ggplot2} package):

library(ggplot2)
dat <- diamonds

test <- chisq.test(table(dat$cut, dat$color))
test

## 
##  Pearson's Chi-squared test
## 
## data:  table(dat$cut, dat$color)
## X-squared = 310.32, df = 24, p-value < 2.2e-16

library(pander)
pander(test)

Pearson’s Chi-squared test: `table(dat$cut, dat$color)`
Test statistic	df	P value
310.3	24	1.395e-51 * * *

All information that you need are displayed in an elegant table. The pander() function works on many statistical tests (not to say all of them, but I have not tried it on all available tests in R) and on regression models:

# Linear model with lm()
model <- lm(price ~ carat + x + y + z,
  data = dat
)
model

## 
## Call:
## lm(formula = price ~ carat + x + y + z, data = dat)
## 
## Coefficients:
## (Intercept)        carat            x            y            z  
##      1921.2      10233.9       -884.2        166.0       -576.2

pander(model)

Fitting linear model: price ~ carat + x + y + z
	Estimate	Std. Error	t value	Pr(>\|t\|)
(Intercept)	1921	104.4	18.41	1.977e-75
carat	10234	62.94	162.6	0
x	-884.2	40.47	-21.85	2.317e-105
y	166	25.86	6.421	1.365e-10
z	-576.2	39.28	-14.67	1.277e-48

The pander function also makes datasets, tables, vectors, etc. more readable in R Markdown output. For example, see the differences below:

head(dat)[1:7] # first 6 observations of the first 7 variables

## # A tibble: 6 × 7
##   carat cut       color clarity depth table price
##   <dbl> <ord>     <ord> <ord>   <dbl> <dbl> <int>
## 1  0.23 Ideal     E     SI2      61.5    55   326
## 2  0.21 Premium   E     SI1      59.8    61   326
## 3  0.23 Good      E     VS1      56.9    65   327
## 4  0.29 Premium   I     VS2      62.4    58   334
## 5  0.31 Good      J     SI2      63.3    58   335
## 6  0.24 Very Good J     VVS2     62.8    57   336

pander(head(dat)[1:7])

carat	cut	color	clarity	depth	table	price
0.23	Ideal	E	SI2	61.5	55	326
0.21	Premium	E	SI1	59.8	61	326
0.23	Good	E	VS1	56.9	65	327
0.29	Premium	I	VS2	62.4	58	334
0.31	Good	J	SI2	63.3	58	335
0.24	Very Good	J	VVS2	62.8	57	336

summary(dat) # main descriptive statistics

##      carat               cut        color        clarity          depth      
##  Min.   :0.2000   Fair     : 1610   D: 6775   SI1    :13065   Min.   :43.00  
##  1st Qu.:0.4000   Good     : 4906   E: 9797   VS2    :12258   1st Qu.:61.00  
##  Median :0.7000   Very Good:12082   F: 9542   SI2    : 9194   Median :61.80  
##  Mean   :0.7979   Premium  :13791   G:11292   VS1    : 8171   Mean   :61.75  
##  3rd Qu.:1.0400   Ideal    :21551   H: 8304   VVS2   : 5066   3rd Qu.:62.50  
##  Max.   :5.0100                     I: 5422   VVS1   : 3655   Max.   :79.00  
##                                     J: 2808   (Other): 2531                  
##      table           price             x                y         
##  Min.   :43.00   Min.   :  326   Min.   : 0.000   Min.   : 0.000  
##  1st Qu.:56.00   1st Qu.:  950   1st Qu.: 4.710   1st Qu.: 4.720  
##  Median :57.00   Median : 2401   Median : 5.700   Median : 5.710  
##  Mean   :57.46   Mean   : 3933   Mean   : 5.731   Mean   : 5.735  
##  3rd Qu.:59.00   3rd Qu.: 5324   3rd Qu.: 6.540   3rd Qu.: 6.540  
##  Max.   :95.00   Max.   :18823   Max.   :10.740   Max.   :58.900  
##                                                                   
##        z         
##  Min.   : 0.000  
##  1st Qu.: 2.910  
##  Median : 3.530  
##  Mean   : 3.539  
##  3rd Qu.: 4.040  
##  Max.   :31.800  
##

pander(summary(dat))

Table continues below
carat	cut	color	clarity	depth
Min. :0.2000	Fair : 1610	D: 6775	SI1 :13065	Min. :43.00
1st Qu.:0.4000	Good : 4906	E: 9797	VS2 :12258	1st Qu.:61.00
Median :0.7000	Very Good:12082	F: 9542	SI2 : 9194	Median :61.80
Mean :0.7979	Premium :13791	G:11292	VS1 : 8171	Mean :61.75
3rd Qu.:1.0400	Ideal :21551	H: 8304	VVS2 : 5066	3rd Qu.:62.50
Max. :5.0100	NA	I: 5422	VVS1 : 3655	Max. :79.00
NA	NA	J: 2808	(Other): 2531	NA

table	price	x	y	z
Min. :43.00	Min. : 326	Min. : 0.000	Min. : 0.000	Min. : 0.000
1st Qu.:56.00	1st Qu.: 950	1st Qu.: 4.710	1st Qu.: 4.720	1st Qu.: 2.910
Median :57.00	Median : 2401	Median : 5.700	Median : 5.710	Median : 3.530
Mean :57.46	Mean : 3933	Mean : 5.731	Mean : 5.735	Mean : 3.539
3rd Qu.:59.00	3rd Qu.: 5324	3rd Qu.: 6.540	3rd Qu.: 6.540	3rd Qu.: 4.040
Max. :95.00	Max. :18823	Max. :10.740	Max. :58.900	Max. :31.800
NA	NA	NA	NA	NA

table(dat$cut, dat$color) # contingency table

##            
##                D    E    F    G    H    I    J
##   Fair       163  224  312  314  303  175  119
##   Good       662  933  909  871  702  522  307
##   Very Good 1513 2400 2164 2299 1824 1204  678
##   Premium   1603 2337 2331 2924 2360 1428  808
##   Ideal     2834 3903 3826 4884 3115 2093  896

pander(table(dat$cut, dat$color))

	D	E	F	G	H	I	J
Fair	163	224	312	314	303	175	119
Good	662	933	909	871	702	522	307
Very Good	1513	2400	2164	2299	1824	1204	678
Premium	1603	2337	2331	2924	2360	1428	808
Ideal	2834	3903	3826	4884	3115	2093	896

names(dat) # variable names

##  [1] "carat"   "cut"     "color"   "clarity" "depth"   "table"   "price"  
##  [8] "x"       "y"       "z"

pander(names(dat))

carat, cut, color, clarity, depth, table, price, x, y and z

rnorm(4) # generates 4 observations from a standard normal distribution

## [1]  1.3709584 -0.5646982  0.3631284  0.6328626

pander(rnorm(4))

0.4043, -0.1061, 1.512 and -0.09466

This trick is particularly useful when writing in R Markdown, as the generated document will look much nicer.

Another trick for the aesthetics is the report() function from the {report} package.

Similar to pander(), the report() function allows to report test results in a more readable way—but it also interprets results for you. See for example with an ANOVA:

# install.packages("remotes")
# remotes::install_github("easystats/report") # You only need to do that once
library("report") # Load the package every time you start R

report(aov(price ~ cut,
  data = dat
))

## The ANOVA (formula: price ~ cut) suggests that:
## 
##   - The main effect of cut is statistically significant and small (F(4, 53935) =
## 175.69, p < .001; Eta2 = 0.01, 95% CI [0.01, 1.00])
## 
## Effect sizes were labelled following Field's (2013) recommendations.

In addition to the p-value and the test statistic, the result of the test is displayed and interpreted for you.

Note that the report() function can be used for other analyses. See more examples in the package’s documentation.

Extract equation model with `{equatiomatic}`

If you often need to write equations corresponding to statistical models in R Markdown reports, the {equatiomatic} will help you to save time.

Here is a basic example with a simple linear regression using the same dataset as above (i.e., diamonds from {ggplot2}):

# install.packages("equatiomatic")
library(equatiomatic)

# fit a basic multiple linear regression model
model <- lm(price ~ carat,
  data = dat
)

extract_eq(model,
  use_coefs = TRUE
)

\[ \operatorname{\widehat{price}} = -2256.36 + 7756.43(\operatorname{carat}) \]

If the equation is long, you can display it on multiple lines by adding the argument wrap = TRUE:

model <- lm(price ~ carat + x + y + z + depth,
  data = dat
)

extract_eq(model,
  use_coefs = TRUE,
  wrap = TRUE,
  terms_per_line = 2
)

\[ \begin{aligned} \operatorname{\widehat{price}} &= 12196.69 + 10615.5(\operatorname{carat})\ - \\ &\quad 1369.67(\operatorname{x}) + 97.6(\operatorname{y})\ + \\ &\quad 64.2(\operatorname{z}) - 156.62(\operatorname{depth}) \end{aligned} \]

Note that:

If you use it in R Markdown, you need to add results = 'asis' for that specific code chunk, otherwise the equation will be rendered as a LaTeX equation
At the time of writing, it works only for PDF and HTML output and not for Word
The default number of terms per line is 4. You can change that with the terms_per_line argument
{equatiomatic} supports output from logistic regression as well. See all supported models in the vignette
If you need the theoretical model without the actual parameter estimates, remove the use_coefs argument:

extract_eq(model,
  wrap = TRUE
)

\[ \begin{aligned} \operatorname{price} &= \alpha + \beta_{1}(\operatorname{carat}) + \beta_{2}(\operatorname{x}) + \beta_{3}(\operatorname{y})\ + \\ &\quad \beta_{4}(\operatorname{z}) + \beta_{5}(\operatorname{depth}) + \epsilon \end{aligned} \]

In that case, I prefer to use $\beta_0$ as intercept instead of $\alpha$. You can change that with the intercept = "beta" argument:

extract_eq(model,
  wrap = TRUE,
  intercept = "beta"
)

\[ \begin{aligned} \operatorname{price} &= \beta_{0} + \beta_{1}(\operatorname{carat}) + \beta_{2}(\operatorname{x}) + \beta_{3}(\operatorname{y})\ + \\ &\quad \beta_{4}(\operatorname{z}) + \beta_{5}(\operatorname{depth}) + \epsilon \end{aligned} \]

Print model’s parameters

Thanks to the print_html() and model_parameters() functions from the {parameters} packages, you can print a summary of a model in a nicely formatted way to make the output more readable in your HTML file. See for instance with the multiple linear regression presented above:

library(parameters)
library(gt)

print_html(model_parameters(model, summary = TRUE))

Parameter	Coefficient	SE	95% CI	t(53934)	p
(Intercept)	12196.69	367.64	(11476.10, 12917.27)	33.18	< .001
carat	10615.50	63.81	(10490.43, 10740.56)	166.37	< .001
x	-1369.67	43.48	(-1454.89, -1284.45)	-31.50	< .001
y	97.60	25.76	(47.10, 148.10)	3.79	< .001
z	64.20	44.75	(-23.51, 151.91)	1.43	0.151
depth	-156.62	5.38	(-167.16, -146.09)	-29.13	< .001
Model: price ~ carat + x + y + z + depth (53940 Observations) Residual standard deviation: 1512.175 (df = 53934) R2: 0.856; adjusted R2: 0.856

Pipe operator `%>%`

If you are using the {dplyr}, {tidyverse} or {magrittr} packages often, here is a shortcut for the pipe operator %>%:

command + Shift + M on Mac
Ctrl + Shift + M on Windows

Others

Similar to many other programs, you can also use:

command + Shift + N on Mac and Ctrl + Shift + N on Windows to open a new R Script
command + S on Mac and Ctrl + S on Windows to save your current script or R Markdown document

Conclusion

Thanks for reading.

I hope you find these tips and tricks useful. If you are using others, feel free to share them in the comment section. See this starting guide in R Markdown if you are not familiar with it.

As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion.

Tips on Stats and R

10 potential career options with a degree in statistics

Introduction

What types of jobs are available?

Statistician

Data scientist, data/business analyst, data engineer or machine learning engineer

Actuary or actuarial analyst

Financial (risk) analyst, investment analyst, financial trader, financial manager or quantitative analyst

Business intelligence analyst

Operational researcher or quality control analyst

Market or survey researcher

Economist or econometrician

(Freelance) consultant

Teacher

PhD student

Conclusion

Top 10 errors in R and how to fix them

Introduction

1. Unmatched parentheses, curly braces, square brackets or quotes

Solution

2. Using a function that is not installed or loaded

Solution

3. Typos in function, variable, dataset, object or package names

Solution

4. Missing, incorrect or misspelled arguments in functions

Solution

5. Wrong, inappropriate or inconsistent data types

Solution

6. Forgetting the + sign in ggplot2

Solution

7. Misunderstanding between = and ==

8. Undefined columns selected

Solution

9. Problem when importing or using the wrong data file

10. Problem when using the $ operator

$ operator is invalid for atomic vectors

Solution

object of type ‘closure’ is not subsettable

Warnings

NAs introduced by coercion

Removed … rows containing non-finite values (stat_bin())

Conclusion

How to keep yourself updated with the latest R news?

Introduction

How do I keep track?

Twitter

Newsletters

Conclusion

How to do a t-test or ANOVA for more than one variable at once in R?

Introduction

Perform multiple tests at once

Concise and easily interpretable results

T-test

Additional p-value adjustment methods

ANOVA

To go even further

Update with the {ggstatsplot} package

Conclusion

References

An efficient way to install and load R packages

What is an R package and how to use it?

Inefficient way to install and load R packages

More efficient way

Most efficient way

{pacman} package

{librarian} package

Conclusion

RStudio addins, or how to make your coding life easier?

What are RStudio addins?

Installation

Addins

Esquisse

ggThemeAssist

Questionr

Recoding factors

Reordering factors

Categorize a numeric variable

Remedy

Styler

Snakecaser

Update with the `{ggstatsplot}` package

`{pacman}` package

`{librarian}` package

`{pander}` and `{report}` for aesthetics

Extract equation model with `{equatiomatic}`

Pipe operator `%>%`