R Handbook: Paired t-test (2024)

The paired t-test is commonly used. It compares themeans of two populations of paired observations by testing if the differencebetween pairs is statistically different from zero.

Appropriate data

• Two-sample data. That is, one measurement variable in twogroups or samples

• Dependent variable is interval/ratio, and is continuous

• Independent variable is a factor with two levels. That is,two groups

• Data are paired. That is, the measurement for each observationin one group can be paired logically or by subject to a measurement in theother group

• The distribution of the difference of paired measurements isnormally distributed

• Moderate skewness is permissible if the data distribution isunimodal without outliers

Hypotheses

• Null hypothesis: The population mean of the differences between paired observations is equal to zero.

• Alternative hypothesis (two-sided): The population mean of the differences between paired observations is not equal to zero.

Interpretation

Reporting significant results as “Mean of variable Y forgroup A was different than that for group B.” or “Variable Y increased frombefore to after” is acceptable.

Other notes and alternative tests

• The nonparametric analogue for this test is the two-samplepaired rank-sum test.

• Power analysis for the paired t-test can be found at Mangiafico (2015) in the “References”section.

Packages used in this chapter

The packages used in this chapter include:

• psych

• rcompanion

• lsr

The following commands will install these packages if theyare not already installed:

Paired t-test example

In the following example, Dumbland Extension had adultstudents fill out a financial literacy knowledge questionnaire both before andafter completing a home financial management workshop. Each student’s scorebefore and after was paired by student.

Note in the following data that the students’ names arerepeated, so that there is a before score for student a and an afterscore for student a.

Since the data is in long form, we’ll order by Time,then Student to be sure the first observation for Before isstudent a and the first observation for After is student a,and so on.

Check the number of paired observations

It is helpful to create a table of the counts of observationsto be sure that there is one observation for each student for each time period.

xtabs(~ Student + Time,
data = Data)

Histogram of difference data

A histogram with a normal curve imposed will be used tocheck if the paired differences between the two populations is approximatelynormal in distribution.

First, two new variables, Before and After, are created byextracting the values of Score for observations with the Timevariable equal to Before or After, respectively.

Note that for this code to make sense, the first observationfor Before is student a and the first observation for Afteris student a, and so on.

Plot the paired data

Scatter plot with one-to-one line

Paired data can visualized with a scatter plot of the pairedcases. In the plot below, points that fall above and to the left of the blueline indicate cases for which the value for After was greater than for Before.

Note that the points in the plot are jittered slightly sothat points that would fall directly on top of one another can be seen.

First, two new variables, Before and After,are created by extracting the values of Score for observations with the Timevariable equal to Before or After, respectively.

Note that for this code to make sense, the first observationfor Before is student a and the first observation for Afteris student a, and so on.

A variable Names is also created for point labels.

Bar plot of differences

Paired data can also be visualized with a bar chart ofdifferences. In the plot below, bars with a value greater than zero indicatecases for which the value for After was greater than for Before.

New variables are first created for Before, After,and their Difference.

Note that for this code to make sense, the first observationfor Before is student a and the first observation for Afteris student a, and so on.

A variable Names is also created for bar labels.

Paired t-test

Note that the output shows the p-value for the test,and the simple difference in the means for the two groups.

Note that for this test to be conducted correctly, the firstobservation for Before is student a and the first observation forAfter is student a, and so on.

t.test(Score ~ Time,
data = Data,
paired = TRUE)

Effect size

Cohen’s d can be used as an effect size statistic fora paired t-test. It is calculated as the difference between the meansof each group, all divided by the standard deviation of the data. The standarddeviation used could be calculated from the differences between observations,or, for observations across two times, the observations in the “before” group.

It ranges from 0 to infinity, with 0 indicating no effectwhere the means are equal. In some versions, Cohen’s d can be positiveor negative depending on which mean is greater.

A Cohen’s d of 0.5 suggests that the means differ byone-half the standard deviation of the data. A Cohen’s d of 1.0suggests that the means differ by one standard deviation of the data.

Interpretation of Cohen’s d

Interpretation of effect sizes necessarily varies bydiscipline and the expectations of the experiment, but for behavioral studies,the guidelines proposed by Cohen (1988) are sometimes followed. They shouldnot be considered universal.

____________________________

Source: Cohen (1988).

Cohen’s d for paired t-test

Standard deviation calculated from differences in observations

Note that for this code to make sense, the first observationfor Before is student a and the first observation for Afteris student a, and so on.

Or we can calculate the value manually.

mean(After)

mean(Before) - mean(After)

[1] -10.2

( mean(Before) - mean(After) ) / sd(Difference)

Standard deviation calculated from the “before” observations

Note that for this code to make sense, the “before”observations must correspond to the second level of the Timevariable, corresponding to y.sd in the function.

( mean(Before) - mean(After) ) / sd(Before)

Optional readings

“Paired t–test” in McDonald, J.H. 2014. Handbookof Biological Statistics. www.biostathandbook.com/pairedttest.html.

References

“Paired t–test” in Mangiafico, S.S. 2015. AnR Companion for the Handbook of Biological Statistics, version 1.09. rcompanion.org/rcompanion/d_09.html.

Exercises Q

2. Residential properties in Dougal County rarely need phosphorus for goodturfgrass growth. As part of an extension education program, Early and RustyCuyler asked homeowners to report their phosphorus fertilizer use, in pounds ofP₂O₅ per acre, before the program and then one yearlater.

Date Homeowner P2O5
'2014-01-01' a 0.81
'2014-01-01' b 0.86
'2014-01-01' c 0.79
'2014-01-01' d 0.59
'2014-01-01' e 0.71
'2014-01-01' f 0.88
'2014-01-01' g 0.63
'2014-01-01' h 0.72
'2014-01-01' i 0.76
'2014-01-01' j 0.58
'2015-01-01' a 0.67
'2015-01-01' b 0.83
'2015-01-01' c 0.81
'2015-01-01' d 0.50
'2015-01-01' e 0.71
'2015-01-01' f 0.72
'2015-01-01' g 0.67
'2015-01-01' h 0.67
'2015-01-01' i 0.48
'2015-01-01' j 0.68

For each of the following, answer the question, and showthe output from the analyses you used to answer the question.

b. Is this an increase or a decrease?

e. What do you conclude practically? As appropriate, reportthe means before and after, the mean difference, the effect size and interpretation,whether the difference is large relative to the values, anything notable onplots, and your practical conclusions.