Abstract

In a Monte Carlo analysis of single-subject data, Type I and Type II error rates were compared for various statistical tests of the significance of treatment effects. Data for 5,000 subjects in each of 6 treatment effect size groups were computer simulated, and 2 types of treatment effects were simulated in the dependent variable during intervention phases, resulting in mean change in level or mean change in slope. Significance test statistics were based on explained variance indicated by squared multiple correlations using multiple regression models that were closely specified to the treatment effect (termed "specific" tests) or that modeled effects beyond those in the data (termed "general" tests). These tests were applied as both parametric and nonparametric (randomization) tests of treatment effects. Results indicate that parametric tests exhibit Type I error control and superior power for independent data, but fail to control Type I error rates for dependent data with autocorrelated observations. In contrast, randomization tests exhibit Type I error control even with serially correlated data, but provide inadequate power for detecting treatment effects and become increasingly conservative with increasing autocorrelation. Implications for analysis of single-subject data series are discussed. (Contains 4 figures, 5 tables, and 38 references.) (Author/SLD)

Citation

Aaron, B.C. & Kromrey, J.D. (1998). Randomization Regression Tests for Single-Subject Data. Presented at Eastern Educational Research Association 1998. Retrieved August 14, 2024 from https://www.learntechlib.org/p/89683/.

This record was imported from ERIC on April 18, 2013. [Original Record]

ERIC is sponsored by the Institute of Education Sciences (IES) of the U.S. Department of Education.

Copyright for this record is held by the content creator. For more details see ERIC's copyright policy.

Keywords

• Computer Simulation
• Effect Size
• Monte Carlo Methods
• Nonparametric Statistics
• Regression (Statistics)