Skip to contents

Sensitivity Analysis for Sample Size Planning for the Standardized ANOVA Contrast From the Accuracy in Parameter Estimation (AIPE) Perspective

Source: R/ss_aipe_sc_sensitivity.R
ss_aipe_sc_sensitivity.Rd

Performs a sensitivity analysis when planning sample size from the Accuracy in Parameter Estimation (AIPE) Perspective for the standardized ANOVA contrast.

Usage

ss_aipe_sc_sensitivity(
  true_psi = NULL,
  estimated_psi = NULL,
  c_weights,
  desired_width = NULL,
  n_per_group = NULL,
  assurance = NULL,
  conf_level = 0.95,
  G = 10000,
  print_iter = TRUE,
  filename = NULL
)

Arguments

true_psi

population standardized contrast

estimated_psi

estimated standardized contrast

c_weights

the contrast weights

desired_width

the desired full width of the obtained confidence interval

n_per_group

selected sample size to use in order to determine distributional properties of at a given value of sample size

assurance

parameter to ensure that the obtained confidence interval width is narrower than the desired width with a specified degree of certainty (must be NULL or between zero and unity)

conf_level

the desired confidence interval coverage, (i.e., 1 - Type I error rate)

G

number of generations (i.e., replications) of the simulation

print_iter

to print the current value of the iterations

filename

an optional path for a comma separated file recording every replication (the realized standardized contrast, the full and the two one-sided interval widths, the three non-coverage indicators, and the two confidence limits): nothing is written when filename is NULL (the default), a new file with a header row is created otherwise, an existing file at that path is appended to, and a throwaway run should point it at tempfile(fileext = ".csv")

Value

A data.frame with columns term and value summarizing the Monte Carlo sensitivity analysis across the G replications. The term entries are: mean_psi, median_psi, sd_psi (summaries of the realized standardized contrast); mean_ci_width, median_ci_width, sd_ci_width (summaries of the full interval widths); mean_ci_width_lower and mean_ci_width_upper (mean one-sided widths, measured from the observed contrast to each limit); pct_ci_less_w (proportion of intervals at or below the target width); pct_ci_miss_low and pct_ci_miss_high (tail-specific empirical non-coverage of true_psi); total_type_I_error (overall empirical non-coverage, the sum of the two tails); and the input echoes n_per_group, total_N, true_psi, estimated_psi (NA when n_per_group was supplied instead), width, conf_level, and assurance (present only when an assurance was supplied). The proportion and Type I error rows are proportions on the 0 to 1 scale, not percentages.

References

Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61(4), 532–574. doi:10.1177/0013164401614002

Hedges, L. V. (1981). Distribution theory for Glass's Estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.

Kelley, K. (2007). Confidence intervals for standardized effect sizes: Theory, application, and implementation. Journal of Statistical Software, 20(8), 1–24. doi:10.18637/jss.v020.i08

Kelley, K., & Rausch, J. R. (2006). Sample size planning for the standardized mean difference: Accuracy in Parameter Estimation via narrow confidence intervals. Psychological Methods, 11(4), 363–385. doi:10.1037/1082-989X.11.4.363

Lai, K., & Kelley, K. (2012). Accuracy in parameter estimation for ANCOVA and ANOVA contrasts: Sample size planning via narrow confidence intervals. British Journal of Mathematical and Statistical Psychology, 65, 350–370. doi:10.1111/j.2044-8317.2011.02029.x

Steiger, J. H., & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical methods. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 221–257). Mahwah, NJ: Lawrence Erlbaum.

See also

ss_aipe_sc, ss_aipe_c, ci_nc_t

design_consequences for what a chosen design delivers: power, the Type S (sign) and Type M (exaggeration) errors of the significance filter, and the expected confidence interval width.

Author

Ken Kelley kkelley@nd.edu

Examples

# Sensitivity analysis for a standardized three-group ANOVA contrast
# (-1, 0, 1) at psi = 0.5 and target full width 0.40. G is kept small
# here so the example runs quickly; raise it for a stable sweep.
set.seed(113)
ss_aipe_sc_sensitivity(
  true_psi = 0.5, estimated_psi = 0.5,
  c_weights = c(-1, 0, 1),
  desired_width = 0.40,
  conf_level = 0.95, G = 50, print_iter = FALSE
)
#>  term                value  
#>  mean_psi            0.495  
#>  median_psi          0.493  
#>  sd_psi              0.103  
#>  mean_ci_width       0.399  
#>  median_ci_width     0.399  
#>  sd_ci_width         0.00164
#>  mean_ci_width_lower 0.2    
#>  mean_ci_width_upper 0.199  
#>  pct_ci_less_w       0.72   
#>  pct_ci_miss_low     0.02   
#>  pct_ci_miss_high    0.04   
#>  total_type_I_error  0.06   
#>  n_per_group         197    
#>  total_N             591    
#>  true_psi            0.5    
#>  estimated_psi       0.5    
#>  width               0.4    
#>  conf_level          0.95   
#> 
#> Confidence level: 95%