| Title: | Common Multiple Testing Procedures and Gatekeeping Procedures |
|---|---|
| Description: | Implementation of commonly used p-value-based and parametric multiple testing procedures (computation of adjusted p-values and simultaneous confidence intervals) and parallel gatekeeping procedures based on the methodology presented in the book "Multiple Testing Problems in Pharmaceutical Statistics" (edited by Alex Dmitrienko, Ajit C. Tamhane and Frank Bretz) published by Chapman and Hall/CRC Press 2009. |
| Authors: | Alex Dmitrienko, Eric Nantz, and Gautier Paux, with contributions by Thomas Brechenmacher |
| Maintainer: | Eric Nantz <[email protected]> |
| License: | GPL-2 |
| Version: | 0.1.1 |
| Built: | 2025-02-12 05:41:01 UTC |
| Source: | https://github.com/cran/multxpert |
Computation of adjusted -values for commonly used parametric
multiple testing procedures (single-step and step-down Dunnett procedures).
paradjp(stat,n,proc)paradjp(stat,n,proc)
stat |
Vector of test statistics. |
n |
Common sample size in each treatment group. |
proc |
Vector of character strings containing the procedure name.
This vector should include any of the following: |
This function computes adjusted -values for the single-step Dunnett procedure
(Dunnett, 1955) and step-down Dunnett procedure (Naik, 1975; Marcus, Peritz
and Gabriel, 1976) in one-sided hypothesis testing problems with a balanced
one-way layout and equally weighted null hypotheses. For more information on the
algorithms used in the function, see Dmitrienko et al. (2009, Section 2.7).
A list with the following components:
proc |
Name of procedure used. |
result |
A data frame with columns for the test statistics,
one-sided raw |
http://multxpert.com/wiki/MultXpert_package
Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L.,
Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology.
Multiple Testing Problems in Pharmaceutical Statistics.
Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and
Hall/CRC Press, New York.
Dunnett, C.W. (1955). A multiple comparison procedure for
comparing several treatments with a control. Journal of the American
Statistical Association. 50, 1096–1121.
Marcus, R. Peritz, E., Gabriel, K.R. (1976). On closed testing
procedures with special reference to ordered analysis of variance.
Biometrika. 63, 655–660.
Naik, U.D. (1975). Some selection rules for comparing processes
with a standard. Communications in Statistics. Series A.
4, 519–535.
# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute one-sided adjusted p-values for the single-step Dunnett procedure paradjp(stat, n, proc="Single-step Dunnett") # Compute one-sided adjusted p-values for the single-step and # step-down Dunnett procedures paradjp(stat, n, proc=c("Single-step Dunnett", "Step-down Dunnett"))# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute one-sided adjusted p-values for the single-step Dunnett procedure paradjp(stat, n, proc="Single-step Dunnett") # Compute one-sided adjusted p-values for the single-step and # step-down Dunnett procedures paradjp(stat, n, proc=c("Single-step Dunnett", "Step-down Dunnett"))
Computation of simultaneous confidence intervals for commonly used parametric multiple testing procedures (single-step and step-down Dunnett procedures).
parci(stat, n, est, stderror, covprob, proc)parci(stat, n, est, stderror, covprob, proc)
stat |
Vector of test statistics. |
n |
Common sample size in each treatment group. |
est |
Vector of point estimates. |
stderror |
Vector of standard errors associated with the point estimates. |
covprob |
Simultaneous coverage probability (default is 0.975). |
proc |
Vector of character strings containing the procedure
name. This vector should include any of the following:
|
This function computes lower one-sided simultaneous confidence limits for the single-step Dunnett procedure (Dunnett, 1955) and step-down Dunnett procedure (Naik, 1975; Marcus, Peritz and Gabriel, 1976) in one-sided hypothesis testing problems with a balanced one-way layout and equally weighted null hypotheses.
The simultaneous confidence intervals are computed using the methods developed in Bofinger (1987) and Stefansson, Kim and Hsu (1988). For more information on the algorithms used in the function, see Dmitrienko et al. (2009, Section 2.7).
A data frame result with columns for the test statistics, point estimates,
standard errors, adjusted -values, and lower simultaneous confidence limits
for the specified procedure.
http://multxpert.com/wiki/MultXpert_package
Bofinger, E. (1987). Step-down procedures for comparison with a control.
Australian Journal of Statistics. 29, 348–364.
Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L.,
Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology.
Multiple Testing Problems in Pharmaceutical Statistics.
Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and
Hall/CRC Press, New York.
Dunnett, C.W. (1955). A multiple comparison procedure for
comparing several treatments with a control. Journal of the American
Statistical Association. 50, 1096–1121.
Marcus, R. Peritz, E., Gabriel, K.R. (1976). On closed testing
procedures with special reference to ordered analysis of variance.
Biometrika. 63, 655–660.
Naik, U.D. (1975). Some selection rules for comparing processes
with a standard. Communications in Statistics. Series A.
4, 519–535.
Stefansson, G., Kim, W.-C., Hsu, J.C. (1988). On confidence sets in multiple comparisons. Statistical Decision Theory and Related Topics IV. Gupta, S.S., Berger, J.O. (editors). Academic Press, New York, 89–104.
# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute lower one-sided simultaneous confidence limits # for the single-step Dunnett procedure parci(stat,n,est,stderror,covprob=0.975,proc="Single-step Dunnett") # Compute lower one-sided simultaneous confidence limits # for the single-step and step-down Dunnett procedures parci(stat,n,est,stderror,covprob=0.975,proc=c("Single-step Dunnett", "Step-down Dunnett"))# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute lower one-sided simultaneous confidence limits # for the single-step Dunnett procedure parci(stat,n,est,stderror,covprob=0.975,proc="Single-step Dunnett") # Compute lower one-sided simultaneous confidence limits # for the single-step and step-down Dunnett procedures parci(stat,n,est,stderror,covprob=0.975,proc=c("Single-step Dunnett", "Step-down Dunnett"))
Computation of adjusted -values for multistage parallel
gatekeeping procedures.
pargateadjp(gateproc, independence, alpha, printDecisionRules)pargateadjp(gateproc, independence, alpha, printDecisionRules)
gateproc |
List of gatekeeping procedure parameters
in each family of null hypotheses, including the family label, vector of
raw |
independence |
Boolean indicator (TRUE, Independence condition is imposed (i.e., inferences in earlier families are independent of inferences in later families); FALSE, Independence condition is not imposed). |
alpha |
Global family-wise error rate (default is 0.05). Note that
this argument is not needed if the function is called to compute
adjusted |
printDecisionRules |
Boolean indicator for printing the decision rules for the gatekeeping procedure (default is FALSE). |
This function computes adjusted -values and generates decision rules for multistage parallel
gatekeeping procedures in hypothesis testing problems with multiple families
of null hypotheses (null hypotheses are assumed to be equally weighted within
each family) based on the methodology presented in Dmitrienko, Tamhane
and Wiens (2008) and Dmitrienko, Kordzakhia and Tamhane (2011). For more
information on parallel gatekeeping procedures (computation of adjusted -values,
independence condition, etc), see Dmitrienko and Tamhane (2009, Section 5.4).
A data frame result with columns for the family labels, procedures, procedure
parameters (truncation parameters), raw -values, and adjusted -values.
http://multxpert.com/wiki/MultXpert_package
Dmitrienko, A., Tamhane, A., Wiens, B. (2008). General multistage
gatekeeping procedures. Biometrical Journal. 50, 667–677.
Dmitrienko, A., Tamhane, A.C. (2009). Gatekeeping procedures in
clinical trials. Multiple Testing Problems in Pharmaceutical
Statistics. Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors).
Chapman and Hall/CRC Press, New York.
Dmitrienko, A., Kordzakhia, G., Tamhane, A.C. (2011). Multistage and mixture parallel gatekeeping procedures in clinical trials. Journal of Biopharmaceutical Statistics. To appear.
# Consider a clinical trial with two families of null hypotheses # Family 1: Primary null hypotheses (one-sided p-values) # H1 (Endpoint 1), p1=0.0082 # H2 (Endpoint 2), p2=0.0174 # Family 2: Secondary null hypotheses (one-sided p-values) # H3 (Endpoint 3), p3=0.0042 # H4 (Endpoint 4), p4=0.0180 # Define family label and raw p-values in Family 1 label1<-"Primary endpoints" rawp1<-c(0.0082,0.0174) # Define family label and raw p-values in Family 2 label2<-"Secondary endpoints" rawp2<-c(0.0042,0.0180) # Independence condition is imposed (Families 1 and 2 are tested # sequentually from first to last and thus adjusted p-values # in Family 1 do not depend on inferences in Family 2) independence<-TRUE # Define a two-stage parallel gatekeeping procedure which # utilizes the truncated Holm procedure in Family 1 (truncation # parameter=0.5) and regular Holm procedure in Family 2 (truncation # parameter=1) # Create a list of gatekeeping procedure parameters family1<-list(label=label1, rawp=rawp1, proc="Holm", procpar=0.5) family2<-list(label=label2, rawp=rawp2, proc="Holm", procpar=1) gateproc<-list(family1,family2) # Compute adjusted p-values pargateadjp(gateproc, independence) # Generate decision rules using a one-sided alpha=0.025 pargateadjp(gateproc, independence, alpha=0.025, printDecisionRules=TRUE)# Consider a clinical trial with two families of null hypotheses # Family 1: Primary null hypotheses (one-sided p-values) # H1 (Endpoint 1), p1=0.0082 # H2 (Endpoint 2), p2=0.0174 # Family 2: Secondary null hypotheses (one-sided p-values) # H3 (Endpoint 3), p3=0.0042 # H4 (Endpoint 4), p4=0.0180 # Define family label and raw p-values in Family 1 label1<-"Primary endpoints" rawp1<-c(0.0082,0.0174) # Define family label and raw p-values in Family 2 label2<-"Secondary endpoints" rawp2<-c(0.0042,0.0180) # Independence condition is imposed (Families 1 and 2 are tested # sequentually from first to last and thus adjusted p-values # in Family 1 do not depend on inferences in Family 2) independence<-TRUE # Define a two-stage parallel gatekeeping procedure which # utilizes the truncated Holm procedure in Family 1 (truncation # parameter=0.5) and regular Holm procedure in Family 2 (truncation # parameter=1) # Create a list of gatekeeping procedure parameters family1<-list(label=label1, rawp=rawp1, proc="Holm", procpar=0.5) family2<-list(label=label2, rawp=rawp2, proc="Holm", procpar=1) gateproc<-list(family1,family2) # Compute adjusted p-values pargateadjp(gateproc, independence) # Generate decision rules using a one-sided alpha=0.025 pargateadjp(gateproc, independence, alpha=0.025, printDecisionRules=TRUE)
Computation of adjusted -values for commonly used multiple testing
procedures based on univariate -values (Bonferroni, Holm, Hommel, Hochberg,
fixed-sequence and fallback procedures).
pvaladjp(rawp, weight, alpha, proc, printDecisionRules)pvaladjp(rawp, weight, alpha, proc, printDecisionRules)
rawp |
Vector of raw |
weight |
Vector of hypothesis weights whose sum is equal to 1 (default is a vector of equal weights). |
alpha |
Familywise error rate (default is 0.05). Note that this argument
is not needed if the function is called to compute adjusted |
proc |
Vector of character strings containing the procedure name.
This vector should include any of the following: |
printDecisionRules |
Boolean indicator for printing the decision rules for
each of the procedures specified in |
This function computes adjusted -values and generates decision rules for the Bonferroni,
Holm (Holm, 1979), Hommel (Hommel, 1988), Hochberg (Hochberg, 1988),
fixed-sequence (Westfall and Krishen, 2001) and fallback (Wiens, 2003;
Wiens and Dmitrienko, 2005) procedures.
The adjusted -values are computed using the closure principle
(Marcus, Peritz and Gabriel, 1976) in general hypothesis testing
problems (equally or unequally weighted null hypotheses).
The decision rules are generated only in hypothesis testing problems
with equally weighted null hypotheses.
For more information on the algorithms used in the function, see
Dmitrienko et al. (2009, Section 2.6).
A data frame result with columns for the raw -values, weights, and adjusted
-values for each of the procedures.
http://multxpert.com/wiki/MultXpert_package
Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L.,
Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology.
Multiple Testing Problems in Pharmaceutical Statistics.
Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and
Hall/CRC Press, New York.
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple significance testing.
Biometrika. 75, 800–802.
Holm, S. (1979). A simple sequentially rejective multiple test procedure.
Scandinavian Journal of Statistics. 6, 65–70.
Hommel, G. (1988). A stagewise rejective multiple test procedure based on a
modified Bonferroni test. Biometrika. 75, 383–386.
Marcus, R. Peritz, E., Gabriel, K.R. (1976). On closed testing
procedures with special reference to ordered analysis of variance.
Biometrika. 63, 655–660.
Westfall, P. H., Krishen, A. (2001). Optimally weighted, fixed
sequence, and gatekeeping multiple testing procedures. Journal of
Statistical Planning and Inference. 99, 25–40.
Wiens, B. (2003). A fixed-sequence Bonferroni procedure for
testing multiple endpoints. Pharmaceutical Statistics. 2, 211–215.
Wiens, B., Dmitrienko, A. (2005). The fallback procedure for evaluating a single family of hypotheses. Journal of Biopharmaceutical Statistics. 15, 929–942.
# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Null hypotheses of no treatment effect are equally weighted weight<-c(1/3,1/3,1/3) # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute degrees of freedom nu<-2*(n-1) # Compute raw one-sided p-values rawp<-1-pt(stat,nu) # Compute adjusted p-values for the Bonferroni procedure pvaladjp(rawp, weight, proc="Bonferroni") # Compute adjusted p-values for the Hommel and Fallback procedures pvaladjp(rawp, weight, proc=c("Hommel", "Fallback")) # Generate decision rules for the Holm procedure # using a one-sided alpha=0.025 pvaladjp(rawp, weight, alpha=0.025, proc="Holm", printDecisionRules=TRUE)# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Null hypotheses of no treatment effect are equally weighted weight<-c(1/3,1/3,1/3) # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute degrees of freedom nu<-2*(n-1) # Compute raw one-sided p-values rawp<-1-pt(stat,nu) # Compute adjusted p-values for the Bonferroni procedure pvaladjp(rawp, weight, proc="Bonferroni") # Compute adjusted p-values for the Hommel and Fallback procedures pvaladjp(rawp, weight, proc=c("Hommel", "Fallback")) # Generate decision rules for the Holm procedure # using a one-sided alpha=0.025 pvaladjp(rawp, weight, alpha=0.025, proc="Holm", printDecisionRules=TRUE)
Computation of simultaneous confidence intervals for selected multiple testing
procedures based on univariate -values (Bonferroni, Holm and
fixed-sequence procedures).
pvalci(rawp, est, stderror, weight, covprob, proc)pvalci(rawp, est, stderror, weight, covprob, proc)
rawp |
Vector of raw |
est |
Vector of point estimates. |
stderror |
Vector of standard errors associated with the point estimates. |
weight |
Vector of hypothesis weights whose sum is equal to 1 (default is a vector of equal weights). |
covprob |
Simultaneous coverage probability (default is 0.975). |
proc |
Vector of character strings containing the procedure name.
This vector should include any of the following: |
This function computes one-sided simultaneous confidence limits for the Bonferroni, Holm (Holm, 1979) and fixed-sequence (Westfall and Krishen, 2001) procedures in in general one-sided hypothesis testing problems (equally or unequally weighted null hypotheses).
The simultaneous confidence intervals are computed using the methods developed in Hsu and Berger (1999), Strassburger and Bretz (2008) and Guilbaud (2008). For more information on the algorithms used in the function, see Dmitrienko et al. (2009, Section 2.6).
A data frame result with columns for the raw -values, point estimates,
standard errors, weights, adjusted -values, and simultaneous confidence limits
for each of the procedures.
http://multxpert.com/wiki/MultXpert_package
Dmitrienko, A., Bretz, F., Westfall, P.H., Troendle, J., Wiens, B.L.,
Tamhane, A.C., Hsu, J.C. (2009). Multiple testing methodology.
Multiple Testing Problems in Pharmaceutical Statistics.
Dmitrienko, A., Tamhane, A.C., Bretz, F. (editors). Chapman and
Hall/CRC Press, New York.
Guilbaud, O. (2008). Simultaneous confidence regions corresponding to
Holm's stepdown procedure and other closed-testing procedures.
Biometrical Journal. 5, 678–692.
Holm, S. (1979). A simple sequentially rejective multiple test procedure.
Scandinavian Journal of Statistics. 6, 65–70.
Hsu, J.C., Berger, R.L. (1999). Stepwise confidence intervals without
multiplicity adjustment for dose-response and toxicity studies.
Journal of the American Statistical Association. 94, 468–482.
Strassburger, K., Bretz, F. (2008). Compatible simultaneous lower confidence
bounds for the Holm procedure and other Bonferroni based closed tests.
Statistics in Medicine. 27, 4914–4927.
Westfall, P. H., Krishen, A. (2001). Optimally weighted, fixed
sequence, and gatekeeping multiple testing procedures. Journal of
Statistical Planning and Inference. 99, 25–40.
# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Null hypotheses of no treatment effect are equally weighted weight<-c(1/3,1/3,1/3) # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute degrees of freedom nu<-2*(n-1) # Compute raw one-sided p-values rawp<-1-pt(stat,nu) # Compute lower one-sided simultaneous confidence limits # for the Bonferroni procedure pvalci(rawp,est,stderror,weight,covprob=0.975,proc="Bonferroni") # Compute lower one-sided simultaneous confidence limits # for the Holm and Fixed-sequence procedures pvalci(rawp,est,stderror,weight,covprob=0.975,proc=c("Holm", "Fixed-sequence"))# Consider a clinical trial conducted to evaluate the effect of three # doses of a treatment compared to a placebo with respect to a normally # distributed endpoint # Three null hypotheses of no effect are tested in the trial: # Null hypothesis H1: No difference between Dose 1 and Placebo # Null hypothesis H2: No difference between Dose 2 and Placebo # Null hypothesis H3: No difference between Dose 3 and Placebo # Null hypotheses of no treatment effect are equally weighted weight<-c(1/3,1/3,1/3) # Treatment effect estimates (mean dose-placebo differences) est<-c(2.3,2.5,1.9) # Pooled standard deviation sd<-9.5 # Study design is balanced with 180 patients per treatment arm n<-180 # Standard errors stderror<-rep(sd*sqrt(2/n),3) # T-statistics associated with the three dose-placebo tests stat<-est/stderror # Compute degrees of freedom nu<-2*(n-1) # Compute raw one-sided p-values rawp<-1-pt(stat,nu) # Compute lower one-sided simultaneous confidence limits # for the Bonferroni procedure pvalci(rawp,est,stderror,weight,covprob=0.975,proc="Bonferroni") # Compute lower one-sided simultaneous confidence limits # for the Holm and Fixed-sequence procedures pvalci(rawp,est,stderror,weight,covprob=0.975,proc=c("Holm", "Fixed-sequence"))