Skip to main content

Table 3 Simulation results for the acrylamide dataset (reference model: Logistic)

From: Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Exclusiona Selectionb Reliabilityc (Mean distance) Rank Validityd (%) Rank BMDL calculabilitye
(%)
Non-exclusion and BMDL calculationf (%) True dose-responseg (%)
None Lowest BMDL 0.4 17 99.9 1 99.8 99.8 0.0
Lowest BMD 0.3 13 99.7 5 99.8 99.8 0.1
Lowest AIC 0.1 2 89.0 16 99.8 99.8 38.3
MA-all 0.2 10 98.2 11 99.8 99.8 NA
MA-3 0.1 1 93.9 14 100.0 100.0 NA
MA AIC < 3 0.2 8 97.8 13 100.0 100.0 NA
KS Lowest BMDL 0.4 17 99.9 1 99.8 99.8 0.0
Lowest BMD 0.3 13 99.7 5 99.8 99.8 0.1
Lowest AIC 0.1 2 89.0 15 99.8 99.8 38.3
MA-all 0.2 9 98.4 10 99.8 99.8 NA
KS, BMD/BMDL Lowest BMDL 0.3 16 99.9 1 99.8 98.7 0.0
Lowest BMD 0.3 12 99.7 5 99.8 98.7 0.2
Lowest AIC 0.1 5 88.7 17 99.8 98.7 38.7
MA-all 0.2 7 98.7 9 99.8 98.7 NA
KS, BMDU/BMDL Lowest BMDL 0.3 15 99.9 1 99.8 93.9 0.0
Lowest BMD 0.2 11 99.6 8 99.8 93.9 0.2
Lowest AIC 0.1 4 88.5 18 99.8 93.9 39.0
MA-all 0.2 6 98.2 11 99.8 93.9 NA
  1. aExclusion criteria: KS, Kolmogorov-Smirnov test of goodness-of-fit; BMD/BMDL, ratio of benchmark dose (BMD10) to benchmark dose lower bound (BMDL10) with values > 10 excluded; BMDU/BMDL, ratio of benchmark dose upper bound (BMDU10) to BMDL10 with values > 10 excluded. bModel selection criteria: Lowest BMDL, model with the lowest value of BMDL10; Lowest BMD, model with the lowest value of BMD10; Lowest AIC, model with the lowest AIC value; MA-all, model averaging of all converged models. MA-3, model averaging of three models with three smallest AIC values. MA-AIC, model averaging of all models with AIC values < 3 compared with the best model that yielded the minimum AIC. cReliability (Mean distance), measured as the mean distance between unbiased BMDL10 and calculated BMDL10 followed by rank. dValidity (%), measured as the iterations that satisfied calculated BMDL10 lower than unbiased BMD10 followed by rank. eBMDL calculability (%), measured as the iterations that yielded BMDL in the model selection criterion. fNon-exclusion and BMDL calculation (%), measured as the iterations that yielded BMDL in the model selection criterion along with exclusion criteria. gTrue dose response (%), measured as the default model selected by the model selection criterion. Note: Validity (%), BMDL calculability (%), non-exclusion and BMDL calculation (%), and true dose response (%) were converted into rates of iterations divided by 9000, nine models in 1000 simulation data. NA, not applicable