Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Yoshii, Keita; Nishiura, Hiroshi; Inoue, Kaoru; Yamaguchi, Takayuki; Hirose, Akihiko

doi:10.1186/s12976-020-00131-w

Theoretical Biology and Medical Modelling

Table 1 Simulation results for the 1-aminoanthraquinone dataset using the benchmark dose method (reference model: Probit)

From: Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Exclusion criteria^a	Selection^b	Reliability (Mean distance)^c	Rank	Validity^d (%)	Rank	BMDL calculability^e (%)	Non-exclusion and BMDL calculation^f (%)	True dose-response^g (%)
None	Lowest BMDL	0.4	5	100.0	1	95.6	95.6	0.1
	Lowest BMD	0.3	2	100.0	1	95.6	95.6	0.1
	Lowest AIC	120.9	15	88.4	15	95.6	95.6	34.1
	MA-all	6.2	9	99.6	8	95.6	95.6	NA
	MA-3	4.7	7	99.8	7	100.0	100.0	NA
	MA AIC < 3	9.0	11	98.8	11	100.0	100.0	NA
KS	Lowest BMDL	0.4	5	100.0	1	95.6	95.6	0.1
	Lowest BMD	0.3	2	100.0	1	95.6	95.6	0.1
	Lowest AIC	120.9	15	88.4	15	95.6	95.6	34.1
	MA-all	6.1	8	99.6	8	95.6	95.6	NA
KS, BMD/BMDL	Lowest BMDL	0.3	4	100.0	1	95.6	79.1	0.5
	Lowest BMD	0.2	1	100.0	1	95.6	79.1	0.5
	Lowest AIC	121.0	17	88.4	15	95.6	79.1	38.4
	MA-all	6.3	10	99.3	10	95.6	79.1	NA
KS, BMDU/BMDL	Lowest BMDL	27.2	13	91.0	12	95.6	49.0	18.4
	Lowest BMD	27.1	12	91.0	12	95.6	49.0	18.4
	Lowest AIC	148.3	18	79.4	18	95.6	49.0	35.3
	MA-all	33.9	14	90.2	14	95.6	49.0	NA

^aExclusion criteria: KS, Kolmogorov-Smirnov test of goodness-of-fit; BMD/BMDL, ratio of benchmark dose (BMD₁₀) to benchmark dose lower bound (BMDL₁₀) with values > 10 excluded; BMDU/BMDL, ratio of benchmark dose upper bound (BMDU₁₀) to BMDL₁₀ with values > 10 excluded. ^bModel selection criteria: Lowest BMDL, model with the lowest value of BMDL₁₀; Lowest BMD, model with the lowest value of BMD₁₀; Lowest AIC, model with the lowest AIC value; MA-all, model averaging of all converged models; MA-3, model averaging of three models with the three lowest AIC values; MA-AIC, model averaging of all models with AIC values < 3 compared with the best model that yielded the minimum AIC. ^cReliability (Mean distance), measured as the mean distance between unbiased BMDL₁₀ and calculated BMDL₁₀ followed by rank. ^dValidity (%), measured as the iterations that satisfied calculated BMDL₁₀ lower than unbiased BMD₁₀ followed by rank. ^eBMDL calculability (%), measured as the iterations that yielded BMDL in the model selection criterion. ^fNon-exclusion and BMDL calculation (%), measured as the iterations that yielded BMDL in the model selection criterion along with exclusion criteria. ^gTrue dose response (%), measured by the default model selected by the model selection criterion. Note: Validity (%), BMDL calculability (%), non-exclusion and BMDL calculation (%), and true dose response (%) were converted into rates of iterations divided by 9000, nine models in 1000 simulation data. NA, not applicable

Back to article page

ISSN: 1742-4682

Contact us

Submission enquiries: Access here and click Contact Us
General enquiries: info@biomedcentral.com