Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Yoshii, Keita; Nishiura, Hiroshi; Inoue, Kaoru; Yamaguchi, Takayuki; Hirose, Akihiko

doi:10.1186/s12976-020-00131-w

Theoretical Biology and Medical Modelling

Table 3 Simulation results for the acrylamide dataset (reference model: Logistic)

From: Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Exclusion^a	Selection^b	Reliability^c (Mean distance)	Rank	Validity^d (%)	Rank	BMDL calculability^e (%)	Non-exclusion and BMDL calculation^f (%)	True dose-response^g (%)
None	Lowest BMDL	0.4	17	99.9	1	99.8	99.8	0.0
	Lowest BMD	0.3	13	99.7	5	99.8	99.8	0.1
	Lowest AIC	0.1	2	89.0	16	99.8	99.8	38.3
	MA-all	0.2	10	98.2	11	99.8	99.8	NA
	MA-3	0.1	1	93.9	14	100.0	100.0	NA
	MA AIC < 3	0.2	8	97.8	13	100.0	100.0	NA
KS	Lowest BMDL	0.4	17	99.9	1	99.8	99.8	0.0
	Lowest BMD	0.3	13	99.7	5	99.8	99.8	0.1
	Lowest AIC	0.1	2	89.0	15	99.8	99.8	38.3
	MA-all	0.2	9	98.4	10	99.8	99.8	NA
KS, BMD/BMDL	Lowest BMDL	0.3	16	99.9	1	99.8	98.7	0.0
	Lowest BMD	0.3	12	99.7	5	99.8	98.7	0.2
	Lowest AIC	0.1	5	88.7	17	99.8	98.7	38.7
	MA-all	0.2	7	98.7	9	99.8	98.7	NA
KS, BMDU/BMDL	Lowest BMDL	0.3	15	99.9	1	99.8	93.9	0.0
	Lowest BMD	0.2	11	99.6	8	99.8	93.9	0.2
	Lowest AIC	0.1	4	88.5	18	99.8	93.9	39.0
	MA-all	0.2	6	98.2	11	99.8	93.9	NA

^aExclusion criteria: KS, Kolmogorov-Smirnov test of goodness-of-fit; BMD/BMDL, ratio of benchmark dose (BMD₁₀) to benchmark dose lower bound (BMDL10) with values > 10 excluded; BMDU/BMDL, ratio of benchmark dose upper bound (BMDU₁₀) to BMDL₁₀ with values > 10 excluded. ^bModel selection criteria: Lowest BMDL, model with the lowest value of BMDL₁₀; Lowest BMD, model with the lowest value of BMD₁₀; Lowest AIC, model with the lowest AIC value; MA-all, model averaging of all converged models. MA-3, model averaging of three models with three smallest AIC values. MA-AIC, model averaging of all models with AIC values < 3 compared with the best model that yielded the minimum AIC. ^cReliability (Mean distance), measured as the mean distance between unbiased BMDL₁₀ and calculated BMDL₁₀ followed by rank. ^dValidity (%), measured as the iterations that satisfied calculated BMDL₁₀ lower than unbiased BMD₁₀ followed by rank. ^eBMDL calculability (%), measured as the iterations that yielded BMDL in the model selection criterion. ^fNon-exclusion and BMDL calculation (%), measured as the iterations that yielded BMDL in the model selection criterion along with exclusion criteria. ^gTrue dose response (%), measured as the default model selected by the model selection criterion. Note: Validity (%), BMDL calculability (%), non-exclusion and BMDL calculation (%), and true dose response (%) were converted into rates of iterations divided by 9000, nine models in 1000 simulation data. NA, not applicable

Back to article page

ISSN: 1742-4682

Contact us

Submission enquiries: Access here and click Contact Us
General enquiries: info@biomedcentral.com