好奇threshold p-values 这个是怎么选的呀？ 背后有没有什么原理

问题如下：

Based on Exhibit 3, which threshold p-value indicates the best fitting model?

选项：

0.57

0.79

0.84

解释：

B is correct. The higher the AUC, the better the model performance. For the threshold p-value of 0.79, the AUC is 91.3% on the training dataset and 89.7% on the cross- validation dataset, and the ROC curves are similar for model performance on both datasets. These findings suggest that the model performs similarly on both training and CV data and thus indicate a good fitting model.

A is incorrect because for the threshold p-value of 0.57, the AUC is 56.7% on the training dataset and 57.3% on the cross- validation dataset. The AUC close to 50% signifies random guessing on both the training dataset and the crossvalidation dataset. The implication is that for the threshold p-value of 0.57, the model is randomly guessing and is not performing well.

C is incorrect because for the threshold p-value of 0.84, there is a substantial difference between the AUC on the training dataset (98.4%) and the AUC on the cross- validation dataset (87.1%). This suggests that the model performs comparatively poorly (with a higher rate of error or misclassification) on the cross- validation dataset when compared with training data. Thus, the implication is that the model is overfitted.