在插入符号中设置重采样的“LOOCV”时,是否可以输出重采样汇总指标?

Can I output resampling summary metrics when setting "LOOCV" for resampling in caret?

提问人:Papyrus 提问时间:8/8/2023 最后编辑:desertnautPapyrus 更新时间:8/8/2023 访问量:20

问:

我用于拟合某些模型,使用或用于重采样。我还想为使用重采样构建的模型输出模型指标和预测。当我使用 时,我确实得到了一个包含此数据的对象,内部和 .但是,当使用 : 仅包含 1 次迭代(我猜是最终模型)并且是空的时,不会发生这种情况。caretCVLOOCVtrainControl(method = "cv", returnResamp = "all", savePredictions = "all")object$predobject$resampletrainControl(method = "LOOCV", returnResamp = "all", savePredictions = "all")object$predobject$resample

有没有办法获得 LOOCV 的这些预测和指标?

下面我展示了我正在使用的代码。

# LOOCV
set.seed(99)
seedy <- lapply(1:(nrow(phy2)+1),function(x){sample.int(1000, 1)})

# Also set global resampling parameters
fitControl <- trainControl(
  method = "LOOCV", # leave-one-out cross validation
  savePredictions = "all",
  returnResamp = "all",
  classProbs = T ,
  seed = seedy,
  summaryFunction = multiClassSummary
)

set.seed(99)
fit1 <- train(x = bty2,
                    y = phy2$grouping,
                    method = "lda", 
                    metric = "Kappa",
                    trControl = fitControl
)

# K-fold CV

index.folds <- createFolds(phy2$grouping, k = 5)
folds <- 5
fitControl <- trainControl(
  method = "cv", # k-fold cross validation
  number = folds,
  savePredictions = "all",
  returnResamp = "all",
  classProbs = T ,
  seed = as.list(rep(99,folds+1)),
  summaryFunction = multiClassSummary,
  index = index.folds
)

set.seed(99)
fit2 <- train(x = bty2,
                 y = phy2$grouping,
                 method = "lda", 
                 metric = "Kappa",
                 trControl = fitControl
)
机器学习 R-CALET

评论


答: 暂无答案