---
title: "Multiple Imputation and Cross-validation - method cv_MI"
author: "Martijn W Heymans"
date: "`r Sys.Date()`"
output:
  bookdown::html_document2
vignette: >
  %\VignetteIndexEntry{cv_MI}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

# Introduction

This page contains information of the `cv_MI` method that combines 
Multiple Imputation with Cross-validation for the validation of 
logistic prediction models. This cross-validation method is based 
on the papers of [Mertens BJ](https://onlinelibrary.wiley.com/doi/full/10.1002/bimj.201800289) 
and [Miles A](https://journals.sagepub.com/doi/abs/10.1177/0049124115610345).
The `cv_MI` method is implemented in the function `psfmi_perform`.
An explanation and examples of how to use the method can be found below.

# Method cv_MI

With this method imputations are implemented as part of the cross-validation
procedure. Within each cross-validation fold, imputation
is done once. By repeating this process over multiple imputation runs,
multiply imputed training and test sets are generated. Model performance 
is evaluated and tested on the training and test sets respectively.
The method can be performed in combination with backward selection
in the training set and subsequently testing the performance in the test set.
The method can only be performed when the outcome data is complete and
the original data, that contains the missing values has to be included. 

How this method works is visualized in the Figure below. 

```{r , echo = FALSE, fig.cap="Schematic overview of the cv_MI method", out.width='100%', fig.align='center'}
knitr::include_graphics("cv_MI.png")
```

# Examples

* [Method cv_MI]
* [Method cv_MI including BW selection]

## Method cv_MI

To run the cv_MI method use:

```{r}

library(psfmi)
pool_lr <- psfmi_lr(data=lbpmilr, formula = Chronic ~ rcs(Pain, 3) + JobDemands + Tampascale +
             factor(Satisfaction) + Smoking + factor(Satisfaction)*rcs(Pain, 3) ,
             p.crit = 0.05, direction="BW", nimp=5, impvar="Impnr",
             method="D1")

set.seed(200)
res_cv <- psfmi_perform(pool_lr, val_method = "cv_MI", data_orig = lbp_orig, folds = 5,
                     p.crit=1, BW=FALSE,  nimp_cv = 3, miceImp = miceImp, printFlag = FALSE)

res_cv

```

Back to [Examples]

## Method cv_MI including BW selection

To run the cv_MI method including BW selection use:

```{r}

library(psfmi)
pool_lr <- psfmi_lr(data=lbpmilr, formula = Chronic ~ rcs(Pain, 3) + JobDemands + Tampascale +
             factor(Satisfaction) + Smoking + factor(Satisfaction)*rcs(Pain, 3) ,
             p.crit = 0.05, direction="BW", nimp=5, impvar="Impnr",
             method="D1")

set.seed(200)
res_cv <- psfmi_perform(pool_lr, val_method = "cv_MI", data_orig = lbp_orig, folds = 5,
                     p.crit=0.05, BW=TRUE,  nimp_cv = 3, miceImp = miceImp, printFlag = FALSE)

res_cv

```
