Page 42 - Risk quantification and modification in older patients with colorectal cancer

Chapter 2
Critical appraisal
Also shown in Tables 4A-C is the quality assessment of the studies. The risk for bias can be subdivided into selection bias (participant selection and sample attrition), information bias (predictor and outcome assessment concerns) and analysis concerns.
Selection bias
The risk of selection bias for the prediction models studied was rated moderate to high; eight studies rated ‘high’ for risk of selection bias. In two studies, participant selection was unclear.19,27 For these studies and five others that did not report loss of follow up, there were high attrition concerns.14,16,17,20,31 In only four prediction model studies, there was no loss of follow-up.13,20,21,29
Information bias
In the majority of the studies, the risk of information bias related to the outcome was considered low. Three studies15,19,27 did not use data-driven predictor selection, but predictor selection was based on a meta-analysis or Delphi Round. In the studies that did not have mortality as an outcome, the risk was higher due to the unclear measurement of the outcome, lack of blinding or non-standardised timing of the outcome.
Analysis bias
Lastly, for the risk related to the analysis, all studies were rated ‘moderate’ to ‘high’. In the majority of the studies, the number of missing values was not reported, and predictors were included not independent of the p-value. Other concerns were related to the small sample sizes for estimation of the predictor effect; the event/ predictor ratio being less than ten events per predictor in seven studies19,22,23,27,30,34
In 6 out of the 25 studies, internal or external validation was not performed or reported.14,15,22,24,25,27 Therefore assessment of potential overfitting and optimism could not be assessed.
40