Page 120 - ON THE WAY TO HEALTHIER SCHOOL CANTEENS - Irma Evenhuis

Chapter 7. General Discussion
in the cafeteria. The scan also showed a substantial to good reliability and validity for the measurement of the food availability in vending machines and food accessibility when assessed by an expert (school canteen advisor). When the food availability in vending machines and accessibility was assessed by a canteen employee, reliability and validity were fair. This suggests that the Canteen Scan is a valid and reliable tool for all assessors to assess the offered food in the cafeteria, but to evaluate the offer in the vending machines and the accessibility validly and reliably an expert has to fill in the scan. Canteen employees needed more explanation of the accessibility criteria in particular. These results were used to further improve the scan.
Textbox 7.1. English summary of the study to the reliability and validity of the Canteen Scan.
Objective:
The aim of the study was to investigate the inter-rater reliability and criterium validity of the Canteen Scan in Dutch schools by assessing the availability and accessibility of food and drink products. Inter-rater reliability was investigated to examine whether the score remains the same even when assessed by different people. Criterium validity was measured to determine whether the Canteen Scan is able to measure a canteen in accordance with the Guidelines for Healthier Canteens. For additional information about the use of the tool in sports canteens and worksite cafeterias, it was also pilot tested in those two settings.
Methods:
A canteen employee, a school canteen advisor (SCA) of the Netherlands Nutrition Centre and a researcher filled out the Canteen Scan (CS) and took pictures independently in 50 school canteens, including vending machines. In the first 25 schools, a second SCA also filled out the CS. Due to the lack of a comparable tool suitable for measuring compliance with the guidelines, a reference score was created through consensus between the first SCA and the researcher. In addition, “remote scans” were performed independently by two SCA’s, i.e. they adjusted scans according to pictures of the canteens. This method with “remote scans” is frequently used in practice by SCA’s, as they are not able to visit all locations. Scores of different users were compared and analysed separately for food availability in cafeterias and in vending machines (VM), and for food accessibility. Inter-rater reliability and criterium validity were calculated with Weighted Cohen’s Kappa coefficients (using Landis and Koch’s interpretation with <0.2 slight; ≥0.20 fair; ≥0.40 moderate; ≥0.60 substantial; ≥0.80 almost perfect agreement) [207].
Results:
Food availability in cafeterias showed substantial to almost perfect reliability and validity for all users (K>0.70). The reliability analyses of food availability in VM and food accessibility showed substantial reliability between SCA’s (K>0.63), but fair reliability between SCA’s and canteen employees (K≥0.2). Food availability in VM and food accessibility revealed good validity when filled in by SCA’s (K>0.72), but fair agreement when filled in by canteen employees (K≥0.20). Canteen employees scored accessibility structurally more positively. Comparison of the scores in the canteen with the scores of remote scans showed substantial to almost perfect reliability and validity (K>0.63) when a different SCA took the pictures and filled out the scan independently.
118
textbox continues