Page 116 - Getting the Picture Modeling and Simulation in Secondary Computer Science Education

114
Chapter 5
students in defining and analyzing the consequences of manipulation of factors in problem contexts due to their ample preparation in scientific thinking. One- tailed t-tests for independent samples were used to evaluate these expectations. We expect that the majority of scores vary between score 0 (prestructural) and 3 (relational). If the projects of VWO students are more often awarded a score of 4 (extended abstract) than those of HAVO students, this too can be regarded as (partial) evidence that the instrument differentiates satisfactorily between students’ level of learning outcomes. In order to see whether the students’ results meet our expectations and in search of possible explanation for the observed differences in the performance between HAVO and VWO groups, we additionally performed qualitative in-depth analysis of the students’ projects.
5.5 Results
All the results of the assessment using the grading rubrics are visually presented in figure 5 and figure 6, and in aggregate form as mean values of scores per school type (HAVO or VWO) per criterium in table 6, where the significance levels of the t-tests are presented too. In the figures, each row represents the results of one student group for the nine criteria assessed, and the height of each block represents the score, ranging from 0 for prestructural level to 4 for extended abstract level.
To assess the inter-rater agreement, we computed Krippendorf’s alpha assuming ordinal scale from the scores of the twelve 12th grade VWO project assessed by both teachers. The result is 0.78 which is a satisfactory value. Then, we used t-tests to find out whether the achievements of all the VWO groups taken together differ significantly from the achievements of the HAVO groups. We used a one-tailed t-test with significance level of 0.05. The results show that the groups differ significantly (printed bold in Table 6) for all the criteria except designing and implementing model, and for reflection.
We now examine the students’ projects per criterium and per type of school, i.e. HAVO vs. VWO. We state our findings in general terms and illustrate them with characteristic text segments taken from the data.