Page 115 - Getting the Picture Modeling and Simulation in Secondary Computer Science Education

Assessment of Modeling and Simulation
that were taught by two teachers: the thesis author and her colleague who worked in both schools. All the students have previously learned to program in Python or a similar high-level textual programming language. The course on Computational Science lasted eight weeks in total. The first five weeks were dedicated to instruction using the teaching material we developed for our curriculum intervention, and during the last three weeks, the students formed groups of two or three (a few students choose to work alone) and worked on the practical assignment. After choosing the cases to model, the students went on to answer the questions form the assignment and to develop their models in NetLogo. The students from the two 12th grade VWO classes presented their models in the classroom and got feedback from other students. In other classes there was no opportunity to organize presentations. Finally, the students answered the last questions from the assignment and turned in their documentation and models, i.e., Netlogo code.
We analyzed only the completed projects turned in by sixteen students forming eight groups in 11th grade HAVO, fifteen students forming five groups in 11th grade VWO and twenty-four students forming twelve groups in 12th grade VWO.
5.4.7 Data Collection 5 Both the teachers assessed the students’ work — project documentation and
program code — using the rubrics presented here, assigning 0 up to 4 points for
the prestructural up to extended abstract level, respectively, First, they separately
assessed work of two groups. They compared their scores, and then the scoring
guidelines and the interpretation of the rubrics were fine-tuned where necessary. Additionally, they agreed to take into account the answers students supplied while
answering other questions — e.g., elaborating on the research of the phenomenon
under scrutiny while answering the question about validation. Then, one teacher
assessed the work of all the groups while the other teacher assessed only the work
of the 12th grade VWO groups in order to establish the inter-rater agreement.
5.4.8 Analysis
Our aim was to investigate the inter-rater agreement and the discriminative validity of instrument. The inter-rater agreement was evaluated by computing Krippendorf ’s alpha for each of the criteria. Discriminative validity was assessed by comparing the scores of HAVO students and VWO students. While it is to be expected that both HAVO and VWO students can be taught to design and implement a model equally well, VWO are expected to outperform HAVO
113