Page 159 - Getting the Picture Modeling and Simulation in Secondary Computer Science Education

necessary research; to design, implement and validate the model; to perform experiments by executing the model, to analyze the outcomes and to answer the research question; and finally, to reflect on the entire process. This results is a portfolio containing documentation and the implemented computational model. The grading rubrics classifies the learning outcomes for each part of the portfolio using the Structure of the Observed Learning Outcome (SOLO) taxonomy which describes the learning progress through five levels of understanding: prestructural, unistructural and multistructural — which are considered to be quantitative — and relational and extended abstract — which indicate a qualitative change (Biggs & Tang, 2011). Our assessment instrument in the form of a practical assignment and accompanying rubrics based on the SOLO taxonomy proved to be reliable, as indicated by a high rate of inter-rater agreement. Its validity is corroborated by exposing the significant differences in the performance levels of the HAVO26 students compared to the VWO27 students: as expected, the performance levels of the VWO students were significantly higher for almost all the criteria.
So far, assessing computational thinking has received a lot of interest, as reported in mapping and review studies by de Araujo et al. (2016), Martins Pacheco et al. (2019) and Tang et al. (2020). A typical example is provided by Roman-Gonzalez et al. (2017) who present their Computational Thinking Test. This multiple-choice test assesses students’ knowledge of computational concepts and is as such focused on programming, independent of any specific context. A number of other examples of assessing computational thinking focus specifically on modeling.
To compare our results to these other studies, we discuss our results across
three dimensions: the context where assessment instrument is used, the quality of 7 the rubrics used and the aspects of the modeling cycle involved. Inevitably, some
of the discussion will touch upon the nature of the modeling process itself, since
assessment is inseparable from it.
Teaching computational modeling is often situated in a specific context where attention is given to the learning objectives related both to the subject matter and to the computational aspects. Consequently, both of these learning objectives are assessed. For example, Caballero et al. (2012) described students who developed computational models of the motion of a craft orbiting Earth by completing a partially completed program. Incorrect programs were analyzed to unveil
26 HAVO: in Dutch: hoger algemeen voorbereidend onderwijs: senior secondary education
27 VWO: in Dutch: voorbereidend wetenschappelijk onderwijs: pre-university education
General Conclusions and Discussion
157