Page 91 - Balancing between the present and the past

1. What is the observation instruments’ dimensionality when used to observe how history teachers promote historical contextualization?
2. What are the reliability outcomes when the observation instrument is used to observe how history teachers promote historical contextualization?
3. How many lessons and observers are necessary to establish a reliable and optimal scoring design?
4.4 Method
4.4.1 Structure of the observation instrument 4 To design and construct our observation instrument, we used the guidelines
described by Colton and Covert (2007), which focus on the development of valid and
reliable instruments in social sciences. Our instrument could be characterized as a high-inference observation instrument. In contrast to low-inference instruments
(such as time sampling and time logs), high-inference instruments provide a more qualitative verdict (Chávez, 1984). However, these instruments are more susceptible to subjectivity; therefore, thorough inter-rater reliability procedures are necessary.
We modeled our instrument on Van de Grift’s (2007, 2009) International Comparative Analysis of Learning and Teaching (ICALT) observation instrument. We chose this instrument’s format because it also seeks to observe teachers’ professional strategies and calculate scores based on these strategies. Similar to the ICALT instrument, our instrument utilizes a 4-point Likert scale to score the items. In our instrument, scores 1 and 2 represent a negative verdict, while scores 3 and 4 represent a positive verdict. Score 1 should be used only if teachers do not use a particular strategy in their lessons.
4.4.2 Formulating and refining the items
Based on the four main strategies identified in our theoretical framework (reconstructing the historical context, fostering historical empathy, performing historical contextualization to explain the past, and raising awareness of a present-oriented perspective) and a review of literature on teaching historical contextualization, we formulated observable items to assess classroom teachers’ behavior in regards to historical contextualization. Furthermore, during two national teacher professionalization conferences, we asked 25 history teachers (after an
Testing an observation instrument
89