Page 99 - Balancing between the present and the past

expert panels, we found positive indicators of the instrument’s content validity. Furthermore, generalizability theory analysis provides indicators that the instrument
is one-dimensional when used to evaluate how history teachers promote historical contextualization. Generalizability theory analysis also showed that a large proportion
of the instrument’s variance was explained by the differences between the observed
teachers and a small proportion of the variance was explained by the differences in
lessons and observers, which demonstrates the instrument’s reliability (Brennan,
2001; Hill et al., 2012; Shavelson & Webb, 1991). Our D-study showed a reliable scoring
design, with one observer evaluating two lessons as the most effective method for
research purposes. For formative teacher evaluations, a reliable scoring design in
which two observers each evaluate two lessons or three observers each evaluate one 4 lesson is most effective.
Van Hover et al. (2012) noted that instruments that provide “useful discipline- specific feedback to preservice (and inservice) history teachers” (p. 604) are lacking. Additionally, Darling-Hammond, Amrein-Beardsley, Haertel, and Rothstein (2012) emphasized that most current teacher evaluation programs do little to help teachers improve their teaching. The FAT-HC instrument could provide insight into teachers’ subject-specific needs, resulting in a valuable addition to existing generic observation instruments (Grossman & McDonald, 2008). For example, if a teacher obtains low scores on the instrument, attention could be devoted to specific items of the instrument in teacher education or professional development programs. The pre-observation and post-observation interviews also could be structured based on the instrument’s items, resulting in more concrete feedback for the observed teacher.
The instrument could also help researchers examine the instructions and methods that teachers employ to promote historical contextualization in classrooms. In the history education literature, there is a clear view of high-quality teaching and learning of history; however, research instruments that capture this view when observing history teachers while they work do not exist (Van Hover et al., 2012). Furthermore, our instrument could be used to gain more insight into the association between history teachers’ instructions and methods and student achievement. Do teachers who activate their students to reconstruct a historical context better promote students’ historical understanding than teachers who do not? The instrument could also be used to evaluate intervention studies, for example, to examine the effects of training teachers in the use of instructions incorporated into the observation instrument.
Testing an observation instrument
97