Page 117 - ART FORM AND MENTAL HEALTH - Ingrid Pénzes

Inter-rater reliability
In round two, the inter-rater reliability for “rhythm” and “movement” was fair (κ = .34 and κ = .29, respectively), while the reliability for “contour” (κ = .57), “color saturation” (κ = .43), and “mixture of colors” (κ = .47) was moderate. Reliability for “suggestion of space” (κ = .65), “number of colors” (κ = .65), “color brightness” (κ = .64), “symmetry” (κ = .68), “linearity” (κ = .74), “line quality” (κ = .64), “dynamic” (κ = .71), and “texture” (κ = .65) was substantial, while reliability for “filled area” (κ = .82), “used space” (κ = .86), “geometry” (κ = .83), and “hue” (κ = .83) was almost perfect (Viera & Garrett, 2005).
The evaluation of round two highlighted three drawbacks of rating. First, the definitions of scores 3, 4, and 5 for “number of colors” overlapped. Second, raters found the definition of the item “rhythm” too complex and the definition of “movement” was found to be unclear and incomplete. This could explain the relatively low reliability for these items. All other definitions were clear and the art product examples were found to be illustrative. Third, it was questioned whether the art materials might influence the ratings of the formal elements. Based on these drawbacks, the definitions of “number of colors,” “rhythm” and “movement” were adjusted and tested for inter- rater reliability in a third and final round. The inter-rater reliability varied between moderate and substantial for all items except “texture” which had poor inter-rater reliability (see Table 2).
The inter-rater reliability for the adjusted formal elements “number of colors”, “rhythm” and “movement” increased in the third round. The inter- rater reliability of all other formal elements remained similar. Only the inter- rater reliability for “texture” decreased significantly, therefore this item was not included in the subsequent studies1.
Table 2. Final inter-rater reliability scores for each formal element
Formal elements
Filled area
Used space Suggestion of space Geometry
κ
.64** .79** .75** .86**
1
round. In the second round, art products made with diverse art materials were included, whereas in the third round, the art products were made with acrylic paint only (see discussion).
The inter-rater reliability of “texture” decreased dramatically between the second and third
Formal elements of art products indicate aspects of mental health | 115
table continues
5