How Student Performance Was Evaluated

An independent evaluator, an experienced school teacher trained in assessment methods, scored the projects. The evaluator had no prior knowledge of either the study or any of the participating schools and was given the mandate to score the projects fairly and without bias. In order to ensure the integrity of the scoring process, a CAST researcher trained the evaluator using assessment consensus strategies. Both the CAST researcher and the evaluator initially scored the same set of five projects, compared their scores, and then discussed the evaluations each had given. This process along with detailed explanations for each of the 9 learning measures allowed the evaluator to develop a consistent set of standards with which to score the remaining projects.

As an additional means of control, over a quarter of the projects were randomly selected (half from experimental classes, half from control classes) and scored by a second external evaluator. Correlations of the two external evaluators' scores substantiate the consistency of the main evaluator's project ratings.

Teachers submitted both the completed student projects and the materials students used to develop their projects. This work was evaluated on a four-point rating scale (0 to 3) according to the following 9 learning measures:

• Effective of presentation (e.g., interesting, informative, creative)
• Effectiveness of stating a civil rights issue
• Accuracy of information in relation to selected issue
• Presentation of a full picture (e.g., who, what, when, where, why, and how)
• Demonstration of insight into civil rights
• Effectiveness of bringing together different points of view
• Completeness
• Organization
• Demonstration of "best work" (e.g., well planned, neat, showing initiative)

Additional data on student performance and attitudes and teacher behavior and attitudes were collected from pre-study and post-study questionnaires, in-class observations, teacher telephone interviews, and records of time spent online. Use of standardized test scores was not appropriate, given that the core intervention took place over a two-month period and differences in standardized test scores cannot be expected within such a short time frame.