Week 10: Reflection Post

I'm going to do my reflection post a bit early this time because I have a few more assignments to complete before the end of the week. I think the main thing that's on my mind now, coming to the end of the course is the question of evaluation: How do we evaluate a produsage-centered course? Do we evaluate it like how we would an instructional course? Using Kirkpatrick's 4 levels of evaluation:

Level 1 - Trainee sentiments of the course.

• I would imagine that this would examine:
• Metrics examining platforms' usability.
• Metrics examining instructor proficiency and helpfulness.
• Metrics examining effectiveness of material.
• Metrics examining relevance of material.
• Largely survey-based I think on a usual likert scale.

Level 2 - Trainee Proficiency

• This would probably examine:
• Trainee's understanding of civil military relations.
• Trainee's understanding of military law.
• Trainee's understanding of just war doctrine and application of Jus ad bellum. 
• This could come in the form of quick quizzes and a review of the groups' defence of their actions in the role-playing game. 

Level 3 - Actual improvement in job performance in the field

• This would probably examine:
• Number of infractions in the field.
• Case studies.
• Reported offences.
• This could either be a data collection effort, plowing all the post-patrol, combat reports, disciplinary reports or another survey with colleagues and superiors on the effectiveness of the individual post-training. I think the latter will be tricky to do though, due to the fact that people generally tend to cover up war crimes. 

Level 4 - Does this improvement in CMR proficiency lead to greater mission effectiveness in the military organization?

• I would imagine the main determinant would be to examine the compliance and support of the military's actions among the local population.
• This would probably be an anonymous survey with qualitative interviews done to support the findings.