Peer Grading in Astronomy Massive Open Online Course

In this work we thoroughly investigate the peer grading process as it happened in the University of Arizona session based MOOC “Astronomy: Exploring Time and Space” offered during Spring 2015 through Coursera. Overall, 25400 learners from over 100 countries registered for this course. Of those, 14900 accessed at least one part of the course and 1332 users engaged in the peer grading. First of all we provide description of the peer graded assignments and we identify trends in behavior of people who participated in these exercises. E.g. time they spent on grading, number of assignments graded and patterns arising from comparing all three assignments. Furthermore, for the second assignment, we graded random sample of 300 essays by a group of trained undergraduate students and a group consisting of one of the course instructors together with graduate TAs and we compared results with grades from the peer grading. Specifically we look on Intraclass Correlation Coefficients for all three groups of graders to determine reliability of each group and correlations between final grades. Finally we assess factors influencing reliability of the peer graders participating in the MOOC based on the difference from our grades.