Editor’s Note: Peer assessment has been used to increase student autonomy and promote active learning. It has positive results when used as a collaborative process, especially for team assignments. This study looks at pros and cons of various approaches to make it’s research findings.
The Implementation and Evaluation of
In addition to the Table 1, to demonstrate the level of team participation in class groups, Table 2 is given. It seems that Team A produced the highest rank in the Main Room even though they had four members. On the other hand, Team C had the lowest rank in the Team Room and the Chat Room. It is interesting to see that while Team B had the lowest rank in the Main Room, they generated the highest rank in the Team Room.
Differences between team ranks in the three different rooms are tested by using Kruskal-Wallis H test (Siegel & Castellan, 1988). According to results represented in Table 3, there is a significant difference in the Main Room submissions (Chi2 2, 14; p<0.05). Also, the Team A members submitted more messages than other team members. While in the Team Room, there is a significant difference (Chi2 2, 14; p<0.05), there is no significant difference in the Chat Room (Chi2 2, 14; p=0.25).
After weekly team assignments, students were asked to submit their evaluations on team members’ performance. When students were evaluating the peers’ performance, they considered six items including preparation, presence, contribution, timeliness, interpersonal relations, and feedback. For each item, a 4-point Likert type scale (1: Strongly disagree, 2: Disagree, 3: Agree, and 4: Strongly agree) was used to evaluate a student performance. The average of Team A Peer Evaluation is 3.958333, the average of Team B Peer Evaluation is 3.8, and the average of Team C Peer Evaluation is 3.986667. It is interesting to have these results. Although Team C had the lowest mean rank which is 3.00, they had the highest average peer evaluation result which is 3.958333. On the other hand, while Team B produced the highest mean rank, 10.80, the same team had the lowest average peer evaluation result which is 3.8.
Students were required to submit their individual assignments to their team for review. Based on peer reviews, they were supposed to make some improvements in their assignments before submitting them to the course instructor for grade. 14 students were asked to submit their peer evaluations regarding the most important concepts that they learned from the peer review process. Time was revealed as one of the most important concepts from these evaluations. This concept becomes very important when students work for their team assignment from different regions and time zones. One student summarized this view:
I have really learned that time is of the essence. It is better to complete your work early allowing time for changes and or peer review. Also, it is amazing how we are all working in different parts of the country, yet we are completing thorough, cohesive assignments through collaboration.
Another student confirmed this opinion: “Everyone finishing their parts on time so the whole assignment can be reviewed before the assignment is due”.
The second most important concept that emerged in the peer evaluations was importance of feedback. A majority of students stated giving and receiving feedback was crucial for the success of collaborative team work, even though some students may not be willing to share what they know with others. As one student explained:
We are all willing to see others view point and be willing to put ourselves into their classroom situations. I sometimes think we all take pride in our work and it is very hard to put a lot of effort into something and then have it changed.
The same student also added: “I learned that feedback is so key into really understanding items that can confusing”. On the other hand, one student put a negative side of giving feedback into consideration: “It is uncomfortable to point out mistakes in other people’s work but my group members seemed to take it in stride”. Benefits of feedback were expressed by students as “it is helpful to assist the writer in fixing his or her paper”, “reviewing others work helps you to review your own work with a more critical eye”, and “it assists everyone in writing a clear, precise paper”.
During the peer evaluation process, students stated that they learned why a team requires understanding and commitment, the power of leadership, organization, and communication. One student explained how communication plays an important role in a team: “Sharing information about your personal life helps to establish the lines of communication and I believe that my team members feel confident in each other and his/her abilities. Everyone has really “stepped up” to the plate”. Since team work requires different people in terms of gender, age, and culture to work together, a team work becomes a melting pot for these people. It was interesting to have one student stated: “Patience is a very important concept to keep in mind when working with a group. As we are all busy individuals, coming together as a group can be difficult due to individual time constraints”.
In addition to concepts that students learned from the peer review process, students were also asked to reflect how they will use this learning in their future team work. A majority of students stated that time management at the online environment and for the peer review process was important. One student reflected this issue: “I must be apt to getting my work done in a more timely fashion rather than waiting a day or two before its due. I plan to really set a schedule and space out my work so it does not affect the team’s peer review of my work”. Another student confirmed this student: “I plan to work on my time management so I can put my input earlier for a better discussion”.
Another learning experience that arose in the peer evaluations was being flexible while working at the asynchronous environment. One student asserted that “Hope to give my students a better understanding of why they should be flexible when working with new people”. One female student considered the peer review process as a good learning opportunity for herself and she reflected: “Working with people who have different priorities in life and different styles is helpful in any educational or work setting”.
In addition to the peer evaluation reflections, students were asked to write a paper to summarize what they learned by evaluating and reviewing other students` instructional plans. A majority of the students declared that reviewing and evaluating peers’ work enabled them to learn creating a well-documented instructional plan. In addition, they liked the process of reviewing and evaluation other’s work. One student stated: “Reviewing and evaluation other students’ instructional plans is an important tool in reflecting and evaluating my own work. By reflecting and evaluation my own instructional plan, I am able to determine the strength and weaknesses within my own instructional designs”. Another student confirmed this view: “Evaluating and reviewing other student’s instructional plans is a very valuable and enriching experience. Since each student’s plan applies the principles of instructional design to a different content area, peer review offers the opportunity to develop a deeper understanding and insight into the design process. By discovering strengths and weaknesses in each plan, reviewers are able to reevaluate their own understanding of the process and improve their own instructional plan. Although the strengths and weaknesses of the instructional plans varied based on the individual student, some commonalities became apparent”.
One student pointed a negative side of the reviewing and evaluating others’ instructional plans: “Reviewing and commenting on a peer’s work product can be difficult. Finding a balance between making suggestions for improvement and being supportive is a skill teachers employ daily with their learners in the classroom but seldom practice with their peers. Some reviewers skirted this difficult task by concentrating on mechanics rather than content”.
In this paper, peer assessment and collaborative learning have been argued to implement successful online courses in which the instructor is concerned with how well students are learning new concepts and skills. Students were required to assess both their own performance and peers’ performance to learn concepts and skills and suggest necessary corrections to peers. In addition, peer assessment is the way to evaluate the collaborative team assignments and of how much and each student has contributed to the final of the team assignments. Overall, students participated more in the Main Room to give responses to the questions that were submitted by the instructor were accounted to the final class grade. However, team members who participated in less than other teams tend to give more points, when they evaluate peer performance in the team. A reason for this tendency may be that students achieved the team objective with less but constructive participation or they may have more motivation to complete team assignments (Fisher, Phelps, & Ellis, 2000) because the same team participated more in the Main Room.
It seems that a leadership role plays an important role in the online courses. In each team one or two students got involved more either in the Main Room or the Team Room and acted a leadership role in completing assignments, giving feedback, and submitting the weekly team assignments to the instructor. Therefore, the online course structure needs to encourage students to take steps in the online teams. Also, class policies, information about participation, attendance, team construction and dynamics are important to provide students beforehand that the class starts. During the peer assessment process, to prevent reliability and validity problems, it is crucial to provide students with rubrics that communicate with students and explain what students are expected when using rubrics to evaluate team members (Grieves, McMillan, & Wilding, 2006).
The concepts of timeliness, constructive feedback, being flexible and patient, and communication have been raised in this study. A majority of the students stated that being on time is the essence of online courses so the online class implementation should consider this issue and encourages students extrinsically such as giving extra points or sending warning messages to all students. Another issue is giving feedback and interacting with class and team members via constructive messages. Students need to differentiate between class related submissions and personal submissions. In order for students to consider these differences, the online environment needs to be divided into several parts including the Main Room, the Team Room, and the Chat Room. In addition, there should be another room for the assignments that is restricted to students to see and open others’ documents. Before the class starts, it is important to provide students with information about how team works and how communication occurs and what they need to do in case of team conflicts. This support can be done by providing some tutorials and readings, as well as proactively sending messages or giving some directions regarding these issues to the class by the instructor.
Ashton, S., T. Roberts, & L. Teles. (1999). Investigating the role of the instructor in collaborative online environments. Poster session presented at Computer Support For Collaborative Learning (CSCL) 1999 Conference, Stanford University, CA.
Berge, Z. (1995). Facilitating computer conferencing: Recommendations from the field. Educational Technology, 35, 22-30.
Boud, D., & Falchikov, N. (Eds.). (2007). Rethinking assessment for higher education: Learning for the longer term. London: Routledge
Chen, C.-h. (2010). The implementation and evaluation of a mobile self- and peer-assessment system. Computers & Education, 55, 229–236.
Davies, P. (2006). Peer assessment: Judging the quality of students' work by comments rather than marks. Innovations in Education and Teaching International, 43 (1), 69-82.
Elliott, N. and Higgins, A. (2005) Self and peer assessment: Does it make a difference to student group work?. Nurse Education in Practice, 5(1), 40-48.
Erickson, F. (1998). Qualitative research methods for science education. In B. J. Fraser & K. G. Tobin (Eds.), The international handbook of science education (1155-1173). Dordrecht, The Netherlands: Kluwer.
Fisher, K., Phelps, R., & Ellis, A. (2000). Group processes online: Teaching collaboration through collaborative processes. Educational Technology & Society, 3(3), 1-16.
Grieves, J., McMillan, J. & Wilding, P. (2006). Barriers to learning: Conflicts that occur between and within organisational systems. International Journal of Learning and Intellectual Capital, 3(1),
McDowell, L., & Mowl, G. (1996). Innovative assessment: Its impact on students. In G. Gibbs (Ed.), Improving student learning through assessment and evaluation (131-147). Oxford: The Oxford Centre for Staff Development.
McLuckie, J. & Topping, K. J. (2004). Transferable skills for online peer learning. Assessment & Evaluation in Higher Education, 29(5), 563-584.
Murphy, K. L., S. E. Mahoney, & T. J. Harvell. (2000). Role of contracts in enhancing community building in web courses. Educational Technology and Society, 3(3), 409-421.
Sahin, S. (2008). An application of peer assessment in higher education. The Turkish Online Journal of Educational Technology, 7(2), 5-10.
Siegel, S. & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences (second edition). New York: McGraw-Hill.
Sluijsmans, D. (2006). A conceptual framework for integrating peer assessment in teacher education. Studies in Educational Evaluation, 32(1), 6-22.
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68, 249-276.
Van den Berg, I. (2006). Design principles and outcomes of peer assessment in Higher Education. Studies in Higher Education, 31(3), 341-356.
Dr. Harun Yilmaz is a scientific programs expert in the Social Sciences and Humanities Research Group (SOBAG) at the Scientific and Technological Research Council of Turkey (TÜBİTAK).
His research interests include e-learning, instructional design of online courses, teacher education, and peer assessment. He received his Ph.D. in Instructional Design and Technology from Virginia Tech, USA.