Editor’s Note: Interaction and collaborative learning play an important role in distance education. This study correlates participation scores with student achievement in a graduate research seminar.

Assessing Student Participation
in an Online Graduate Course

Stafford A. Griffith

Jamaica

Abstract

This article is concerned primarily with assessing student participation as part of the cooperative learning experience in a Research Methods graduate course. It assesses the extent of student participation in online conferences and the relationship between student participation and achievement. Based on the assessment, mean participant scores of students in each of three classes, and the correlation between participation scores and achievement on a final assignment requiring students to prepare a research proposal, the author concludes that evidence reinforces the view that when students are assigned specific roles, are carefully monitored and are rewarded for their online contribution, a satisfactory level of participation will result. The author further suggests that the low, non-significant as well as negative correlations observed between participation and achievement on the final assignment may need further exploration. It may be due to the ability of students to use computers and the web and accessibility of these resources.

Keywords: cooperative learning, participation, achievement, online, correlation, graduate course, assessment, internet, group work, research methods.

Introduction

Li (2002) notes that the use of computer-mediated communication as a teaching and learning tool has increased dramatically in recent years as a result of the increased number of schools and educational institutions that are able to access the Internet. This has led to what Patel and Patel (2006) characterize as a “radical innovation” which has seen a proliferation of online distance education initiatives in Higher Education. Patel and Patel cite estimates indicating that nearly half of all U.S. colleges and universities provided online educational offerings and served nearly two million students in 2002. Growth is projected to be five million students by 2006 (Wechsler, 2002). A 2006 Sloan Consortium report revealed that in the United States alone, almost 3.5 million students were taking at least one online course during the fall 2006 term (Allen & Seaman, 2006) while the 2008 report put that number at 3.9 million for the fall of that year (Allen & Seaman, 2008).

With the rapid growth in the number of online course offerings, issues about the quality of these offerings are increasingly raised (Muirhead, 2000, 2001). Among those issues is the level of interactivity between students and between teachers and students (McNabb, 1994; Sherry, 1996).

Participation is an important dimension of that interactivity. Lam (2004) pointed out that while online discussion forums are potentially excellent for peer learning, one of the problems is that these online forums can suffer from lack of student participation. Further, Moallem (2003) expressed the view that if interaction is not an integrated, essential and graded aspect of the online learning environment, the anticipated benefit of such interaction will not be realized.

For the Research Methods graduate course with which this article is concerned, student participation was considered an important element of student learning. Marks were also awarded for participation and these marks contributed to the overall grade of the student.

Purpose

This article is concerned primarily with assessing student participation as part of the cooperative learning experience in a Research Methods graduate course. It seeks to assess:

1.      the extent of student participation in online conferences; and

2.      the relationship between student participation and achievement.

Sample

This study used data from participants in the three separate offerings of the Research Methods course in the School of Education, Mona Campus. The first course, offered in the January to May 2007 semester, was completed by 33 graduate students; the second course, offered in the January to May 2008 semester, was completed by 21 graduate students; the third course, offered in the August to December 2008 Semester, was likewise completed by 21 graduate students.

Organization and delivery of the Course

The Research Methods course was delivered through a web-based course management and online conferencing facility used by the University of the West Indies (UWI), Mona Campus, Jamaica. Course Delivery was entirely online using asynchronous communication. Materials and other items were made available over the web to students connected from remote locations in a number of countries, mainly in the Caribbean.

The course was delivered over a 13 week period. The topics covered included:

1. Ways of knowing: the characteristics and value of research.

2. Overview of educational research, including ethics in research.

3. Selection and definition of a research topic, including research questions, hypotheses and variables.

4. Review of the literature, including in-text citation and references.

5. Quantitative and qualitative research – the differences.

6. Sampling: quantitative and qualitative.

7. Data collection methods: questionnaire, interviews, scales, reliability and validity.

8. Survey/Descriptive research, correlational research and ex post facto/causal-comparative research.

9. Experimental research, including single subject experiments.

10. Narrative research, ethnographic research and mixed method research.

11. Descriptive statistics.

12. Selected inferential statistics.

13. Selected techniques in qualitative data analysis.

Students registered for the course were divided into three groups of approximately equal numbers. Johnson, Johnson and Smith (1991) classified groups involved in group work into three general types: informal cooperative learning groups, formal cooperative learning groups and cooperative base groups. Informal learning groups involve the ad hoc or temporary assignment of students together, within a single class session. These groups are used to help focus students' attention on materials to be learned, set a mood conducive to learning, help organize in advance the material to be covered in a class session, ensure that students cognitively process the material being taught, and provide closure to a class sessions. Formal learning groups are teams established to complete a specific task. In these groups, students work together for one or several class sessions to achieve shared learning goals and complete, jointly, specific tasks and assignments. Base Groups are long-term groups, usually existing over the course of a semester. The responsibility of such groups is primarily to provide members with support, encouragement, and assistance in completing course requirements.

In the case of the online Research Methods course with which this article is concerned, the groups to which students were assigned exemplified the combined attributes of the three types of groups. Although students were assigned to groups for the duration of the course and performed the functions of the base group as previously described, they also served the functions identified for the formal learning group in working together over a period to complete, jointly, specific tasks and assignments. Additionally, they engaged in activities associated with the informal group by working within specific class sessions to focus on materials to be learned and to undertake the other related activities of the informal learning group.

Student participation, which involved working cooperatively online within groups, was an important requirement of the Research Methods course. Palloff and Pratt (2001) provide some guidelines for ensuring that students participate in a way that will optimise their benefit from the online interactions. They point out that the minimum acceptable number of posting to be made, the time period over which the postings should be made and the types of postings that meet the required standard should be specified for students.

Lam (2004) also provided guidelines for the delivery of online courses which included, inter alia:

1.      the early setting of expectations and laying the ground rules;

2.      the assigning of students to lead the discussion; and

3.      the assessment of online participation.

These guidelines were followed in the delivery of the Research Methods course online. The aim was not only to achieve a high level of participation, but also to ensure that the participation was of high quality (Lam, 2004).

As earlier indicated, the course was divided into 13 topics. For the first topic, scheduled for the first week of the course, students were required to undertake individual reading. During that week, they introduced themselves to the rest of the class. The course instructor provided guidance for the individual reading and clarified any issues raised by students.

For the other twelve weeks, students in the three groups into which the class was divided assumed the lead roles. Each of the three groups was assigned the role of Discussant for four separate weeks, covering four separate topics. In addition, each student served once either as a Moderator or as a Researcher for at least one conference. Each conference was served by a Moderator and one or two Researchers, depending on the demands of the topic and the number of students in the class. Moderators and Researchers for a conference were drawn from a group that was different from the one serving as Discussant. Students who were not assigned to any of these roles for a particular conference were required to follow the conference by reading the material posted.

It should be noted that in each of the offerings of the Research Methods course, two to four students dropped out after the first week or two, for a variety of reasons, including inability or failure to complete the registration process, or for other personal reasons.

In keeping with guidelines developed by the Office of the Master of Education Online, School of Education, UWI, Mona, for the various roles, the Moderator was required to lead the discussion for the week on the assigned topic. More specifically, the Moderator was required to:

1. provide guidance on the reading and research for the week, identifying the particular focus to be pursued;

2. pose questions to guide the on-line discussion;

3. initiate and stimulate discussion for the topic under consideration; and

4. guide discussion by (a) logging on daily to integrate and advance the discussion, (b) ensuring that all questions were discussed and (c) encouraging each participant to arrive at some closure.

The Researcher was required to find and post relevant materials, including materials from various websites and to give a summary of the content and its importance to the topic under discussion. More particularly, the Researcher was required to:

1. find materials on the Internet and other sources that that relate to the topic for the week;

2. call attention to relevant UWI, Mona Library resources available online;

3. share understanding and insights derived from the material found; and

4. pose questions that would advance the discussion on the topic.

Discussants were each required to participate fully in the discussions. More specifically, they were required to:

1. read the assigned materials;

2. respond to the Moderator’s questions and discussions;

3. respond to comments from other participants; and

4. contribute further insight that would explain and expound on issues under discussion.

As noted earlier, the first week was devoted to introductions, individual readings and clarifications, all guided by the instructor. The instructor monitored the discussions in subsequent conferences and made a number of inputs to provide periodic guidance, to compliment and encourage students and to encourage the correction of any apparent misunderstanding or misconceptions of any aspect of the subject matter under discussion. This type of active involvement of the instructor is held to be important to the maintenance of student interest, motivation and learning satisfaction (Anderson, Rourke, Garrison & Archer, 2001; Russo & Benson, 2005). However, an effort was made to ensure that the discussions in the conferences were not instructor driven. This took into account the research findings about the possible limitations of over-reliance on the instructor in online conferences and the possible impairment of student confidence which an overwhelming instructor presence in such conferences may bring (Light, Nesbitt, Light & White, 2000; Nickel, 2002; Pearson, 1999).

The role of the student Moderator was paramount among student responsibilities. This is clearly highlighted in the definition of the tasks associated with that role. It was the student Moderator and not the instructor who assumed leadership in the conferences, except in the limited instances already specified. The student Moderator provided guidance on the required reading and research for class participants, posed questions to guide the direction of the discussions, stimulated student participation and ensured that the various contributions of students were integrated into the discussions in a way that advanced understanding of the subject matter under discussion.

The assignment of these important functions to the student Moderator is well supported in the literature (Hara, Bonk & Angeli, 2000; Leh, 2000; Poole, 2000; Tagg, 1994; Veen, Lam & Taconis, 1998). As Seo (2007) noted from a review of the research literature: “student moderators can lead discussions more effectively and foster greater student comprehension than the instructor because student moderators better understand their peers’ way of thinking” (p. 22). The assignment of critical responsibilities to student Moderators in the Research Methods course took these findings into account.

Analysis of Participation

Student performance in the Research Methods course was graded as follows: participation was given a weight of 15 percent, a written group assignment scored up to 25 percent, and an individual assignment up to 60 percent.

The group assignment required students to consider a list of 10 issues in education provided by the instructor based on excerpts from a recent Education Task Force Report on Educational Reform in Jamaica. Students were required to define a research topic derived from one of the issues, outline the purpose, justification, research questions and/or hypotheses and define the key variables. Each of the three groups submitted a single report. For this assignment, each student received a group mark out of 20 and an individual mark out of 5 which took into account peer ratings based on the contributions of each student in the group.

The individual assignment required students to develop a comprehensive research proposal suitable for the research project which they would have to complete for the award of the Master of Education degree in their respective specializations. This individual assignment benefitted from the learning that took place through participation in the online conferences and the group assignment. The individual assignment was intended to measure how well students were able to apply the knowledge and skills acquired in the Research Methods course in preparing a research proposal. The research proposal was regarded as an independent measure of individual student achievement and was used as the basis for examining student achievement in relation to participation in the online conferences.

For the online participation component of the course, students earned a maximum of 5 marks for the one time they served either as a Moderator or a Researcher. They received a maximum of 2.5 marks for each of the four times they served as Discussant or a total of 10 marks for that role. These together made up the maximum of 15 marks awarded for participation.

The distribution of marks earned for participation and for the individual assignment for each student in each of the three offerings of the Research Methods course taught online for the periods January to May 2007, January to May 2008 and August to December 2008 are shown in Table 1.

Students’ participation level appeared to be very good. Students in the three offerings of the Research Methods course online - in January to May 2007, January to May 2008 and August to December 2008 - scored an average of 12.2, 12.4 and 11.1, respectively, out of 15.0 or 81.3%, 82.7% and 74.0%, respectively, for participation.

The level of participation that these mean and percentage scores reflect may be attributed to the measures taken to ensure optimal online participation of students. As earlier noted, these measures included:

1.      the assignment of each student during specified weeks to the role of
(a) a Moderator or a Researcher or (b) a member of a team of Discussants;

2.      the proper definition of each role;

3.      the careful monitoring of levels of participation of each student; and

4.      the assessment of student participation.

These measures were in keeping with the advice of both Palloff and Pratt (2001) and Lam (2004).

Table 1

Participation Scores and Individual Assignment Scores of Students in the Research Methods Course Taught Online on Three Different Occasions

In all three instances, the mean participation scores were depressed by a few students who scored poorly on this dimension of course. This was due mainly to the inability of these students to undertake fully in the roles they were assigned because of illness or other emergencies which interfered with their availability at the times that they were scheduled to make their contributions. This information was conveyed to the tutor either during the period of the assigned tasks or immediately thereafter.

Table 1 shows that correlations between scores obtained for participation and scores obtained for the individual assignment were all small and non-significant for the students in each of the three classes (p > .05). For the class of January to May 2007, there was a low positive correlation of 0.01; for the class of January to May 2008, there was a low negative correlation of –0.04; and for the class of August to December 2008, there was also a low negative correlation, -0.22.

Davies and Graff (2005), in a study which compared the frequency of online interaction of 122 undergraduate students and their grades at the end of the year, noted that students who interacted and participated more in online discussions in their study did not necessarily achieve higher grades, despite the wealth of research about the benefits of online interaction. More recently, Chang (2008) found mixed results when examining the correlation between participation measured variously and achievement on project work. Weak and negative correlations were obtained in a few instances. Based on the research, Chang concluded that backgrounds of participants, such as “abilities of using a computer and the web, experiences of using a computer and the web, facilities of a computer and an Internet connection, and preference and custom of using the web” (pp. 505-506), may affect participation in an online course.

The varying backgrounds of students in the three classes in this investigation, as well as their varying technological competencies and accessibility to a computer and the Internet may be mediating factors affecting the results. The variation in backgrounds, technological competencies and accessibility to a computer and the internet was evident from the feedback obtained during the orientation programme to familiarise students with the way the course would be delivered. It is likely that a number of students capable of producing high quality work may have had limitations in their technological competencies and accessibility to the equipment and Internet services needed to facilitate a level of participation comparable to that of other student, some of whom may be among those who are less able. This may account for the low, as well as negative correlations that were evident between participation and achievement.

It is worth noting that marks awarded for online participation in the Research Methods course for the three classes in the investigation, were influenced more by the frequency of student participation than by the quality of the participation. It is possible that the correlation between participation and achievement would have been greater, had there been a heavier weighting on the quality of participation rather than the frequency of participation. The importance of quality of online interaction has been raised by both Chong (1998) and Davies and Graff (2005). This is an area that requires further research.

Summary and Conclusions

The articles explains how students in three separate offerings of a Research Methods course taught online were assigned to roles of Moderator, Researcher and Discussant for specified tasks and how these roles were defined to optimise participation. The results indicate that the participation level was high level of participation for students in all three classes. It was found, though, that the correlation between participation and achievement on the individual assignment to develop a research proposal, which was the main measure of student accomplishment in the course, was low and non-significant.

Based on the assessment of the mean participant scores for the three classes and the correlation between participation scores and scores on the individual proposal writing assignment, the author concludes that:

1.      The evidence from this study would support the view that when students in an online course are assigned specific roles, are carefully monitored and are rewarded for their online contribution, a satisfactory level of participation will result.

2.      Factors such as facility with computers and the web as well as accessibility to both may affect participation and the correlation between participation and achievement.

It is recommended that future studies should give greater attention to:

1.      exploration of the variables affecting levels of participation online; and

2.      assessment of the quality of online participation.

References

Allen, I. E. & Seaman, J. (2006). Making the grade: Online education in the United States, 2006. Needham, MA: Sloan Consortium.

Allen, I. E. & Seaman, J. (2008). Staying the course: Online education in the United States, 2008. Needham, MA: Sloan Consortium.

Anderson, T., Rourke, L., Garrison, D. R. & Archer, W. (2001). Assessing teaching presence in a in a computer conferencing context. Journal of Asynchronous Learning Networks, 5, 1-17.

Chang, C. (2008). A case study of the relationship between participation in online discussion and achievement of project work. Journal of Educational Multimedia and Hypermedia, 17 (4), 477-509.

Chong, S. (1998). Models of asynchronous computer conferencing for collaborative learning in large college classes. In C.J. Bonk & K.S. King (Eds.), Electronic collaborators: Learner centred technologies for literacy, apprenticeship and discourse (pp. 157 – 182). Mahwah, NJ: Lawrence Erlbaum Associates.

Davies, J. & Graff, M. (2005). Performance in e-learning: online participation and student grades. British Journal of Educational Technology, 36 (4), 657-663.

Hara, N., Bonk. C.J. & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28, 115-152.

Johnson, D., Johnson, R., and Smith, K. (1991). Cooperative Learning : Increasing College Faculty Instructional Productivity, ASHE-ERIC Higher Education Report 4, Washington, D. C.: George Washington University.

Lam, W. (2004). Teaching tip: Encouraging online participation. Journal of Information Systems Education, 15 (4), 343-348.

Leh, A. (2000). Action research on hybrid courses and their online communities. Educational Media International, 39, 31-38.

Li, Q. (2002). Exploration of collaborative learning and communication in an educational environment using computer-mediated communication. Journal of Research on Technology, 34 (4), 503 – 516.

Light, P., Nesbitt, E., Light, V. & White, S. (2000). Variety is the spice of life: Student use of CMC in the context of campus-based study. Computers and Education, 34, 257-267.

McNabb, J. (1994). Telecourse effectiveness: Findings in the current literature. TechTrends, 39 (4), 39-40.

Moallem, M. (2003). An interactive online course: A collaborative design model. Educational Technology Research and Development, 51(4), 85–103

Muirhead, B. (2000). Interactivity in a graduate distance education school. Educational Technology & Society, 3 (1). Retrieved November 13, 2007, from http://www.ifets.info/journals/3_1/muirhead.htm

Muirhead, B. (2001). Enhancing social interaction in computer-mediated distance education. Edat a Distance Journal, 15(4). Retrieved November 13, 2007, from http://www.usdla.org/html/journal/APR01_Issue/article02.html

Nickel, T. B. (2002). Student-to-student interaction in online discussions: The role of moderator status. Dissertation Abstracts International, 63 (1), 152.

Palloff, R. M. & Pratt, K. (2001). Lessons from the Cyberspace Classroom: The realities of online teaching. San Francisco, CA:Jossey-Bass.

Patel, C. & Patel, T. (2006). Exploring a Joint Model of Conventional and Online Learning Systems. e-Service Journal 4:(2), 27-46.

Pearson, J. (1999). Electronic networking in initial teacher education: Is a virtual faculty of education possible? Computer and Education, 32, 221-238.

Poole, D. (2000). Student participation in a discussion-oriented online course: A case study. Journal of Research on Computing in Education, 33, 162-177.

Russo, T. & Benson, S. (2005). Learning with invisible others: Perceptions of online presence and their relationship to cognitive and affective learning. Educational Technology & Society, 8, 54-62.

Seo, K. K. (2007). Utilizing peer moderation in online discussions: Addressing the controversy between teacher moderation and nonmoderation. The American Journal of Distance Education 2 (1), 21-36.

Sherry, L. (1996). Issues in distance learning. International Journal of Educational Telecommunication, 1 (4), 337-365.

Tagg, A.C. (1994). Leadership from within: Student moderation of computer conferences. The American Journal of Distance Education 8, (3), 40-50.

Veen, W., Lam, I. & Taconis, R. (1998). A virtual workshop as a tool for collaboration: Towards a model of telematic learning environments. Computers in Education, 30, 31-39.

Wechsler, A. 2002. Online Learning Casts an Ever-widening Net. Times Union; Albany, NY; March 3, s3.

About the Author

Stafford A. Griffith is Professor of Research, Measurement and Evaluation in the Institute of Education at the Mona Campus (Jamaica) of the University of the West Indies. Professor Griffith holds graduate degrees in education with specialized study in research, measurement and evaluation, and curriculum development. He also holds a LL.B. degree. Professor Griffith has over 40 years of combined work experience in teaching, curriculum development, measurement and evaluation, and programme planning and management. He previously served as Pro-Registrar of the Caribbean Examinations Council. Professor Griffith’s research interests include public examinations, large scale testing programmes, formative assessment and assessment in a constructivist learning environment.

Mailing Address: Professor Stafford A. Griffith, Institute of Education, University of the West Indies, Mona Campus, Kingston, JAMAICA.

E-Mail: drstaff@yahoo.com