Editor’s Note: This pilot study shows a relationship between participation in web forums and final letter grades. It also suggests that the web forum supports learning better than other forms of electronic messaging such as email. Additional research is needed to provide conclusive data about asynchronous communication tools for learning.

Online Writing as an Indicator of Student Performance

Michael K. Barbour and Michael A. J. Collins

Abstract

In this paper, the authors consider student use of a web-based discussion forum in a second year, non-major Biology course. The authors discuss how meaningful participation in the forum is a form of public writing and may be an indicator of overall student success in the course. The authors also discuss how this success in the course is not tied to the students’ previous performance at the post-secondary level.

Keywords: computer mediated-communications, web-based discussion forum, electronic messaging, student achievement, student performance, technology integration

Introduction

Over the past decade, the researchers have been investigating the effects of student participation in electronic messaging (i.e., electronic mail, asynchronous discussion forums and instant text messaging). This investigation has evolved as both the technology has changed and the research trends in the field have adjusted to reflect both these new technologies and current theories behind learning in technology enhanced environments. As the focus of this investigation has evolved, it has settled on the question of whether students’ participation in a web-based discussion forum has an effect on their final course grade.

In this article, the researchers review the evolution of this study over the past decade, tracing the changes in both the technology and the researchers’ own thinking. They also explore the question of whether the nature of student participation in a web-based discussion forum, accounting for the individual abilities of the student, play a role in overall student performance in the course.

Literature Review

Berge and Collins (1993) indicated that the main benefits of web-based discussion forums were the convenience for students, the time and place independence that it created for students, and the potential for students to become part of an online learning community. In addition to these primary benefits, there has also been considerable research into whether or not students participation in asynchronous means of communication, such as electronic mail and web-based discussion forums, have an effect on students’ performance.

In 1989, Slovacek reported that

there appeared to [be] a positive correlation between students' use of e-mail to augment normal in-class communication with their instructors and final course grades [specifically] that each e-mail message initiated by the students was associated with a 1.781 point increase in final course grade on average (pp. 113-114).

Conversely, Collins (2000a) reported little difference between the final course scores of e-mail users and non-users. However this same study showed that there was a positive relationship existed between the level of Web forum use and final course scores.

This distinction may be explained by Piirto (1998), who reported that approximately half of the students surveyed responded "never" or "not often," when asked if they proofread and/or edited their electronic mail. This was compared to 90% of students who responded that they proofread and /or edited their written documents "every time" or "most of the time." (p. 28) According to Piirto, the level of care that university students place into their composing of an electronic mail message was very low. This was supported by Collins and Barbour (2001a), who speculated that while e-mail messages are often short messages about non-content queries which are “private” and only for the instructor's eyes, postings to the Web forum are “public” and open to the scrutiny of all class members. Students are more likely to be careful and deliberate about what they write on the web forum because they are for public consumption (p. 8).

The careful and deliberate writing by students having an effect on student performance is supported by Ambron (1987), who reported

student response [was] extremely favourable; … most mentioned the value of writing in helping them understand [the subject.]” (p. 266). Moore (1993) reported that “learning improves … when writing assignments are complemented with instruction about how to use writing as a tool to learn [a subject.] (p. 217).

In addition, Chickering and Gamson (1987) reported that one of the seven principles for good undergraduate education was that students “must talk about what they are learning [and] write about it” (p. 5). They also detailed that interaction between students and the professor and between students and other students is a key mechanism in enhancing learning. This is further supported by Moore (1994), who reports that

asking students to write is an effective way of making students think because writing involves more than putting down what’s already in your head. Rather, writing is a powerful tool for learning because it helps you discover, develop, and organize your ideas (p. 290).

Many of the advocates, like Moore, claim that student learn as they write, that ideas form as pen hits paper and that writing about a subject is a way of knowing the subject because writing creates meaning.

This line of inquiry is supported by the research into microthemes. “Microthemes are a special kind of student writing whose length is strictly limited, usually to 150-400 words” (Collins, 2004, p. 7). These written assignments have been used in many larger undergraduate classes in the hard sciences. “Microthemes are marked for their ideas rather than their grammar and spelling” (Collins, 2004, p. 7). Over a four year period, Collins (2000) found that there was an overall mean gain of 0.67% in student scores between those students who completed microthemes and those students who completed a term paper. However, the average test scores of students who completed 9-11 microthemes was 13.2% higher than the average test scores of students who completed only 1-6 microthemes.

The Study

The courses considered in this study are Biology 2040 – Modern Biology and Human Society I (Human Biology) and Biology 2041 – Modern Biology and Human Society II (Environmental Science), large enrolment second-year, non-major Biology courses which were regularly offered in on-campus lecture, off-campus correspondence and Web-based formats. During the period 1994-2001, student contributions to a discussion forum and e-mail messages sent to the instructor were collected.

This study initially began as a professor “searching for a way to improve student-student and student-professor interaction in a large second year biology course… [that had] increased from 40, when [he] first taught it, to over 170 students in the 1995 winter semester” (Collins, 1995a, p. 1). The first system that was utilized for this project was a text-based environment, “DEC Notes would allow students to post and reply to notices, or to just read notes and their responses, although they could still contact [the professor]” (Collins, 1997, p. 189). Research into this system was focused upon how the students used the bulletin board, considering many of the variables raised by Berge and Collins (1993). For example, Collins (1995b) stated that one of the perceived benefits of the system was “its availability around-the-clock so that students can ask questions, make comments, etc. at any time that suits them rather than having to wait for class or an opportunity to talk to the professor” (p. 189). Collins (1998) reported that “two thirds of the student use was for on-going discussions of a variety of topics such as the ethics of genetic engineering, the right to die, AIDS and youth, and the Red Cross and blood donations” (p. 80).

Collins (2000b) marked the next phase of this study, when the authors began to consider the effect of participation in the Web forum upon a students’ performance in the course. “There [did] not appear to be a relationship between e-mail use and final course score” (p. 7). However, there appeared to be a positive relationship between Web forum use and final course score. Collins and Barbour (2001b) reported

only ‘A’s were very frequent users, and only ‘A’s and ‘B’s were frequent users. Only about one-third of ‘C’s, ‘D’s and ‘F’s were infrequent users while two-thirds made no use of the Web forum. Students achieving an ‘A’ in the course were much more likely to be Web forum users (21 of 42) than ‘B’s (12 of 29), who, in turn were more likely to be users than ‘C’s, ‘D’s, and ‘F’s (only 7 of 20) (p. 7).

This relationship was further explored in Barbour and Collins (2002a, 2002b, & 2003a).

However, this exploration led to the question of whether “higher levels of motivation or scholastic achievement may also lead some students to participate in electronic messaging more than others” (Althaus, 1996, p. 14). It also presented another “question of whether it is the act of writing which accounts for this enhanced learning or whether that interaction is meaningfully based upon the content area” (Barbour & Collins, 2004).

Data and Findings

Earlier studies have indicated a relationship between frequency of use of the web forum and final letter grade, as is summarized in Table 1.

Table 1

Frequency of use of the web forum and final letter grade
from 1997 academic year

However, this still does not address the question of whether it is interaction or the act of writing which accounts for this enhanced learning or should that interaction be meaningful based upon the content area. In order to answer the above question each student message was assigned a value based on the following scale.

0 – No content basis

1 – Administrative

2 – Content-based question or message

3 – Content-based question or message with brief explanation

4 – Content-based question or message with substantial, but incomplete explanation

5 – Content-based question or message with complete or near complete explanation.

For this analysis, it was determined that the Spring 1999 semester of Biology 2040 was the only class that had enough messages to provide an adequate sample for this scale to be utilized effectively.

While, the vast majority of messages that were posted to the web forum were rated in the lower three categories, with approximately 55% of the posts being assigned a rating of 0 or 1, there were five posts assigned a rating of 4 or 5. The majority of the messages analyzed were of an administrative nature, such as questions about the timing or format of assignments or exams, however, 46% of the messages did have some content-basis (i.e., were rated 2 or higher). When the scale value for each message is averaged on a student by student basis (e.g., Student 11 posted eight messages which were rated 2/4/2/2/1/1/3/1 and would fall in the 1.51-2.00 range as the average value of these messages is 2), the following results are obtained.

Table 2

Value of use of the web forum and final letter grade

This table indicates a similar pattern to the one found earlier by Barbour and Collins (2003b). The only student who had an average message value of higher than 2.0 scored an “A” in the course. The majority (5 out of 6) students who had average message values of 1.51 to 2.0 scored an “A” or a “B”. The only students who scored an “F” in the course did not use the web forum at all.

The next data set addresses the issue raised by Althaus (1996), whether students do well because they participate more in the Web forum or whether stronger students are simply the ones who participate the most in the Web forum. The table below provides the difference between the student’s overall average at the university and the student’s average in the course based upon their level of participation in the Web forum.

Table 3

Students mean adjusted score
based upon level of web forum usage

As is indicated in the table, the eight students who did not use the Web forum at all had an average in Biology 2040 that was 1.99 percent less than their overall university average. However, the five students who were high users of the Web forum had an average in Biology 2040 that was 15.40 percent higher than their overall university average.

Barbour and Collins (2003b) “indicated that there existed a positive, but not conclusive, relationship between the number of times students posted to the Web forum and the grade that the student received. This study has found similar results, not solely based upon simply interaction, but on meaningful, content-based interaction.” The data presented above illustrates that in addition to there being a positive relationship between students’ meaningful content-based participation in a Web forum and their final course grade, the relationship does not appear to be dependent upon the students’ higher levels of motivation or scholastic achievement. This analysis is also supported by the findings of Wu and Hiltz (2003). Wu and Hiltz found that “students felt that they learned a great deal from their peers through online discussion… [and that] online discussion increased their learning quality” (p. 691).

Conclusions

Initial studies into the relationship between students’ participation in electronic messaging and students’ final course grades indicated that there was a positive relationship between participation in the web forum and final course grade, but no relationship between the use of e-mail and their final course grade. Later studies found that it wasn’t simply participation in the Web forum, but meaningful content-based participation that also showed a positive relationship to students’ final course grades. Based upon these findings, the researchers speculated that the public act of writing in a web-based discussion forum had a positive affect on student performance.

The data presented in this article indicates that in addition to these earlier findings, the researchers speculation appears to be correct, in that students do well because they participate more in the Web forum, as opposed to the notion that stronger students are simply the ones who participant the most in the Web forum. However, it should be noted that from five years, representing dozens of classes, there was only one class in which there was enough student participation in the Web forum to conduct this analysis. In addition, this one class only represented twenty-four different students (compared to the over one thousand students who have taken Biology 2040 or 2041 during the period studied by the researchers).

Bibliography

Althaus, S. (1996). Computer-mediated communication in the university classroom: An experiment with on-line discussions. Annual Meeting of the American Political Science Association, San Francisco, California.

Ambron, J. (1987) Writing to improve learning in biology. Journal of College Science Teaching, 16(4), 263-266.

Barbour, M., & Collins, M. (2002a). Online writing as a form of electronic communication in a second year biology course. Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare and Higher Education (2544-2545). Norfolk, VA: AACE.

Barbour, M., & Collins, M. (2002b). Electronic messaging and student achievement in second-year science classes. The Morning Watch, 30(1-2). Retrieved March 17, 2004 from http://www.mun.ca/educ/faculty/mwatch/fall02/BarbourCollins.htm

Barbour, M., & Collins, M. (2003a). Online writing as a form of electronic communication in a second year biology course. Media and Technology for Human Resource Development: Journal of Educational Technology, 14(1-2), 33-42.

Barbour, M., & Collins, M. (2003b). Online writing as a form of electronic communication in a second year biology course. Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare and Higher Education (1491-1494). Norfolk, VA: AACE.

Barbour, M., & Collins, M. (2004). The effects of online writing in student performance in a second year biology course. The Assembley on Computers in English (ACE) On-Line, 1(1). Retrieved December 1, 2004 from http://faculty.gvsu.edu/patterna/aceonline/barbour/barbour-collins.htm

Berge, Z. & Collins, M. (1993). Computer conferencing and online education. The Arachnet Electronic Journal on Virtual Culture, 1(3). Retrieved on August, 28 2004 from http://www.infomotions.com/serials/aejv/aejvc-v1n03-berge-computer.txt

Chickering, A. & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. American Association Higher Education Bulletin, March, 3-7.

Collins, M.A.J. (1995a). The electronic bulletin board. Memorial University of Newfoundland Teaching and Learning Newsletter, 12(2), 1-4.

Collins, M.A.J. (1995b). Using electronic bulletin boards with college biology classes. American Biology Teacher, 57(3), 188-189.

Collins, M.A.J. (1997). A successful experiment with an electronic bulletin board in a large class: Computer conferencing promotes cooperation and interactivity among students both inside and outside of the classroom. Journal of College Science Teaching, 26(3), 189-191.

Collins, M. A. J. (1998). The use of email and electronic bulletin board in college-level biology. Journal of Computers in Mathematics and Science Teaching, 17(1), 75-94.

Collins, M.A.J. (2000a). Comparing web, correspondence and lecture versions of a second-year non-major biology course. British Journal of Educational Technology, 31(1), 21-27.

Collins, M.A.J. (2000b). Do microthemes improve student learning of biology. Paper presented at the annual National Science Teachers Association National Convention, Orlando, FL.

Collins, M.A.J. (2000c). The importance of electronic communications in successful wed-based courses. Proceedings of the annual International Conference on Advances in Infrastructure for E-Business, Science, and Education on the Internet (CD-Rom). L'Aquila, Italy: Superiore G. Reiss Romoli (SSGRR).

Collins, M.A.J. & Barbour, M.K. (2001a). Some characteristics of student use of electronic communications in second-year science classes. Proceedings of the annual conference on ED-MEDIA (309-310). Norfolk, VA: AACE.

Collins, M., & Barbour, M. (2001b). Some observations on student use of electronic communications. Proceedings of the annual International Conference on Advances in Infrastructure for E-Business, Science, and Education on the Internet (CD-Rom). L'Aquila, Italy: Superiore G. Reiss Romoli (SSGRR).

Collins, M.A.J. (2004). Using short pieces of writing (microthemes) to improve student learning. In M. Druger, E.D. Siebert, L.W. Crow (Eds), Teaching tips: innovations in undergraduate science instruction (pp. 7-8). Arlington, VA: National Science Teachers Association Press.

Leahy, R. (1994). Microthemes: An experiment with very short writings. College Teaching, 42, 15-18.

Moore, R. (1993). Does writing about science improve learning about science? Journal of College Science Teaching, 22(4), 212-217.

Moore, R. (1994). Writing to learn biology. Journal of College Science Teaching, 23(5), 289-295.

Piirto, J. (1998). University student attitudes towards e-mail as opposed to written documents. Computers in the Schools, 14(3/4), 25-32.

Slovacek, S. (1989). Electronic mail use and grades. Western Education Computer Conference, San Francisco, CA.

Wu, D. & Hiltz, S.R. (2003). Online discussions and perceived learning. Paper presented at the Ninth Americas Conference on Information Systems, Tampa, FL. Retrieved on August 28, 2004 from http://www.alnresearch.org/Data_Files/articles/full_text/wu_Hiltz(2003).pdf.

About the Authors

Michael Barbour is a Ph.D. student at the University of Georgia. A social studies teacher by training, having taught in the traditional classroom and virtual high school environments, Michael is interested in the use of virtual schools to provide learning opportunities to rural secondary school students.

Michael K. Barbour
Department of Educational Psychology and Instructional Technology
University of Georgia
Athens, Georgia, United States 30602

706-542-4025         mkb@uga.edu
 

Dr. Michael Collins has been teaching at Memorial University of Newfoundland since 1969, and is currently involved in a number of international projects. His interest in the natural history of Newfoundland has resulted in the publication of several books including Life on the Newfoundland Seashore in 1993, and The Plants and Wildflowers of Newfoundland in 1994. Dr. Collins is actively involved in computer-based education and has given a number of presentations on the subject at the local, national and international level. His work has been published in a variety of educational and scientific journals. He is currently serving as the Associate Vice-President (Academic).

Michael A. J. Collins
Department of Biology
Memorial University of Newfoundland
St. John’s, Newfoundland, Canada A1B 3X9

709-737-3259         collinsm@mun.ca