Editor’s Note: Formulative evaluation enables the instructor to assess the success of learning strategies and make mid course corrections to optimize results. The Midcourse Feedback procedure applied here was taken from The Online Teaching Guide by Ken White and Bob Weight.

Implementing a Midcourse Feedback Procedure
in the Online Learning Environment

Kelly Bruning

Abstract

The onset of the information age has ushered in a new revolution of learning. This learning takes the form of asynchronous communication between learner and educator using a technical platform that allows global access. This teaching process brings with it new assessment tools in evaluating student learning and the educational objectives of an online course. Student assessment and feedback is critical to the online environment and developing the learning community. Research indicates that formative evaluation tools serve as a justifiable assessment tool that allows for honest student feedback and an opportunity for the instructor to redirect the class based upon that feedback while also nurturing the learning.

Keywords: Online assessment, online learning, online student feedback, community.

Growth of Online Distance Education

The growth of the Internet and its impact on society has often been compared to the impact of the industrial revolution. The Internet's effect on online education is now being considered by academia as a reputable mode of educational delivery with global reach. Online instruction is the fastest growing field in distance education, making use of the Internet and other communication technologies to reach wide-spread audiences of distance learners (Scagnoli, 2002). Universities and colleges worldwide are taking advantage of this trend. Scagnoli (2002), states, "Universities embrace this new trend and adopt this medium to offer their traditional distance courses or to promote new programs or courses" (p. 19).

Evaluation Concepts

Course and student progress evaluation generally take two forms: formative and summative evaluation. Formative evaluation is an ongoing process that can occur at any point throughout the course; it can surface gaps in course material or in the learners' ability to grasp the material. Formative evaluation gives instructors a way to shift the focus of the course if the course is not proceeding as the instructor expected. Evaluations conducted to assess feedback prior to the course end in order to further develop the class are know as formative evaluation because they influence how the learning community develops (Preece, 2000).

Evaluations also play a validation or summative role. Summative evaluation assess the completed course and is the model most often used for evaluation in academic institutions (Palloff & Pratt, 1999). Stephen Brookfield (1995) as cited by Palloff & Pratt, 1999, in commenting on the reliance of summative evaluation, states this form of evaluation is really a measure of student satisfaction with the course and the instructor not a measure of student learning.

Many academic institutions require the use of a course evaluation format that is standardized across the organization. These evaluation forms rarely address whether or not a class has supported students in achieving their learning objectives. Instead, they tend to evaluate whether the student liked the instructor or the course (Brookfield, 1995). According to Palloff & Pratt (1999), if instructors are truly establishing a collaborative transformative process, the formative as well as summative evaluation must be used. Formative evaluation helps determine to what extent instructors are successfully facilitating reflection on the course material under study, reflection on the means of learning, and reflection on self as a learner as the course progresses. Summative evaluation helps to assess how well the instructor and students achieved the goals and learning outcomes established in the course outline.
 

Using Midcourse Feedback to Improve Online Teaching and Learning

The concept and procedure for the midcourse feedback evaluation tool is outlined in the text entitled The Online Teaching Guide A Handbook of Attitudes, Strategies, and Techniques for the Virtual Classroom written by Ken White and Bob Weight who are members of the University of Phoenix faculty. White and Weight (2000), state,

"In order for online teaching to be effective it needs to include the feedback of student opinions….it is really the online students who are in a better position to comment on classroom teaching skills, course difficulty, and online instructor-student interaction" (p. 175). Pioneered for traditional onsite courses as a means of evaluating teaching effectiveness, online education can be greatly enhanced by the use of midcourse feedback (White & Weight, 2000).

The purpose of the midcourse feedback procedure project was to gain an honest assessment by students of the course at midpoint in order for the instructor to redirect the class. Fostering an open and nurturing learning environment will aid in soliciting honest student feedback. Many times, participants are unwilling to be completely honest about their evaluation of a course or an instructor due to fear of repercussions (Brookfield, 1995). Consequently, the willingness of participants to be honest will be related to one that includes the instructor as an equal member (Palloff & Pratt, 1999).

The Midcourse Feedback Process for BUS105

The first step in the process is to introduce the idea of the Midcourse Feedback Procedure as early in the course as possible, preferably in the syllabus. The second process is to secure a volunteer from the course who would be willing to collect the individual e-mails and cut and paste them into a word document to protect student anonymity. In this case, a freshman level Business Mathematics Course was selected for the process by the instructor who created and piloted the course during the summer (taught one time prior to the fall class providing the assessment feedback).

Figure 1. Blackboard Posting to the Class From Instructor (step 1)

Figure 2. Blackboard Posting to Class from Instructor
after volunteer secured (step 2)

 

Figure 3. Blackboard posting of Responses (step 3) by Instructor

1. List three things that are working for you in this class:
 

Student Responses:

"I really enjoy Kelly's class. She helps out so much. She is always available. Now if we could talk about some other internet classes.

I wouldn't change anything."

"I like that we can work ahead if we want to.

I like the prompt responses from the teacher.

I like the CP interaction with my classmates."

"Ability to access site.
Great communications link
Instructors aid with problems prompt.

No thoughts on how to improve procedures. Structure of class is excellent."

"Three things that are working well are the discussion boards, the ease of the website, the layout and easy access to information (grades, assignments, so forth) I also like that our teacher communicates with us often."

"The student/teacher interaction in this course is unmatched. Great job!

The discussion forums for HW help and the whole Create-a-Problems concept is very good.

I think the deadline dates and times are right where they need to be." 

Instructor Reply:

I am glad that most of you enjoy the format of the class. It has taken a long time to develop. I believe the CP problems add to student understanding and communication among learners. I make it a priority to be in contact with you on a regular basis, so I am glad to learn I am doing a good job! J
 

2. List three ways to improve the course:

Student Response:

"I think more time is needed in the quiz portion. I run right to the wire on most of the answers trying to do the calculations. Also as far as tests go I wish we could use our handouts or sample tests. It is hard to keep everything straight. Also the CP are good but some of the questions are confusing or misleading they are a good learning tool but I will use myself as an example I have a hard enough time without confusing others too."

Instructor Response

The quizzes are timed 30 minutes the same as the “live class.” That group also takes the same quizzes and the time limit is adhered to. Those that aren’t finished (which are only a few) turn it when the time limit is up. The quiz is timed at an interval to measure comprehension and understanding at an acceptable level (meaning that 98% -100%of the class is done by that time).

Student Response:

"I think that it is difficult (for some) to get to campus for the exams.

Instructor Response

It is the Instructor’s prerogative to request exams to be taken on campus. In the introductory material sent to all learners and in orientation it was stated that this was a requirement. To protect the integrity of the exams, I take the option of on campus testing and make it a point to inform students as early as possible (and it’s in the course description about 4 exam weeks as well). I feel I’ve allowed flexibility in the exam week leaving students plenty of options on dates and times. I do not plan on having exams online and feel that I’ve done a good job in incorporating test integrity with on-campus exams in a flexible time-frame for students.

Student Response:

I think that it would be easier if things were due on the same day instead of some things this day and some things another day.

Instructor Response:

The create-a-problem exercises do not allow everything to be due on the same day since your partner needs time to respond. Also, students need time to review the homework sheet with the correct answers before taking their quiz. I felt that the Wed./Sunday due dates were well spaced out and easy to remember while also incorporating time for students to learn from reworking homework problems and from the Create-a-Problems (that’s why I don’t have everything due on Wed). To have HW, CP, Quiz, and CP responses all due on Wed. I think would be too much too soon without allowing time for things to “sink in.”

Student Response:

I cannot think of a third, all in all this class runs pretty smoothly."

Instructor Response:

Good ….. that makes me happy!

Student Response:

"I feel a time limit on quizzes is little too much. The purpose of the quiz is to help you do better it seems some instructors are so geared on punishment and not exactly helping you. Homework assignments that need to be handed in at about 10:00.

Instructor Response:

I’m not really sure on what this comment means. The quizzes are timed just as in a regular course. As well, deadlines are in place just as they are in a regular course. The homework assignments need to be passed in by 11:00 p.m. on Wed. This deadline shouldn’t be a problem unless students work last minute. The 10:00 p.m. deadline was for questions to the homework on the discussion link, not the homework to the drop box (you still have that extra hour). This is because I will reply to HW questions to the discussion link up until 10:00 p.m. on Wednesday nights. This resulted when a HW question was posted at 10:55 p.m. on a Wed. with an expectation for an immediate response from the instructor. If students want a response, then courtesy needs to play a role as well. I’ll check up until 10:00 p.m. and answer.

The purpose of the quiz is as a learning tool. That’s why they are only weighted at 10% and that’s also why you get an immediate response to the questions you had incorrect. I believe by allowing answers to appear helps the learner understand errors and students can also e-mail me any questions regarding the quiz as some of you have. The time limit is in place as in the regular classes.

Some students have an expectation that Internet courses don’t have due dates or time limits and that they are self-paced. That myth is dispelled in my introductory letter sent to students before orientation and also on the Announcement page. This posting and direct mail letter to students informs students before orientation and class start that there are due dates and time limits and exam weeks so the student can make an informed decision on if the platform of the course meets their expectations.

I hope this has served as a useful learning tool for all of us. Again, thank you for your participation in the midcourse feedback process.

Sincerely,

Kelly Littlefield

Figure 4. Blackboard posting from Instructor

 

Learning Outcome of Midcourse Feedback Process

The learning outcome from this process is that the instructor was able to assess the development of the course. The instructor did not implement change, since the majority of the comments provided positive feedback. It reinforced to the instructor that the learning platform, navigation of the course content, and assignments were all contributing to the success of the course.

A personal reflection by the instructor was that the feedback procedure reinforced the concept of the Business Math Create-A-Problems which was a learning tool developed by the instructor to form a learning community. The students interact with one another using the discussion thread and create mathematical word problems based on course concepts that a fellow learner needs to solve. The partner solves the problems and sends them back to the author via the discussion thread. The author then must correct any mistakes in detail and/or provide positive feedback to the partner. Meanwhile, the author also receives problems from another learner which he/she is responsible to solve and the student that created those problems must provide feedback in a positive tone.

The Midcourse Feedback Procedure provided additional feedback to the Discipline Chair on the student assessment of this class. As indicated in Figure 4, the Midcourse Feedback Procedure was copied and sent to the Discipline Chair for his review and comments. The procedure also nurtured the development of the learning community for the course. By posting the student comments verbatim as well as the instructor's response to the course room, it allowed other learners to read the comments from their peers. The outcome has been more interaction by the learners in the discussion thread. As well, more positive e-mails were received by the instructor regarding student issues. According to White & Weight (2000) the midcourse feedback is especially useful in online classrooms because it is more personal than individual student ratings and allows student to read what other students are thinking (p. 181).

Benefits and Limitations of Midcourse Feedback

The process of the midcourse feedback builds the online learning community. Its timing allows instructors to make changes during the same class, and its feedback contains specific suggestions on how to make those changes. The procedure is a tool that emphasizes to students tat they have a role in reshaping the method of instruction and subsequent learning outcomes (White & Weight, 2000). The process allows the instructor to gain insight from the perception of the student on how the class is going before it is too late to restructure the class. Continual evaluation is essential and faculty must be willing to adjust their course materials to accommodate what students are telling them while still achieving the learning outcomes (Bauman, 2002).

"Maintaining the midcourse feedback as a formative process-voluntary, anonymous and confidential helps to encourage open communication and a meaningful online student feedback" (White & Weight, 2000, p. 181). If the feedback tool is forced upon students or the students feel threatened with an instructor's recourse and/or anonymity is breeched, the feedback process will not produce honest, reliable student assessment of the class.

Conclusion

The Midcourse Feedback Procedure provided the opportunity to receive feedback from online students at a point mid-way through the course enabling the instructor to gauge the learning methods in the course and redirect if necessary. The formative evaluation tool was outlined in the course text entitled The Online Teaching Guide written by Ken White and Bob Weight, both faculty members from the University of Phoenix.

The Midcourse Feedback Procedure is a formative evaluation tool that allows the instructor to solicit student assessment of the online class in a non-threatening way. The process is voluntary by students, the students' comments remain anonymous and confidentiality by the volunteer that assimilates the comments is guaranteed. This creates a secure environment in which students feel safe in sharing their true assessment of the course.

The Midcourse Feedback Procedure starts when the instructor explains to students as early as possible the purpose of the feedback tool and stresses the importance that their feedback will help shape the course for the remainder of the semester. An e-mail message goes out to the students asking for a volunteer. The first person that responds is usually the volunteer. The instructor composes two questions that comprise the evaluation tool. They consists of (1) List three things that are working for your in the course, and (2) List three things to improve the course. The volunteer collects the e-mail and copies and paste them to word to protect student anonymity. The instructor receives the student responses unedited by the volunteer, in one document, with no names. The instructor can choose to assimilate and reflect upon the feedback and not post a response to the students. However, it is strongly recommended by White and Weight that the instructor constructively responds to each and every comment and posts it to the public platform entity of the learning community. This provided an opportunity for other learners to read the comments from their peers along with the instructor's response, which leads to developing the learning community.

The outcome of the Midcourse Feedback Process provided the instructor with feedback to make changes to the course if warranted. In this case, it reinforced receptivity by students with the Create-A-Problem interaction and the way that the course was designed into four modules allowing for easy navigation to find assignments, quizzes, homework, and helpful hints on specific chapters. It also allowed the instructor to share the feedback with the Discipline Chair who also suggested a way of improving student time on quizzes that was subsequently posted to the course room for students to read.

Another outcome of this Midcourse Feedback Process was that the instructor went on to use the concept in all classes (meaning face-to-face classes) and provided the same type of feedback to students in an oral discussion after the comments were reviewed. This too, has nurtured a more positive classroom environment since student feel that their input makes a difference in how the rest of the class is conducted.

The Midcourse Feedback Process is a formative evaluation tool that can change the direction of the course well in advance of the end of the semester. The instructor gains honest student assessment feedback in a non-threatening way and the students gain a sense of input on their learning outcomes while also developing either a learning community or a more positive classroom camaraderie. The Midcourse Feedback Process outlined by White and Weight in The Online Teaching Guide is an excellent formative student assessment tool when put into practice in both online classes and face- to-face traditional classes.

References

Bauman, P. (2002). Student retention: what you can control, and how.
Distance Education Report, 6, p. 8.

Brookfield, S.D. (1995). Becoming a critically reflective teacher. SanFrancisco: Jossey-Bass, Inc.

Palloff, R., and Pratt, K., (1999). Building learning communities in cyberspace: effective strategies for the online classroom. San Francisco: Jossey-Bass, Inc.

Preece, J. (2000). Online communities designing usability, supporting sociability.
New York: Hohn Wiley & Sons, LTD.

Scagnoli, Norma, I., (2001). Student orientation for online programs. Journal of Research on Technology in Education, 34, 19-27.

White, K., and Weight, B. (2000). The online teaching guide: a handbook of attitudes, strategies and techniques for the virtual classroom. Needham Heights: Allyn & Bacon.

About the Author

go top

 June 2005 Index

Home Page