Editor’s Note: This paper raises questions as to the comparability of the groups. Did the same presenter record Fall and Spring Lectures? And if so, to what extent, if any, did this affect the outcomes?

Does Source of Recorded Lectures Affect Exam Performance of Distance-Based Pharmacy Students in a Therapeutics Course?

Eric Hoie, Gary Elsasser

Abstract

Objective: Determine if the source of recorded lectures to distance pharmacy students in a therapeutics course affects exam performance compared to on-campus learners.

Design: Distance students were given audio files of lectures, recorded during the previous year, for the fall semester. For the spring semester they had access, within 24 hours, to lectures given to campus students. Exam performance of distance students was compared to campus students both semesters and distance students were surveyed for their preference of audio file source.

Assessment: Campus students scored significantly higher on 4 of the 5 fall semester exams than distance students (p<0.05). In the spring semester distance students and campus students each scored significantly better on one exam with no difference on the remaining three exams. Twenty-five of forty-seven distance students enrolled in the course completed a survey. Previously recorded lectures were preferred by 13 (52%) students with 12 (48%) students preferring recordings of the current semester’s lectures. Eighteen students (72%) said recordings of current semester's campus lectures better prepared them for exams while 7 (28%) students said previously recorded lectures were better preparation.

Conclusion: Recordings of the current semester's lectures given on campus may improve exam performance of distance students.

Keywords: Distance-based, lecture, therapeutics, exam

Introduction

Creighton University’s School of Pharmacy and Health Professions enrolled its first pharmacy distance pathway class in the fall of 2001¹. The distance-based Pharm.D. pathway offered by Creighton University is the first of its kind. The majority of the didactic work is provided over the internet, although unlike most distance-based programs our distance students complete the majority of their didactic work at the same time and pace as their campus classmates. Lab courses and other practical experiences are completed during intensive summer sessions. With the introduction of the distance pathway, efforts were made to ensure campus students and distance students received similar educational experiences. Distance and campus students use the same course materials and the same instructors were responsible for course content and exams for both pathways.

During the first two years the distance pathway was offered, all required material was provided by means of handouts, slides, and textbooks. Lectures were not recorded as it was expected that all required material would be on a course website. Performance of the distance students was compared to campus students in each course and at the end of the second year the students had completed fourteen courses. There was one course in which distance students performed statistically better than the campus students with no difference in the other thirteen. Similar results have been reported previously at other institutions for individual courses^{2, 3}.

Prior to the fall 2003 semester it was decided to provide distance students enrolling in therapeutics with lecture recordings in the form of synchronized slide and audio files on a CD. The audio portion of the files was from the previous year’s lecture or was recorded during the summer by the faculty member who would lecture in the fall. As the fall semester progressed, we observed that distance students were not performing as well as campus students on the therapeutics exams. This was unexpected as the two groups had performed similarly during the first two years. Since the only major difference between the campus and distance students was access to “real time” lectures, it was decided to record lectures during the spring semester and provide them to distance students. As that was the only change made from fall to spring we compared exam results from both semesters to determine if the source of the audio files might affect exam performance of the distance students.

Design

In the fall semester of 2003, 47 students in the distance pathway and 97 students in the campus pathway enrolled in the therapeutics course. The course is a two semester sequence, in the third professional year, worth seven credit hours each semester and is team-taught with 10-12 instructors each semester. All students, distance-based and campus-based, enrolled in therapeutics were provided with the same lecture handouts for both semesters of the course. Before the first semester, distance students were provided with CD’s containing slides, handouts and previously recorded lecture audio files. The slides and audio files were synchronized so distance students could view and listen to lectures in a similar fashion to campus students. In most cases the audio files were from lectures given on campus the previous year. If a recording of the previous year’s lecture was not available, or if a new instructor was scheduled to lecture in 2003, the lecture was recorded during the summer of 2003 in the faculty member’s office.

Distance students enrolled in the spring semester of therapeutics accessed handouts and slides from the course website. These materials were identical to those available to campus students. All lectures given to the campus students were digitally recorded, compressed, and placed on the course website the same day.

There were five exam dates each semester. Both groups of students were administered identical exams on each of the 5 dates. All campus students completed their exams during a three hour period on paper. Distance students were given a 36 hour window to take their exams which were completed on computer under the supervision of a proctor. Following each exam, the performance of distance and campus students were compared. Mean scores for each of the exams were compared using the Student’s t-test with a p value <0.05 considered statistically significant.

Near the end of the spring semester distance students were asked to complete a two question survey on their preference of synchronized lectures on a CD versus having access to the most recent lectures and on which method better prepared them for exams.

Assessment

Campus students achieved significantly higher scores on four of the five exams during the fall semester and had a significantly higher semester average compared to distance students (Table 1). During the spring semester, campus students achieved a higher score on one exam, distance students achieved a higher score on one exam, and there were no differences on the other three exams (Table 2). There was also no difference in the overall semester average between the two groups.

Twenty-five distance students completed the survey. Thirteen students (52%) preferred the synchronized files provided on a CD prior to the semester and twelve (48%) preferred having access to current campus lectures. Eighteen students (72%) said recordings of current lectures better prepared them for exams and seven (28%) felt synchronized files better prepared them.

Table 1

Fall Semester Exam Performance

 All values expressed as percentage
^* p<0.05

Table 2

 Spring Semester Exam Performance

All values expressed as percentage

^* p<0.05

Discussion

The impact of audio files on distance student learning has been studied in a variety of settings^4-6. Embi et al. reported that medical students preferred the ability to download and review slides with audio over audio alone⁴, while Solomon et al. found that medical students who viewed digital lectures performed as well on examinations as a similar group of students who viewed the lectures live⁵. Spickard et al. randomized medical students to view a lecture with or without audio⁶. Students expressed a benefit from audio files and a trend towards improvement in student learning was noted.

What makes Creighton University’s pharmacy program unique is that two groups of students, campus and distance; complete their courses in the same time-frame, with the same instructors and materials, and both groups of students take the same exams and quizzes on the same days. These two groups of students completed the same 14 courses during the first two years of the program and in only one course was a difference in performance noted, and in that course the distance students performed better than campus students. During these two years, distance students did not have access to audio files of lectures and it is possible instructors wrote exam questions based only on handouts, slides, and assigned readings. Therapeutics instructors, knowing that distance students had audio files of their lectures, may have included exam questions from lecture material that might not have been in their notes.

Why campus students performed better on exams in the fall semester and exam performance improved for distance students in the spring semester cannot be determined with certainty, but it is possible that having access to the lectures given to campus students was a significant component. For the fall and spring semesters both groups of students had access to the same handouts and slides and while campus students can ask questions in class distance students will frequently email instructors with their questions. The only change made between the two semesters was the source of the audio files. From year-to-year, content of lectures may not change significantly but points of emphasis may. An instructor may emphasize content that a previous class struggled with on exams. Questions from students during lecture likely differ from one year to the next and may lead to a more thorough discussion of a topic. If instructors are in the habit of providing tips or clues for an exam these may also differ. While points of emphasis or tips given during a lecture may not be significantly different from year-to-year, it may account for the difference we observed in exam performance.

There was no clear preference for one delivery method. The most common reason given for preference of synchronized lectures on a CD was convenience. No internet access was needed, the slides and audio were synchronized, and students could work at a faster pace if desired. Having access to the current lectures was more time consuming for students. The files required downloading and the students no longer had synchronized files which required them to determine when they should advance slides while listening to the lecture. The majority of students thought access to current lectures better prepared them for exams. The most common reason given for preference of the current semester’s lectures was having access to the most current information. Students heard the same lectures as the campus students and had the most current information provided by instructors.

With the improvement in distance student performance during the spring semester we conclude that providing access to the most current lectures improves distance student examination performance.

References

1.  Malone PM, Glynn GF, Stohs SJ. The development and structure of a web-based entry level doctor of pharmacy pathway at Creighton University Medical Center. Am J Pharm Educ. 2004;68:Article 46

2.  Wiecha JM, Chetty VK, Pollard T, Shaw PF. Web-based versus face-to-face learning of diabetes management: The results of a comparative trial of educational methods. Fam Med. 2006;28(9):647-52

3.  Johnson SD, Aragon SR, Shaik N, Palma-Rivas N. Comparative analysis of learner satisfaction and learning outcomes in online and face-to-face learning environments. J of Interactive Learning Research. 2000;11(1):29-49

4.  Embi PJ, Biddinger PW, Goldenhar LM, Schick LC, et al. Preferences regarding the computerized delivery of lecture content: A survey of medical students. AMIA 2006 Symposium Proceedings: 916

5.  Solomon DJ, Ferenchick GS, Laird-Fick HS, Kavanaugh K. A randomized trial comparing digital and live lecture formats. BMC Medical Education. 2004;4:27

6.  Spickard A, Smithers J, Cordray D, Gigante J, Wofford JL. A randomized trial of an online lecture with and without audio. Medical Education. 2004;38:787-90

About the Authors

Dr. Eric Hoie Pharm.D., is an Associate Professor of Pharmacy Practice at Creighton University School of Pharmacy and Health Professions – Omaha, NE and the coordinator of the therapeutics class for the distance pathway students.
ehoie@creighton.edu

Eric Hoie, Pharm.D.
2500 California Plaza, HLS 147
Omaha, NE 68178

Tel: 402-280-3795

Fax: 402-280-1268 

Dr. Gary Elsasser Pharm.D. is an Associate Professor of Pharmacy Practice and Family Medicine at Creighton University. He is also Vice Chair of the Pharmacy Practice department. elsasser@creighton.edu