| May 2009 Index | Home Page |
Editor’s Note: In a discipline responsible for the well-being of others, the quality of education and training is a major concern for both teachers and learners. Does distance learning create anxiety because teacher and learner are physically separated? And does this impact the quality of learning? This study explores and compares depression, anxiety and academic performance for traditional and distance learning.
The Effects of Synchronized Distance Education (SDE)
on Anxiety, Depression, and Academic Achievement
in a Physician Assistant Studies Program
Carroll-Ann W. Goldsmith, Kathryn E. Miller, Louise Lee,
Tom Moreau, Susan White, Scott Lee Massey
USA
Abstract
Purpose: This pilot study examined depression, anxiety, and academic achievement among two cohorts of Physician Assistant (PA) students; one cohort received the majority (>80%) of class content via synchronous distance education (SDE), while the other received the majority (>80%) of class content onsite via traditional delivery (TD).
Methods: Depression, using the Beck Depression Inventory-II (BDI); anxiety, using the Global Severity Index (GSI) from the Brief Symptom Inventory-18 (BSI); and academic achievement, measured by test scores across the curriculum, were studied. The BSI and BDI were administered four times during the students’ first didactic year (N>9, Jan., May, Sept. and Dec. 2008).
Results: Mean BDI and GSI scores were not significantly different between SDE and TD cohorts at each survey administration, but depression and anxiety scores significantly increased for all students over the course of the academic year. Academic achievement was not significantly different between the SDE and TD cohorts.
Conclusions: These results suggest that PA students receiving their classroom content via SDE were as academically successful as their TD peers, but that over time both cohorts experienced statistically significant increases in depression and anxiety. These data can assure PA students, faculty, and administrators that SDE is an effective model for didactic education, while suggesting that interventions be considered to alleviate the anxiety and depression that accompany intensive study during PA didactic course work.
Introduction
In recent years, communications technology and infrastructure have developed to the point where interactive educational content can be delivered effectively in real-time across broad geographical terrain. The use of synchronous distance education (SDE) and virtual classrooms have been shown to be effective for delivering content in a variety of educational settings.1-3 The impact that such technological innovations has on student levels of depression and anxiety remains largely unexplored. In addition, the Accreditation Review Commission on Education for the Physician Assistant Inc.-Physician Assistants (ARC-PA) standard B1.10 states “The [PA] program must assure educational equivalency of course content, student experience and access to didactic and laboratory materials when instruction is conducted at geographically separate locations” http://www.arc-pa.org/Standards/3rdeditionwithPDchangesandregionals4.24.08a.pdf. This study provides valuable data to demonstrate equivalency of academic achievement between two geographically separated cohorts.
Post-graduate education is typically more intense than undergraduate coursework. This has been shown to be particularly true for students actively engaged in professional educational programs such as medical school. Often, considerable intellectual challenges are encountered at a time when the individual is subjected to an incursion of significant financial debt and alterations of personal and family lives. It is no surprise then that several studies have shown elevated measures of depression and anxiety in graduate, law, and medical students.4-8
The intensive PA professional program is likely to expose students to stressors similar to those seen among medical students and students enrolled in other professional education programs. Despite a call for studies of affective parameters of stress within individuals who are learning the PA profession, no published data exists.9 Furthermore, no studies of stress symptoms have been found for health care professionals who receive the majority of their content education via synchronous distance education.
To answer this call, two surveys were utilized to measure anxiety and depression in the PA students enrolled in this study, the Brief Symptom Inventory-18 (BSI) and Beck Depression Inventory-II (BDI), respectively. The BDI, a tool for assessing depression, utilizes a four point scale, ranging from 0 to 3, for each of 21 questions.10,11 These scores may be presented as means, medians, or categorized according to severity of depression (total score of 0 – 13 is considered minimal, 14 – 19 mild, 20 – 28 moderate, and 29 – 63 severe). Medical students at the onset of their first year of medical school in the fall had a mean BDI score of 3.28 + 4.41; this score had increased to 6.14 + 6.22 by the spring of their first year of medical school.12 Over the course of the 4-year study mean scores ranged from 3.03 + 3.94 to 8.27 + 8.55.12
The BSI measures anxiety and depression, utilizing a five point scale, ranging from 0 to 5, for each of 18 questions. The BSI can be broken into three subscales of six questions each, anxiety, depression and somatization.13 This work focused on the total score produced by the BSI, referred to as the Global Severity Index (GSI), an overall measure of stress. The mean GSI score can vary from one population to another. For example, Cochran and Hale examined cohorts of male and female college students and compared their levels of distress to adolescents and adults.14 The average GSI for college men and women in the Cochran study was significantly different from that of adult women and men.14 According to the BSI manual, the average community GSI is 8 for women and 5 for men.15
The primary objectives of this pilot study were to examine levels of depression, anxiety, and academic achievement among two cohorts of PA students, one that received the majority (>80%) of its course content via SDE, while the other received the majority (>80%) of its course content via TD. It was hypothesized that measures of depression and anxiety, using the established self-reporting instruments, the BDI and the BSI10,11,13, would be different for students participating in SDE and TD. It was expected that academic achievement would be similar for the two cohorts.
Methods
Subjects: Subjects who elected to participate in the study came from two different student cohorts in the PA Program at two of the three campuses at the Massachusetts College of Pharmacy and Health Sciences (MCPHS). The pool for this research study included 43 incoming students of the class of 2009 at the Manchester Campus and 24 students at the Worcester Campus. Thirty-three students from the
In accordance with MCPHS institutional review board (IRB) guidelines and recommendations from the MCPHS IRB committee, participants were educated about the study at the time of orientation. All appropriate consent forms were distributed at both campuses by a designated individual who explained the reason and rationale for the study and was not involved in the students’ educational process. The same script was utilized for both PA-M and PA-W students to avoid bias. Students were given the option not to participate in the study, and both cohorts of students were informed that they could withdraw from participation at any time. Though 33 subjects in the PA-M cohort and 19 in the PA-W cohort initially enrolled, study subject attrition was high, as demonstrated by the total number of participants who filled out the BSI and BDI at all four administrations (PA-M=14 and PA-W=9). This attrition was addressed statistically, as described below.
Materials: The Brief Symptom Inventory-18 (BSI) and Beck Depression Inventory-II (BDI) were from Pearson Education, Inc. (Minneapolis, MN) and Harcourt Brace & Company (San Antonio, TX), respectively. The instruments were used according to manufacturer instructions.
Hypotheses and statistical analyses: The first research hypothesis was that there would be a statistically significant difference in levels of anxiety and depression between the cohort of students educated in the SDE format (PA-W) and the cohort of students receiving a majority of education with TD (PA-M) at each survey administration after the first, baseline, administration. The second research hypothesis was that there would be a statistically significant effect of time when comparing the BDI score and GSI score from the BDI over the three semesters that encompass the didactic year. The final research hypothesis was that differences in academic achievement between these groups would not be statistically significant.
Study data were generated from January 2008 until December 2008 using two validated surveys to measure anxiety and depression, the BSI and BDI, respectively.10,11,13 The BDI utilizes a four point scale, ranging from 0 to 3, for each of 21 questions. These scores may be presented as means, medians, or categorized according to severity of depression; mean scores were utilized for this work. The BSI measures anxiety and depression, utilizing a five point scale, ranging from 0 to 5, for each of 18 questions. The BSI can be reported as a total score, referred to as the Global Severity Index (GSI), or it can be broken into and reported separately as three subscales (of six questions each), anxiety, depression and somatization. The GSI, the sum of all these subscales and an overall measure of stress, was analyzed in this investigation. All scales (anxiety, depression, and somatization) on the BSI can be converted to standardized area T scores, characterized by a mean of 50 and standard deviation of 10. Using community norms, as reported by the manufacturer, for females and males combined, the mean T score of 50 equates to a raw GSI score of 5, with a raw GSI score of 15 being within one standard deviation, and a score of 33 being within two standard deviations.15
Students’ levels of anxiety and depression were measured four times using the BDI and BSI instruments (baseline: January 2008; after semester one: May 2008; after semester two: September 2008; and at the end of year one: December 2008). Data were analyzed separately for BDI and GSI and a Bonferroni adjustment was used to control for Type I error; all tests used the adjusted alpha .025. Data were analyzed using a mixed between-within subjects analysis of variance to evaluate the influence of the two educational delivery models (TD and SDE) on students levels of depression and stress, as measured by the BDI and the GSI, respectively, over time. The assumption of sphericity was violated, so the Greenhouse-Geisser correction was used.
A series of post-hoc dependent samples t-tests were conducted to locate marginal mean differences on the BDI and the GSI over the four administrations. The Bonferroni adjustment was used to control family-wise Type I error and all tests used the adjusted alpha critical value of 0.008.
Course grades and practical exam grades were used to measure student achievement and compare the performances of the SDE and TD students. At the end of the spring (May 2008), summer (August 2008), and fall (December 2008) terms, grades were recorded and compared across delivery models. Non-directional independent samples t-tests were conducted to test for difference in academic achievement between the groups.
No variables were manipulated during the study nor did one cohort receive benefits that the other did not receive. No specific interventions were involved in the study. Participants had full access to counseling services at all times and any student who was identified as being depressed, anxious or in need of academic support was promptly referred. This study maintained strict adherence to the institution’s protocols for studies involving human subjects.
Results
Did the method of delivery, TD or SDE, affect anxiety and depression over time?
There was no significant interaction between BDI score over time and delivery model [F (1.764, 42.343) = 0.522, p = 0.572], indicating that delivery model was not a significant factor contributing to the students’ depression scores on the BDI over time. There was also no interaction between educational delivery model and the GSI (the summary index produced from the BSI) over time [F (3, 81) = 0.542, p = 0.655], indicating that delivery model was not a significant factor contributing to the students’ GSI scores over time.
Was there a statistically significant difference in anxiety and depression between students educated in an SDE setting versus a TD setting?
The main effect, delivery model, was not statistically different in anxiety or depression scores, suggesting no difference between the SDE and TD groups. [BDI: F (1, 24) = 0.005, p = 0.945; GSI: F (1, 27) = 0.104, p = 0.749]. In other words, both the SDE and TD groups were statistically equivalent at each of the four administrations, indicating that stress and depression scores were the same regardless of delivery model. Means and standard deviations are reported for the BDI and GSI in Tables 1 and 2, respectively.
Was there a statistically significant change in anxiety and depression over time in PA students, regardless of method of delivery?
The main effect, time, was statistically significant for both the BDI and GSI, suggesting that all students experienced more anxiety and depression as they progressed through the didactic year. Scores over time for the BDI and GSI are shown in Figures 1 and 2, respectively. The main effect for mean BDI score over time was statistically significant [F (1.764, 42.343) = 5.712, p = 0.008], as was the GSI score over time [F (3, 81) = 6.699, p < 0.0005], suggesting time was a significant factor affecting both BDI and GSI scores.
Figure 1. Beck Depression Inventory (BDI) Scores
Figure 2. Brief System Inventory (BSI) Scores
Comparisons of mean BDI score changes over time are located in Table 3, while those for the GSI are in Table 4. While none of the comparisons were statistically significant at the adjusted alpha of 0.025, there may be some practical significance worth noting, especially considering the relatively low p-values for both delivery modes from the January to the May administrations. As more cohorts are tracked and sample sizes increase, it is expected that significant results for both cohorts will be seen.
The attrition rate for both groups (TD and SDE) was approximately 50%, contributing to error in the study. In an attempt to control for this, independent samples t-tests were conducted to test whether those who completed all four BDI administrations differed at baseline (January) from those who did not complete all four administrations of the BDI. Tests showed no statistically significant differences in baseline from those who completed all four administrations of the BDI and those who did not, for either delivery method. Independent samples t-tests were also conducted to test whether those who completed all four BSI administrations differed at baseline from those who did not complete all four administrations. Tests showed no statistically significant differences in baseline from those who completed all four administrations of the BSI and those who did not, for either delivery method. Results are shown in Table 5.
Was there a statistically significant difference in academic achievement between students educated in the SDE setting compared to the TD setting?
Academic achievement was not statistically significantly different between the SDE and TD cohorts on any of the 13 course grades or 8 practical/lab grades measured; thus, the SDE group achieved grades and scores statistically similar to their TD classmates. Figures 3 (spring 2008), 4 (summer 2008), and 5 (fall 2008) show comparisons and respective p-values for the two cohorts.
Table 6 shows mean grades for each course and lab/practical scores for both delivery methods.
Discussion
These results indicate that there were no statistically significant differences between method of delivery, SDE and TD, on measures of anxiety and depression in cohorts of first year PA students. Scores of depression and anxiety, as measured by the BDI and the GSI, were statistically equivalent for both the SDE and TD groups at each of four administrations, indicating that stress and depression were the same regardless of delivery model. These results do demonstrate that PA students, regardless of mode of delivery, experience increased anxiety and depression as they progress through the didactic year.
Mean differences in academic achievement, measured by using final course grades and practical exam grades over the course of the PA didactic year, were not statistically different between the two groups, SDE and TD, on any of the 13 course grades or 8 practical/lab grades. Thus, despite the SDE cohort having most of their didactic lessons taught via synchronized distance education, this group was able to achieve grades and scores statistically equivalent to their TD classmates. This measure can assure students, faculty, and administrators that SDE is an effective model for didactic education.
It should be noted that these results represent one class progressing through a PA program and that small subject numbers likely contributed to lack of significance in some of the parameters studied. In order to increase subject numbers and more precisely reflect the influence of mode of didactic delivery on depression and anxiety, the study was expanded in September 2008 to include doctor of pharmacy (PharmD) students at the MCPHS Manchester and Worcester campuses. The PharmD program has a similar cohort structure, though the locations of the SDE and TD cohorts are reversed in comparison to the PA cohorts, with the PharmD SDE cohort located in Manchester and the TD cohort located in Worcester. It is believed that a larger subject number and the ability to longitudinally follow two cohorts in a different discipline will provide further statistical validity to the data and valuable insights to plan interventions that can be implemented to define emerging best practices for student services, academic support services, and pedagogy for students in programs that utilize SDE.
References
1. Allen M, Bourhis J, Burrell N, Mabry E. Comparing student satisfaction with distance education to traditional classrooms in higher education: A meta-analysis. American Journal of Distance Education. 2002;16(2):83-97.
2. Bernard RM, Abrami PC, Lou Y, et al. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research. 2004;74(3):379-439.
3. Childers JL, Berner RT. General education issues, distance education practices: Building community and classroom interaction through the integration of curriculum, instructional design, and technology. Journal of General Education. 2000;49(1):53-65.
4. Finkelstein C, Brownstein A, Scott C, Lan Y. Anxiety and stress reduction in medical education: An intervention. Med Educ. 2007;41(3):258-264.
5. Helmers KF, Danoff D, Steinert Y, Leyton M, Young SN. Stress and depressed mood in medical students, law students, and graduate students at McGill University. Acad Med. 1997;72(8):708-714.
6. Levine RE, Litwins SD, Frye AW. An evaluation of depressed mood in two classes of medical students. Acad Psychiatry. 2006;30(3):235-237.
7. Niemi PM, Vainiomaki PT. Medical students' distress--quality, continuity and gender differences during a six-year medical programme. Med Teach. 2006;28(2):136-141.
8. Vitaliano PP, Maiuro RD, Russo J, Mitchell ES, Carr JE, Citters RL. A biopsychosocial model of medical student distress. J Behav Med. 1988;11(4):311-331.
9. Brown MH. Perspective on Physician Assistant Education. 2004;15(2):116-120.
10. Beck AT, Steer RA, Ball R, Ranieri W. Comparison of beck depression inventories -IA and -II in psychiatric outpatients. J Pers Assess. 1996;67(3):588-597.
11. Steer RA, Clark DA, Beck AT, Ranieri WF. Common and specific dimensions of self-reported anxiety and depression: The BDI-II versus the BDI-IA. Behav Res Ther. 1999;37(2):183-190.
12. Clark DC, Zeldow PB. Vicissitudes of depressed mood during four years of medical school. JAMA. 1988;260(17):2521-2528.
13. Derogatis LR, Melisaratos N. The brief symptom inventory: An introductory report. Psychol Med. 1983;13(3):595-605.
14. Cochran CD, Hale WD. College student norms on the brief symptom inventory. J Clin Psychol. 1985;41(6):777-779.
15 . Derogatis LR. BSI-18 Brief Symptom Inventory 18, Administration, Scoring, and Procedures Manual. Minneapolis, MN: NCS Pearson, Inc.; 2001.
About the Authors
Carroll-Ann W. Goldsmith, DSc, Massachusetts College of Pharmacy and Health Sciences Manchester.
Kathryn E. Miller, MA, Massachusetts College of Pharmacy and Health Sciences - Boston.
Louise Lee, MHS, PA-C, Massachusetts College of Pharmacy and Health Sciences - Manchester.
Tom Moreau, MS, PA-C, Massachusetts College of Pharmacy and Health Sciences - Manchester.
Susan White, MD, Massachusetts College of Pharmacy and Health Sciences - Manchester.
Scott Lee Massey* PhD, PA-C, Massachusetts College of Pharmacy and Health Sciences – Manchester.
corresponding author:
Scott Lee Massey
Massachusetts College of Pharmacy and Health Sciences
1260 Elm Street
Manchester, NH 03101
Phone: 603-314-1708
Fax: 603-314-0147
Email: scott.massey@mcphs.edu