Editor’s Note: This study raises some interesting questions about the level of learning that results from distance learning as compared to face-to-face instruction. Other research needs to be conducted in other settings with different students and motivations to determine if this is the tip of a yet to be explored iceberg, or an isolated incidence.

“No Significant Difference” Only on the Surface

Thomas K. Ross and Paul D. Bell

United States

Abstract

The purpose of this study was to compare the effectiveness of an asynchronous online platform and its traditional F2F classroom based counterpart in promoting deep learning or higher levels of learning according to Bloom’s taxonomy. Equivalency between the two modes of instruction was assessed using course grades as the outcome measure. The ANOVA results suggested that on-line education is comparable to classroom education at lower levels of abstraction (recollection) but as work progresses to higher orders of abstraction (application) significant performance differences arise that place on-line learners at a disadvantage. Regression analysis was then employed to understand why the performance difference between the two groups existed or if it existed when other explanatory variables were introduced. Finally, regression results were analyzed and reasons offered for why this difference in learning outcomes occurred for the two instructional delivery modes.

Keywords: surface learning, deep learning, no significant difference, asynchronous web based learning, face to face learning, Bloom’s taxonomy, learning outcomes, ANOVA and regression.
 

Introduction

The growth in asynchronous web based undergraduate courses has been dramatic. It has, in large part, been spurred by the motivation to hold down costs while simultaneously expanding access to higher education (Jones, 2003). However, the legitimacy of this expansion in Asynchronous Web-Based Learning (AWBL) has been based on a genre of academic research known as “media comparison studies” (MCS). This genre of research is based on the finding of no significant difference between average achievement scores for learners in online versus classroom-based versions of the same course. The conclusion reached based on this lack of difference in student achievement is that online learning is as effective a modality for learning as is conventional face to face learning (Bell, 2001; Hadidi & Sung, 2000).

Some have argued that MCS (Diaz, 2002; Bendixen and Hartley, 2001) are inappropriate for establishing the effectiveness of asynchronous online courses. This is because such studies fail to assess the learning environment according to some specific blueprint or definition of effectiveness. Biggs, 1999; Ramsden, 1992; and Weigel, 2002 have further argued that the true measure of course effectiveness is related more to whether it promotes deep learning over surface learning than to whether a particular medium serves as the learning platform. They have defined deep learning as the critical analysis of new ideas and linking them to already known concepts and principles. It leads to understanding and long-term retention of concepts that can be used for problem solving in unfamiliar contexts. Deep learning promotes understanding and application for life. In contrast, surface learning is the tacit acceptance of information and memorization as isolated and unlinked facts. It leads to superficial retention of material for examinations and does not promote understanding or long-term retention of knowledge and information. Thus, a better gauge of course effectiveness would appear to be how well it promotes deep learning over surface learning. Accordingly a better method of comparing different course learning environments should include a comparison of how successful each is in promoting deep rather than surface learning.

Bloom’s taxonomy provides a practical rubric for measuring deep learning. It specifies six types of learning (or levels of abstraction) that can be used to describe one’s depth of learning. These include knowledge, comprehension, application, analysis, synthesis, and evaluation (Huitt, 2004) In the current study learners were compared on five of the six types of learning. They included knowledge, application and synthesis.

As different levels of abstraction require different study skills, the depth of student learning can be assessed according to the study skill employed. For example, Bloom’s taxonomy is used to categorize test questions that are reflective of a particular taxonomy or level of learning. Furthermore, if a student successfully answers a question at a certain taxonomy level then it can be inferred that specific study skills have been mastered. The mastery of certain study skills can, in turn, give information about the particular level of learning achieved according to Blooms taxonomy. Thus, an analysis of the number and type of questions and problems correctly answered by both face to face (F2F) learners and distance education (AW) learners can then be used to compare the depth of learning in each group.

The purpose of this study was to assess the ability of an asynchronous online platform and its traditional classroom counterpart to promote deep learning or higher levels of learning according to Bloom’s taxonomy. Equivalency between the two modes of instruction will be assessed using course grades as the outcome measure.

Method

Course Materials

A Quality Management course delivered in Spring 2007 provides an appropriate case for determining if asynchronous web based (AW) online courses are equivalent to face to face classroom based (F2F) education. In this course the only difference between the AW and F2F students was the latter group came to class and the AW students viewed the in-class lectures via the internet. The class was built on the Blackboard course management system and all lectures were recorded using Mediasite. PowerPoint slides to accompany the lectures were provided in the Course Docs section of Blackboard and available to all students prior to the lectures. The Mediasite lectures were streamed almost simultaneously with the class lectures and remained on-line for the duration of the course. F2F students had the option to: 1) attend class, 2) view the lectures on Mediasite or 3) attend class and review lectures via Mediasite. Class attendance was not mandatory and was not considered in calculating student grades.

Class lectures began with a review of the homework assigned in the prior class (if applicable) and proceeded to new material. The new material drew heavily on the text (theory) and was documented in the PowerPoint slides. Class lectures provided the opportunity to flesh out ideas, draw additional examples of applications, and demonstrate cross-industry uses of quality techniques. Some of the lecture material may not have been covered or only introduced briefly in the PowerPoint slides.

The study group/descriptive statistics

There were 79 students enrolled in the course, 44 F2F and 35 AW.  All but two of the students were junior or senior majors in the department offering the course and the two non-majors were enrolled in a five year accelerated Masters program within the same college.

Exhibit 1

Descriptive Statistics

* Final sample reduced by four due to withdraws and non-majors.

Student performance in the course was based on ten homework assignments (10% of total course grade), a midterm exam (40%), and a final exam (50%). Assignments and due dates were identical for both groups. The same exams were given to both groups and both groups completed the tests on-line. Grades on exams were determined largely on mastery of the subject rather than effort while timeliness and effort were major components of homework grades.

Two AW students withdrew after the midterm exam due to low scores. The average grade for this course was 76.11% and the standard deviation was 8.31%. On the standard four point GPA scale the grade was 2.16. Exhibit 2 displays the distribution of scores. The grades show wide variation in performance providing a good sample to study differences in performance between groups and identify the potential explanatory variables that could impact grades.
 

Exhibit 2

Grade Distribution by Setting

Results

ANOVA - Overall Course Grade

ANOVA analysis of final course scores reveal that the F2F students scored on average 6.1 points higher than their AW counterparts and the difference was statistically significant (p = 0.001). Not only were the F2F scores higher but the scores suggest that F2F students are also more consistent performers. The standard deviation for the F2F group was 0.0608 versus 0.0949 for AW students. An F test shows the variance difference is statistically significant (p = 0.001). The difference in performance is evident in exhibit 2, while the F2F students range from 88.9 to 65.2 (23.7 points), the AW earned both higher and lower scores, 90.8 to 54.2 (36.6).

Given the significant difference between the two groups on the overall course grade as well as in the consistency of performance, further ANOVA analyses were undertaken to identify the source or sources of the performance difference. First the scores for the midterm and final exams were analyzed. F2F students scored 8.28 points higher on the midterm (p = 0.003) and the difference narrowed to +6.55 on the final exam but remained statistically significant (p = 0.002). The narrowing of performance was expected given the withdrawal of two low performing AW students after the midterm.

Given the consistency of the higher performance of the F2F group across exams and using Bloom’s taxonomy as a guide, performance on the various components of the exams was analyzed. The midterm was composed of 25 multiple choice (knowledge/ comprehension), a cause and effect diagram (application/analysis/synthesis) and chart construction and interpretation (application/analysis/synthesis). F2F students scored 1.56 points higher on the 40 point multiple choice section (+3.9%) but the difference was insignificant (p = 0.232).

The cause and effect diagram (20 points) and charting problems (40 points) required a higher level of abstraction and it is this type of work where the impact of different educational settings may be apparent. On the cause and effect diagram, the F2F students scored 2.20 points (+11.0%) higher than the AW students (p = 0.003). Similar results occurred on the charting problem, F2F students scored 2.97 points (+7.4%) higher (p = 0.014).

The components of the final exam were similar to the midterm with multiple choice questions (30 points), essays (16 points), and charting and statistical process control (SPC) problems (54 points). Results for the multiple choice questions followed the midterm, F2F students performed marginally better (1.15 points or 3.8% higher) than their DE counterparts and the difference remained insignificant (p = 0.191).

On the essays, F2F students scored 0.01 points (-0.1%) lower than AW students but the difference was not significant (p = 0.973). The application, analysis, and synthesis problems (charting and SPC) accounted for the majority of the performance differential and was the only component statistically significant (p = 0.000). F2F students scored 5.43 points (+10.6%) higher than their AW counterparts. This difference in deep learning accounted for 82.9% of the total performance difference on the exam.

The remaining 10% of the course grade was based on homework assignments and there was no significant performance difference between the two groups on the ten assignments leading up to the exams. The F2F group scored 1.61 points (+1.6%) higher on these assignments but this difference was not statistically significant (p = 0.550). This result was unsurprising as grading of homework was designed to encourage students to attempt and turn in assignments versus measuring mastery. Subsequent in-class review of assignments was designed to highlight and correct student errors. Based on the data and grading scheme, both groups submitted similar assignments.

Regression

Regression analysis was used to understand why the performance difference between the two groups (F2F and AW) exists or if it exists when other explanatory variables are introduced. Regression provides a mechanism to study the impact of educational setting on student performance when other variables are controlled. According to previous research studies other variables including age (Alstete & Beutell, 2004), prior educational performance, (Garavilia & Gredler, 2002; Wang & Newlin, 2000), and utilization of course materials including completion of homework (Wang & Newlin, 2002) may affect performance. Regression was used to determine if age, GPA, homework performance, and use of on-line lectures explain any of the observed difference in performance between the two groups of students and their impact, if any, on overall course grade.

Statistics were gathered on whether or not a student viewed the on-line lectures – these statistics did not record how long the lectures were viewed but simply if any portion of the lecture was viewed. AW students on average accessed 68.3% of the posted lectures; in contrast F2F students accessed 22.5%. While the F2F students use of the on-line lectures was much lower than that of the AW students this was expected given that the on-line lectures were supplements to class attendance for F2F students and not the primary vehicle for delivering information.

Independent regressions were run on each group as the effect of viewing the lectures should be different for each. The Mediasite on-line lectures were the exclusive means of viewing the lecture material for the AW students while the Mediasite lectures were a supplement to classroom attendance for F2F students. The regression samples were reduced by two because of the absence of age and GPA information for the two non-majors, one F2F and one AW.

For AW students, age, GPA, homework performance, and number of lectures viewed on-line explained 70.1% of the difference in course grade, Exhibit 3. All variables were significant. The regression shows course grade increased by 1.10 points for every lecture viewed on-line (p = 0.000). Course grade was also positively correlated with performance on homework assignments. Course grade increased by 0.36 points for every point earned on homework (p = 0.000). Homework was a component of the overall course score but the magnitude of the coefficient, 0.36, greatly exceeds the percent of the overall grade contributed by homework performance, 10% or 0.10.

Exhibit 3

Dependent variable = course score

As expected course grade was positively correlated with GPA, and overall course score increased by 5.66 points for each one point change in GPA (p = 0.012). Unexpectedly overall course grade decreased by 0.37 points for every one year increase in age (p = 0.007). This is unexpected as the descriptive statistics, exhibit 1, demonstrate that the older group of students, the AW students, had the higher GPA, 3.23 versus 2.78 for the younger F2F students. The intercept, the expected grade when the explanatory variables are zero, was 26.25 and significant (p = 0.001).

For the F2F students, the four independent variables explained 36.3% of the difference in course grades. All variables were significant. Unlike the AW results course grade decreased by 0.54 points for every lecture viewed on-line (p = 0.040) by F2F students. The relationship between homework performance and overall course grade was positive and significant (p = 0.015) but the coefficient was much lower for F2F students. F2F student grades increase by 0.18 points for every point earned on homework assignments versus 0.36 for AW students.

The coefficients on GPA and age were in the same direction and had a similar magnitude to those observed in the AW class.  The coefficient for GPA and probability was 6.58, p = 0.002 for F2F students versus 5.66, p = 0.005 for AW students.  The coefficient for age and probability was -0.37, p = 0.020 for F2F students versus -0.36, p = 0.007 for AW students. The intercept was 55.71, considerably higher than the AW class, and significant (p = 0.000).

The major differences between the two groups were the impact of on-line lectures and homework assignments and other unmeasured variables. In both regressions the intercept is positive and significant. The intercept for the F2F class is more than twice as large as the AW coefficient (55.72 versus 26.25). Obviously excluded variables have a major impact as the intercept predicts much higher performance in the F2F class.

Discussion

The ANOVA results suggest that on-line education is comparable to classroom education at lower levels of abstraction (recollection) but as work progresses to higher orders of abstraction (application) significant performance differences arise that place on-line learners at a disadvantage.

This finding may be a result of the differences between how the two groups of learners chose to engage in their learning. That is, whether or not learners attended lectures, accessed them via Mediasite electronic recordings, and/or whether they completed homework assignments.

Course content was presented in face-to-face class lectures, video recordings of class lectures, and summarized in PowerPoint slides. Moreover students had the opportunity to practice and learn course material via homework assignments. These different modes of learning content can be conceptualized in terms of how well they support active learning. If these modes of learning are viewed on a continuum of active to passive learning the most active form of learning would be doing homework assignments followed by attending the in-class lectures, viewing the recorded lectures, and lastly, reading the PowerPoint slides for course information content. This is because each type of learning listed above requires less purposeful educational activity on the part of students. Prior research on college student development shows that the time and energy that students devote to educationally purposeful activities is the single best predictor of their learning success (Kuh, Kinzie, Schuh,Whitt et al, 2005) In keeping with this prior research, the current results suggest that those students who took advantage of opportunities to engage in active learning were the most successful learners regardless of whether they were AW or F2F students.

Differences between F2F and AW students

Despite this global finding, a key difference between the two classes is the adjusted R² (coefficient of determination) statistic. The adjusted R² for AW students is nearly twice as large as that for the F2F students. However, it should be recognized that the decrease in the adjusted R² in the F2F group is probably related to the fact that classroom attendance was not required or recorded. Had classroom attendance been recorded we would expect class performance would vary positively with attendance and a larger percentage of the variance in course grade would be explained. Furthermore, while the larger adjusted R² for the AW group might have been impacted by viewing on-line lectures, it should be noted that other non-quantified factors such as reading the textbook and amount of time spent viewing lectures, could affect student performance.

While viewing on-line lectures was a positive and important factor in predicting an AW student’s grade, the negative coefficient for F2F students suggests that viewing on-line lectures is not a good substitute for class attendance. For example, a AW student who viewed all the lectures and received the maximum available points from homework assignments could expect to score 13.3 points lower than a F2F student who viewed no on-line lectures holding other variables constant. The difference in course grades would narrow as F2F students view the on-line lectures that is, substitute on-line lectures for class attendance. If a F2F student elected to view all the lectures on-line rather than attend class the difference in course score would shrink to 6.3 points.

As stated earlier AW students only accessed 68.3% of the recorded lectures indicating that the on-line lectures were frequently not used to obtain course material. Many students preferred to access course material via the PowerPoint slides. The low use of on-line lectures by the AW students may be attributed to a difference in opinion between the students and professor concerning the most effective and efficient means for learning course material. The professor viewed on-line lectures as the primary information delivery vehicle for AW learning; the on-line lecture usage statistics indicate that AW students often felt the PowerPoint slides were the primary vehicle for learning course material. The problem and risk of relying on this passive method of obtaining information is that students may believe that a cursory reading of slides is sufficient for attaining mastery of the subject. Yet, if they do not attend lectures or at least view the video recordings, they will miss extemporaneous remarks made during the lectures that do not appear in the prepared PowerPoint slides. Relying solely on PowerPoint slides, failing to attend or view entire lectures, and inability to recognize the importance of in-class remarks presented a major obstacle to successful test performance.

On the other hand, in-class students who were physically present for the lectures not only were exposed to in-class remarks but could also pick up the verbal and non-verbal clues from the instructor and/or their classmates as to the usefulness of these remarks. Obviously the availability of the professor’s notes via PowerPoint and the internet have increased the ease and risk of passive acquisition of information. Class notes, whether those of their fellow students or the professor, are inferior substitutes for actively attending class or viewing a lecture.

In addition to how course material was accessed (in person or via video recording), course scores were also correlated with homework assignments. As stated earlier both groups had similar performance on homework assignments but similar to the utilization of on-line lectures, regression shows these assignments had a larger impact on the overall course score for AW students. An AW student earning 90 points on homework could expect this performance to translate into a 32.4 point increase in their overall course grade, a F2F student earning the same number of homework points could only expect a 16.2 increase in their course grade.

The larger impact for AW students may emanate from the reduced opportunity for exposure to course material in this group. F2F students were observed working together in the classroom, consequently, it was probably easier for students to build study and support groups in the classroom setting. Expanded opportunity for extra-classroom cooperation may improve outcomes given the greater ability to learn from one’s peers or the opportunities for guiding and coaching one’s fellow students (and thus reinforce material in the student’s mind) in a face to face setting. The reduced opportunities for exposure and interaction (especially those of a spontaneous nature) in on-line environments may place additional importance on those instructional devices that are readily available to AW students.

Another prime difference between F2F and AW students involves the student’s ability to selectively view course materials. Students who attend class have little choice but to review prior assignments and be exposed to the elaborations on the new material (obviously daydreaming would be one way of maintaining physical presence without paying attention). In the asynchronous on-line environment a student who assumes he or she knows the prior material could skip that part of the lecture or put it on fast play in addition to not attending (viewing) or paying attention. Similarly AW students could decide after the basic material was introduced that further elaborations were not needed and terminate the session. On the other hand, premature termination in the classroom, i.e. walking out, is constrained by social convention.

Reason for selecting one medium vs. another

Previous research concerning the reasons for taking an on-line course has demonstrated an association between learning achievement and student reason for selecting a particular medium for his/her learning. Studies involving on-line learners [Roblyer 1999 and Collins & Pascarella 2003] showed that those learners who earned the highest learning outcome scores in on-line learning environments belonged to those who actively selected on-line learning because they wanted to regulate their own learning. In the group of AW learners some actively chose to learn on-line instead of taking a face-to-face class. On the other hand, other students had little or no choice of their learning medium because they lived at a great geographic distance from campus or were full time working adults with little opportunity to come on campus for their learning. Regardless of the reason, self selection by learners into one medium of learning versus another may have played a role in affecting the outcomes achieved by both groups of learners.

Asynchronous web-based learning is different than learning in a classroom!

Another factor that may have affected the study results is on-line learning is different than learning in a traditional classroom setting. That is, processing information from on-line lectures is more challenging than from classroom lectures. For example, in the classroom the number of distracters is deliberately kept to a minimum ( no TV, computer, phone, refrigerator although we are aware that students surf the internet and use cell phones in class). Each lecture was approximately three hours and students in the classroom setting were committed to spending this time in their learning with classroom breaks allotted in a controlled fashion. Moreover, the presence of other students paying attention in class provided an example and support for maintaining attention in class. To the contrary, AW students have to rely on their own capacity to self regulate and control their learning. Not all students are prepared to exercise such control (Turnbull, 2003). Therefore, some AW students may not have been prepared to control their learning and needed external control imposed by the instructor in the classroom to focus their attention. Furthermore, these AW students may not have anticipated putting in the same number of hours as “traditional” classroom students. Trouble in self regulating their learning is evidenced by their belief that reviewing the PowerPoint slides was sufficient to master the course material.

On the other hand, many AW students may have approached their learning in the same manner as they would traditional classroom courses. They recognized that PowerPoint slides were supplements to lectures and invested time and effort in viewing the on-line lectures. Differences in the willingness to learn from on-line lectures demonstrated that some students were adequately prepared for the independent environment of on-line learning while others were less capable of handling the demands of self-regulation required by on-line classes. This could partially explain why variance in learning outcomes was greater for AW students than F2F students.

Age

Many educators assume that older students are more diligent in their study habits than younger students because they are more autonomous, self directed, and goal oriented in their learning compared to younger students (Knowles, 1984). As a result they are expected to succeed at a higher rate in their learning than their younger cohorts. However, previous research in on-line learning environments has yielded mixed results in this area. Some studies indicate that older learners are more successful learners (Alstete & Beutell, 2004),while other studies find no difference in rates of learning achievement among age groups (Tucker, 2001). In the current study older students had lower course scores than their younger colleagues. The age coefficients in the regression analyses were negative, similar in magnitude, and significant for both groups of students. This finding may reflect the higher burdens that older students face when they go back to school. As opposed to traditional college students, these students may have full-time jobs and families that require attention.

Previous academic achievement

Prior academic achievement was associated with higher course scores in the present study. Prior course grades, as measured by academic GPA, were a strong predictor of grades in the course: the difference between a B student (3.0) and a C student (2.0) translated into an additional 5.66 points on course grade for AW students and 6.58 for F2F students. This supports earlier research findings that demonstrate importance of previous academic achievement for predicting subsequent academic learning achievement.(Bell, 2006; Ishitani, 2003; Garavila & Gredler, 2002)

Conclusion

The present study demonstrates a large and statistically significant performance difference between AW and F2F students where the only difference between the two groups was the means by which lectures were delivered. F2F students performed better than their AW counterparts which is attributed to the more active learning environment provided by the traditional classroom setting versus computer based, on-line learning. Additionally it was shown besides performing at a higher level, F2F students were more consistent performers than AW students.

Similar to previous studies that found “no significant difference” between traditional and on-line class environments, this study found comparable performance for certain levels of learning. Specifically at lower levels of learning, knowledge and comprehension, performance was comparable across learning environments. However outcome differences arose when students were asked to apply, analyze, and synthesize information. While we show that higher level performance is positively correlated to use of course materials, completion of homework assignments, and prior academic performance, the learning environment continues to play a large and significant role in student performance.

We cannot conclude that the learning environments are comparable at anything other than the lower levels of learning. If we wish to achieve “no significant difference” between environments this study suggests that we will need to implement additional activities into the on-line environment to more actively engage AW students or become more selective in accepting students into on-line degree programs. This study shows identical activities (except for in person versus recorded lectures) do not result in similar student outcomes so more activity will be required in the less active, on-line learning environment. The study also demonstrates that certain members of the on-line community are well prepared for the demands of on-line learning so another possible step toward “no significant difference” may be to identify and accept only those students who have a high probability of succeeding in this environment. This study demonstrates that using the same learning and student selection techniques in traditional classroom and on-line learning environments produces dramatically different student outcomes when higher order learning is required.  

References

Alstete, J.W., Beutell, N. (2004), "Performance indicators in online distance learning courses: a study of management education", Quality Assurance in Education, Vol. 12 No. 1, pp. 6-14.

Bell, P.D. (2001). Learning medical terminology in a traditional classroom versus the World Wide Web: Is there a difference in PA student performance? Perspectives on Physician Assistant Education, 12(4), 232-235.

Bell, P. D., (2006). Can factors related to self-regulated learning and epistemological beliefs predict learning achievement in undergraduate asynchronous Web-based courses? Proquest Digital Dissertations, retrieved on July 13, 2007 from
http://wwwlib.umi.com/dissertations/fullcit/3233020  .

Bendixen, L., & Hartley, K. (2003). Successful learning with hypermedia: The role of epistemological beliefs and metacognitive awareness. Journal of Educational Computing Research, 28(1), 15-30.

Biggs, J. (1999). Teaching for Quality Learning at University, SHRE and Open University Press

Collins, J., & Pascarella, E. (2003). Learning on campus and learning at a distance: A randomized instructional experiment. Research in Higher Education, 44(3), 315-325.

Diaz, D. (2002, May/June). On-line drop rates revisited. Technology Source. Retrieved November 14, 2004, from http://ts.mivu.org/default.asp?show=article&id=981.

Garavilia, L. S., & Gredler, M. E. (2002). Prior achievement, aptitude and use of learning strategies as predictors of college student achievement. College Student Journal, 36(4), 616-625.

Hadidi, R., & Sung, C. (2000, August). Pedagogy of on-line instruction: Can it be as good as face to face? Paper presented at the Americas Conference on Information Systems, Long Beach, CA.

Huitt, W. (2004). Bloom et al.'s taxonomy of the cognitive domain. Educational Psychology Interactive. Valdosta, GA: Valdosta State University. Retrieved [date], from http://chiron.valdosta.edu/whuitt/col/cogsys/bloom.html.

Ishitani, T. (2003). A longitudinal approach to assessing attrition behavior among first-generation students: Time varying effects of pre-college characteristics. Research in Higher Education, 44(4), 433-449.

Jones, R. (2003). A recommendation for managing the predicted growth in college enrollment at a time of adverse economic condition. On-line Journal of Distance Learning Administration, 6(1), 1-12.

Knowles, M. (1984). The Adult Learner: A Neglected Species (3rd Ed.). Houston, TX: Gulf Publishing.

Kuh, G. D., Kinzie, J., Schuh, J. H., Whitt, E. J. & Associates (2005). Student success in college: Creating conditions that matter. San Francisco: Jossey-Bass.

Ramsden, P. (1992). Learning to Teach in Higher Education, Routledge .

Roblyer, M. D. (1999). Is choice important in distance learning? A study of student motives for taking Internet-based courses at the high school and community college levels. Journal of Research on Computing in Education, 32(1), 157-171.

Tucker, S. (2001) Distance Education: Better, Worse, Or As Good As Traditional Education? Online Journal of Distance Learning Administration, Volume IV, Number IV, Winter 2001  Retrieved 7/12/07.

Turnbull, E. D. (2003). Investigating the potential for improving experiential undergraduate curriculum through the concept of personality. Unpublished doctoral dissertation. University of Western Sydney, Sydney, Australia.

Wang, A. Y., & Newlin, M. H. (2000). Characteristics of students who enroll and succeed in psychology Web-based courses. Journal of Educational Psychology 92(1), 137-143.

Wang, A. Y., Newlin, M. H. (2002). Predictors of performance in the virtual classroom [Electronic version]. T.H.E. Journal, Retrieved July 13, 2007, from http://www.thejournal.com/magazine/vault/M2643.cfm.

Weigel, V. (2002). Deep learning for a digital age: Technology's untapped potential to enrich higher education. San Francisco: Jossey-Bass.

About the Authors

Thomas K. Ross, PhD, is Assistant Professor of Health Services and Information Management in the School of Allied Health Sciences at East Carolina University in Greenville, NC. Email: rossth@ecu.edu

Paul D. Bell, PhD, RHIA, CTR, is Associate Professor of Health Services and Information Management in the School of Allied Health Sciences at East Carolina University in Greenville, NC. Email: bellp@ecu.edu

Paul David Bell, PhD, RHIA, CTR
Associate Professor
East Carolina University
Department of Health Services & Information Management
4340E Health Sciences Building
Greenville, North Carolina 27858

Tel: 252.744.6171

Fax: 252.744.6179

E-Mail: bellp@ecu.edu