Editor’s Note: Bruce Joyce’s Models of Teaching, now in its 8^th edition, is the classic text on methodologies such as Advance Organizers (AO). This research by Chen and Hirumi is a gift to update the repertoire of instructional designers and writers of online curriculum. I had assumed AOs worked for everybody, ignoring the role of learning styles and levels of experience. Now I have to rethink, as you will, when and how to use this tool most effectively. It would be interesting to have readers of this article respond with their comments pro and con on their experiences with Advance Organizers for distance learning.

Effects of Advance Organizers on Learning for Differentiated Learners in a fully Web-Based Course

Baiyun Chen, Astusi Hirumi

USA

In online education, the use of multimedia resources often brings challenges of cognitive overload and learner disorientation (Dias & Sousa, 1997), especially for learners with low learning ability or poor prior knowledge. While learners enjoy the flexibility and abundance of Internet resources, they may also be overwhelmed with multiple tasks and sources of information. Effective online teaching and learning strategies, including advance organizers, debate, case studies, and guest experts, have been widely perceived as potential solutions to the learning challenges (Bonk & Dennen, 2003). However, there is limited research on integrating teaching and learning strategies in fully Web-based environments. The present research is an endeavor to explore the potentials of using an instructional strategy – advance organizers (AOs) – in a fully Web-based course.

Keywords: Online education; advance organizer; Web-based learning; instructional strategy; instructional designer; learning ability; experimental research; ANOVA; concept map; learning outcome; higher education

Purpose of the study

An advance organizer (AO) is relevant introductory materials presented in advance in any format of text, graphics, or hypermedia (Ausubel, 1968). A concept map was used as a graphic AO, and a text outline was used as a textual AO. As the outcome of the study, students’ learning achievement in knowledge acquisition and application was tested both immediately and four weeks after the experiment. In analysis, students of the treatment groups and the control group were divided into two sub-groups based on their scores. Specifically, this study investigated if the use of AOs would improve learning outcomes differently for learners of differentiated learning ability levels in a fully Web-based course.

Theoretical framework

The rationale for using AOs is rooted in cognitive learning theories. Cognitive theories state that learning performance depends on processing capacity and prior knowledge. With the aid of AOs, learners are able to link what they already know to new information and apply it to new contexts.

Ausubel first introduced the concept of AOs in his assimilation theory of meaningful learning and retention. Based on his theory and early experiments, Ausubel (1968) asserted that the use of AOs helps learners activate prior knowledge in the new instructional context and makes the instructional process meaningful. Moreover, one of his assumptions was that learners having either low ability or low prior knowledge of the material should be helped more by AOs than other learners are.

Mayer (1979) reinterpreted Ausubel’s theory in terms of his assimilation encoding theory. He contended that if the learner already possesses a rich set of relevant past experiences and knowledge and has developed a strategy for using it (eg, a high-ability learner), the AO would not be effective. The results of his analyses demonstrated that AOs more strongly aid performance when material is poorly integrated and for inexperienced learners.

However, later studies were not consistent in supporting that AOs are especially effective for low-ability learners. On one hand, a study on graphic AOs supported the assumption for learners of differentiated learning ability (Alvermann, 1988). The study found a facilitative AO effect for self-perceived low-ability students. But for the self-perceived high-ability students, the AO appeared not to help but to interfere with their learning. On the other hand, two meta-analyses were conducted on extensive AO research in the early 1980s. Both analyses found AOs are effective for all ability levels, and they have no special facilitation for low-ability learners (Luiten, Ames, & Ackerson, 1980; Stone, 1983).

In the 1990s and 2000s, many researchers conducted studies on AOs of various formats, such as graphic AOs (DaRos & Onwuegbuzie, 1999; Hirumi & Bowers, 1991) and multimedia AOs (Calandra, Lang, & Barron, 2002; Tseng, Wang, Lin, & Hung, 2002; Yeh & Lehman, 2001). However, in the recent studies, the variables such as learning ability or prior knowledge were not controlled. Thus, there is no evidence to suggest if the use of AOs would have special facilitation for learners of high or low learning ability in Web-based learning environments.

Hypotheses

Two hypotheses were posited for this study.

Hypothesis I.   Students of high ability using an advance organizer (either a concept map or a text outline) will achieve higher learning outcomes in both short-term and long-term knowledge-based and performance-based assessments than those in the control group.

Hypothesis II.  Students of low ability using an advance organizer (either a concept map or a text outline) will achieve higher learning outcomes in both short-term and long-term knowledge-based and performance-based assessments than those in the control group.

Methods

Subjects

The population of this study consisted of 166 undergraduate students enrolled in a fully Web-based health-related ethics class in fall 2006 at a southeastern university in the United States. The students were mostly above 21 years old in either their junior or senior years. One hundred and twelve of the total 166 students voluntarily participated in the experimental activities.

Research Design

This study used an experimental control-group posttest-only design with a random assignment to examine the effects of AOs on learning achievements as illustrated in Figure 1.

          R  E₁          X₁ (Graphic Organizer)                        O₁                    O₂

          R  E₂          X₂ (Text Organizer)                           O₃                    O₄

          R  C           (No Advance Organizer)                     O₅                     O₆

1: Research Design Diagram

The 166 participants were randomly (R) assigned to three groups, two treatment groups and one control group. AOs were the intervention in this experimental design. The experimental group (E₁) reviewed a concept map (X₁), a form of graphic AO, before reading a textbook. The comparison group (E₂) reviewed a text outline (X₂), a textual AO, and the control group (C) did not read any AO. During the course of the study, all three groups completed an immediate posttest I (O₁, O₃, O₅) and a delayed posttest II (O₂, O₄, O₆).

Dependent and Independent Variables

One of the dependent variables in this study is students’ learning achievement, encompassing their short-term and long-term knowledge acquisition and application learning achievements. The short-term and long-term knowledge acquisition was tested with two parallel 9-item knowledge quizzes. The short-term and long-term knowledge application was tested with problem-based scenario essay questions.

The independent variable is the treatment of AOs. The three groups had the same instruction, except for the treatment of AOs. The experimental group was intervened with a graphic AO; the comparison group was intervened with a textual AO; and the control group had no AO exposure before textbook reading. The graphic AO is a flash-based interactive concept map. The textual AO is a text outline. Both AOs present the same concepts with a different presentation of the relationship among the concepts.

Instruments

This study utilized two major instruments: posttest I and posttest II. Posttest I is comprised of Quiz 1 and Scenario 1. Posttest II is comprised of Quiz 2 and Scenario 2. Both posttests are parallel in content and format, with a 9-multiple-choice-question quiz examining concept acquisition and three open-ended questions based on a scenario, testing knowledge application. Posttest I measures students’ short-term learning achievement, and posttest II measures their long-term achievement.

Procedures

This study lasted for six weeks. During the first week of the fall semester, 2006, participants were randomly assigned into three groups. The experimental module was open to the students for seven days in week two. The students were suggested to first review the AOs to gain an overall idea of the key concepts, if they had one AO available. The experimental group (group 1) reviewed the concept map before reading the book. The comparison group (group 2) reviewed the text outline. The control group (group 3) proceeded directly to textbook reading without reading an AO.

After textbook reading, the students were instructed to complete posttest I. The knowledge quiz of posttest I was a timed WebCT quiz. For the second part of posttest I, the students answered three questions based on a scenario using Microsoft Word and submitted the assignment to the WebCT Assignment tool.

Four weeks after the module, in week six, posttest II was administered through WebCT. With
all other assignments, posttest II, including a quiz and three scenario questions, was open for students. Students completed posttest II with the knowledge they learned in week two module.

To back up the results from the quantitative analyses, student surveys and interviews were conducted to further explore students’ attitudes toward using AO and how they used AO.

Analyses

Statistical procedures, including descriptive analysis and Analysis of Variance (ANOVA), were performed to study the research findings. Descriptive analysis was used for scores in the posttests. Means, standard deviations, and students’ learning achievement scores were computed for each quiz and scenario questions. Based on Ausubel’s assimilation theory, students having low learning ability should benefit from using AOs more than their peers (Ausubel, 1968). To validate this theoretical proposition, students were divided into two sub-groups based on the average mean scores of posttest I, and ANOVA analysis was conducted on the high-scorers and the low-scorers.

Findings

Hypothesis I - high-scorers

Knowledge-Based Quiz. ANOVA was performed on the quiz results of students with scores over or equal to 60 in quiz 1. There is no statistically significant difference either in quiz 1 scores (F_{2, 85}=0.329, p>0.05) or in quiz 2 scores (F_{2, 74}=1.055, p>0.05) among the high-scorers of the three groups. Table 1 demonstrates means, standard deviations, and effect sizes of the higher-scorers.

Table 1

 Descriptive analysis of quiz scores (high-scorers)

Among high-scorers, group 1 using a concept map achieved the highest mean score (M=72.14), with group 3, the control group, in the middle (M=71.43), and group 2 using a text outline the lowest (M=70.36) in quiz 1. In quiz 2, group 3 (M=67.14) outperformed group 1 (M=62.14) and group 2 (M=60.71). The effect sizes are small in quiz 1 and small to medium in quiz 2.

Performance-based scenario questions. ANOVA was performed on the scenario question results of students with a score over 22.5. There is no statistically significant difference either in scenario 1 scores (F_{2, 88}=0.165, p>0.05) or in scenario 2 scores (F_{2, 76}=0.013, p>0.05) among the high-scorers of the three treatment groups. Table 2 demonstrates means, standard deviations, and effect sizes of the scenario scores for higher-scorers.

Table 2

Descriptive analysis of scenario scores (high-scorers)

In scenario 1 questions, there is little variation in scores among the three groups. Group 2 achieved the highest mean score (M=24.04), with group 3 in the middle (M=23.86), and group 1 the lowest (M=23.74). In scenario 2 questions, group 1 (M=22.55) and group 2 (M=22.54) outperformed group 3 (M=22.43). The effect sizes are quite small between the treatment groups and the control group in both posttests.

Hypothesis II - low-scorers

Knowledge-based quiz. The ANOVA analysis of quiz results for the low-scorers demonstrates no statistically significant difference either in quiz 1 scores (F_{2, 53}=0.495, p>0.05) or in quiz 2 scores (F_{2, 47}=0.208, p>0.05) among the three groups. Table 3 demonstrates the means, standard deviations, and effect sizes for the low-scorers.

Table 3

Descriptive analysis of quiz scores (low-scorers)

Among the low-scorers, both AO treatment groups achieved the same mean score (M=42.94), considerably higher than that of group 3 (M=39.33) in quiz 1. In quiz 2, group 1 earned the highest scores (M=55.29), with group 2 the second (M=53.53), and group 3 lowest (M=52.67). Both effect sizes are small to medium between the treatment groups and the control group.

Performance-based scenario questions. The ANOVA analysis of scenario questions results of the low-scorers demonstrates no statistically significant difference either in scenario 1 scores (F_{2, 37}=0.373, p>0.05) or in scenario 2 scores (F_{2, 30}=0.676, p>0.05) among the three groups. Table 4 demonstrates the detailed means, standard deviations, and effect sizes of the low-scorers.

Among the low-scorers, group 2 achieved the highest mean score (M=20.18), higher than that of group 3 (M=19.91) and that of group 1 (M=19.60) in scenario 1 questions. In scenario 2 questions, group 3 (M=21.09) scores the highest, with group 2 (M=20.21) the second, and group 1 (M=19.75) the lowest. Most of the effect sizes are negative between the treatment groups and the control group, indicating a negative effect of the treatment.

Table 4

Descriptive analysis of scenario scores (low-scorers)

Discussions

Hypothesis I

Hypothesis I failed to be rejected. Students of high ability using an AO did not achieve higher learning outcomes in either short-term or long-term knowledge-based or performance-based assessments than those in the control group. The high-scorer subgroup consists of fewer than 80 students, which is a small population and underpowered in terms of significance tests. For most of the test analyses, there was little difference in scores among groups, especially for the learning outcomes of the scenario questions, and no AO effect was found.

The use of AOs interfered with students of high ability in learning, especially in the long-term knowledge-based learning achievements. In quiz 2, the control group scored considerably higher than the other two treatment groups by over five points out of a full score of 90. Both AO effect sizes were negative and ranged from small to medium. Even though the differences among the groups were not statistically significant, the control group outperformed the treatment groups considerably, given the small sample size in the sub-group analysis.

In summary, AOs do not assist students of high learning ability in this study for their knowledge acquisition or retention. Moreover, students expressed a desire to use student-constructed AOs in the survey results. It is estimated that the use of teacher-constructed AOs, as the ones utilized in this study, might have restrained their long-term knowledge retention.

Hypothesis II

Overall, hypothesis II failed to be rejected. According to Ausubel’s assimilation theory (1968), it is anticipated that the low-scorers would benefit more from the AOs than the high-scorers did. In the present study, taking into consideration that the low-scorer subgroup consists of fewer than 50 students in the analyses, even though a statistical significance was not reached, AO benefits were demonstrated by better quiz performances of low-ability students who had used an AO.

In both the short-term and long-term quizzes, students in the treatment groups outperformed the control group in mean scores. The effect sizes of the AO groups were small to medium. Although no statistical significance was found among the three groups, given the small sample size in the analysis on low-scorers, the small to medium effect sizes indicate considerable AO benefits with helping low-scorers in knowledge acquisition. The findings are in agreement with prior research (Alvermann, 1988; Ausubel 1968), demonstrating that AOs, especially the graphic AOs, might assist students of low ability in knowledge acquisition. Compared with the high-ability peers, the low-ability subgroup had more problems with taking an initiative when organizing new information. The AOs, especially the concept map, helped them scaffold the new knowledge and thus made it easier for them to process the information deeply while they were reading.

However, the results in scenario 2 questions showed an opposite trend. The control group outscored the concept map group. The effect size of the concept map is medium and negative. A detailed analysis of the data shows that the negative effect might be caused by skewed and underpowered data, not by a negative impact of the concept map for low-scorers. Small sample size and measurement error might be important attributors for such a high negative effect size.

In spite of the non-statistically significance from the quantitative results, the student interviews illustrate how AOs facilitate learning in this study. The interviewees described that AOs provided them with a general overview of the main topics which prepared them to be more involved in their own reading and learning. They pointed out that AOs refreshed their memory of the declarative knowledge in assessments and helped them relate important concepts with real-life scenarios.

Implications

This study is an attempt to validate Ausubel’s AO theory in fully Web-based learning environments. The original AO model was first developed for the face-to-face classroom setting where the blackboard is the main teaching medium. The framework had been constantly modified by later researchers to further investigate the methods for constructing and applying an AO in a computer-based instructional environment in the late 80s and early 90s. In the new century, school learning is enhanced and optimized with the explosive development of emerging Internet technologies and diversified digital media. However, the research on AOs in fully Web-based learning is very limited. The current study expands the AO framework to fully Web-based environments. The use of AOs is a good teaching and learning practice in the context of self-paced online learning. Students might benefit from using AOs not only in a traditional classroom, but also in the ever-growing Web-based learning environment.

The assumption that AOs helped low-ability learners is suggested in the present study. Students of low ability performed better with an AO in both the short-term and long-term tests than those without an AO. The use of AOs helped them cultivate a meaningful learning process by well organizing the relevant knowledge structure, and develop an emotional commitment by integrating new knowledge with existing knowledge.

The results of this study suggest that integration of AOs for online student remedial programs may be beneficial. Since the No Child Left Behind Act was signed into law in the United States in 2002, the American schools have tried every means to help students of low learning abilities to catch up with their peers. Many at-risk or dropout students are given another chance to make up for their school credits by taking online remedial courses or programs. It might be helpful to incorporate AOs, especially an interactive multimedia concept map, into self-paced Web-based remedial courses. AOs help learners identify large general concepts prior to instruction of more specific details, and assist them in sequencing learning tasks with progressively more explicit knowledge that can be anchored into developing conceptual frameworks, if they cannot make sense of the materials by themselves. Moreover, graphic and interactive AOs may strengthen students’ motivation to choose to learn by attempting to associate new meanings with their prior knowledge, rather than simply memorizing concept definitions, propositional statements or computational procedures.

AOs may be helpful Web-based learning devices for new online learners as well. Nearly 96% of the very largest institutions have some online offerings (Allen & Seaman, 2006). Yet online learning can be intimidating and disorienting for laymen. Instructors and course designers can use AOs to point out course contents and instructional activities relative to their educational goals. With the aid of a graphic or textual AO, students can visualize the course and connections among subtopics its entirety. It is easier for new learners to navigate through different course components with a bigger picture of the course contents and clearly-delineated objectives in mind.

Limitations and Recommendations

Even though the differences between the treatment groups and the control group are considerable, a statistically significant difference was not obtained based on a small sample size in the study. There were only fewer than 30 students in each group among the high-scorers and fewer than 20 among the low-scorers. It is anticipated that a significant result might be generated from a larger population in the future.

Also, the limited intervention duration might be a major factor that negatively influenced the effectiveness of AO in this study. The current AO intervention lasted for one week. However, one week is not long enough for students to fully master the AO strategy in online classes. Longer intervention time is highly recommended for AO research. Future studies should be extended to semester-long interventions. Additionally, students’ performance with the aid of AOs can be monitored and measured in multiple posttests throughout the semester.

The assessment instruments for this study can be improved. One of the issues that the researcher had found in the study is that an online quiz is difficult to monitor. Though the quizzes had been instructed as closed-book tests and questions were randomized in order, it was impossible to prevent students from referring to their lecture notes or textbooks while they were taking the online quizzes. This might seriously threaten the validity of the test instruments. An important implication for further research is to develop measures to prevent students from online cheating. Another reason for the non-significant result in the current study might be the lack of measurement of students’ analytical and critical thinking abilities. The scenario questions may lack sensitivity and discrimination, since there is little differentiation in results for both performance-based tests. The standard deviation for the scores is very low and the average mean scores are approaching the full score. There is little room for differentiation or improvement in both scenario-question tests. Future studies need to develop more strict rubrics and assessment instruments to differentiate students’ learning application outcomes.

For learners of high ability or with ample prior online experiences, the use of teacher-constructed AOs, as the ones utilized in the present study, might not help, but interfere with their learning. The high-scorers are capable of taking a structured and deliberative approach without the assistance of a pre-existing organizer. It is worth trying to engage them with a participatory organizer (student-constructed organizer) for future studies. However, the participatory organizer might be a new direction for future studies on instructional strategies in Web-based learning. Learners can use participatory AOs to create concept maps or outlines of their own. According to the generative learning hypothesis (Kenny, 1993), participatory organizers may improve students’ information retention and learning transfer by encouraging them to explore and construct the connections among concepts. In this way, students may interact with the learning materials in great depth, thus making the materials easy for them to comprehend and use.

Recently, new instructional concept mapping tools have become available for instructors and students to create digital organizers in computer-assisted instruction and online education. For example, the Visual Understanding Environment (VUE) and the C-Map are two free information management applications that provide an interactive concept mapping interface. Future Web-based AO research studies can take advantage of these free concept mapping tools, focus on helping students generate their own organizers, and measure the effectiveness of participatory organizers in both face-to-face and Web-based educational settings.

References

Allen, I. E., & Seaman, J. (2006). Making the Grade: Online Education in the United States, 2006.   Retrieved Feb. 6, 2007, from http://www.aln.org/publications/freedownloads.asp

Alvermann, D. E. (1988). Effects of spontaneous and induced lookbacks on self-perceived high and low-ability comprehenders. Journal of Educational Research, 81(6), 325-331.

Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart, & Winston.

Bonk, C. J., & Dennen, V. (2003). Frameworks for research, design, benchmarks, training, and pedagogy in Web-based distance education. In M. G. Moore & B. Anderson (Eds.), Handbook of distance education (pp. 331-348). Mahwah, NJ: Lawrence Erlbaum Associates.

Calandra, B. D., Lang, T. R., & Barron, A. E. (2002). Assessing Holocaust education: Preservice teachers' knowledge and attitude. Paper presented at the Annual Meeting of the Eastern Educational Research Association, Sarasota, FL.

DaRos, D., & Onwuegbuzie, A. J. (1999). The effect of advance organizers on achievement in graduate-level research methodology courses. National Forum of Applied Educational Research Journal-Electronic, 12E(3), 83-91.

Dias, P., & Sousa, A. P. (1997). Understanding navigation and disorientation in hypermedia learning environments. Journal of Educational Multimedia and Hypermedia, 6(2), 173-185.

Driscoll, M. P. (1999). Meaningful learning and schema theory. In Psychology of learning for instruction (2nd ed., pp. 113-151). Needham Heights, Massachusetts: Allyn & Bacon, A Pearson Education Company.

Hirumi, A., & Bowers, D. R. (1991). Enhancing motivation and acquisition of coordinate concepts by using concept trees. Journal of Educational Research, 84(5), 273-279.

Kenny, R. F. (1993). The effectiveness of instructional orienting activities in computer-based instruction. Paper presented at the Association for Educational Communications and Technology, New Orleans, LA. from http://search.epnet.com/login.aspx?direct=true&db=eric&an=ED362172.

Luiten, J., Ames, W., & Ackerson, G. (1980). a meta-analysis of the effects of advance organizers on learning and retention. American Educational Research Journal, 17(2), 211-218.

Mayer, R. E. (1979). Can advance organizers influence meaningful learning? Review of Educational Research, 4(2), 371-383.

Stone, C. L. (1983). A meta-analysis of advanced organizer studies. Journal of Experimental Education, 51(4), 194-199.

Tseng, C., Wang, W., Lin, Y., & Hung, P.-h. (2002). Effects of Computerized Advance Organizers on Elementary School Mathematics Learning. Paper presented at the International Conference on Computers in Education.

Yeh, S.-W., & Lehman, J. D. (2001). Effects of learner control and learning strategies on English as a foreign language (EFL) learning from interactive hypermedia lessons. Journal of Educational Multimedia and Hypermedia, 10(2), 141-159.

About the Authors