Editor’s Note: Teachers have a unique opportunity to research alternative methods of teaching and learning. The data collected from this pilot study raise interesting questions for those who instruct and those who design instruction. Rather than “teach the test”, can we motive students to think critically about the subject matter to promote higher levels of learning. Interactive study questions are a way to focus student research and explore subject matter in greater depth. The retrospective questions gave valuable information about the students’ thought processes and perceptions.

Interactive Study Questions: A Pilot Study

Brent Muirhead
USA

The author will discuss a pilot project involving two study participants who used interactive web based WWII study guides. Study participants comments related their learning experiences with the interactive exercises. The project reflects insights into cognitive psychology and computer technology.

Taking American history tests can be a challenging task for high school and undergraduate college students. Students wonder how to focus their study efforts to prepare for class discussions, quizzes and tests that demand having a good working knowledge of specific information. Textbook publishers are making more web based study materials such as quizzes, flash cards and outlines.  Salomon (1988) recognized how computer technology can help learners in several ways:

1. assume part of the intellectual burden by handling lower-level functions of the task, thus enabling learners to work at higher-levels:

2. provide learners with guidance by raising questions, signaling errors, suggesting moves, etc.;

3. display intermediate states and processes en route to the final solution; and

4. provide models of how information can be represented and processed (computers & learning, para 2).

Teachers (K-12 and university) are integrating more technology into their courses as relevant ways to promote critical thinking skills. Lipman (1995) states “….critical thinking is skillful, responsible thinking that facilitates good judgment because it (1) relies upon criteria, (2) is self-correcting, and (3) is sensitive to context” (p. 146). The definition reveals the dynamic nature of critical thinking with a strong connection to megacognition. Livingston (1997) defined as “thinking about thinking” (para 2) and involves the executive control or self-regulation of the cognitive information processing.  Flavell (1979) has described metacognition in three basic categories: individual knowledge about learning, knowledge of variables to complete a task and learning tactics. Metacognition skills play a major role in a student’s ability to analysis their learning needs and create plans to meet them. Students must make a diversity of learning decisions based on their understanding of their skills and study habits (Livingston, 1997).

 The project focused on testing the effectiveness of two interactive WWII study guides that had multiple-choice, short answer and matching questions. Two WWII interactive exercises built with Hot Potatoes (2006) software (e.g. Appendix C). The first exercise involved 25 multiple-choice questions and five short answer questions (Appendix A) and the second exercise included 30 matching questions (Appendix B). The author developed study materials that reflected intellectually challenging knowledge expectations of students who could take a high school AP US history class or a college introductory course to American history. The project had two major aims: to identify if there was a particular question type that was more effective for learning or reviewing American history material and what were the advantages and disadvantages to interactive study guides. Designing the two study guides did consider three usability issues:

1. The number of errors and time to learn how to use the study guides.

2. Time to complete the two quiz exercises.

3. The user satisfaction with the study guide (Shneiderman & Plaisant, 2004).

The study participants for this project were two females (ages 19 and 53) who used their computers to take the two interactive quizzes. The 19-year old participant is a college student and the 53-year old participant has completed a college degree. They did have prior knowledge about the content of the WWII questions.  Study participants went to the website www.life-longstudy.com  (Figure 1) and clicked the ISR tab to begin work on the two study guides: WWII Study Guide Matching Questions and WWII Study Guide. There were no time limits for completing the two exercises.

Figure 1: Students click ISR tab on website www.life-longstudy.com
 

The author posed four questions to the study participants after they had completed their study guide exercises which took 40 minutes. First question, what questions (short answer, matching or multiple-choice) do you prefer to help you learn and review history information? Why this question format is more effective for you? Study participant A related that “multiple-choice; it was effective because my mind works best when eliminating answers to see the most logical response. The short answer is the least effective because it is subjective.” This response highlights how students can sometimes view short answer and essay questions as being more subjective which complicates their preparation efforts. Study participant B affirmed that “multiple-choice; eliminating the wrong answer to be more effective way of learning for me.” Teachers and technology designers who are preparing virtual study guides often include only multiple-choice and matching questions. Research literature suggests information recall can depend on the individual’s emotional state and the circumstances that they learned the material. Bruning, et al (2004) encourages teachers to provide testing opportunities that are similar to the classroom. Therefore, whenever students take a test in an unfamiliar room, it could have a negative impact on student performance. “Contextual changes in a person’s environment can cause added stress to the testing situation. Stressors cause a person’s attention to shift, lapse, or narrow. The person’s decision speed can be influenced as well” (Jacobs & Troester, 2004, Introduction, para 2). Research literature indicates that virtual study guides should closely align with quizzes and tests to enhance the student recall of information.

The second project question; what are the educational advantages and disadvantages are there to using interactive study guides? Study participant A shared that “there are major advantages such as instant yes or no response, you can keep taking the quizzes over and over tell you get 100, and less boring way of studying than staring at a book. The disadvantage is you do not have the human contact if you think the quiz is wrong.” Study participant B noted that “being able to keep working at getting the correct answers helps me learn and retain information than staring at pages in the book. The lack of human contact is the main disadvantage; can’t discuss answers you feel are incorrect.” The responses reflect an appreciation for learning material in a different format from textbooks. Textbook publishers have developed more web based study guides that recognize students being more technologically sophisticated and enjoy a diversity of learning tools. Jacobs and Troester (2004) tested the context-dependent theory of learning by having students take tests in regular desks and beanbags. The results revealed that student accuracy for recall questions was higher in the beanbags which were more comfortable and less stressful for the students. The study participants did describe some concerns about the absence of human interaction with a teacher that could have a negative impact on their accuracy of their knowledge. Students have diversity of educational needs and cognitive maturity. For instance, some students lack confidence in their academic abilities and need more individual attention, while others are highly autonomous and have different kinds of needs. Teaching responsibilities have moved away from knowledge transmission to a stronger emphasis on guidance.

The third project question asked what are some ways that interactive study guides can be improved to meet your learning needs? Study participant A noted that “this answer depends on how the study guide parallels the test or final. If the test is close to the study guide form then there is little room for improvement. If the test is an essay based one then include more short answer.” Study participant B related that “if essay questions use more short answers. If multiple-choice then the study guide is adequate.”  The research project comments affirmed creating relevant study guides that align properly with tests. Also, research studies indicate the value of stressing the understanding major concepts over memorization of detailed facts and teaching knowledge in a variety of contexts to promote a greater transfer of learning. Teachers should avoid presenting knowledge that is too strictly “context bound” (Branford, et al, 2000, p. 236). It could restrict students when trying to apply knowledge in different circumstances. Teachers who prepare students for SAT tests should focus on recreating the testing circumstances such as using time constraints and questions that are similar to the exam. The wise teacher will always strive to use the best techniques that promote enduring learning in their students. Teachers can aid students by helping them broaden the range of cues associated with encoding and retrieval of information and improves their ability to recall the information in different circumstances. 

Craik and Lockhart (1972) argue from their studies that deeper cognitive interactive with the material is essential for long-term recall. The elaboration process must be intentional to help individuals to trigger memory traces from their senses to their long-term memory. Anderson (2005) relates that "the theory called depth of processing held that rehearsal improves memory only if the material is rehearsed in a deep and meaningful way" (p. 178). Therefore, individuals must engage in elaboration of ideas in a way that makes the material relevant to them. Students can use basic strategies such as asking themselves questions before reading the material to increase their understanding (Anderson, 2005).

A fourth project question asked, what type of Internet resources are useful or have been useful in the past to assist you in learning new information? Study participant A shared that she used the following resources, “Wikepedia.org, online textbooks, GALILEO, google.com, findarticles.com, WebCT Vista, Google book search, textbook web sites, gsu.edu, Owl Perdue Writing Center and easybib.com.” Study participant B related that “google.com, WEDMD, ETenet.com and yahoo.com.” The comments reflect how the web search engines and specific knowledge sites are becoming an integral part of a student’s educational experience. Universities are developing more digital databases that contain journal articles and doctoral dissertations. Information is accessible but it is important to remember that “teachers must not confuse familiarity with knowledge or in-depth understanding. They must constantly evaluate their instruction to ensure they are building on what their students know, not just "giving information." (Critical Issue, 1995, para #9)

Interactive study guides are useful tools that should reflect testing expectations. Teachers who assign essay tests should develop essay oriented study guides. Future research projects could use a different set of survey or interview questions to discover how study guides can address specific misconceptions and inaccurate knowledge about WWII. Also, exploring peer coaching with interactive study guides is a promising study area. Research studies point out students can improve their metacognition skills by having a combination of direct instruction and observing others (Bruning et al, 2004).

In closing, investigating educational technology continues to be a challenging endeavor. Study participants noted the individual who had not a history class for over 30 years had recorded 89% correct answers on the two study guides and the person who had history class two years ago had 70% correct answers. Shneiderman and Plaisant (2004) relate “… predicting performance on complex cognitive tasks (combination of subtasks) is especially difficult because of the many strategies that might be employed and the many opportunities for going astray” (p. 83). Cognitive research continues addressing these dynamic issues as educators strive to create learning environments that help individuals develop productive studying tactics.
 

References

Anderson, J. R. (2005). Cognitive psychology and its implications (6th ed.). New York, NY: Worth Publishers.

Branford, J. D. Brown, A. L., Cocking (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press.

Bruning, R. H., Schraw, G. J., Norby, M. N., & Ronning, R. R. (2004). Cognitive psychology and instruction (4th ed.). Upper Saddle River, NJ: Pearson.

Craik, F. I. MI., & Lockhart, R. S.  (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684. In Anderson, J. R. (2005). Cognitive psychology and its implications (6th ed.). New York, NY: Worth Publishers.

Critical Issue: Building on prior knowledge and meaningful student contexts/cultures. (1995). Available: http://www.ncrel.org/sdrs/areas/issues/students/learning/lr100.htm

Emmerson, P. G. (1986). Perceptual and Motor Skills, 63, 1047-1050. In Shanks, D. (2004). C530: Human learning and memory. Available: http://www.psychol.ucl.ac.uk/david.shanks/C530/C530.html

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906-911.

Hot Potatoes (2006). Available: http://hotpot.uvic.ca/

Jacobs, L., & Troester, E. (2004). Context-dependent memory: The effects of seating on recall and recognition. Available: http://www.personal.psu.edu/users/l/m/lmj131/index.htm

Lipman, M. (1995). Critical thinking - what can it be? In A. L. Ornstein & L. S. Behar (Eds.) Contemporary Issues in Curriculum, Boston, MA: Allyn & Bacon, 145-152.

Livingston, J. (1997). Metacognition: An overview. Available: http://www.gse.buffalo.edu/fas/shuell/cep564/Metacog.htm

Ogle, D. S. (1986). K-W-L group instructional strategy. In A. S. Palincsar, D. S. Ogle, B. F. Jones, & E. G. Carr (Eds.), Teaching reading as thinking (Teleconference Resource Guide, pp. 11-17). Alexandria, VA: Association for Supervision and Curriculum Development.

Salomon, G. (1988). AI in reverse: Computer tools that turn cognitive. Journal of Educational Computing Research, 4, 123-133. In  Kozma, R. (1991). Computer-based writing tools and the cognitive needs of novice writers. Available: http://www.hu.mtu.edu/~candc/archives/v8/8_2_html/8_2_3_Kozma.html

Shneiderman, B. & Plaisant, C. (2004). Designing the User Interface: Strategies for Effective Human-Computer Interaction (4^th ed.). New York:  Addison Wesley

Sorden, S. D. (2005). A cognitive approach to instructional design for multimedia learning. Informing Science Journal, 8, 263-279.