Editor’s Note: Teaching/Learning experiences may sometimes extrapolate well from a face-to-face live classroom to the distance learning experience. The bibliography for this paper is of special interest – widely varied and creatively demonstrated?
A Case Study on Effectiveness of Debriefing as an Instructional Strategy in Web-Based Instruction
Cheng-Chang Pan, Michael Sullivan
This investigation is concentrated on the use of debriefing in a graduate distance education course within three semesters from a southern state university in the United States. Using content analysis, six themes are identified: learned decisions, reinforcement, reciprocity, anticipation, perception change, and positive attitude. Results suggest that debriefing be considered instrumental in sustaining students’ endeavors and allowing for group synergy.
Keywords: Debriefing, instructional strategy, online course, elearning, distance education, distributed learning, Web-based instruction, synchronous communications.
The present study is intended to explore the potential use of “debriefing” as an instructional strategy in an online graduate practicum course within a state university in southern Texas, USA. The problem that we encounter as cyber instructors is the lack of efficacy for learner reflection, which is considered a personal process and may or may not involve a dialogue with an enabler or a peer.
The goals of this investigation are twofold: first, we want to examine how “debriefing” is used in other disciplines, e.g., military and corporate; second, we wish to examine how debriefing strategies can be used in an online course in order to ascertain what potential benefits, if any, can be gleaned for students, instructors and the instructional package itself. Cyber instructors and course developers/instructional designers are expected to benefit from this study/presentation.
Let us begin by asking ourselves, “What is debriefing?”
We can begin by defining debriefing by a quick look at its origins as a means for gathering and sharing information in an interactive fashion, in a variety of situations. Debriefing originated with the United States Air Force during World War II, and is still a commonly used means by which pilots are interviewed after returning from an assigned mission and an account of their operation is then elicited (Colby, 1980). In the medical field, debriefing is adopted as a widely accepted intervention for traumatized victims (Deahl, 2000; Smith & Roberts, 2003). Additionally, health professionals regard debriefing as common practice in a focus group dealing with using adolescents as standardized patients (Blake, Gusella, Greaven, & Wakefield, 2006).
Debriefing is also seen in the corporate world, particularly in the training area (Chernick, 1992; Geber, 1994). Chernick (1992) states that upon completion of a telecommunication training course, a debriefing session is usually held. The debriefing session tends to include a group of participants who respond to a pre-defined list of unstructured questions altogether. Such activity can be followed by a clarification session if needed. Two frequently asked questions during the debriefing are, “What worked well?” and “What would you change to make it [the course] better (Chernick, 1992, p. 72)?” Further specifics of an algorithmic approach were prescribed by Thiagarajan (as cited in Gebber, 1994) in which he proposes five stages of debriefing (p. 11):
Stage One: How do you feel?
Stage Two: What happened?
Stage Three: Do you agree?
Stage Four: Has this ever happened to you in the workplace?
Stage Five: What if?…
Advocates of debriefing seem to go beyond personal reflection practice and emphasize a purposive conversation (as opposed to a casual conversation or an intrapersonal dialogue) within a group setting.
An important distinction is made by Andrusyszyn and Yankou (2004) who go to great length to distinguish debriefing from reflection. The two Canadian researchers state that reflection is an intrapersonal activity which is generally used to enable an individual learner to review his/her courses of conduct and to clarify his/her own experience of interacting with the external world. Debriefing is a different activity and for different purposes than the reflective gaze in the mirror of one’s psyche. Debriefing, for them, is a specific form of reflection for a particular purpose that may involve a group. It also serves as a strategy to foster reflection. Drawn from the experiential learning theory, Menaker, Coleman, Collins, and Murawski (2006) acknowledge the significance of revisiting personal experience and its influence on both the individual and group levels. They further assert that such practice can allow learners to strengthen their knowledge, skills, and attitude and to promote deep learning.
Russell (as cited in Collis, De Boer, and Slotman, 2001) separates debriefing from feedback. According to Russell, feedback is a process of measuring learner performance on standards. This process is a type of comparison that occurs in a learning context where a pre-determined set of correct or incorrect answers exists. Conversely, debriefing does not deal with “a guaranteed outcome and the purpose of debriefing is to explore options” (p. 307). These options suggest a new understanding of the learning experience.
Apparently, literature appears to suggest that debriefing can be an effective method for learning transfer--the optimal objective of most course (activity) designs.
This inquiry is a qualitative study in nature. Three classes of graduate students (with a total of 38 students) enrolled in the practicum class between Fall 2006 and Summer 2007 within an online graduate program participated in the study. The practicum course is only offered to those who are finishing the last six hours in the 36-hour Educational Technology program within a state university in southern Texas. The course is delivered via Blackboard course management system (CMS) through a state telecourse department that coordinates with other campuses with the same state university system and makes this joint offering of a completely online MEd a reality. The state university confers the degree of Master’s of Education in Educational Technology.
Data were collected from students’ biweekly reflective writing assignments. Only those reflection papers submitted right before and immediately after the scheduled midterm debriefing are selected for further analysis. A total of 17 papers are selected because midterm debriefing was mentioned in the writing. Two of them were submitted before the midterm debriefing. Data were analyzed using content analysis. The content analysis procedure consisted of taking all written reflections and “color coding” key words, phrases, examples of self dialogue (or self questioning), questions to the instructor (classified as wanting further clarification, advice on possible changes in strategies and directions of individual projects), etc. The results of the content analysis revealed six major themes which will be briefly discussed at the conclusion of this paper. A description of the course content and course activities follows.
Course Description and Design
Below is the course description adapted from the course in Spring 2007:
This is your “capstone” course for the Educational Technology program. In this course you will synthesize your skills and conceptual background you have gained (or refined) as you have progressed through the program. You will produce both a product and demonstrate a process. The product will be a topic of your choosing. In the event you cannot select a topic, or would prefer not to select a topic for your project, the instructor will do so for you. Please refer to the “Projects” link for further information about what is expected for your project. The process upon which this course will focus is self-reflection. This course will ask you to answer the question, “How will you continue your education after you have completed your formal coursework and have your Master’s degree?” By systematically reflecting upon how you identify problems and develop strategies, how you gather and relegate resources, and how you monitor your progress as you grapple with a solution to an instructional problem, you will be able to identify your areas of strength, and those areas you may want to further develop after you have obtained your degree. Please see calendar and refer to the “Reflections” link for further information on this part of the course.
As far course design goes, the practicum class entails two required meetings: midterm debriefing and final oral presentation in Horizon Wimba Live Classroom, a conference management system.
Voluntary meetings are also scheduled, and they are orientation (at the beginning of the semester) and Design Document meeting (scheduled in between midterm debriefing and final oral presentation). Unscheduled meetings are available in response to emerging needs of class participants. For instance, two of the three classes had an unscheduled meeting on conducting instructional (task) analysis and diagramming selected tasks.
Regarding the coursework, the course consists of one capstone project and six reflection papers, plus the two mandated meetings. The capstone project is divided into three parts:
Part 1: Proposal
Part 1 Project is a proposal where students identify a performance problem with an instructional solution to it. A template is furnished (see Appendix A). Students will present their proposal to the instructor for approval in the first two weeks.
Part 2: Instructional materials (i.e., deliverables) and design document
The next 12 weeks are used to develop Part 2 project, which is the bulk of the capstone project. Using the Dick and Carey model (Dick, Carey, & Carey, 2006) as a guide, students will proceed with the proposal accordingly. A grading system is provided for students in advance.
Part 3: Lessons learned
Part 3 Project is concerned with the lessons learned. An 800-word formal writing is assigned, which is a synopsis of the practicum project and summative reflective writings. No colloquialism, slang, or idioms are allowed. A grading system is also provided for students to prepare this document.
The two mandatory meetings, midterm debriefing and final oral presentation were facilitated using Horizon Wimba Live Classroom, embedded in this Blackboard practicum course. Using VoIP technology, the conference management system enables the class to speak, present their PowerPoint slides, and share the desktop of their computers with other class members. Typically class members would convene for approximately 90 minute to 150 minute sessions. In “round robin” style, class members would present their project and answer a set of questions, designed as a format to debrief presenters. Debriefing exercises were not “tied” to the preset question bank, and the debriefing was purposely designed to be open-ended if the exercise moved away from the prepared question set. The questions prompting the midterm debriefing follow.
One week prior to the scheduled meeting, students received the following lead questions from the instructor:
What is the terminal or ultimate goal of the project (within the timeframe of this course)? Tell us again what you plan to accomplish by April 11, 2007.
What does the deliverable look like now? Is it taking shape? What does your end-product look like at this point?
How is the culminating project going? If you recall, you are also putting together a website that demonstrates all the reflections and projects. The website serves as a portal, where viewers can click and read your work. How is this part going?
What lessons have you learned?
What is the chance of finishing the project in about a month?
Is there any other concern or issue that may interest the class as a whole?
These questions are intended to assist the students in preparing for the midterm debriefing.
Final oral presentation
Below are questions sent by the instructor to the students one week before the scheduled meeting:
What is the terminal or ultimate goal of the project (within the timeframe of this course)? Tell us again what you planned to accomplish.
What have you accomplished in your Part II project? Describe it.
How does the end product differ from your initial plan?
Tell us about your rationale for the instructional design by using one of your lessons/modules as an example.
What lesson(s) have you learned from this capstone project?
Likely, these questions are intended to lead the students at the final presentation.
It is worth noting that, although the class orientation is involuntary, students are greatly encouraged to take part in the event on the first day of the class or manage to listen to the archived session. Due to the nature of this cyber class, reminders are provided at the event:
Stick to the goal.
Keep up with the timeline.
Inventory available resources and assets.
Use a previously learned repertoire of skill sets.
Prepare for the unexpected.
Seek support from others.
Remain calm under all circumstances.
Allow time for self-reflection.
Results and Interpretation
With the help of the midterm debriefing, students acquired a new understanding of the purpose of the practicum course. Their overall perception of this type of instructional strategy was positive. Our preliminary analysis suggests that six themes have emerged.
The midterm event allowed for learned decisions. For instance, Student A made the following remarks in the reflection assignment:
“After feedback from my last reflection paper and comments made during the midterm debriefing I realized that I need to elaborate further on the development of this project…”
At the debriefing, not just the feedback from the instructor and the peers but also questions they asked made the student realized that something needed to be changed or fine-tuned. This realization may have happened as a result of self reflection or metacognition because of the debriefing. This is a case where the student was a presenter.
From a participant’s (listener’s) perspective, another female student (Student B) reported the following in the assignment:
“I have also learned a lot from the previous mid-term meeting. While listening to others speak about their project, I was able to picture how other projects were being created and managed. I originally was creating everything on PowerPoint, but quickly realized that most of my classmates were creating their projects on the internet. I decided to also add some interaction to the project and can only hope for a thriving project for this course.”
Apparently, she participated in the debriefing and listened carefully to how her peers developed and managed the capstone project. She then decided to change her courses of action by switching to PowerPoint and adding more interaction design features to her work. All of this change was in a hope for a successful experience.
This individual learner (Student C) viewed the debriefing as an opportunity to determine his progress in relation to others’ and he was assured that compared to others, he had made substantial progress. This reinforcement not only strengthened his self confidence for the completion of the project but also empowered him intellectually to try out new ideas on the project. The debriefing event seemed to reinforce his cognition and affection associated with this capstone project (see below).
“The midterm debriefing was a good check point to see where I stood and to have a sense of what other people were doing and how they felt about the project. I feel like I’m doing pretty well on my project. I have a lot of the work done but there is a lot more to be done.”
Another male student (Student D) made a similar comment:
“…The ability to see other formulations and foundations has sparked other ideas that I plan to implement on my project, too. This is true, I was able to benefit from the mid-term debriefing and reflect upon my ideas. This process has directed me in a path that will continue to add on to my final vision.”
Benefit from the debriefing should not be exclusively for the presenter or the participant (audience). As a matter of fact, the midterm debriefing appeared to benefit both sides. Student E made the following comment in his reflection paper:
“It was good to see the other projects at the mid-term debriefing. I will be able to use their work to guide me as I create my own project and hopefully mine will provide some assistance for them.”
Students’ anticipation for the midterm debriefing had a symbolic meaning. Their expectation suggested an intrinsic value of the instructional event. It is worth noting that these students were not instructed to discuss specifically the midterm debriefing in the biweekly assignment. Yet, they talked about it, which suggests they be interested in the debriefing and looking forward to it. Below are quotes from two students’ reflection papers prior to the scheduled event:
“As the mid-term debriefing draws near, I am interested to see the class projects that will be introduced on Wednesday of this week. This process will guide me in the proper direction and will help me complete my vision.” (Student E)
“My short-term objectives for this next two weeks are first to start working on the performance objectives and then prepare for the mid-term [debriefing] presentation. I think it will be interesting to see where the other students in class are within their projects.” (Student F)
They were interested in the event partially because they were able to take advantage of the event and to check or confirm where they were standing in the project. These students used this event as a feedback to (re)adjust and (re)affirm their work experience.
Students’ perception may have changed due to the debriefing. Student G mentioned in her reflection paper that the class unanimously recognized the gravity of this practicum project. Because there were a great number of little pieces of work involved in this project, students initially encountered difficulty in keeping track of the pieces. The midterm debriefing helped these students re-examine the basics of the project and stick to the approved proposal (blueprint). It could be this collective understanding of the nature of the project that changed these students’ perception that this humongous task is achievable. Here is what Student G said,
“The best thing I’ve learned since the last reflection is that I’m not alone. There seemed to be an echo in the room when we were doing our Midterm Debriefing….The more I do, the more it seems I need to do… I stated in my first reflection that I thought I had possibly bitten off more than I could chew, but it is becoming more achievable as I continue to work on it. Like XXX [Another classmate] said, ‘The pieces are coming together.’”
Furthermore, this change of perception may, in turn, have increased the students’ confidence level. Below is how Student H described his thoughts in this regard:
“What did I learn this past two weeks? The first thing I learned is that it is possible to complete this project. Even though I said it with confidence in the mid-term debriefing, I was not confident that I would finish. Now I believe that I can see the light at the end of the tunnel.”
Debriefing appeared to promote a positive attitude of students toward the capstone project. The midterm activity did convey a message to the class: “Indeed, it is a lot of work, and you are not alone” (see Student G’s comment previously mentioned). Although overwhelmed, students encouraged one another to accept the emotional setback, turn it around, and make a positive breakthrough. Student I presented her comment on this:
“I was unable to participate in the midterm debriefing with the class live but I did listen to the archived session the next evening. I actually felt better about my progress as I feel that we all are feeling that we are in the same boat.”
Student J’s reflection below also supports our interpretation here. She even patted herself and her classmates on their backs by acknowledging the hard work they had accomplished thus far.
“I have learned after Monday’s debriefing that I am not alone in this endeavor and that all of us are working diligently and laboriously. I will continue to work on the rest of the project with intentions to finish it by this weekend.”
Summary and Conclusions
This qualitative study has two major purposes. First, we intend to explore how psychological debriefing is used in the other disciplines. Second, we are interested in investigating to what extent a debriefing strategy can be incorporated in an online practicum course as an instructional activity in light of benefit for our students. Three graduate classes during Fall 2006 and Summer 2007 participated in the research with a total of 38 students. The students’ biweekly reflection papers were reviewed. Seventeen of the papers were selected for further content analysis.
Six major themes emerged: Learned decisions, reinforcement, reciprocity, anticipation, perception change, and positive attitude.
Students reported that the midterm debriefing activity enabled them to make learned decisions thanks to all the information being presented during the session by the class and the instructor.
Students’ thoughts of the project design were reinforced by listening and picturing what their peers did to their own work. For some, debriefing was associated with a mutual benefit for the development of the capstone project. Both participants and presenters benefited from this reciprocal activity.
Results also indicate that students considered the debriefing activity potentially instrumental, so they had a keen expectation for this event to assist them in guiding them and completing the project.
Students’ perception may have changed due to the class event. The class gathered for the debriefing close to the midterm of the semester during which each student was informing and being informed of the fundamental issues related to the capstone project. In the process, a collective perception of the achievement of this project is (re)affirmed.
Lastly, the debriefing was able to promote a positive attitude change. This practicum project was intended to allow students to demonstrate all the competences they have acquired throughout the Educational Technology program. The capstone project was expected to be more extensive than any other previous project in terms of its scope and size. All this suggests that the practicum can be overwhelming for some. Regardless, the midterm activity seemed to exert a positive influence on students in regard to their attitudes toward the capstone project itself. Instead of taking an emotional “hit” by the workload of the project, students turned the hit around and made the best out of it.
We learned that students generally favored psychological debriefings as an instructional strategy. They came to the debriefing session with an eager expectation to learn from the group and to explore new options to cope with problems to come. While they were presenting their unfinished projects, they were holding back one step and looking at what they did. This is a matter of reflection. While they were listening to others’ presentations, they were comparing and contrasting their works with one another. The comment students provided for one another not only guided their projects but also helped them explore new methods to deal with content-specific issues. And that is what debriefing can offer.
Andrusyszyn, M., & Yankou, D. (2004). Attention fatigue and the effect of debriefing in a Webenhanced graduate nursing course. International Journal of Instructional Technology and Distance Learning, 1(4), 61-66.
Blake, K. D., Gusella, J., Greaven, S., & Wakefield, S. (2005). The risks and benefits of being a young female adolescent standardised patient. Medical Education, 40, 26-35.
Chernick, J. (1992, April). Keeping your pilots on course. Training and Development, 46(4), 69-73.
Colby, E. (1980). New army talk. American Speech, 55(4), 307-308.
Collis, B., De Boer, W., & Slotman, K. (2001). Feedback for web-based assignments. Journal of Computer Assisted Learning, 17, 306-313.
Deahl, M. (2000). Psychological debriefing: Controversy and challenge. Australian and New Zealand Journal of Psychiatry, 34, 929-939.
Geber, B. (1994, April). Let the games begin. Training, 10-15.
Menaker, E., Coleman, S., Collins, J., Murawski, M. (2006, December). Harnessing experiential learning theory to achieve warfighting excellence. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC, 2006), Orlando, FL. Retrieved February 27, 2007 from http://www.iitsec.org/documents/Ed_2974.pdf
Smith, A., & Roberts, K. (2003). Interventions for post-traumatic stress disorder and psychological distress in emergency ambulance personnel: A review of the literature. Emergency Medicine Journal, 20, 75-78.
About the Authors