Editor’s Note: The gulf between classroom learning and real world practice is a challenge in professional training. This is especially true where human life and health are involved. This research addresses the learner motivation and experience through role-playing games to provide experience beyond the classroom.
The Design and Evaluation of Virtual Situation Role-Playing Learning Game (VSRPG)
-- Legend of Shannon --
Ju-Ling Shih, Bai-Jiun Shih, Rey-Long Chen
Conventional education of medicine is mostly based on lectures because it is a subject with structural knowledge. The mastery of the subject relies heavily on learners’ memorization and use of references. The content is mostly categorical and indexical, compiled in bulky professional directories and dictionaries within its own chapters and sections so that it can be easily searched and referenced. In the medical profession, precision is a necessity and an obligation because human life is at stake. However, the customary repetitive style of teaching, and progressive recitation toward effectual learning can easily cause the regression of learning motivation.
Limited opportunities for students to transfer textual knowledge to practical situations and integrate knowledge and applications into real-world environments can result in learning that is static and ineffective. Digital technology provides educators an alternative to fill this gap. Using it as an appropriate supplement to the classroom lectures, learning can be more interesting and motivating. The goal is to find a balance between conventional and innovative forms, in which the value of traditional teaching methods can be preserved, at the same time maximizing opportunities for knowledge exploration and expansion.
As the trend of technology-mediated education continues, the Taiwan government is taking actions to promote digital learning projects in all fields, such as the National Digital Archive Program (http://www.ndap.org.tw/). In the realm of health education, the Department of Health, Executive Yuan in Taiwan has established an informational medical education network to prevent abuse of club drugs. It integrates e-learning materials and activities in the form of virtual simulation. It replicates the feelings, symptoms, behaviors, and the side-effects of misapplied drugs, and suggests methods and channels to the users for solutions. Simulation is used to attract youths to foresee the results of the misuse of drugs, the e-learning materials help them retrieve more information online and indirectly reduce these medical problems (Hsu, Wang, & Chien, 2004). In a related area, development of an E-learning-based rehabilitation system for mental disorders is in construction supported by government funds (Chian, Sung, Yang, & Yei, 2004).
In response to the need to implement a digital system for the education of medicine, this research posits the development of a simulated situation learning model. Knowles (1996) suggests that adults are more self-directed, goal-oriented, intrinsically motivated than children. The target is a fundamental college course of Chinese medicine to focus on Chinese herbal medication safety. The simulation is a multimedia game based on constructive learning theories to provide situations in learning environments that embed “learning in realistic and relevant contexts” (Hobebein, 1996, p. 11). The goal is to stimulate motivation and learning and create opportunities for students to synthesize their textual knowledge into life situations. Empirical studies of the implementation of situated learning in professional training (e.g. Artemeva, Logie, & St. Martin, 1999; Wolfson & Willinsky, 1998) confirm its effectiveness for helping participants to translate knowledge and skills to the future real works.
Games used in education must be high-quality to motivate and interest users, otherwise their effect will be lost (Elliot, Adams, & Bruckman, 2002). Therefore, a central concern of this research is how to design a successful digital game for educational purposes. A Virtual Situation Role-Playing Game (VSRPG) was designed to explore innovative ways for teaching and learning to supplement classroom learning. This paper describes the processes of design, development, implementation and evaluation to demonstrate the potential efficacy of a VSRPG as a tool for medical education.
There are four basic reasons why we introduce computer games to educational practices: natural activity and tendency for learning, content-based scenario, social functions, and active participation. All of these require elaboration and discussion.
First, let us delve into natural activity and tendencies for learning. Play, in its diverse forms, constitutes an important part of human cognitive and social development (Csikzentmihaly, 1990; Rogoff, 1990), and can be considered as an excellent example of situated learning. It has a strong influence, especially on children (Piaget, 1951; Vygotsky, 1976). However, “games” in this research refer to not aimless “play” but a goal oriented set of properties, rules and processes. A mental framework is generated in the gaming process when players deal with goals, conditions, rivals, and problems in the game. An understanding of the underlying concepts of games plays an important organizing role in cognition, similar to that of a story schema (Schank, 1990).
In America, digital games are more prosperous in business and entertainment than in education; nevertheless, a few games, such as SimCity, Carmen, SanDiego, and various Multi-User Dungeons/Dimensions (MUDs/MOOs) have been used effectively for education (Perrone, Clark, & Repenning, 1996). WebQuest, for example, a learning environment combining the Internet and virtual reality, is widely used in various schools for inside and outside of class activities. Even Nintendo has demonstrated its effects in education in assisting students to improve their literary and algorithm ability (Rosas et al., 2003).
The second reason for using computer games in education involves content-based scenarios. Scholars (e.g. Ausubel, Novak, & Hanesian, 1978; Papert, 1980) propose that people best construct new knowledge when they are engaged in personally- meaningful tasks that corresponds with students’ interests. Therefore, the instructional tools must approach students from an angle that seems interesting and relevant to them (Kafai, 1995; Provenzo, 1991)
Computer games were traditionally classified as strategy games, multiple-user dungeon (MUD) games, simulation games, role-playing games, shooting games, and online games. Basic games, such as dartboard, crossword puzzle or roulette wheel, are extrinsic games that have nothing to do with the content of a course. They are fun because they are competitive and add tension to the training experience, but they add little if anything to the skills or information being taught (Fister, 1999). Brandt (1996) recommended that program content and activities should be based on typical practice situations, such as discussion of case vignettes with multiple interacting variables, and working through complex problem-based cases. “True learning involves experimentation and reflection both in the virtual world and the real world” (Senge & Fulmer, 1993). An intrinsic approach, which uses the content as the basis for a game, is linking the fun to the content in such a way that playing the game actually increases the learning effectiveness instead of just testing recall of the information delivered earlier (Fister, 1999). As McFarlane et al. (2002) state, “computer games provide a forum in which learning arises as a result of tasks stimulated by the content of the games, knowledge is developed through the content of the game” (p. 4).
The third reason arguing for computer game use in education revolves around social functions. In the game world, the drive for realism and dramatic fantasy demands some of the most sophisticated software, hardware, animation and graphic design performance standards in the information and communication technology domain (Kim, Park, Kim, Moon, Chun, 2002). Many game designs simply focus on creating the excitement that game-play brings. In fact, designers need to pay more attention to building psychological involvement in online games. “In single-player game, players are looking for a good solo experience, on the other hand, online players are looking for opportunities to have a good time competing and/or cooperating with other humans” (Mulligan and Patrovsky, 2003, p.9). Social functions are key characteristics that appeal to online game players. The social behaviors exhibited by players within games have made game design even more challenging.
The fourth reason is based on active participation in tasks, which is the main characteristic of computer games. Pea (1987), addressing the problem of knowledge transfer, argued that formal education tends to separate knowledge from its everyday use without linking the content of first acquisition to the use of knowledge beyond the contexts. It causes students to only be able to solve specific problems they have been taught. He suggested that educators ensure the construction of learning settings to support the use of concepts, skills, and strategies in everyday life and work situations. Therefore, by using role-playing games, every player plays his or her own part, takes individual responsibilities, practices real-life simulated tasks, and experiences the consequences of every decision, rewards or punishments, in the games. Learning by doing and learning from mistakes is how humans learn since infancy. Building an “edutainment” environment facilitates learning because students direct their own goals, progress in their own way, in their own speed, and practice successful transfer of knowledge and skills from classrooms to the real world. Computer games are a means to build that bridge and channel it.
Friedl (2003) thinks all gaming environments can provide learners opportunities to participate and discover knowledge in different fields with various aspects. In the game, they learn functions, features, and concepts, and try to connect them forming a cognitive process. Such gaming environments, a kind of virtual reality, let participants practice dangerous tasks in a safe setting (p. 26). This learning process can induce an “immersion effect” (Hubbard, 1991) where players submerge themselves in the virtual environment, progressively increasing their levels of attention and concentration toward obtaining their goals. Relating to Csikszentmihalyi’s flow theory (1990), players get into a state in which satisfaction occurs while they are absorbed by a certain activity. Although sometimes it can be alienating, it can also be a genuine opportunity for users to engage in the educational contents (Lepper & Malone, 1987).
The most common form of active participation is role-playing. Role-play has meaningful social functions. Due to the limited time and space of schools, education can hardly match the rhythm and range of real life situations. Role-plays in class increase students’ participation in learning. However, it is not easy to provide all students the opportunity to engage in role-play activities. Now with technology, simulated real life situations can be experienced on the computers. Computer games have created a unique world to simulate the real one. All students can fully emerge in the situation and participate in the problem-solving process. Contrary to the passive mode of learning, learning becomes active and meaningful for each individual to create and construct his or her own meaning toward the knowledge they learned.
Role-playing games are often simulations with common characteristics such as a familiar and appealing theme, an interactive interface, a self-exploring environment, a strong and rich database of meaningful materials, imbedded professional knowledge, a requirement for diverse skills, and interesting and challenging features. However, simulation specifically refers to the similar environment and/or situations mimicking the real world. Although many simulations have the element of a role-play feature, many do not. On the contrary, in role-play games, most situations are simulated to the real world in terms of the character definition, social organizations, and conflict resolutions. Often they are a microcosm of real life.
The game designed for this research, Legend of Shannon, is semi-fictional. Although most of the professional materials are based on historical documents and medical facts, many parts of the scripts are based on legends passed down from six thousand years ago. Consequently, the game produced by this research is defined as a “virtual situation game” instead of “virtual simulation” in order to emphasize its distinction. And to further stress its feature of role-playing, it is called a virtual situation role-play game (VSRPG).
This research followed the standard instructional systems design model, ADDIE, (Dick & Carey, 2005) having gone through the process of analysis, design, development, implementation and evaluation. Since this research is supported by the National Science Committee fund and involves personnel in cross-disciplines, we organized a series of design meetings and called for frequent small consultations as needed. For formative and summative evaluation of the production of the game, we used descriptive, quantitative and qualitative methods. A mixture of questionnaires and interviews were used during and after the formation of the game.
The first phase of the research was the analysis, design, and development of the game. The design team used existing game design guidelines to sketch Legend of Shannon. In order to obtain the original pedagogical goals, we started with the learner and content analysis to define the needs by discussing the content with course teachers and content experts. Learning content and gaming elements were planned throughout the game in the form of storyboards. Game maps, characters, and amenities were designed and developed, together with a series of formative evaluations and feedbacks. The major formative evaluation of the game was to ensure the quality of the product using heuristic evaluation.
The second phase was summative evaluation of the implementation and evaluation of the game. Students’ learning was assessed in-class with 135 college freshmen in the Department of Pharmacy, Tajen University, who took their first fundamental course, Introduction to the Chinese Herbs. The pre-test pinpointed the students’ background knowledge level. Then, they scheduled a 3-hour session to play the game after some fundamental classroom lessons on Chinese herbs. One week after they played the game, a post-test was conducted to evaluate their learning outcome. Scores were compared and students were categorized into “low-achievement”, “average”, and “high-achievement” students.
Four to five students from each of the three groups were selected to join the focus groups to explore intangible effects of learning activities embedded in the game. They were encouraged to freely talk about their feeling toward the game so that researchers could observe their meta-cognition of learning from the game.
All 135 students completed a questionnaire for game appraisals. In order to reach fuller understanding toward the intangible meaning of the effectiveness of the design, and at the same time triangulate the research finding, we coordinated reviews and interviews with experts in information management, educational technology, and pharmacy, to generate a list of recommended revisions.
Phase One. Production of VSRPG
Scholars (e.g. Fletcher, 1986; Malone, 1980; Provenzo, 1991; Turkle, 1984) named a few elements that are important for an enjoyable and successful computer learning game. Synthesizing from their research, we generate a list to include:
A) a decision maker, which is the player. Sometimes it can be done with fictional characters with whom players can identify or role-play;
B) a set of rules to define the choices of behaviors;
C) a set of meaningful, specific, and multiple goals that users must try to reach from which they can receive rewards and punishments;
D) incorporated instructions that users understand while playing the game without having to read textual instructions;
E) competition between players;
F) freedom of users of different levels to choose their own favorite playing modes -- an adequate level of complexity which is highly challenging but rarely totally mastered is an important component;
G) independence from physical laws of the universe where objects can fly, spin, change shape or color; use of emotionally appealing fantasy and metaphors;
H) random and uncertain outcomes;
I) an information system which can show the condition of playing.
This research is intended to design the learning environment of the digital learning game, Legend of Shannon, to provide opportunities for students to play roles in the virtual situation and explore the game using knowledge they learn in the classroom.
Certain key design features of computer games are found to affect student learning more strongly (Avezedo & Bernard, 1995; Sivin-Kachala & Bialo, 1994; Hung & Liu, 1992; Baltra, 1990; Lepper & Malone, 1987; Malone, 1980). A designer must pay attention to the following:
A) perceive students’ knowledge background;
B) provide adequate, adaptive, and constant feedback;
C) incorporate cognitive strategies such as repetition, rehearsal, paraphrasing, outlining, cognitive mapping, drawing of analogies and inferences, and
D) use animated graphics, which increase achievement and reduce task time;
E) stimulate learners’ motivation with two types of curiosity: sensory curiosity (audio and visual effects) and cognitive curiosity (surprises and constructive feedback);
F) give opportunity for users to apply learning skills and knowledge they gained.
The theme of this game integrates myths, legends, historical documents, and medicinal resources, using Shannon as the main character combating monsters and helping people along his journey, which though intended to aid his escape from the throne, actually results in his becoming more qualified to be the king. Shannon compiles his herbal journal along the journey, and passes it onto the posterity as Shannon Sutra. His story is called the Legend of Shannon.
Lave & Wenger (1991) said that the situated learning perspective emphasizes that learning processes are related closely to participation in the practices of a community. Shannon in the story tested Chinese herbs everywhere, risked his own life exposing himself to the poisonous plants, dangerous ground, and vicious monsters, but does not forget to take notes as he experimented with prescriptions on himself for the welfare of people. In order to shape students with a positive therapeutic attitude, the gaming nature is not only to kill monsters but also to heal villagers’ sickness. The empathy toward others is very important for students in the profession of medicine. Having compassion to patients is much more important than mastering professional knowledge and skills, so Shannon can be called a good role model for students in the department of medicine.
Brown et al. (1989) have argued that the socio-cultural context in which learning is embedded is of crucial importance. The game is anchored on a complete story, hoping to connect students’ classroom experience to the real life situation, and at the same time to stimulate players’ motivation to continue try out their classroom knowledge in the game. Designing to suffice students’ knowledge application ability, the game is depicted to occur on Wenshan Island (see Figure 1; Figure 2), where along the adventure, Shannon passes mountains, rivers, caves, and villages. In the journey, he interacts with the villagers, helping them to cure their diseases. Just as Lave and Wenger (1991) said, “Social learning is a creative achievement, therefore, which involves a degree of personal investment; it can only be achieved by active participation.”
Figure 1：Game Map
* The game map describes Wenshan Island into which Shannon has gone for his escape. Five stages are located around the island. As Shannon gone through all five stages, he happened to return to the starting point where he finally comes to his throne.
The game is designed for college freshmen who are in their first school semester learning Chinese Herb Medication, thus the five stages of the game are based on their course content, and in progressive level of difficulties, within which some of the herbs are repeatedly used in different prescriptions. The herbs are preliminary designed into five stages. The gate key in each stage is an herbal medicine prescription, namely Ephedra Decoction, Four Formulae Soup, Four-Articles Soup, Six Flavor Glutinous Rehmannis Pill, and Ten Assorted Tonic Soup. Players need to use their background knowledge and classroom knowledge on combining herbs to pass.
As the game progresses, the players repeatedly encounter these herbs. They must be very familiar with the herbs’ respective effects and functions so they can apply it whenever it is appropriate in the tasks. As Schwen, Kalman, Hara, and Kisling (1998) refer to learning as a process of acquiring knowledge, these Chinese herbal medicinal, prescriptions, stories, and knowledge are embedded in the characters and tools. Other than that, in each stage, there is one big monster, such as Ephedra, Dragora, Ginsengia, Mo-White, and Queen NuWa (see Figure 3), and a few small monsters such as Almondia, Cinnamomia, Glycyrrhizia, and poisonous snakes. Spread out in the journey, there are also treasures, such as map, medical guide, red lash, Yun Nan White, etc.; and ploys, such as tharm-cutting grass, Chinese Soapberry Seed, and Chloranthi Serrati Radix, etc. Such design is to require users to apply their learned skills. When correct prescriptions are made, players game the pass to go the next stage, otherwise, their life power got deducted as they failed to make distinctions or over the challenges, as the system offers appropriate feedbacks to users of their progress.
Figure 2: Game Interface 1
* The interface shows Shannon beginning to enter Wenshan Island. The game is designed by Virtool with which players can see 3D animations. Superimposed on top of the screen is the map of the island which shows the players location. In the middle of the dialogue box, it shows how much of each herb players has collected. Function control buttons are at the bottom of the screen.
Figure 3: Game Interface 2
* Shannon combating herb monster in order to retain “it” as part of his prescription.
B. Formative Evaluation
A good interface design comes from the designers’ understanding of both the technology and the users (Hackos & Redish, 1998). In the process of development, the heuristic evaluation was performed for formative usability tests which provided the team with constructive suggestions for improvements.
Heuristic Evaluation is a usability inspection method that targets interface design by judging compliance to certain “heuristic” principles for successful design. It is a type of formative evaluation, where users review the product's usability using the heuristics. They explore the system freely in their own styles according to individual interests and needs. The severity and extent of each problem was rated and hit statistics were generated to identify problems that were not previously perceived by the design team. The scales of severity and extent are as follows.
1. Major usability problem: imperative to fix now
2. Major usability problem: important to fix now
3. Minor usability problem: give low priority
4. Cosmetic problem only: need not be fixed now
5. I don't agree this is a usability problem at all
2. Several places
3. Single case
We generated a customized heuristic for instructional technology design which was integrated and configured from the Heuristic Evaluation Guideline of Jakob Nielsen (1994), Levi and Conrad (2002), and Stanford University Information and Technology Systems and Services (2001). There are five aspects to observe, namely appearance, languages, functionality, structure, and assistance. The evaluation is not performed by programming technicians or interface designers, which would introduce unavoidable bias, but instead used selected target users to ensure the credibility of the evaluation. There are total of 25 heuristics.
The heuristic evaluation allowed us to find usability problems in the user interface design of the VSRPG so that these problems were taken into consideration in the design process. The formative evaluation cycles that led to detailed feedbacks and design adjustments were continued until the design was sufficiently mature for official implementation. The score comparison between first and last evaluation in severity are presented in Table 1.
1. Appealing interface. Colors and layout designs appeal to eyes, especially for long time viewing.
2. Aesthetic and minimalist design. Minimal irrelevant information in a dialogue to increase the visibility of options.
3. Readability. Text has high contrast for easy reading.
4. Intuitiveness of label functions. Symbols and features match user’s expectations. Metaphors of icons, actions, and interface in general that enable users to instantly grasp the conceptual model.
5. Consistency in visual presentation within the site. Consistency between elements that have the same functionality.
6. Consistency of the site with platform conventions. Consistency of symbols and features between the platform and the interface.
7. Clarity of terminologies and symbols. Terminologies and symbols written in a natural and logical way without using jargons. Language using real-world conventions to prevent misunderstanding and confusion in the navigation process.
8. Consistency in terminology usage. Same terms are used to address the same meanings all the time.
9. Availability of descriptive information. Sufficient descriptions to words and symbols to assist the understanding of the nature of links and categories.
10. Options availability. Visible options to expedite the browsing process by providing instructions and options whenever possible.
11. Anticipation. Anticipates the user’s needs by bringing up the tools that he may need.
12. Tailoring of frequent actions. Implementing documentation facilities to tailor frequent actions.
13. Tracing ability. Documentation of search process and task steps to enable users to trace their footprints.
14. Adaptation to user request type. Recognition of entries in different approaches, such as words, concepts, symbols, data, and etc to access purposeful search of information.
15. Display progress indicators. Instant feedback to user interaction and display processing information.
16. Backup memory. Prevent users from losing their work as a result of error.
17. Hierarchical organization of information. Logical and retrievable information hierarchies to accelerate searching speed. General information appears before specific details.
18. Indication of user location within the site. Layout designs and mapping tools provide users stable perceptual cues for the sense of location.
19. Misspelling recognition capability. Prompt alternatives for misspelled words.
20. Intelligent and customizable defaults. Provide meaningful defaults and facilitate easy customization.
21. Automatic response system. Availability of timely technical and technological assistance.
22. Procedural assistance. Concrete step guidelines focused on the user's tasks that are tight and responsive to problems.
23. Mistake solving support. Assistance for accidental mistakes by prompting “emergency exit” and supporting “undo” and “redo” functions.
24. Plain language for assistance. Use easy-to-understand language to help users recognize, diagnose, and recover from errors. Error messages should precisely indicate the problem and constructively suggest a solution.
25. Reporting website modifications. Make announcement/news about the change of the website.
* The scores represent the severity and extend of each item evaluated for the game. In severity, the higher the score, the less serious the problem. In extend, except scored zero, the higher the score, the worse the problem.
In the process, some items that might influence the learning are put in the earlier priority for modification, which include game appearance, symbols and language use, content organization, query responses, and procedure assistance. At the end of modification, we find that most questions in appearance and language have fewer problems, except few aesthetics designs and descriptive information that guide users through the game. On the other hand, the functionality and structures of the game have more problems, especially tracing ability, progress indicator, and location indicator. In the assistance aspect, intelligent defaults and automatic response system appear to be more serious than other items. These aspects and items are mostly related to the limited technical expertise of the design team. Among all, readability, visual presentation, terminology use, organization of information, and recognition to mistakes are rated highest, which means need less modifications.
Phase Two. Evaluation of the VSRPG
A. Student Assessment
Like problem-based learning and cooperative learning, situated learning instruction is the deliberate arrangements of events to facilitate learning (Driscoll, 1994; Heinich et al., 1999), which include three components: the desired outcomes, learning activities, and methods of evaluation. Unfortunately, although games are one of the most important human learning models that can intrigue learners’ motivation, game learning is difficult to evaluate since learning effects are not its immediate results, and cognition transformation and motivations are intangible. Therefore, in this research we used a combination of descriptive quantitative and qualitative method to see how this digital learning material is taking place in the physical courses.
The result shows that the students’ test score rise 12% after playing the game. Among the group, 87 students (64%) can totally understand the content, and only 2 students (1.4%) failed the test.
B. Focus Group
Focus groups are conducted to reach an understanding of students’ learning process and reflections. The groups are led with only a few guided questions. After the session, the conference notes are analyzed to generate a conclusion.
In terms of cognition, the students think that they can practice the professional knowledge they previously learned in class, realize the content of lectures in real-life situations, and attempt to produce prescriptions in the virtual situation is a fresh and exciting experience. In the process, they are able to perceive more questions toward the content, and are more critical when facing the tasks. The digital game has provided a target for discussion, which they often talk about it between classes. It gives a context for students and teachers to confer about the characteristics of different herbs and identify the possible inter-relational effects in prescriptions. However, to low-achievement students, they tend to focus more on the fun, and ignore the herb prescriptions. In other words, since the game presented limited sorts of herbs, players can easily produce prescriptions in each stage with trial and errors until they find the solution.
In terms of skills, since paper and pencil case studies are common in classroom teachings and this game is a multimedia version of lively situations, after students diagnose the diseases the village people have from the symptoms they have in the game, almost all students confirmed that they can better apply their classroom knowledge into real life than before. The distributed thinking effect has demonstrated itself here, and has made the conventional textual learning more practical.
In terms of the emotional aspect, because the digital game is very different from the classroom lectures, the students like its interesting plots and challenges in the story. It helps to extend their learning motivation. However, since it is the first version of the game, there are places where a passageway to the exit is hard to find. This often eliminates their drive to continue.
From these focus groups, we understand that this game can lead to positive learning outcomes, but the production quality can often reduce its effectiveness. Moreover, since the game plot is in linear structure, there is only one difficulty level. For some, it is too easy, and for others, it can be too hard. If a game can offer multiple entry levels and points, the game can be more challenging. Another key weakness of the game is that there are so few herbs in each stage that the collection and prescription of herbs is effortless. For some, this allows them to capture the key item in a prescription; for others, the analytical challenge is insufficient.
To review the key elements of an effective learning game, there should be functions to support students’ psychological development, allow them to immerge in the meaningful content, and induce their active participation into instructional tasks. Legend of Shannon has achieve its preliminary goal, and with more future development guidance toward perfection.
C. Game appraisals
At this stage, a questionnaire for game appraisals is distributed. It is a usability evaluation that aims to understand the influence of the game on the students’ cognition, skills, and emotions.
1. The interface and icon arrangements are easy to understand and manage.
2. Interface design is appealing and motivational.
3. Functional control mechanism is user-friendly.
4. User guide is accessible, searchable, and readable.
5. Multimedia presentation is rich and appropriate.
6. Content is coherent to the learning objectives.
7. Knowledge level is proper.
8. Learning resources is rich.
9. Software structure is clear, and fitting to learning.
10. Situation is real and natural, and helpful to understanding.
11. Role-play increases learning participation level.
12. Game plot is interesting, and stimulate motivation.
13. Game stages are challenging, and raise sense of learning achievement.
14. Gaming process can inspire analytical and critical thinking.
15. Gaming process is interactive.
The questionnaires (see Table 2) include system functionalities, learning content, and learning models. 5 levels of Likert Scale of “totally agree,” “agree,” “average,” “not agree,” and “totally not agree” are used in the test. From the total 126 effective questionnaires, most of the evaluation items received positive feedback.
Looking at the technical aspect, the result shows that most users think the design of the game is appealing, and stimulated motivations. The control buttons are easy to use and quickly become a habit. The function guide is easy to understand and for search, and the multimedia presentation is appropriate. The test scores are between 3.1 and 3.4. Only the interface icon design is 2.9, a little lower than other functionalities. This means the interface design needs to be improved.
In terms of the instructional aspect, students think the game is coherent to the overall learning objectives; content level is appropriate, and resources are rich. The structure is clear and can match the learning progress. The situation is close to real life, and can raise the comprehension level. The average score is 3.28.
From the motivational aspect, we found users liked the plot and think it can stimulate learning motivations. The game is challenging, and helps them to have the sense of achievement. The game can strengthen the analysis and application ability, and is highly interactive. Additionally, the role-play modality increases learners’ participation - the test score, 3.6, is higher than others.
D. Expert Interviews
From the expert interviews, we perceived the implicit problems hidden from statistics. Several suggestions are iterated for the future development of the materials.
First of all, most people think that a single-player game lacks the challenges of making collaborative efforts and fun. If the game could have multiple-players mode, it might be more interactive. At the same time, the game is designed in a linear one-story structure, and can be more challenging. Players should be able to choose the difficulty levels, including the dynamic of rewards and punishments to have a higher degree of complexity. Then, the game can appeal to and be effective for users in a wider ability range.
The current character in the story a player can role-play with is Shannon. If in the future the game can provide more characters for the players to choose from, it can be more appealing and fun. Other than that, the game can strengthen its “empowerment” dynamic. If the user plays the game with more frequency, the system can increase the character’s power in some way so that the players would be willing to get into the system more, helping to increase the motivation.
These suggestions are mostly laid on one important element, the multiplicity. A good game needs multiple roles, modes and levels for the users to choose and interact from. It can create an environment that is even close to real life, enhancing the cooperative learning mechanism, and suffice to more diverse users. Because this digital game is only at the prototype stage, it neither has the needed nor the possible complexity yet. However, in the future development, it is necessary for the design team to work toward a more complete implementation. The second version of the game will be designed as Inheritance War of Shannon’s Posterities. The intent is for the content to match up the learning objectives and instructional design ideals better than in the prototype.
Different from other digital games, Legend of Shannon is based on professional knowledge of Chinese herbal medication. Other than parts of the plots in the game being purely fictional, that is created from folklores and imaginations, including the legend of Shannon, the geographical portrait, and many tales and monsters, all other information, especially the herbs and medication facts, are real to provide accurate, correct professional education. In order to give it high creditability, the content of the game is evaluated by the experts in the Chinese Herbal Medication Safety Information Committee led by the Department of Health, Executive Yuan in Taiwan.
The main contribution of this research is that the development of an innovative VSRPG digital learning model demonstrated the implementation of activity-based learning with meaningful learning resources and the use of role-play mechanism in accelerating students’ learning motivation. The digital game presents the construction of a student-centered learning environment that supplements the classroom teaching to reinforce students’ learning experience. We learned that the design of games for learning needs to eliminate visual distraction, without losing their fun and challenges. There should be sufficient knowledge content to include course-related materials, and opportunities for students to synthesize and apply learned-knowledge in the tasks.
In cooperation with the establishment of Chinese Herbal Medication Safety Information Center by the Department of Health, Executive Yuan in Taiwan, this research with the experiment of design and implementation of the game, Legend of Shannon, explored the opportunities to use digital technology to create virtual situations for students to learn professional knowledge in constructive ways. With this VSRPG, we tested an interactive multimedia learning environment for the education of medicine, and aimed to promote the knowledge to the general public. It opens up a channel for the government to market the college digital content to the wider Chinese market either in Taiwan or internationally. The design and implementation process of this research hopes to provide experiences to academics with their similar attempts.
This research was supported by National Science Committee in Taiwan under the category of National Technology Plan for Digital Learning, and as a part of the result of the project “Shannon Sutra: Digital Learning Content and Experimental Research on Chinese Herbal Medication Safety” (NSC-93-2524-S-127-001). Special thanks are owed to the research team in Digital Content and
e-Learning Laboratory in Tajen University, Taiwan.
Artemeva, N., Logie S. & St. Martin, J. (1999). From page to stage: How theories of Genre and situated learning help introduce engineering students to discipline-specific communication. Technical Communication Quarterly, 8(3), 301-316.
Ausubel, D. P., Novak, J. D., Hanesian, H. (1978). Educational Psychology: A Cognitive View (2nd Ed.). New York: Holt, Rinehart and Winston. Reprinted, New York: Werbel & Peck, 1986.
Avezedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedbacking computer-based instruction. Journal of Educational Computing Research, 13(2), 111–127.
Baltra, A. (1990). Language learning through computer adventure games. Simulation and Gaming, 21(4), 445–452.
Brandt, B. L. (1996). Cognitive learning theory and continuing health professions education. Journal of Continuing Education in the Health Professions, 16(4), 197-202
Brown, J., Collins, A., & Duguid, P. (1989). Situated Cognition and the Culture of Learning. Educational Researcher, 18(1), 32-42.
Chian, C, -F., Sung, W, -S., Yang, Y., -K., & Yei, S, -L. (2004). Development of E-learning-based Rehabilitation System for Mental Disorders. Taiwan: National Science Committee, NSC93-2213-E214-031.
Csikszentmihaliy, M. (1990). Flow: the psychology of optimal experience. New York: Harper & Press.
Dick, W. & Cary, L. (2005). The Systematic Design of Instruction, (6rd Ed.). Boston: Allyn & Bacon.
Driscoll, M. (1994). Psychology of learning for Instruction. Boston: Allyn & Bacon.
Elliot, J. Adams, L., & Bruckman, A. (2002). No Magic bullet: 3D video games in education. In Proceedings of International Conference of the Learning Sciences. Seatle, WA: Oct. 23-26.
Fister, S. (1999). CBT fun and games. Training, 36(5), 68-78.
Fletcher, P. & Collins, M. (1986/87). Computer-administered versus written tests‹advantages and disadvantages. Journal of Computers in Mathematics and Science Teaching, 6(2), 38-43.
Friedl, M. (2003). Online Game Interactivity Theory. MA: Charles River Media, Inc.
Hackos, J. T., & Redish, J. C. (1998). User and Task Analysis for Interface Design. New York: John Wiley & Sons, Inc.
Heinich, R., Molenda, M., Russel, J, and Smaldino, S. (1999). Instructional media and Techologies for Learning. (6th ed.). Columbus, OH: Prentice Hall.
Hobebein, P. C. (1996). Seven Goals for the Design of Constructivist Learning Environments. In B. Wilson (Ed.), Constructivist Learning Environments: Case Studies in Instructional Design. NJ: Educational Technology Publications, Inc.
Hsu, C, -Y., Wang, C, -C., Chien, W, -S. (2004). Building an E-Learning Model for Preventing Abuse of Club Drugs. Taiwan: Department of Health, Executive Yuan, DOH93-NNB-1018.
Hubbard, P. (1991). Evaluating computer games for language learning. Simulation and Gaming, 22(2), 220–223.
Hung, R, -C., & Liu, M, -C. (1992). The Design Principles and Applications of Computer-Assisted Instruction. Taipei: Shtabook.
Kafai, Y. (1995). Minds in play: Computer game design as a context for children’s learning. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Kim, K. H., Park, J. Y., Kim, D. Y., Moon, H. I,, & Chun, H. C. (2002). E-lifestyle and motives to use online games. Irish Marketing Review, 15(2), 71-77.
Knowles, M. (1996) Adult learning. In R. L. Craig (Ed.) The ASTD training and development handbook (4th ed.). NY: McGraw-Hill.
Lave, J., & Wenger, E. (1991). Situated learning and legitimate peripheral participation. Cambridge. UK: Cambridge University Press.
Lepper, M. R., & Malone, T. W. (1987). Intrinsic motivation and instructional effectiveness in computer-based education. In R. E. Snow, & M. J. Farr (Eds.), Aptitudes, learning and instruction, III: Cognitive, and Affective Process Analysis. (pp. 255-286). Hillsdale, NJ: Lawrence Earlbaum Association.
Levi, M. D., & Conrad, F. G. (2002). A Heuristic Evaluation of a World Wide Web Prototype. Washington, DC: Bureau of Labor Statistics. Retrieved July 1, 2004, from http://stats.bls.gov/ore/htm_papers/st960160.htm
Malone, F. (1980). Toward a theory of intrinsically motivation instruction. Cognitive Science, 5(4), 333–369.
McFarlane, A., Sparrowhawk, A. & Heald, Y. (2002). Report on the educational use of games: An exploration by TEEM of the contribution which games can make to the education process. (On line). Available: http://reservoir.cent.uji.es/canals/octeto/es/440.
Mulligan, J., & Patrovsky, B. (2003). Developing Online Games: An Insider’s Guide. Indiana: New Riders Publishing.
Nielsen, J. (1994). Heuristic Evaluation. Retrieved January 8, 2002, from http://www.useit.com/papers/heuristic
Papert, S. (1980). Mindstorms: children, computers, and powerful ideas. New York: Basic Books.
Pea, R. D. (1987). Socializing the knowledge transfer problem. International Journal of Educational Research, 11, 639-663.
Perrone, C., Clark, D., & Repenning, A. (1996). WebQuest: Substantiating education in edutainment through interactive learning games. Computer Networks and ISDN Systems, 28(7-11), 1307-1319.
Piaget, J. (1951). Play, Dreams and Imitation in Childhood. New York: Basic Books.
Provenzo, E. (1991). Video kids. Cambridge: Harvard University Press.
Rogoff, B. (1990). Apprenticeship in Thinking: Cognitive Development in Social Context. New York: Oxford University Press.
Rosas, R., Nussbaum, M., Cumsille, P., Marianov. V., Correa, M., Flores, P., et al. (2003). Beyond Nintendo: design and assessment of educational video games for first and second grade students. Computers & Education, 40(1), 71-94.
Schank, R. C. (1990). Tell Me a Story: A New Look at Real and Artificial Memory. New York: Scribner.
Schwen, T. M., Kalman, H. K., Hara, N., & Kisling, E. L. (1998). Potential knowledge management contributions to human performance technology research and practice. Educational Technology Research and development, 46(4), 73-89.
Senge, P. M., & Fulmer, R. M. (1993). Simulations, Systems Thinking and Anticipatory Learning. Journal of Management Development, 12(6), 21-33.
Sivin-Kachala, J., & Bialo, E. R. (1994). Report on the effectiveness of technology in schools 1990–1994. Washington, D.C.: Software Publishers Association.
Stanford University, Information and Technology Systems and Services (2001). Heuristic Evaluation. Retrieved July 1, 2004, from the Stanford University, Information and Technology Systems and Services Web site: http://www.stanford.edu/group/web-creators/heuristics.htm
Turkle, S. (1984). The Second Self: Computers and the Human Spirit. New York: Simon & Schuster, Inc.
Vygotsky, L. (1976). Play and its role in the mental development of the child. In Bruner, J., Jolly, A., & Sylva, K. (Eds.) Play- Its Role in Development and Evolution, (pp. 537-554). New York: Basic Books.
Wolfson, L., & Willinsky, J. (1998). Situated Learning in High School information technology management. Journal of Research on Computing in Education, 31(1), 96-110.
About the Authors
Ju-Ling Shih is Associate Professor, Dept. of Educational Technology at Tamkang University.
Bai-Jiun Shih is Associate Professor, Dept. of MIS at Tajen University.
Rey-Long Chen is Professor and President of Tajen University.
Department of Educational Technology, Tamkang University
151,Yin-Chuan Road, Tamsui, Taipei Hsien 251, Taiwan, R.O.C.
Office: 886-2-26215656 ext. 3528 Fax: 886-2-86315377