Editor’s Note: Individual and group reflective thinking are the subject of this paper. Reflection supporting tools for computer-supported collaborative learning provide learning opportunities that are parallel to individual and collaborative activities in a classroom setting. Collaborative reflective thinking is a product of group sharing through discussion and other reflection supporting tools. Results show significant support for the reflective process, especially in computer-supported collaborative learning environments.

Design and Analysis of Reflection-Supporting Tools in Computer-Supported Collaborative Learning

Seung-hee Lee

Abstract

This study proposes design principles for reflection-supporting tools in computer-supported collaborative learning (CSCL) environments as well as ways to analyze how such tools influence group performance and the perception of learners on group learning. The functions of reflection-supporting tools suggested in the study were group workplace, thinking sharing board, reflective journal, and reflective scaffolding. When employed in group learning within CSCL environments, reflection-supporting tools turned out to positively foster learners’ performance and process. In addition, learners felt that these tools made a significant contribution to meaningful reflection and inquiry within CSCL.

Keywords: Computer-supported learning, Collaboration, Reflection, Reflection- supporting tool

Background

In computer-supported collaborative learning (CSCL), knowledge construction is produced by learners’ active thinking and reflection. Higher-order thinking is essential for learning cycle in which learners can look back and monitor themselves on their own. The same is true when learners change their own cognitive structure during learning; when learners work in groups, they tend to modify and further expand both individual and group cognitions. Researchers such as Dewey (1933) and Schön (1983) call this type of thinking “reflection”. Constructive learning theories and models in recent times emphasize the power of learners’ reflective thinking.

Some of the CSCL studies (Dillenbourg, 1999; Koschmann, 1994; Stahl, 2002) indicate that meaningful group learning depends on facilitating thinking and idea sharing though peers’ discourse. Throughout group reflection, learners in social contexts can experience negotiation and meaning elaboration. While previous studies have focused individual-oriented reflection, recent CSCL studies stress the roles of collaborative reflection for meaningful group learning.

The studies on reflection have suggested the need to design reflective inquiry fostering environments. However, these studies tend to belong to the categories of theoretical studies, or to focus on technical implementation of tools from the perspective of the developers, not those of the learners or instructors. Further, most studies of reflective inquiry fail to mention that instructional designers should design learning systems and support tools that can assist learners’ reflective thinking in order to facilitate knowledge transfer (Felton & Kuhn, 2002; Häkkinen, Jarvela, & Dillenbourg, 2000; Lin, 2001). In fact, only a few studies (Bell & Davis, 2000; Kolodner & Guzdial, 2000) focused on such key issues.

Since collaborative reflection in group learning is regarded as an important component in CSCL environments, how to support collaborative reflection needs to be explored. Therefore, the purpose of this study is to suggest cognitive tools for supporting learner reflection in CSCL environments. Also, this study intends to empirically verify the effects of reflection-supporting tools on the group learning.

Reflection in Learning

Meaningful learning results from balanced integration of experiences and reflective thinking. Learning activities without enough reflective thinking can bring superficial level of knowledge. In traditional classroom environments, learning sometimes occurs with the limited levels of reflection.

However, reflection plays a pivotal role in group learning where an effective activity is related to the individual cognition as well as the social interactions among learners (Kemmis, 1986). In CSCL environments, learners become naturally experienced not only with internal conflicts, but also with social conflicts from multiple perspectives of their peers. These kinds of socio-cognitive conflicts or psychological burdens occur frequently in CSCL environments, as compared to classroom learning. Reflection can resolve such conflicts and result in the equilibrium of learner’s cognitive structure.

Also, reflection as a constituent in group learning process should be influenced by collaborative interactions. With collaborative reflection, learners can compare their own thinking with those of others, and if appropriate, adapt their abstract thinking towards different and perhaps more appropriate and meaningful learning goals.

The possibility of tool implementation for facilitating reflective thinking has only been recently studied to determine if such technologies or tools can support learning. For instance, some studies have introduced functions of self-monitoring questions (Kolodner & Guzdial, 2000) as well as knowledge representation tools to compare their opinions or solutions with peers by explaining what they understood (Bells & Davis, 2000). In addition, some researchers have provoked reflective thinking by using visualization methods (Kyza, Golan, Reiser, & Edelson, 2002), while still others have employed elaborate reflective devices (Balyor, Kitsantas, & Hu, 2001).

Repeatedly to say in the study, reflection by nature has a social aspect and is strongly influenced by the community activity. As many scholars have pointed out, social learning environments are significant for providing individual-level strategies (Lin, Hmelo, Kinzer, & Secules, 1999). Reflective discourse with peers and more experienced others can improve both self and group actions. Based on the review of previous studies of tool implementation, it seems that there is support for reflection in group learning, but how to support reflection in CSCL environments remains to be revealed.

Design and Development of Reflection-Supporting Tools

Reflection support plays a role as a mediator for learners to construct collective knowledge. To foster depth within such knowledge construction, such cognitive tools should extend beyond traditional discussion board functions. The key design principles for reflection-supporting tools suggested in the study emerge from previous theoretical and developmental studies (Bell & Davis, 2000; Gama, 2000; Gutwin, 1997). The basic principles used for the study are: facilitating social awareness, thinking visualization, learner discourse, and group meta-cognition. Based on these principles, the major functions for reflection-supporting tools were identified and developed to promote collaborative reflection (see Figure 1). The major tool functions are described in the following sections.

Figure 1. Design framework of reflection-supporting tools

Group workplace: The group workplace is a transformed traditional online library, in which learners can categorize the mid-term or final products as well as accumulate project-related information and learning resources. Learners can save their work files in different places, according to different subject matters so that they can identify visually how the project products have developed over time.

Reflection journal: Learners can regularly record daily reviews on reflection journal pads while performing in their respective group projects. They can write what has been done so far, what should have been done better, and how much they have learned. With use of the group reflective journal, they can freely write their feelings, thought processes, and difficulties they experienced during the learning process. In addition, all records can be shared with other learners.

Thinking sharing board: This tool is intended to trigger learners’ convergent thinking as well as divergent thinking. Tagging of ideas as pro, con, or neutral shows what stances learners can have on debate messages, and visualize how their own opinions and thinking in group work flows. Ideas, discussions, and debates of learners can be shared, compared, and contrasted among peer learners in-groups as well as between-groups.

Reflective scaffolding: The purpose of reflective scaffolding is to stimulate learner’s meta-cognitive activities related to problem-solving and for offering informative questions and cognitive assistances. Cues from this tool can be categorized into monitoring two parts such as subject domain and activity process. With reflective scaffolding, learners can look back at what they did while participating in projects. Group performance-related cues for reflection-supporting tools are provided during learning.

Those tools described above were developed to support collaborative reflection. With such tools, all learning products and processes are open and shared with peer learners at any time. For instance, learners can see or edit project products in the middle of working. They can also build their journals by collaboratively writing reflection notes within their journals.

Impacts of Reflection-Supporting Tools

1) Data Collection and Analysis

The study was conducted with adult learners in three universities. For this study, 53, 48, and 50 participants respectively formed three groups and were distributed into groups which were provided with different reflection-supporting tools; (1) a group with collaborative reflection-supporting tools (CRG), (2) a group with individual reflection-supporting tools (IRG)[1], and (3) a group with no reflection support (NRG).

A pre-test was conducted to measure group differences and their homogeneity was confirmed. Data from the evaluations of group performance, surveys and semi-structured focus group interviews were analyzed using the constant comparative approach. After subjects experienced their online projects for five weeks, participants’ accomplishments were evaluated by two content experts. They were also solicited for interviews related to their online experiences and their perceptions of the usefulness of various learning tools.

2) Findings

Within their group tasks, learners were asked to conduct group performance projects. To succeed, the participants of each group worked together to meet the common goal of completing the given projects. As shown in Table 1, the analysis of variance between groups (ANOVA) of the differences among the three groups’ scores in the group performance was statistically significant. A post-hoc test results suggest that the CRG scored statistically higher than the IRG and the NRG.

Table 1

ANOVA results for group performance

Learner perception on performance was assessed with the reflection-supporting tools. Once again, the mean of score for group learning, CRG, was the highest when compared to the IRG and the NRG. Table 2 details the results of the ANOVA tests, which significantly favored the CRG on the awareness of the group performance.

From these results, we can find out that reflective thinking and reflection-supporting tools have a positive impact on group learning. In particular, the tools supporting collaborative reflection have a greater impact than those for individual reflection-support or no support regarding learner performance and perception.

Meanwhile, in order to understand how learners perceived the functions of reflection-supporting tools, the survey results of tool usefulness were analyzed. The findings show that both CRG and IRG were positive on their learning experiences and usefulness of reflection-supporting tools.

Table 2

ANOVA results of perception on group performance

At the beginning, learners with little online group learning opportunities seemed to have psychological and cognitive overload to collaborating online, but they perceived online learning positively as time gradually went by. According to the descriptive response from learners, they seemed to have unexpected meaningful experiences during the group projects. Learners from both GRG and IRG considered positively collaborative activities as learning experiences. The learner responses for user friendliness are noted below:

Group workplace: The CRG indicated that sharing ‘group workplace’ with other group members gave useful guidance and tips for conducting projects (92 percent), compared to the IRG (90 percent). ‘Group workplace’ seemed to help learners conveniently achieve their outcomes. At the same time, some learners were confused by the different boards per project subject. As noted in the quote below, a few learners seemed to feel cognitive overload, since they had to distinguish each of the functions, while conducting group projects.

I think work workplace have effective functions in terms of the fact that this tool gave learners independent boards per the different tasks or different subjects, compared to the traditional boards where all the files usually have to be upload in one board.

Thinking sharing board: CRG mentioned ‘thinking sharing board’ was useful and positive (84 percent). They experienced deep reflection while group opinions were revised and well developed with the use of the embedded function. In particular, with tagging, they could relay the relevant messages under particular discussion topic categories, and that helped group discussions and negotiations become much effective. This is reflected in the following learner quote:

At the beginning, my group members showed different ideas and opinions so that we had a kind of conflicts in making consensus. But as time went by, we could naturally exchange multiple perspectives and accept others’ ideas or revise them, even elaborate our own thinking better. I was quite surprised when I found my fellow’s thinking and I felt the range of my thinking was much wide and reconsidered the project subject with better approaches. In particular, tagging to the messages on the board was a helpful function with that we could see how group ideas were developed and revised and finally consensus made.

Reflective scaffolding: Regarding the preference for ‘reflective scaffolding’, learners responded that reflective scaffolding was helpful. For instance, one learner stated:

Reflective cues, questions or best practice for the group projects, seemed to play a good role for conducting projects. They were useful for us to guide group journal so that we were on the right tracks for learning.

Reflection journal: With ‘reflection journal’, learners became aware of the importance of reflective activities in the group learning projects. The learners in CRG perceived reflective activities important for group performance than learners of IRG (CRG: 76 percent, IRG: 68 percent).

On the other hand, the preference to individual reflection journal writing was slightly higher than collaborative journal writing. However, most of them agreed that it was of vital importance to have both their journal-writing available for critical reading and reviews by peers. As noted by one learner,

Without the reflective activities, we could hardly find out if our group projects have been on the right track, if we understood the essence of the projects or information, or if all the actions have been done as scheduled. I think that reflecting gave us the right judgment for good learning. Particularly with collaborative journal, we could monitor, find and revise what was missing or what was less good. We knew the fact that all the revisions are the required steps in the group learning and the focus of conducting projects was on the learning experiences and learning process.

Interestingly, regarding reflection journal writing in particular, some learners in CRG answered that writing on reflection journals was not comfortable. It can be interpreted in two perspectives. First, they were not accustomed to put their records open to peer learners. These actions are more private in oriental cultural contexts. Therefore, even though learners realized the importance of sharing their reflective thinking, they still showed some feeling of hesitation to share what they had monitored in their online activities, until they got familiar with the benefits of the tool within online collaborative learning environments. Further studies have to address how to encourage learner internal motivation to open their thinking to others with deep consideration of their cultures and backgrounds.

Secondly, there was not full understanding on functions of the tools among learners, which led them unsure of how to apply them in writing journals. One of keys to effective learning performance and process would rely on not only providing useful reflection-supporting tools to learners, but also guiding them enough on how to use them in their group activities.

Conclusion

Reflection is an in-depth practice in which learners participate in social behaviors such as communication or decision-making. They look back on their thinking process or actions, and listen to peer learners in groups or teams throughout their collaborative reflection. The social contextual environments with peer learners can be extremely useful in developing high-level cognitive structures. Previous studies have inclined to focus the area of theoretical reviews or technological implementation, but this study went further to draw out design principles for reflection-supporting tools and to identify the usefulness of several different learning and reflection tools. In this study, collaborative reflection-supporting tools turned out to positively impact group learning as expected.

Repeatedly to say, the findings show that reflection-supporting tools have positive impacts on the group performance as well as the perception of learners on collaborative learning. Also, the reflection-supporting tools in CSCL environments were effective and user friendly for group learning. This study clarified the ideas of Dillenbourg (1999) who argued that there is a need to externalize thinking and ideas with appropriate tools. Tools such as the project workplace, thinking sharing board with tagging, and reflection scaffolding that were proposed in the study were useful for facilitating reflective activities and for conducting group projects. On the basis of the results, follow-up studies on reflection support in CSCL environments are required with qualitative research approaches so that extra efforts should be put to identify changes in both learner behaviors and flow of thinking during group learning.

References

Baylor, A., Kitsanstas, A., & Hu, H. (2001). Introducing the IPSRT and CPSRT(constructivist planning self-reflective tool): Tools to promote instructional planning for pre-service teachers. Paper presented at the Association for Educational Communication and Technology, Atlanta, GA.

Bell, P., & Davis, E. (2000). Designing Mildred: Scaffolding students' reflection and argumentation using a cognitive software guide. In B. Fishman & S. O'Connor-Divelbiss (Eds.), Fourth International Conference of the Learning Sciences (pp. 142-149). Mahwah, NJ: Erlbaum.

Dewey, J. (1933). How we think. Lexington: D.C. Health and Company.

Dillengbourg, P. (1999). Introduction: What do you mean by collaborative learning? In P. Dillengbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1-19). Elsevier Science Ltd.

Felton, M., & Kuhn, D. (2002). Science as argument. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL 2: Carrying forward the conversation (pp. 493-498). London: Lawrence Erlbaum Associates, Publishers.

Gama, C. (2000). The role of metacognition in problem solving: Promoting reflection in interactive learning systems. Retrieved on May 21, 2004 from: http://www.cogs.susx.ac.uk/users/claudiag/edut ech/metacognition.html

Gutwin, C. (1997). Workplace awareness in real-time distributed groupware. Doctoral dissertation. The University of Calgary.

Häkkinen. P., Jarvela, S., & Dillenbourg, P. (2000). Group reflection tools for virtual expert community-REFLEX project. In B. Fishman & S. O'Connor-Divelbiss (Eds.), Fourth International Conference of the Learning Sciences (pp. 203-204). Mahwah, NJ: Erlbaum.

Kemmis, S. (1986). Action research and the politics of reflection. In D. Boud, R. Keogh, & D. Walker (Eds.), Reflection, turning experience into learning. NY: Nichols Pub.

Kolodner, J, & Guzdial, M. (2000). Theory and practice of case-based learning aids. In D. Jonassen & S. Land (Eds.), Theoretical foundations of learning environments (pp. 215-242). Lawrence Erlbaum Associates, Publishers.

Koschmann, T. (1994). Toward a theory of computer support for collaborative learning. Journal of Learning Sciences, 3(3), 219-225.

Kyza, E. Golan, R., Reiser, B., & Edelson, D. (2002). Reflective inquiry: Enabling group self-regulation in inquiry-based science using the Progress Portfolio tool. Proceeding for CSCL 2002 Conference. Retrieved on November 4, 2003 from: http://newmedia.colorado.edu/cscl/165.pdf

Lin, X., Hmelo, C., Kinzer, C., & Secules, T. (1999). Designing Technology to Support Reflection. Educational Technology Research & Development, 47(3), 43-62.

Lin, X. (2001). Designing metacognitive activities. Educational Technology Research & Development, 49(2), 23-40.

Schön, D. (1983). The reflective practitioner: How professionals think in action. NY: Basic Books.

Stahl, G. (2002). Contributions to a theoretical framework for CSCL. Proceedings of the Computer Support for Collaborative Learning (CSCL) 2002 Conference. Retrieved on November 5, 2003 from: http://newmedia.colorado.edu/cscl/81.pdf

Acknowledgements

This work was supported by
Korea Research Foundation Grant (KRF-2003-037-B00071)
 

About the Author

Dr. Seung-hee Lee is a research associate at Kelley Direct Online Programs of Indiana University. She earned her doctorate from Hanyang University in Seoul, Korea in 2003 and has conducted post-doctoral research projects at Indiana University from September 2003-December 2004. Dr. Lee had worked previously in e-Learning Center of Korean National Open University in Seoul, where she had consulted and promoted faculty development for online teaching and learning. Her major research interests are online collaboration, reflective technologies, e-learning in higher education, and online mentoring.

Dr. Lee can be contacted at:

Kelley Direct Online Programs, 777 Indiana Ave. Suite 200, Indianapolis, IN 46202-3135 (Tel: 317-278-9084, E-mail: seuselee@indiana.edu)

End Note

[1] Reflection-supporting tools for IRG feature the same functions but operate only for a particular learner. In such instances, learners might conduct group work but keep their private journals which are not shared with anyone.
 

go top

Mar 2005 Index

Home Page