Editor’s Note: The dynamics of a discussion go beyond the content to interactions based on personalities and relationships within the discussion group. This study breaks new ground in its exploration of latent content and analysis of responses, intent, and paths followed. Data on individual motivations and strategies show how these impact content and direction of the online discussion.

Reading Between the Lines:
Understanding the role of latent content in the
analysis of online asynchronous discussions

Elizabeth Murphy and Maria A. Rodriguez Manzanares

Abstract

This paper reports on an exploratory case study related to analysis of an OAD (online asynchronous discussion) that focuses both on manifest content and latent content. The purpose of the study was to explore the role of latent content, or individuals' intentions and motives, in providing insight into the behaviors of participants in an OAD. Participants were ten graduate students who used an online discussion designed for engagement in Problem Formulation and Resolution (PFR). The transcripts of the discussion were analyzed using an instrument with two categories, five processes and nineteen indicators. In addition, interviews with all participants were conducted at the end of discussion. Analysis of latent content provided additional insight into participants' behaviors in the discussion. In some cases, it confirmed results from analysis of manifest content, such as participants' emphasis on solutions. The focus on latent content also uncovered why they engaged in certain behaviors more than others, for example why they did not engage in critiquing other participants' solutions. Analysis of latent content also offered insight into participants' different ways of conceptualizing the solution process, and their emphasis on use of experience. In other cases, analysis of latent content did not further explain participants' behaviors. Limitations of the approach used to analyzing latent content are presented.

Keywords: online asynchronous discussions; content analysis; distance learning; latent content; problem solving.

Introduction

Content analysis of transcripts of online asynchronous discussions (OADs) can support observation and identification of discussants' engagement in behaviors related to social processes such as interaction, collaboration, and teacher presence. As well, it can provide insight into cognitive processes such as knowledge-building, metacognition, and problem-solving. Regardless of the behaviors or processes targeted, the analysis involves observing instances of behaviors related to a construct. The starting point for the analysis is an analytical model, framework or instrument that defines, characterizes and encompasses the construct (e.g. what is metacognition?, what is interactivity?). These models, frameworks and instruments detail specific indicators or markers of the construct that might be observed and coded for in a transcript. The indicators or markers serve to operationalise the construct in terms of behaviors.

This approach to analysis involves focusing on what can be referred to as the manifest content of the transcript, or "elements that are physically present and countable” (Gray & Densten, 1998, p. 420). Analysis of manifest content is premised on the assumption that discussants' engagement in social or cognitive processes is manifest and observable in the text of the discussion. An alternative perspective argues that discussants have intentions and motives for their behavior which are not necessarily or always observable in the text. We can refer to this type of content as latent content. This distinction between manifest and latent content was highlighted in a general context of content analysis prior to the existence of online discussions. Berelson (1952) argued that content analysis should be limited to analysis of manifest content. Consistent with this perspective, he described content analysis as "a research technique for the objective, systematic, and quantitative description of the manifest content of communication" (p. 18). Content analysis proceeds in terms of "what-is-said", and not in terms of "why-the-content-is-like-that (e.g., 'motives') or how-people-react (e.g. 'appeals' or 'responses')" (p. 16). Hair, Anderson, Tatham, Ronald and Black (1995) argue, like Berelson, that content analysis should focus only on manifest content. They referred to latent content as "hypothesized" and "unobserved" concepts that can only be approximated through observable or measured behaviors.

These arguments in favor of a focus on manifest content may explain why analysis of online discussions often focuses only on observable manifestations or indicators of behaviors in the text. The purpose of the case study reported on in this paper was to explore the role of latent content in providing insight into discussants' behaviors in an OAD. We considered latent content as the discussants' intentions or motives for behavior. To gain insight into latent content in the OAD we relied on interviews with discussants.

Methodology

The volunteer participants in the case study were seven graduate students and three undergraduate students enrolled in Counseling Psychology courses in the Fall of 2004. The unmoderated discussion in which they took part was a component of a four-week-long online learning module designed to engage them in Problem Formulation and Resolution (PFR). The problem or issue discussed was how to promote parental involvement in schools. The discussion consisted of eight tasks. The first five tasks were designed to support engagement in Problem Formulation and the remaining three tasks to support engagement in Problem Resolution. Once participants completed all eight tasks, their transcripts were compiled, then printed off, and manifest content coded for PFR using the second iteration of an instrument created by Murphy (2004). The instrument includes two main categories, Problem Formulation and Problem Resolution, five processes and nineteen indicators of PFR related behaviors. For example, one of the processes within Resolution is "Identifying solutions," which has two indicators, "Proposing solutions" and "Hypothesizing about solutions."

The transcripts of the discussion were grouped by a participant and coded by two independent coders against the nineteen indicators of behavior associated with PFR in the instrument using the paragraph as the unit of analysis. The transcripts were also coded a third time jointly by the two coders and the creator of the instrument and principal investigator. This third coding is used in this study to report aggregate results of engagement in PFR in the online discussion. Cohen's Kappa was used to calculate interrater reliability. Codes assigned by the first coder compared against the third coding gave a value of 0.776, and codes assigned by the second coder compared against the third coding gave a value of 0.773. A total of 80 messages were coded, totaling 355 paragraphs and 35,654 words. The subject headlines of messages were excluded from the coding.

Following analysis of the transcripts for manifest content using the instrument for PFR, individual interviews were conducted to focus on latent content or participants' intentions related to PFR. Each interview lasted approximately one hour. At the beginning of the interview, participants were given a copy of their transcript. They were asked to read their first message. The interviewer then asked them to talk about their intentions or motives in writing the message and about what they were trying to accomplish. Once the participants discussed their first message, they were asked to proceed to the other messages until all eight messages in their transcript had been discussed. The interviewer prompted the participants using the indicators associated with PFR in the instrument. The instrument served as a structured guide for the interviews. Results of the coding of manifest content for each participant were available to the interviewee prior to the interview. These results also guided the interview. For example, if the analysis of one participant's transcript evidenced engagement in one behavior over another or little engagement in one behavior, the interviewee directly asked questions related to the results of the analysis of the transcript, such as "Were you trying to critique other people's solutions?" or "Were you hypothesizing about solutions?" After the interviews were conducted, we looked for patterns using keyword analysis (Miles & Huberman, 1994).

Presentation of results

Results are provided in two separate sections, in relation to the two main categories identified in the instrument: Problem Formulation and Problem Resolution. Each section is further subdivided to present results in relation to manifest and latent content. In this regard, the first part of each section includes results of the coding of the discussion transcripts using the instrument for PFR, and the second part reports on the interviews conducted to gain insight into participants' intentions and motives.

Problem Formulation: Manifest content

The total number of units coded in the discussion transcripts for behaviors associated with Problem Formulation was 188, which corresponds to 53% of the total number of units. Participants privileged two behaviors associated with Problem Formulation, accessing and reporting on sources of information (14%) and identifying causes of the problem (9%). They also engaged in identifying the value of knowledge, and reflecting on one's own thinking (9% and 7% of units coded respectively). Within Problem Formulation, participants engaged the least in agreeing with the problem as presented, specifying ways in which the problem manifests itself, redefining the problem within the problem space, articulating a problem outside the problem space, and minimizing or denying the problem, identifying the extent of the problem, and identifying unknowns in knowledge. Results from coding the transcripts of the discussion in relation to Problem Formulation are presented in Table 1 in the form of aggregate or group measures of participants' engagement.

Table 1:
Results of Analysis of Manifest Content for Problem Formulation

Problem Formulation: Latent content

Participants expressed various goals related to their engagement in Problem Formulation. P10 aimed to "summarize the problem" using his "own words" in his first message, and P11 initially approached the problem by "rephrasing" it because this helped her understand "what exactly is the issue." Other participants referred to their initial behaviors in the discussion as expressing agreement with the problem as given, and recognizing its importance. P13, for example, stated that her intention in her introductory message was "to show the other people that, yes, [parental involvement] is actually a problem." Participants also focused on identifying causes of the problem at the beginning of the discussion. In this regard, P06 commented: “I was trying to get some insight into the problem, what I felt might contribute to challenges parents have with getting involved in their children’s education."

Participants emphasized why they felt it was important to consider the perspectives and experiences of others in order to understand the problem in more detail. They highlighted the importance of illustrating the problem by drawing on their own or others' experiences. With regard to her introduction, P13 commented: "I wanted to show … an example of one of the problems that I face." P09 connected personal experience with other experiences because "different aspects of the problem become more clear" and one realizes "how many pieces of the puzzle there actually are." Some participants mentioned that they specifically engaged in "looking" at other people's viewpoints alongside their own to compare them with their knowledge. Commenting on one of her first messages, for example, P11 stated: "I'm getting more information and I have to look at how information corresponds to what I already know." Similarly, P09 initially "had a picture of how other people see the problem" and then concentrated on "seeing how it relates to some experiences in my work."

In addition to referring to their individual increasing awareness of the different perspectives of the problem, participants also commented on their interest in "sharing" and "collaborating" in relation to the knowledge-building process and to "expanding" and "furthering" knowledge. They not only stressed how they had personally become more aware of different viewpoints and expanded their knowledge, but they also described the various ways they intended to expose other participants to different perspectives. For example, some participants shared experiences to help others broaden their views. P08 shared a personal experience because others "may only have had the perspective from the school." In one of her messages, P08 indicated that she wished to "share the main points" of her article, and that her intent was "to share the article with everybody and summarize what it was saying … so that people would want to read it."
 

Problem Resolution: Manifest content

The number of units coded for behaviors associated with Problem Resolution was 167, which corresponds to 47% of the total number of units in the discussion. Identifying causes of the problem, with 22% of units coded, was the behavior most privileged among all the behaviors associated with Problem Formulation and Resolution. Within Resolution, agreeing with solutions proposed by others was the second most favored behavior (12% of units). Participants also engaged in hypothesizing about solutions (4%) and reaching conclusions (3%). The other behaviors within Resolution, weighing and comparing alternative solutions, critiquing solutions, rejecting solutions judged unworkable and planning to act showed lower percentages of engagement. Results from coding the discussion transcripts in relation to Problem Resolution are presented in Table 2 in the form of aggregate or group measures of participants' engagement.
 

Table 2:
Results of Analysis of Manifest Content for Problem Resolution

Problem Resolution: Latent content

The focus on latent content provided an opportunity to gain insight into why participants focused on Resolution of the problem from the beginning of the discussion, even though the discussion was designed for engagement in Problem Formulation prior to Resolution. P15 described how, in her first posting, she was "looking at the potential for solutions." The rationale P15 offered for this early focus on solutions was based on her approach to problem solving: "Automatically I just think to myself 'What can I do to solve this problem?'" P14 exhibited a similar approach to problem solving. He described himself as being "more of a solution person" because "in life there are always going to be problems." He justified his early focus on solutions by noting that, when confronted with a problem, he "internally" asks himself, "What I think is going on?" and "What do I think we can do about this?"

Participants also provided insight into their approaches to Problem Resolution. P14 described his approach to solutions as "global." He referred to his solution as a "global perspective in prevention," or "preventative medicine." P06 described his approach similarly as an "overall" perspective or umbrella approach from which to identify different solutions which are "all linked:"

I was looking at [the problem] from an empowerment point of view overall, to empower the parents…and things to do in order to meet that …. There is a central bridge that will connect administrators of this program and parents and the bridge is parental self-advocacy.

P09 explained her approach to solutions as one that focused on specific parts of the problem and its causes: "In my plan of action I was trying to tackle as many of the key pieces of the problem as I could." She described her approach in detail:

I felt you needed to break it down, look at the specifics of certain things, why this big problem happened in the first place, and then I broke it down that way and added pieces to the plan of action that I thought would address those smaller issues in detail ... really to make sure that everything was addressed in there.

Rather than addressing different causes of the problem, P13 focused on addressing one specific obstacle to parental involvement in her solution. She proposed organizing a concert as a way of overcoming the obstacle of "get[ting] parents physically into the building." She chose to link her solution to one particular cause, although she also indicated that "it is not the only one." P13 gave a practical rationale for her focus on one cause of the problem: "I thought that trying to find solutions to each one of the specific problems [the other participants] mentioned was way and above the scope of what we could possibly do with the amount of time." P02 used a different approach to identifying solutions that involved redefining the problem as follows: "It is not that parents have to be at the school always," and then proposed a solution based on schools promoting educational opportunities "not necessarily at the school."

Participants also expressed their interest in using experience to identify possible solutions to the problem. For example, P15 stated that, when presented with the problem at the beginning of the discussion, she drew on her "own individual context" to identify possible solutions: "Doing my internship in education, a lot of the teachers didn't show up … so from there I kind of developed my action plan." Participants, such as P09, referred to why she relied on experiences to identify solutions: "I really identified with the problem and different solutions that we come up with at work … [I was] dealing with my own experience and what other people said. I just kind of built on that."

The focus on latent content also highlighted why participants did not engage in certain behaviors. Some participants explained that they tried to critique other people's solutions, or that they felt they could have critiqued more or that they had to make an effort to critique without making others feel uncomfortable. P14 explained that he critiqued another participant's solution by discussing its limitations, but at the same time he "was trying to be nice." Similarly, P10 mentioned with respect to one of his messages: "I disagreed with [P15] to a certain extent so I was trying to get that. I remember when I was writing that comment I spent a lot of time rewording ... so that it wouldn't come off as being too critical." P09 relied more on highlighting the strengths of another participant's solution than its weaknesses: "I could see a whole lot of weaknesses … but other than that I just focused on what I thought was pretty good about her plan." Similarly, P08 explained that, in one message, she gave some suggestions to "energize" people, and "not to insult them or critique them, just put my spin on things," because she preferred to concentrate on the positive:

When I reflect back to people, a lot of times it is to congratulate, them, support them, and compliment them. I might have some suggestions but I like patting them on the back …. If you're working in a team you want to point out to people "Hey, that's a good idea!' … not "That's no good!" That's not the way I operate in real life.

P08 emphasized how the intent of her final message was to implicitly critique. In her message, she concentrated on collaboration in the discussion. However, indirectly she wanted to criticize what she perceived as the need for more collaborative learning environments:

I wanted to express thanks … and my strong thoughts about how teamwork and collaboration should be a part of the program …. My point is that I want to see more of it. I’m expressing an opinion here and maybe a bit of a critique of the faculty at the same time. When I think back to what I was thinking … I was kind of not happy with the way some courses were laid out. I didn’t want to be blatant. I know some of the people I’m talking to think the same way, so I was comfortable doing that.

When describing their reactions to other people's solutions, two participants did not use terms such as "critique" or "criticize." P06 explained that he wished to "further" another participant's explanation in one message, and, with respect to another message, his intention was to "offer additional information." P11 differentiated "giving feedback" from expressing a "reaction" or "criticism," or "breaking someone else's bones." However, she described how she found it was "easy to give feedback" in the discussion, whereas, at her work, she needs to "be careful" when giving feedback.

Discussion

Results are presented with respect to the categories of Problem Formulation and Problem Resolution. Emphasis is placed on the findings within each category in relation to manifest and latent content and on the ways in which analysis of latent content confirmed, supplemented, complemented, or did not further explain results of analysis of manifest content.

With regard to participants' engagement in Problem Formulation, analysis of manifest content of the discussion transcripts using the coding instrument revealed that they engaged little in the behaviors associated with this category. Percentages of units coded for six out of the seven Formulation behaviors were low, ranging from 1% to 9%. Only one behavior, accessing and reporting on sources of information, accounted for a higher percentage, 14%. The exploration of latent content offered some insight into why participants engaged little in Formulation behaviors. Participants focused on solutions rather than on Formulation from the beginning of the discussion, even though it was designed for engagement first in Formulation. Additionally, as participants themselves highlighted, their approach to problem solving was solution-focused. Only P11 declared a preference for Formulation, or "consult[ing] first,” over Resolution. The rationale she offered was that "the greater the understanding of the problem the more chance you have of coming up with effective solutions." She also noted that, whereas half way through the discussion she was still trying to understand the problem, "other people … were already in the solutions."

Analysis of latent content revealed participants' reliance on experience. Participants referred to use of personal experience or to other people's experiences in relation to their engagement in various behaviors associated with Formulation, such as identifying causes of the problem and illustrating the problem. In some cases, participants drew from their experience in order to "show" the others aspects of the problem that they may not be familiar with or have previously considered. In relation to this interest in sharing experiences, participants referred to the importance of seeing others' perspectives and sharing and comparing perspectives in the process of problem solving.

In contrast, analysis of latent content did not seem to give insight into why participants privileged accessing and reporting on sources of information over any other behaviors associated with Formulation. Conversely, it did not explain why they engaged the least within Formulation in identifying unknowns in knowledge. The extent of engagement in these behaviors may be explained by the nature of the tasks included in the discussion. It could also be a result of limitations of the interview protocol. In this regard, eliciting information about intentions and motives in the discussion required participants to bring them to a conscious level, which required them to engage in a form of metacognition. Engagement in metacognition requires self-interrogation and self-regulation (Brown, 1978), skills that may not always be well developed in adult learners such as those participating in the study's discussion (Niehaus, 1995).

With regard to participants' engagement in Problem Resolution, latent content both complemented and supplemented the manifest content with regard to participants' preference for identifying solutions. Analysis of latent content confirmed participants' focus on identifying solutions, which was the behavior favored in the discussion as a whole, with 22% of all units coded for this behavior. It also offered additional information as to why participants privileged solutions. They offered as a rationale for their emphasis on solutions the fact that their approach to solving problems is one focused on solutions. As one participant explained, from the moment he is faced with a problem, what first comes to mind is "What can I do to solve this problem?" An additional insight gained from analysis of latent content was that participants tended to use experience to assist them with identifying solutions.

Analysis of latent content also provided insight into the different ways that participants approached solutions. One approach to solutions identified by participants was to see an overall solution or perspective from which to view the problem and from which to adopt specific solutions. For example, P14 referred to his approach to solutions as a "global perspective," and P06 described his as an "overall" perspective. Analysis of latent content also revealed that different participants related causes and solutions differently in their approach to Problem Formulation and Resolution. For example, P09 tackled specific "key pieces" or causes of the problem and later in another task she tried to match up solutions with the causes she had identified, whereas P13 addressed one specific cause of the problem in her solution.

Regarding the behaviors related to evaluating solutions outlined in the instrument, agreeing with other people's solutions accounted for 12% of units coded, whereas the other three behaviors related to evaluating solutions, critiquing solutions, rejecting solutions judged unworkable, and weighing and comparing solutions, were considerably low, ranging from 0.3% to 3%. This would suggest that, in terms of solutions, participants did not go beyond identifying solutions or agreeing with solutions. The analysis of latent content helped explain results of manifest content related to evaluating solutions, specifically as to why participants agreed with other people's solutions and why they did not critique other people's solutions. Participants provided a variety of reasons for not criticizing others' ideas. For example, they did not want to make others feel uncomfortable. Additionally, they themselves felt uncomfortable about expressing criticism, as evidenced by one participant's comment that he spent a long time rewriting a posting in which he wanted to critique another person's solutions. Another reason for not wanting to critique was that the discussion was not anonymous and participants knew each other from class. Finally, participants preferred to concentrate on the positive rather than the negative, which may explain why they agreed with solutions more than they critiqued solutions. Analysis of latent content did not reveal why participants engaged little in the other behaviors associated with Resolution outlined in the instrument, specifically why they engaged little in hypothesizing about solutions or in behaviors associated with acting on solutions, such as planning to act or reaching conclusions.

The approach to focusing on latent content provided insight into participants' motives and intentions in the online discussion. However, the approach presented some limitations. One limitation related to the retroactivity of the interviews which were conducted one week after the discussion was completed. This retroactivity introduced a special challenge, as in some cases, participants were being asked to think about a message they had posted four or five weeks before. Another limitation related to eliciting information from participants that required focusing on intentions and motives. Asking participants why they posted a particular message or what they intended by the message required them to engage in a form of metacognition. As noted previously, participants may not easily be able to engage in this type of activity.
 

Conclusion

Transcript analysis of online discussions using coding instruments is often concerned with describing what behaviors participants engage or do not engage in a discussion, but not with explaining the intentions or motives driving their behavior. The purpose of the case study reported on in this paper was to explore the role of latent content, understood as participants' intentions and motives, in providing insight into discussants' behaviors in an online asynchronous discussion designed for engagement in PFR. Results of the study highlighted how latent content can help explain why participants did or did not engage in certain behaviors or why they privileged some behaviors over others. In our study, analysis of latent content helped explain participants' emphasis on identifying solutions and their lack of emphasis on criticizing other participants' solutions. The focus on latent content also revealed behaviors related to participants' motives and intentions with regard to PFR which were not evident through a focus on manifest content alone. In this regard, latent content provided insight into participants' different ways of conceptualizing solutions, and their emphasis on use of experience for understanding and solving problems.

To overcome the limitations and challenges associated with requiring participants to focus retroactively on their intentions and motives, the focus on latent content could be conducted after each message is posted or simultaneously, using think-aloud protocols. Alternatively, participants in the discussion could be asked to state their intentions or motives for their behavior in the subject line of their message. These approaches may help overcome the problems associated with interviewing participants once the discussion is over. In spite of the event of these approaches, they may not always be feasible. Likewise they may potentially stifle the discussion or influence it. To engage interviewees in metacognitive thinking, interview techniques may need to rely on protocols, strategies or scaffolds. Results of the study suggest that while latent content can provide insights beyond those gained through a focus on manifest content alone, it requires sophisticated and well-thought out procedures in order to be effectively investigated.

Acknowledgements

This study was made possible by a grant from
the Social Sciences and Humanities
Research Council of Canada (SSHRC).

References

Bales, R. F. (1951). Interaction process analysis. Cambridge, Mass.: Addison-Wesley Press.

Berelson, B. (1952). Content analysis in communication research. Glencoe, Ill: Free Press.

Brown, A. (1978). Metacognition, executive control, self-regulation, and other mysterious Mechanisms. In Weinert, F. E. and R. H. Kluwe (Eds.). Metacognition, motivation, and understanding (pp. 65-116). Hillsdale, NJ: Lawrence Erlbaum Associates.

Gray, J. H., & Densten, I. L. (1998). Integrating quantitative and qualitative analysis using latent and manifest variables. Quality & quantity, 32, 419-431.

Hair, J. F., Anderson, R. E., Tatham, R. L., Ronald, L., & Black, W. C. (1998). Multivariate data analysis (5^th Ed.). Upper Saddle River, NJ: Prentice Hall.

Miles, M. A. & Huberman, A. M. (1994). Qualitative data analysis (5^th Ed.). Thousand Oaks, CA: Sage Publications.

Murphy, E. (2004). Promoting construct validity in instruments for the analysis of transcripts of online asynchronous discussions. Educational media international, 41(4), 346-354.

Niehaus, L. (1995). The readiness of prospective master's degree students in terms of cognitive and other basic skills. Ziff papiere, 97. Retrieved April 17, 2005, from http://www.fernuni-hagen.de/ZIFF/papalle.htm
 

Authors

Elizabeth Murphy (emurphy@mun.ca) is Associate Professor of Educational Technology and Second Language Learning at Memorial University of Newfoundland. She has published widely on the analysis and use of online asynchronous discussions. Her publications in this area have focused on problem formulation and resolution as well as on critical thinking and collaboration. She was the evaluator for MusicGrid which pioneered broadband-enabled learning in Canada as well as for the Telesat Multimedia Satellite Trials. She is the recent recipient of a three-year grant from the Social Sciences and Humanities Research Council of Canada (SSHRC) to study the online teacher in the virtual secondary classroom. She is a former French-Immersion teacher and school principal. Correspondence concerning this article should be addressed to Elizabeth Murphy, Faculty of Education, Memorial University, St. John's, NL, Canada, A1B 3X8.

Dr. Elizabeth Murphy
Associate Professor
Faculty of Education
Memorial University of Newfoundland
St. John's, NL Canada A1B 3X8

Tel.: (709) 737-7634 Fax: (709) 737-2345 E-mail: emurphy@mun.ca
 

Maria A. Rodriguez Manzanares (mariar@mun.ca) received a Bachelor of Arts Degree and a Diploma in Postcolonial Studies from the University of Barcelona, Spain, and the Degree of Master of Arts in English Language and Literature from Memorial University of Newfoundland. She is completing a Master of Education (Second Language Education) program in the Faculty of Education at Memorial University. Her thesis explores the language learning beliefs of university learners of Spanish as a foreign language. She teaches Spanish in the Department of French and Spanish and the Division of Lifelong Learning at Memorial University.

Maria A. Rodriguez Manzanares
Graduate Student
Faculty of Education
Memorial University of Newfoundland
St. John's, NL Canada A1B 3X8

Tel.: (709) 737-4748 E-mail: maria_angeles_rodriguez@yahoo.com
 

go top

 June 2005 Index

Home Page