Editor’s Note: Multi-faceted and reusable learning objects are moving from the laboratory into daily teaching. Earlier research shows instructional designers produce more effective learning objects. This study trains teachers to prepare learning objects and compares the product and results. Teachers produce a different kind of product with less media and more verbal explanations to achieve similar results. This research demonstrates a step forward and opportunities for further study.

Should K-12 Teachers Develop Learning Objects? Evidence from the Field with K-12 Students

Yavuz Akpinar, Huseyin Simsek

Turkey

Abstract

The emergence of learning objects for teachers as a focus of educational concentration is relatively new and much of the discussion has not been based on the actual development of objects, but different definitions, learning theories, properties and standards or decorative packages of learning objects (LOs). In many teacher education programs, prospective teachers take a computer literacy class separate from content methods classes and rarely engage in producing authentic teaching/learning experiences. This research addresses prospective K-12 teachers’ development of learning objects. In this study, a group of prospective K-12 science teachers’ learning objects were examined, evaluated and compared with LOs developed by instructional designers (IDs). A total of forty learning objects were closely investigated and effectiveness of eight of them was tried out with 180 target students in classrooms. Detailed analysis of the LOs demonstrated that while both preservice teachers and the IDs use similar number of instructional elements in their LOs, the IDs represent concepts and procedures with screen objects other than the text and used the text for supporting graphical objects. Both groups developed LOs similar in quality measured with the LORI 1.5. Statistical tests on data obtained from classroom usage of the LOs showed marked improvements in the students’ learning.

Keywords: learning object, prospective teachers, development, evaluation.

Introduction

Teachers are responsible for tailoring instructional activities to meet curriculum standards and the unique interests and educational needs of their students. Teachers decide on "conditions, time, and strategies" of using technology in the classroom. Those decisions may include selecting learning objects that enlarge and enrich their repertoire of instructional techniques for the content to teach (Bratina, Hayes & Blumsack, 2002). E-learning systems replace the teacher as the center for learning, the teacher role shifts from lecturer to that of course developer and, once a course is in session, the learning facilitator (Cohen & Nycz, 2006). Teachers can now engage their students in computer based processes that help them build a personal knowledge base by manipulating aspects of simulated worlds, analyzing and visualizing data. Also computer based modeling tools allow students to express their theories in models that can be simulated and students will be confronted with the consequences of their ideas (Van Joolingen, Jong & Dimitrakopoulout, 2007).

To realize these potentials, the learning object (LO) development for and use in K-12 environments has become popular in related yet varied projects across the globe. In the CELEBRATE project organized by European SchoolNet (2002-2004), many such objects were made to use in classrooms. The UK government, in 2003, initiated a web portal to give teachers easy online access to a range of digital learning resources to support their teaching across the curriculum. One year later, the ARIADNE Foundation of Europe started to create tools and methodologies for producing, managing and reusing computer-based pedagogical elements and ICT supported training curricula. In the USA, Apple’s Learning Exchange is one of the first repositories; the NSF funded SMETE Digital Library project was developed as a learning object repository and is used as a resource and knowledge base by both K-12 and higher education instructors (McGreal, 2004). A third project in the USA, MERLOT consortium, held a repository and uses peer-reviews of learning objects as the basis for inclusion. Australia, Canada and New Zealand have also made efforts to engage K-12 schools in design and development of initiatives (Bennetta & McGeeb, 2005).

As the use of learning objects for teaching via technology became more widespread in educational settings (Conceição & Lehman, 2003), most of the research literature on learning objects has focused on the specifications and potential designs of learning objects. Even the National Educational Technology Standards for Teachers does not require teachers to develop their technology-based learning resources, but asks to use those facilities. Interdisciplinarity in teams of LO development may be necessary for high quality as highlighted by Kay and Knaack (2005; p.231) who stated: “Developing high quality learning objects is a daunting task involving collaboration among subject specialists, programmers, multimedia designers, and evaluators”.

It is often said that we don’t expect teachers to write their own textbooks, so why should we expect them to design their own technology based materials? (Bratina et al., 2002). Ainsworth and Fleming (2006) reply to this question and argue that teachers do customize their textbooks to use in their classroom by suggesting an order to read chapters, explaining difficult terms, providing exercises and worksheets. They propose that much can be gained by providing teachers with simple authoring tools.

Other researchers (Bell, 1999, Boyle, 2003; Merriënboer & Martens, 2002) suggest that instructional software templates may positively affect the efficiency of the development process and compensate for the developers’ lack of experience. This can be beneficial for the authoring of instructional software because more people with low instructional design and software production skills are becoming involved. Further, teacher involvement in the development of online learning resources has received attention only recently (Akpinar & Simsek, 2006; Kay & Knaack, 2005; Muirhead & Haughey, 2005; Lajoie, 2003; Oliver, Harper, Hedberg, Wills, & Agostinho, 2002; Recker et al., 2005) and researchers (Dunning et al., 2004; Jones, 2004) have suggested that with the addition of simple templates, teachers will be able to make their own objects. Haughey and Muirhead (2005) stress that it is likely that teachers will be able to develop objects requiring activities such as drag and drop, or put the items in a sequence.

In developing learning objects, different type of information might be created using traditional tools such as scanner software, spreadsheets, word processing, painting tools, HTML editors, GIFmakers, video editors/capturers and some general and specific purpose software. In LO terms, picture, animation, simulation, sound file, hyperlink, game, video, and downloadable-file are called assets. Assets can be combined to form larger files and sharable content objects (SCO). The number, quality and orientation of screen elements loaded into a lesson are an issue for development of LOs, though Learning Content Management Systems (LCMS) and authoring environments provide many facilities to create and edit screen components. The usage of those facilities should not require experience and expertise but should demand great care because research data (Hannafin & Hooper, 1989; Li, 2006; Stemler, 1997) for possible components of a computer based lesson suggest that for effective learning, screen design decisions should reflect balance among learner attributes, content factors, and processing requirements of the learning task.

Teachers with ready access to learning objects become designers who adapt and customize learning objects to fit their local needs and context (Dede, 2003; Littlejohn, 2004). In this context, learning objects become catalysts for creating locally relevant instructional solutions to support learning (Recker et al., 2005) Unfortunately most LO repositories are in English and that creates a “language divide”. Teachers instructing in other languages have to do more than adaptation and customization to develop their own LOs in the light of instructional theories and available LO repositories. Perhaps LO repositories in languages other than English, especially in the developing countries, can be constructed and enriched by such efforts. Other researchers (e.g., Figg, & Burson, 1999; Oliver et al, 2002; Waddoups & Wentworth, 2002) have also pointed out the importance of including teachers in the development process. While a number of design features have been incorporated by developers of learning objects in the literature, only a few studies did a formal descriptive evaluation of the final learning object product (e.g., Cochrane, 2005; Krauss & Ally, 2005), there are not enough number of studies examining impact of learning objects developed by teachers on students’ achievement .

Problems of the study

This research studied preservice science teachers’ development of learning objects in an LCMS and compared those with instructional designers’ LO development. The study provides a preliminary quantitative measure and evaluation of different authors’ use of assets, organization of assets and instructional directions in a learning object they create. This study aimed to:

(1) compare preservice science teachers’ and IDs’ development of K-12 science learning objects with different (a) number of assets (picture, animation, simulation, sound file, hyperlink, game, video, downloadable-file), (b) text density on each learning objects (small amount, moderate amount and large amount of text), (c) number of instructional elements (advance organizers, questions and didactical directions), (d) number of screen orientations (templates, picture orientation, font types and font sizes, colors, main topics, sub-topics and Sharable Content Objects (SCO)) in their products, (e) the quality of LOs using the Learning Object Review Instrument (LORI, version 1.5 by Nesbitt and Li, 2004), and

(2) investigate the effect of LOs with the targeted students in real classroom environments.

Method

Subjects

To investigate preservice science teachers’ development of learning objects in a LCMS, a series of studies were conducted with 40 subjects (20 preservice science teachers and 20 newly graduated instructional designers). During the study, the preservice science teachers, the experimental group, were studying their final year in a school of education in order to be the teachers of varying fields as Primary and Secondary School Science and Mathematics Education in 2006 spring and fall semesters. They complete their degrees in four or five years after one-year of English Language preparation. The ones who will teach in secondary schools study five years, but others study four years. Selection of the subjects was carried out on the basis of accessing them during the research activities. Before the study, they all studied at least one ICT related course, “e.g. Introduction to Computing”. They were familiar with and users of information and communication technologies.

The instructional designers (IDs), the control group, were new graduates of the same faculty, studied in the Department of Computer Education and Educational Technology, and completed courses including instruction, learning, analyzing performance problems, and design, development, implementation and evaluation of instructional strategies and products. All participants contributed to the study on a voluntary basis.

Procedure and Materials

The materials of this study included “Instructional Materials Development course, LCMS, BU-LeCoMaS, environment, the learning objects for K-12 developed by the subjects using the BU-LeCoMaS, LO Review Instrument and achievement tests used in pre and post testing of K-12 students’ achievement. The preservice science teachers followed a thirteen week Instructional Materials Development course focusing on the development, implementation and evaluation of ICT based instructional materials. Special emphasis was given to the properties of learning objects. In this four-hour per week course (two hours theoretical and two hours practical activities), subjects were given opportunities for intensive experience in web based learning materials; some learning activities in the course were based on developing online support materials and web sites with a commercial web editor to assist K-12 students’ learn content. The course included practical sessions on how to create learning resources in MM Flash and DreamWeaver environments.

When the preservice science teachers completed their course, both the instructional designers and the preservice science teachers were provided with a username and a password to the BU-LeCoMaS server and received one-hour of training in use of the BU-LeCoMaS learning content management system. The training was carried out in two sessions. The lab was equipped with one server and 20 PCs organized in U shape in the room, all connected to the Internet. After training, participants were instructed to select a K-12 science learning task, prepare and bring their materials (assets of learning objects) to the lab in a week time to aggregate those materials and develop learning objects for K-12 science students. They were encouraged to use any sort of learning materials  and assets from text to animations, and from static graphics to video segments. They were allowed to re-use graphics borrowed from Internet; they were free to use anything they found that was appropriate for their instruction.

In the following week, subjects were asked to use the system facilities and to develop a set of web based materials as learning objects for a part of their chosen K-12 science learning unit. They required enough materials to create one lesson hour of study. During their usage of the system, one of the researchers was present in the lab to resolve technical problems but did not intervene in the participants’ work. Each participant developed one learning object, a total of forty, in K-12 science.

The BU-LeCoMaS, learning content development and management system (see Figure 1), is an easy-to-use LCMS, requiring content authors with little or no technology expertise to develop learning objects. It helps online material developers, with time, place and platform independent content authoring. The architecture of BU-LeCoMaS can handle and execute any content input. It facilitates integration of textual content, sound, movie and animations into software packages and enables multimedia platform creation. It has lesson templates, layout templates and information creation and editing tools. Multiple users can easily and collaboratively construct, share and re-use content within the LCMS as well as re-use after development. Further, it supports SCORM standards, allowing developed content to be used in different learning management systems based on the idea of reusable learning content as sharable content object.

To create a small set of learning content, an authorized author can use BU-LeCoMaS to sequence and group learning materials to constitute a learning unit, the size of which varies and depends on its author. A list of available learning units and learning topics is displayed in the root window of the system. A learning topic is a subset of a learning unit. BU-LeCoMaS supports both constructing a learning unit and constructing an asset, a granular learning content. The author specifies the title and description of the material he or she is creating, selects a template, object type, tree-view type, background and foreground colors, and style sheet. The author determines “create as template” or share it as a“public template”, and decides whether ore not to include it in the subject index of the BU-LeCoMaS.

The subject index is used to search the object repository of the BU-LeCoMaS and it is used by authors to manage associations of their materials for learning units. Once the author enters relevant information and selects, for example, LO Template Tutorial-1, the learning unit frame is provided. The author receives a screen where the name of his or her materials appears with four sub-sections: Objectives, Introduction, Read & Study, and Images. The author can select any sections and designs. Finally, the author publishes his or her projects and set viewer permissions. Viewing options include user-only, the author’s students, or anyone browsing the BULeCoMaS. The project can be downloaded as a SCORM-compliant zip archive for use outside BULeCoMaS.
 

Figure 1. BU-LeCoMaS learning object development platform.

Following two consecutive days when participants develop learning objects for K-12 with the BU-LeCoMaS, one for preservicers and one for IDs, they are given a usability questionnaire with 44 five-point Likert type items and two essay items to measure the usability of the content development system, BU-LeCoMaS (five additional questions collect personal information). The scale was previously developed and used elsewhere (Akpinar & Simsek, 2006) and for testing usability of a similar tool. Cronbach's alpha reliability coefficient of the questionnaire was estimated in this study as 0.91. Each participant’s total usability score was estimated. The mean of those scores was 165.25 of a possible score of 220.00.

Review of the Learning Objects and Data

The forty learning objects were analyzed by the two researchers. They studied the LOs to identify patterns and counted elements including (1) number of assets (picture, animation, simulation, sound file, hyperlink, game, video, downloadable-file), (2) text density (small amount, moderate amount and large amount of text) on each learning object, (3) number of instructional elements (advance organizers, questions and didactical directions) and (4) number of screen orientations, sub-topics –Sharable Content Object (SCO)s, templates, picture orientation, font types, font sizes, colors, and main topics in each LO.

Table 1

Data on the two groups’ LOs

To establish a learning object repository for various levels requires criteria to assist teachers develop, submit and assess LOs (Akpinar & Simsek, 2006). These criteria are crucial to ensure quality and accessibility of resources in the repository. For that purpose, Nesbit and Li (2004) developed a Learning Object Review Instrument (LORI 1.5). This study used it based on evidence that LORI can reliably assess some aspects of LOs. LORI 1.5 uses nine items with brief descriptive rubrics associated to each item and Likert-style five point response scale scored from low (1) to high (5). If an item is judged not relevant to the LO, or if the reviewer does not feel qualified to judge that criterion, the reviewer may opt out of that item by selecting “not applicable”. Items of LORI 1.5 are given in the first column of Table 1.

In order to evaluate the LOs developed by the preservice science teachers (see Figure 2 for an example), two researchers reviewed and rated the LOs individually using LORI scoring sheets. Following the reviewing and rating process of 20 LOs, the researchers combined the ratings and estimated average ratings for each of nine issues for a particular LO.

The reviewers’ overall ratings for a LO was obtained through summing up points given to each nine issue for a particular LO. Next, the LOs developed by the control group, IDs (see Figure 3 for an example), were made available in a web server to the IDs who reviewed and rated their twenty developed LOs independently using the LORI and the twenty IDs’ ratings were averaged. The two researchers who rated the preservicers’ LOs also rated the IDs’ LOs.

The correlation between the researchers rating and the IDs’ rating was high (0.96) so LO ratings of the preservice and control groups were combined. A formative reliability analysis of the LORI 1.5 data revealed that the overall internal-consistency reliability (Cronbach’s alpha) of the LORI 1.5 is 0.94.
 

Figure 2. A learning object developed by a preservice teacher.