Editor’s Note: New dimensions of distance learning include remote assistance, monitoring, assessment, and feedback. These functions may mimic traditional procedures, or be enhanced by interactivity, instant replay, and other web features. The logistical advantages – to reduce travel and have real time and recorded access makes the assessment process much more responsive and efficient.

Electronic Assessments for Teacher Interns

Cecil Clark and David A. Falvo

USA

Abstract

Prior to the past decade, most teacher interns were evaluated and mentored using traditional paper-based assessments and face-to-face interactions with mentor teachers and faculty supervisors. Widespread use of the Internet for conducting instruction has recently become popular in academia (Fuller, Rena, Pearce, & Strand, 2000) and the possibility of using online tools for supervising teacher candidates has grown.

This paper explores the experience of university teacher candidates and their university supervisors in a quest to explore use, concerns and attitudes about integrated features and tools in web-based, interactive learning environments for teacher preparation. This study highlights student experiences during their teacher internship placement, their reflections on the challenges and successes using technology support and performance tools, and mostly their perceptions and attitudes about the on-line tools and resources for learning that they used during the experience.

Participants of this study (n=58) consisted of 15 to 20 members in each of three groups: supervising teachers, faculty members, and pre-service teachers. This study shows that supervising teachers, faculty, and teacher candidates (the inclusive teacher education program) have extensive interest in using electronic assessment tools, but they lack confidence that electronic assessments are more effective than traditional paper-based forms. The results indicate that the electronic assessments are much easier to use and likely require less effort to complete.

Introduction

Prior to the past decade, most teacher interns were evaluated and mentored using traditional paper-based assessments and face-to-face interactions with mentor teachers and faculty supervisors. Recently, broad use of the Internet for conducting instruction has become popular in academia (Fuller, Rena, Pearce, & Strand, 2000) and the possibilities of using these online tools for supervising teacher candidates has grown. Although instructional situations and supervision processes are typically conducted in one of three environments: completely on-line without face-to-face interaction; as hybrid situations where the teacher candidates and faculty members meets face-to-face frequently, as well as on-line; and as face-to-face sessions with integrated web-based support materials and activities (Horton, 2000), the tools provide a variety of uses for supervision. The use of online tools for pre-service internship experiences helps not only in areas of assessment data, but also to provide accurate and timely feedback for improved performance and accountability. Recently, much research has been published about the impacts and effects of on-line courses and the pioneering approaches of Web-based pedagogy. The history and adaptation of the Web is labeled as a technological shift and has changed how we think about and use information (Burnett & Marshall, 2003).

Although the Web is an academic village where people learn and produce, a significant number of university administrators and faculty persist in focusing their vision on the physical campus bricks and mortar (Leonard, 2001). Perhaps not knowing how online tools, features and components (Kahn, 1997), are used for teaching and learning causes disinterest in the Web as and academic village.

This paper explores the perceptions of university teacher candidates and their university supervisors in a quest to explore their use of and concerns and attitudes about integrated features and tools in web-based, interactive learning environments for teacher preparation. This study highlights student experiences during their teacher internship placement, their reflections on the challenges and successes using technology support and performance tools, and mostly their perceptions and attitudes about the on-line tools and resources for learning that they used during the experience. Further, this study explores how on-line assessment and integrated learning tools are used not only to deliver the content of a course, but also to help learners explore issues related to the way interactions are developed in the unique environment of field experiences. The data shows how the university supervisors, mentor teachers and candidates reacted to the implementation of online assessment of practicum experiences. Although this study primarily explores teacher education, we conducted this study to explore issues that may contribute to the knowledge base and practice of merging Internet technologies with education and teaching in general.

Early Field Experience and Teacher Education Program

Participants in this research were teacher candidates, university faculty, and mentor teachers at a Mid Atlantic, Historically Black College and University that is a State Land- Grant University. These participants were associated with an NCATE approved Teacher Education Program. This university was originally established as teacher-preparation institution and places a high priority on teacher quality and k-12 student learning. The university continues to be a primary institution for statewide teacher preparation and has cultivated strong working partnerships with local K-12 school districts. As a Historically Black College/University (HBCU), the University has a tradition of quality teacher preparation programs, having received its initial Middle States accreditation in 1947. The institution has been well regarded for its teacher education program and more recently, for development of its Masters and Doctoral studies in Educational Administration, Leadership, and Supervision. The teacher education programs are accredited by National Counsel for Accreditation of Teacher Education (NCATE). The conceptual framework for teacher education affirms the goal of developing highly qualified teachers for the twenty-first century with diverse populations.

The teacher education program is fully described in web based documentation and printed sources of information including the University Catalogs (University Publications, 2010). The program combines a balance of theory and applied practice, while providing quality teaching experiences with an emphasis on diversity and special needs. The learning environment is flexible and student centered where the faculty view candidates as both learners and professionals who contribute to intellectual capital of the university. In addition, the faculty maintains collaborative agreements with schools districts, agencies, and companies who will potentially employ graduates of the program. This helps to keep the curriculum current, innovative and highly engaging for program stakeholders.

All students in this study were admitted to the Teacher Education Program (TEP) as a result of filing an application with the Council for Professional Education (C.P.E.). The Council for Professional Education is an advisory body to all Teacher Education Programs (TEP). The Council is composed of representatives from each department at the University with a teacher education curriculum, the Education Department Chairperson, the Director of Student Teaching, the Coordinator of Field Experiences, Primary Program Coordinator, Early Care and Education Coordinator, Special Education Program Coordinator, Secondary Education Program Coordinator, Content Area Program Coordinator, students and other appointed University representatives. The Education Department is the administrative body for the Professional Education Unit and the Council for Professional Education (CPU).

The Teacher Education applicants were recommended by a faculty member, academic advisor, and respective departmental chair. All applicants were required to have a cumulative grade point average of 2.5 or higher on a 4.0 scale and were expected to take the PRAXIS I by the end of their freshman year and pass all three sections the PRAXIS I by the end of their sophomore year. Satisfactory performance on the PRAXIS-I is a prerequisite for admission to the Teacher Education Program. The chair of the Education Department with a designated committee will reviews each application for admission and submits a list of students for final approval to the Council for Professional Education(University Publications, 2010).

Prior to student teaching, all education majors must participate in early field experiences (EFE). Field experiences are required for designated content method classes and courses within the Education Department. At this stage, these students are introduced to using the online tools for EFE assessments. Students are expected to be professional: dress appropriately, be on time, maintain appointments, and meet expectations of the course given by the instructor. The field experience begins with a course titled Philosophical Foundations of Education. Students are expected to obtain between 60-100 clock hours of field experiences depending on the program and academic department. These hours are in addition to student teaching internship. Students must keep a log of their hours and submit copies to both the Early Field Experience coordinator and to the faculty teaching the EFE course. There are four phases to field experience: Phase 1 - observation, Phase 2 - observation with minimal participation, Phase 3 - practicum, and Phase 4 – student teaching internship placement. Candidates will participate in each phase and have a variety of experiences at different age levels within diverse populations of k-12 students (University Publications, 2010).

The student teaching internship occurs during the last full semester of enrollment prior to graduation and is considered the culminating experience for students in Teacher Education. Internships consist of a 14 week, on-site placement with a minimum of 65 days and or 200 clock hours. Students are not permitted to register for additional courses without approval from the Council for Professional Education (CPE). Candidates who successfully complete student teaching then apply for an Institutional Recommendation for Certification through the Office of Clinical Experiences. Students' preferences are taken into consideration in making student teaching assignments; however, final determinations of placements are at the discretion of the Director of Student Teaching and/or Chair of the Education Department (University Publications, 2010).

Electronic Tools for Assessing Teacher Candidates Competencies

The National Center for Education Research, Institute of Education Sciences is engaged in empirical research about structuring classrooms for improved student learning (USDOE, 2009). Much of this research also supports integrated assessment for better teaching and learning (Darling-Hammond, 2010). For example, tests and quizzes are tools for identifying content that has been learned and content that needs to be learned in a formative way. The empirical research also validates that quizzes help to introduce topics and to promote learning by re-exposing students to information. Researchers are exploring educational tests and measurements as they apply to the modern classroom and how these tools are used to address the individual needs of diverse students and students with special needs (Allen, 2006).

Topics related to this research often include performance and portfolio assessments, best practices in educational measurement, authentic assessments, technology for assessment, and contemporary issues such as the impact of “No Child Left Behind.”

In her study of the use of technology for teacher field experience, Snider found that interns and mentors teachers were receptive to technology integration into the field work. However, many participants stated concerns about the lack of modeling good use of technology in the public school classrooms and the lack of access to instructional technologies by teachers and students in these schools. Snider’s study indicates a transition stage for many teacher preparation programs. This particular transition entails student teachers/interns and faculty mentors moving from traditional tools to web-based and technology-based instructional technologies and assessments.

Educators need skills and knowledge about measurement and assessment to address the complex issues and challenges in today’s classrooms (Astin, 1996). Technology provides various advantages for implementing an assessment plan and for addressing specific needs of each individual learner. Foundational knowledge about the essential psychometric concepts combined with skill in implementing assessment technology enhances the effectiveness of teachers and administrators. Assessments are essential for best practices in education.

Mayer explored using information and communication technologies in the practicum component of pre-service teacher education. Because these interns often feel disconnected from their university faculty and peers, using online discussion forums and web-based interaction tools provides meaningful interactions and support for these student teachers. Mayer describes these learning spaces as resources for personal and professional networking and support. In addition to formal assessments, students are able to reflect and share about their successes and challenges of their field experiences. The challenge for University Supervisors relates to achieving higher levels of reflections that might, for example, examine the outcomes, consequences and equity of professional practices.

Teachers who develop useful assessments, provide corrective instruction, and give students second chances to demonstrate success can improve their instruction and help students learn. Highly qualified and effective teachers have both the knowledge and skill to integrate assessments into their teaching, for enhanced student learning (Guskey, 2003; Reynolds, Livingston, & Willson, 2006).

Additionally, highly skilled and effective teachers differentiating instruction based on student growth, development, and cognitive capabilities (Kirk, Gallagher, Anastasiow, & Coleman, 2006). These teachers know the educational implications of human development throughout life span. There has been much recent research about teaching and the development of school programs, with and emphasis on identification of exceptional learners and the understanding of their educational needs. These issues include Understanding by Design (UbD), best practices for individualized learning, legal implications of inclusion, and contemporary issues in human growth and development in diverse learning environments.

Educators need skills and knowledge about individualizing instruction to meet the specific learning needs and situations in classrooms of diverse learners. Today, educators are challenged with the demands and complexities of a wide variety of diversity among learners and high expectations for results in terms of public opinion and political regulations such as No Child Left Behind. School professionals in leadership positions need to understand the varied theoretical and philosophical orientations that significantly impact upon the principles and practices of individualized education and program development.

Everhart and Hogarty (2009) discuss the use of online assessments to document preparation levels and identify strengths and areas for improvement in teacher preparation programs. In addition to helping interns maximize their experiences in the field, electronic assessment tools have the potential to guide program improvement at the University level. Rieg and Wilson  (2009) found that University faculty members ranked in-class applications and problem solving activities, small group discussions and brainstorming as effective instructional strategies for students engaged in field experiences. However, the results of this study indicate that professors are not always using in their classrooms what they deem to be the most effective of instructional activities, especially in terms of enhancing practicum experiences. The University faculty members in this study rated tests and quizzes as being less effective in assessment when compared to attendance, projects, and class participation.

Ma and Rada (2005) explored the use of a Web-based Accountability Model (WAM) for assessment and leaning in a teacher education program. Their research helped to redesign teacher education in terms of curriculum, program, and operation. In particular, the use of online performance tools provided a systematic form of accountability and assessment of teacher candidates.

Figure 1: TK2O Assessment System Home-Page

TK20 is an assessment, accountability and management system to help colleges and universities meet requirements for accreditation. TK20 stands for "Technology in Kindergarten through Age 20." This system provides a standards-based form of assessment tracking for data management for the PEU. Participants in this study used TK20 as the primary assessment tool in terms of competencies and requirements of the program and eventual certification. Teacher candidates are able to upload artifacts and assignments into the web-based system. Faculty and mentors are able to grade projects and assignment using grading rubrics.

Faculty and mentor teachers also assess student teaching performance and lesson planning using online assessment forms specifically designed for each teacher education program. Additionally, student dispositions are self-assessed and assessed by teacher education faculty and mentor teachers. All data is managed within the TK20 system and is used to assess performance and to assess the education department’s performance in terms of standards for accreditation of their programs.

Teacher Work Samples

The Teacher Work Sample assessment instrument is the product that was produced through the efforts of the Renaissance Partnership Project. The 1999 US Department of Education Title II grant focused on the enhancement of teacher quality for teacher preparation programs. The eleven founding institutions wanted to create a system that would properly measure a candidate’s ability to impact student learning. The senior capstone helps to create an instructional snapshot of a unit that interns deliver to student in the classroom (University Publications, 2010).

Teacher preparation institutions across the nation experience some of the same unique challenges. This collaborative effort from the original members of the Renaissance group is a prime example of a true partnership that has been very successful. Regarding this University, the local and surrounding school districts have embraced the work sample as a valuable teaching tool. The mentor teachers have become very familiar with the components and how it is utilized in the classroom setting.

Figure 2: Sample of TK20 Interface

One of the key segments of the capstone is the learning goals the students. Here the interns must align the goals with the learning goals of the participating district. This ensures that the goals of the work sample are the same goals of those of the classroom teachers. Teachers are very mindful of the standards by which students will be expected to master. State and Federal mandates have prompted teacher to be more aware of student achievement.

The classroom teacher of record is ultimately held accountable for the academic progress of their students. This is one of the realities that we want to infuse into our internship and the minds of our students in the program.

Method

This mixed methods study explored the use of online electronic tools for supervision of pre-service teacher education students while they engage in field experiences including student teaching. Primarily the study investigated how these tools were integrated and how the faculty, supervising teachers, and students perceived the tools in terms of support for the experience. Data for the study were collected via access to reports in the TK20 system, and through a series of surveys conducted to ascertain perceptions about the effectiveness of the electronic support tools.

Results and Conclusions

Participants of this study (n=58) consisted of 15 to 20 members of three groups, which were Supervising teachers, faculty members, and pre-service teachers. Of the 58 participants, 39 were females and 41 participants reported prior experience using electronic assessment tools and most of the participants had experience using educational technology.

Table 1

Results of Likert Scale Questions

Most participants (90%) felt that electronic performance assessments make good use of technology for learning and many (85%) believed that these tools enhance student learning. However, participants were less confident about effectiveness of TK20 as an assessment tool and almost half of them reported that TK20 was difficult to use. Although 64 percent of the participants reported that electronic assessments were easier to use than traditional, paper-based forms, 54% felt that the electronic assessments did not help make the field experience more effective.

Among the three particular groupings (Student Interns, Mentor Teachers, and University Supervisors) results from all three groups were in support of electronic assessment tools. The three groups were less confident in the effectiveness of TK20, where there mean scores were mid-range on the scale. All three groups were in the same range when asked about the ease of use of TK20. Interestingly, only the University Supervisors group felt that TK20 was valuable. Even so, all three groups did not prefer the traditional paper-based assessments. The University Supervisors rated the ease of use of TK20 assessments the highest, while Student Interns gave the ease of use of TK20 the lowest rating. The Student Interns and Mentor Teachers indicated less confidence in TK20 making the experience effective compared to the University Supervisors.

Scores were relatively similar on the last three Likert items on the survey. Both directions and clarity of language in the system were rated slightly above average. Similarly, objectives and creative use of differentiated instruction were both rated slightly about average by all three groups.

Both the Student Interns and the Mentor Teachers highest ratings were for using electronic assessments, stating that these assessments are a good use of technology, and that they enhance the learning experience. The University Supervisors highest ratings were for using electronic assessments, stating that these assessments are a good use of technology and that assessments were easier by using TK20.

This study shows that teachers, faculty, and teacher candidates (the inclusive teacher education program) have much interest in using electronic assessment tools, but they are less confident that electronic assessments are much more effective than traditional paper-based forms. However, the results indicate that the electronic assessments are much easier to use and likely require less effort to complete.

Future research and improvements should focus on how electronic assessment tools may be used to improvement the teacher education program. One idea that has been explored is to expose candidates to our electronic tools much earlier than in previous years. Students would start to submit assignments and learn the critical functions of TK20 data collection system in their freshman courses. This would allow students to build a comfort zone by the time they reach their senior capstones phase.

Mentor teachers that collaborate with an early field experiences may also get an opportunity to participate in lesson preparation that utilize the TK20 system. Perhaps case-study research would be a useful tool in understanding the barriers and challenges of using and developing these tools for teacher education.

Last, because technology serves as a tool for practitioners in the educational professions, it behooves teacher educators to embrace this tool in ways that make their work with interns efficient and effective. Using technology to implement valuable assessment data enhances the teacher intern experience as well as the teacher education program.

References

Allen, M. J. (2006). Assessing general education programs. Bolton, MA: Anker.

Astin, A. W. (1996). Assessment for Excellence: The philosophy and practice of assessment and evaluation in higher education. Portland, OR: Oryx and American Council on Education.

Darling-Hammond, L. 2010. The Flat World and Education. New York: Teachers College Press.

Everhart, B. & Hogarty, K. (2009????) Supporting the Preparation of Beginning Teachers with Online Assessment Products. Education 129 (3) pp 400-411

Fuller, D., Rena, F.N., Pearce, K., & Strand, S., (2000). Internet teaching by style: Profiling the on-line professor. Educational Technology & Society. 3(2)

Guskey, T. R. (2003). How Classroom Assessments Improve Learning. Retrieve From: http://168.169.25.71/AdminIntraNetSite/Cabinet32508/ASCDGuskeyHowClass
AssessImpvLearn.pdf

Khan, B. H. (1997). Web-Based Instruction. Educational Technology Publications, Inc., Englewood Cliffs, New Jersey.

Kirk, Gallagher, Anastasiow, & Coleman. (2006) Educating Exceptional Children. (11thEd). New York: Houghton-Mifflin.

Leonard, D. C. (2001). The Web, the millennium, and the digital evolution of distance education. In Cole, R. A. (2000). Issues in Web-based Pedagogy. Westport: Connecticut: Greenwood Press.

Ma, X. & Rada, R. (2005) Building a Web-based Accountability System in a Teacher Education Program. Interactive Learning Environments. 13 (1-2) pp 93-119.

Mayer, D. (2002) An Electronic Lifeline: Information and communication technologies in a teacher education internship. Asia-Pacific Journal of Teacher Education, 30(2) pp 182-195.

Rieg, S. A., & Wilson, B. A. (2009) An Investigation of the Instructional Pedagogy and Assessment Strategies used by Teacher Educators in two Universities within a State System of higher. Education, Education v130 (2) pp 277-293

Reynolds, C. R, Livingston, R. B., & Willson, V. (2006). Measurement and Assessment in Education. Pearson, Boston. (ISBN: 0-205-37602-9)

University Publication (2010). Web-Based Undergraduate Catalog of the Participating University.

About the Authors