Editor’s Note:  Online assessment with immediate knowledge of results is preferred by most learners. It also makes sense from an “effectiveness of learning” point-of-view. This preliminary study is useful to explore the attitudes of pre-service teachers and to define parameters for further studies.

Pre-Service Teachers’ Attitudes towards Online Assessment: An Initial Component of Teacher Training

Patrice C. Boyles

USA

Abstract

Online education is becoming the preferred mode of learning for many adults.  More institutions are offering a myriad of courses and programs. This research examined perceptions of online assessment of pre-service teachers’ who were enrolled in a class at an urban institution in the Midwestern United States.  The survey focused on student and instructor performance, and future implementation. Data from this study revealed students preferred classes that use online assessment. The survey data uncovered common issues associated with online learning such as the need for educators to focus on learning styles and implement online ‘best practices’. Recommendations can be used by higher education institutions to proactively plan and prepare distance education courses.

Keywords: Assessment, Evaluation, Distance Learning, Perceptions, Pre-Service, Teachers, Technology, Performance, Online Learning, Training.

Introduction

In a classroom without walls, online education has become the desired mode of many adult learners. It has become the anytime, anywhere way of continuing education thus forcing many educational institutions to revise delivery methods and strategies to adapt to e-Pedagogy learning environments.  Those institutions that have chosen to embrace distance education are doing so for a variety of reasons including acquiring the opportunity to reach a more diverse population and expanding their revenue base.  A research report by Allen and Seaman (2008) found almost two-thirds of all higher education institutions in the United States offer some type of online course.

The convergence of curriculum redesign, online methodologies, and new technologies can be difficult for some instructors.  Some instructors are even apprehensive teaching online courses because they require significant planning and preparation. Given that, it has become critical for instructors to maintain instructional quality and implement ‘best practices’ in a virtual environment, which includes online assessment. 

Assessment in any course helps instructors to identify potential learning outcomes.  Online assessment allows for collaborative interaction between students and the instructor; it accommodates students and creates a flexible learning environment. Online Assessment fosters a climate of learning by allowing instructors to provide immediate feedback to students. It allows students to become actively engaged in the assessment process. There are several factors to consider prior to initiating online assessment.

Internal Factors

Internal factors include perceptions and beliefs related to learning, more specifically online learning. Instructors need to become aware of their teaching styles as well as their student’s learning habits. Both instructor and student should displace preconceived notions of distance learning and employ a self-questioning framework that includes determining their motives, goals, abilities, disposition and strategies before embarking in an online course.  More importantly, instructors must create and employ a climate that is conducive to learning while at the same time help students identify their barriers.  Grash and Hicks (2000, p. 3) research concluded teaching styles are centered on the “needs, emotions, and attitudes of the teacher.

External Factors

External factors to be considered include technical support as well as the student demographics. Many schools do not have the technical infrastructure to handle a migration to a distance learning platform. With regards to student demographics, many students enroll in distance learning courses with the preconceived notion that the courses are easier than face-to-face courses; some students are ill-prepared, and not disciplined. Many students enroll in online courses without proper preparation. To add fuel to the fire, some instructors who are facilitating online courses, are not properly trained to teach in an online environment.

The goal is to enhance the student’s experience in virtual environment. A students experience can be enhanced by promoting learning through discussion, feedback and by giving students opportunity for self-direction. Instructors are encouraged to inform students of course expectations and requirements early.  Communication, collaboration and discussion are vital components to a successful virtual environment. While many schools relay primarily on asynchronous communication to deliver instruction, many instructors prefer incorporating synchronous as well as asynchronous instruction. Synchronous lectures allow the students and instructors to chat one-on-one in real-time to facilitate immediate feedback. Given that, the purpose of this study was to examine the degree in which students perceived online assessment and their faculty’s usefulness.

Literature Review

Advances in technology have forced many educational institutions to move beyond face-to-face traditional assessment to one that expands online curriculum.  The goal is to increase student enrollment. There are a growing number of students enrolling in online courses. This was affirmed by Allen and Seaman (2010) report; approximately 5.6 million students were enrolled in at least one online course in fall 2009.  Based on this, educational institutions have been required to prepare their students and instructors for online classes. 

Pecheone and Chung (2006) addressed performance assessment as part of the teaching practice in their research. These assessments inform instructors on needed curriculum changes. In addition, the assessments can contribute to improvement of teaching. Curriculum embedded assessments are stressed more than ever before. These assessments occur throughout teacher preparation. Instructors can embed assessments into instruction in a variety of ways that include case studies of individual students, lessons, unit plans, analysis of student work, observation of student teaching and e-portfolio-based assessment. Research by Quellmalz, Schank, Hinojosa and Padilla’s (1999) research asserted assessment is another means to education reform; it has a direct correlation to budget implications. It allows stakeholders to use data to perpetuate change. Performance-based assessment has become one of the critical issues in educational reform, teacher preparation and educational funding.  Ahn, through her research asserted “Portfolio-based assessment and more recently, the electronic portfolio have been seen as an alternative to standardized tests” (Ahn, 2004, p. 12).  There is growing need for research on online assessment practices from face-to-face to online environments. Their research discussed the pros and cons of online assessment. In addition to instructors acquiring a new technology skill set, using the online medium structure required more time spent streamlining course content, requirements, building test banks, and uploading resources.  Instructors needed to learn how to build a sense of community as they redefine pedagogy to meet the needs of learners in a virtual environment.  This was affirmed by Woods and Ebersole (2010) research using “communal architect” or community-building strategies.  A “communal architect” is someone who erects a communal scaffold for the purpose of community building (Woods and Ebersole, p2).

Method

Research has shown that more educational institutions are offering online courses. Some are even offering entire degrees and certificates online.  Authentic online assessment is at the heart of every learning experience and helps to negate negative student outcomes. Therefore, the purpose of this study was to investigate pre-service teachers’ perceptions of online assessment. The study was conducted at an urban university in the Midwest, where the majority of the students were African-American and all were enrolled in a technology class.

Overview of Design

To better understand teacher candidate’s perceptions of online assessment, a written survey was designed to collect data from the teacher candidates (see Appendix 1). Candidates were asked to reflect and respond to ten questions based on personal beliefs.  All students enrolled in a technology course were given the opportunity to complete the survey. Descriptive and frequency statistics were conducted to analyze the data.  The following describes the procedures used in detail.

Procedure

The procedure used to conduct this study comprised of several phases: 1) a survey was constructed, 2) the sample was selected, 3) data collection, and 4) data analysis. Students were given the option to participate in the study.  Students who elected to participate were given the self-designed survey.  To protect the anonymity of the participants, students were asked to not identify themselves and to return the survey to the instructor’s office or mailbox. Prior to class, students were given the purpose of the study and instructions to complete the survey and return it to the faculty member’s mailbox.  All students completed a self-report questionnaire that gathered information on their perception toward online assessment. Participants were encouraged to reflect on their experience with online assessment.

Survey Construction

The researcher developed a self-constructed survey to gather information on students’ perceptions of online assessment. The instrument consisted of 2-page questionnaire which was divided into three sections: 1) perceptions of online assessment and 2) general comments and suggestions; 3) demographic information.

The first section consisted of ten questions rated on a Likert-type scale where the participants selected from strongly agree to strongly disagree. These questions focused on participants’ preference and use of online assessment. Participants reflected whether online assessment improved their pre-service teaching performance. Participants were also asked if online assessment enhanced their learning experience and if it should be required for all classes.  To ascertain if more training is needed, participants were asked if they perceived instructors used online assessment effectively and if additional training was needed for faculty. To determine technology integration into curriculum planning, participants were asked if they plan to implement online assessment into their instruction after graduation. Participants were also asked if online assessment is an effective teaching method used in education courses.

Section two of the survey was left for general comments from the participants. The researcher was soliciting qualitative data that could be used to improve the use and effectiveness of online assessment at the university.  The last section allowed participants to provide some general demographic information.  (e.g. area of concentration, grade level, gender, age range). This information could be used to plan future online courses.

Data Collection

As the surveys were returned, the researcher sought potential themes from analyzing and sorting the data. The data was entered and analyzed using statistical software to complement descriptive data to provide student perceptions of online assessment. 

Participants

The sample included all students who were enrolled in a technology course at a four-year urban higher education institution in the fall 2009 term.  Students were informed the purpose of the study and procedures as well as the risks and benefits. Students who volunteered for the study were given a survey instrument. The survey was distributed to thirty-eight students. Twenty-three students (60%) agreed to participate.  Nineteen students were female and four were males. For grade level, fourteen identified themselves as graduate students, three were seniors, two were juniors, one was a sophomore and the remainder were freshman.

Data Analysis

The phenomenological approach was used to analyze the experiences of students as they reflected on their perception of online assessment. The researchers intent was to give students a voice to the online assessment process and gain an understanding of past experiences using online assessment. To ensure every student had the opportunity to participate in the research, students were reminded during a face-to-face session that they could still participate in the survey until the end of the term.

Results

Of the 38 students selected for the sample, a total of 23 participated.  This provided a final response rate of 60%.  Of the students who participated in the survey, eighty-two (82%) were female. This coincides with the demographics of the student enrollment of the class and the university population. In 2007, the university reported 72% of its population were female and only 28% were male.  Gender demographics of participants are shown in Table 1.

Table 1

Participant Gender Demographics

The data also revealed that the majority of the students were over 25 years of age. Out of the 23 students who responded to the question regarding age, 12 or 52 % reported they were between the ages of 31 and 40. This data also is reflected in the 2007 university report which indicated only 34% of the student population was between the ages of 22 and 29. Since this is a technology course, many students who enroll in this course are interested in learning how to integrate technology in a variety of majors. The majority of participants did not identify their area of concentration; however 21% identified their majors as elementary; 13% early childhood; 17% as career and technical education; and 8% as special education.  The area of concentration is shown below in Table 2.

Table 2

Age and Area of Concentration

Preference Using Online Assessment

Of the students who participated in the survey, an overwhelming ninety-five (95%) percent agreed they preferred some use of online assessment. Many students prefer the use of online assessment due to the fact they can receive immediate feedback from instructors and have the opportunity to work in an asynchronous learning environment.  The preference of online using online assessment is clearly shown in Table 3.

Table 3

 Preference using Online Assessment.

Participants also responded to whether online learning enhanced their learning experience.  Students enrolled in an online course have more flexibility, and are allowed to reflect on becoming more active participants in the learning process.  Students have the capability of using tools such as threaded discussions, e-mail, boards and chats. The majority of the students, ninety-one (91%) percent agreed online learning enhanced their learning experiences as shown in Table 4 below.

Table 4

Online Assessment Enhanced Learning Experience

Instructors’ usage of online assessment

While more instructors are revising their delivery methods to an online course platform, some instructors as well as students are having difficulty adapting to an online environment.  Of the twenty-three students who participated in the survey, over ninety-five (95%) percent perceived to some degree instructors who use online assessment, use it effectively. This could be attributed to the implementation of best practices being incorporated in online courses.  Instructors are encouraged to foster a climate of learning in a window without walls; and adapt instruction to allow every student the opportunity to express them and offer deeper discussion of topics. Students are encouraged to share experiences and suggestions related to issues being discussed. Instructors are also encouraged to develop social bonds online and switch their roles from content transmission to content acquisition while keeping students on task with relevant topics and promoting cooperative learning.  Participant responses of usage of online assessment is shown in Table 5.

Table 5

Instructors who use online assessment, use it effectively

Need of Online Instruction

While some pre-service teachers twenty-one (21%) do not agree additional instruction is needed using online assessment; a majority of the students seventy-nine (79%) perceive additional instruction is needed.  The complexity of taking an online course can be overwhelming. Students must be able to identify barriers prior to enrolling in an online course. One of the first barriers is that of technology itself. Many students enroll in online courses do so without proper preparation. This could confirm the results of additional instruction responses as shown in Table 6.

Table 6

Additional instruction is needed using online assessment

Usage of Online Assessment

Several survey questions explored the usage of online assessment. Participants in the study were asked if online assessment should be used in all classes; should all pre-service teachers experience it, and if it is an effective teaching method used in education courses. Over half of the participants, seventy-eight (78%) percent agreed online assessment should be used in all classes. All respondents, (100%) agreed all pre-service teachers should experience online assessment and it is an effective method used in education courses. A high number of respondents also agreed  online assessment is used effectively as illustrated in Table 4.

Table 7

Instructors who use online assessment, use it effectively

Future Implementation Using Online Assessment

The data suggest that over ninety-five (95%) percent of the respondents agreed to some extent  they plan on implementing online assessment into their instruction after they graduate. These factors suggest that future teachers are becoming aware of the latest technologies and pedagogical methods to enhance online learning.  Prior to graduation, pre-service teachers are obtaining a first-hand look at what online assessment entails. The impact of technology integration and online assessment is clearly identified in Table 8 shown below.

Table 8

I plan to implement online assessment into my instruction after I graduate

Discussion and Conclusion

The findings from this survey can be divided into three discrete yet equally important outcomes associated with teaching and learning.  The effects of these differing perceptions may explain why assessment and student outcomes go hand-in hand. Factors should be considered include both the pre-service teachers’ preference using online assessment and their perception of instructors using online assessment.  Insight was gained into pre-service teacher perceptions on how online assessment is used and how it impacts teaching and learning.

This study showed that there is a definite correlation between graduate course work and online instruction. The majority of the students participating in the survey were graduate students  that plan to implement online assessment into their instruction upon graduation.  This sets a roadmap for educational institutions to aggressively prepare for online instruction and assessment that is both flexible and authentic.

The first outcome relates to pre-service teacher preference using online assessment. Responses clearly show that the majority of the students prefer online assessment. The findings show it has enhanced their learning experiences and it has improved their pre-service teaching performance.   The findings also suggest that the majority of the students plan to implement online assessment into instruction after graduation.

The second outcome relates to instruction. While more educational institutions are developing new strategies to develop online curriculums and assessments, results show that more training is needed for the instructor.  Instructors are learning to assess themselves as learners as well as their students. “People who rely merely on a few learning strategies are likely to over-use them and apply them in inappropriate situations” (Roth, 1997, p. 2).

The third outcomes related to allowing the pre-service teachers have a voice in the online process. This section allowed respondents to make comments of online assessment and how it is used.  Two students provided narrative feedback on online assessment.

One respondent said, “All teachers at this university need training on how to use Live Text”.
A different respondent said, “Chicago Public Schools has started an on-line assessment called (Scantron).” The responses from these students clearly indicated they are aware of movement towards online assessment.

Limitations and Future Considerations

This study has two major limitations: first, a limited sample.  Data was obtained from a small percentage of students enrolled in a technology course in fall 2009.  Participation in surveys is contingent on participants’ interest in the subject matter. Participants may refuse to participate in surveys due to past experience with previous surveys. The use of a computer generated survey could have generated more responses and limited the amount of time of the respondents.  The second limitation involved setting. This study took place during an evening and weekend technology class.  Participants may feel they were overburdened and did not have sufficient time to complete survey.

When educational institutions elect to implement online learning, it is important to recognize the impact of online assessment.  This study suggest that graduate students, particularly prefer online assessment. It also suggests that pre-service teachers who are exposed to online assessment plan to implement in their future classrooms.  The findings of this study also suggest online assessment has enhanced their learning experience. A broader version of this study would include colleges and universities who offer online assessment exclusively for the graduate programs.   

References

Ahn, J. (2004). Electronic Portfolios: Blending Technology, Accountability & Assessment, T.H.E. Journal, 31:9: 12-18.

Allen, I.E. & Seaman, J. (2008). Online nation. Five years of Growth in Online Learning. Needham, MA: Sloan Consortium.

  Online education in the United States’,2010, Report from the Sloan Consortium, Retrieved December 28, 2010, from [Online], Available: http://sloanconsortium.org/publications/survey/pdf/class_differences.pdf

Anderson, Charles & Johnson (2003). The impressive psychology paper. Chicago:  Lucerne Publishing.

Grash, A.F. & Yangarber-Hicks, N. (2000). Integrating teaching styles and learning styles with instructional technology. College Teaching, 48 (1), 2.

Pecheone, R.L. and Chung R.R. (2006). Evidence in Teacher Education: The Performance Assessment for California Teachers (PACT), 57:1, 22-35.

Roth, G. (1997). Helping adults learn how to learn. The Korean Journal of Lifelong Education, 3(1), pp. 139-153

Quellmalz, E, Schank, P. Hinojosa, T. and Padilla C. (1999). Performance Assessment Links in Science (PALS), Practical Assessment Research & Evaluation. Retrieved February 23, 2011 from http://PAREonline.net/getvn.asp?v=6&n=10.

Woods. R. Ebesole S. (2007). Becoming a “Communal Architect” in the Online Classroom-Integrating Cognitive and Affective Learning for Maximum Effect in Web-Based Environment, Retrieved June 28, 2011 from http://www.westga.edu/~distance/ojdla/spring61/woods61.htm.

About the Author

Dr. Patrice C. Boyles is a teacher, workshop leader and academic advisor with over ten years’ experience in higher education. She has taught graduate courses in business education and technology education.  She has provided professional development training to educators and business professionals across the state. Patrice embraces technology and is highly aware of the impact it has on teaching and learning in the classroom. 

Email: pboyles@gmail.com

Appendix A: Survey Instrument

Pre-Service Teachers’ Attitudes towards Online Assessment:
An Initial Component of Teacher Training

Directions: Indicate your opinion on online assessment by circling the appropriate number on the scale of 5-1.

5-strongly Agree, 4---agree, 3---mildly agree, 2---disagree, 1---strongly disagree

Comments:

                                Please return form to Dr. Patrice C. Boyles, ED 203

Appendix B

Pre-Service Teachers’ Attitudes towards Online Assessment:
An Initial Component of Teacher Training

Directions:  Please complete the personal datasheet and return with the informed consent form

 _____________________________________________________________________________________

Personal Data Sheet

Please Circle Area of Concentration: 

Early Childhood             Elementary Education             Special Education

          Career and Technical Education               Other________________

Grade Level: _____________________

Gender: Female_________Male________

Age Range: 21-30_______, 31-40__________, 41-50______

go top