| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Parameters of |
Profession | n |
State Education agency staff | 10 |
Professors | 6 |
Principals | 6 |
Teachers | 14 |
OSEP | 3 |
Professional Development Specialists | 8 |
Regional Resource Center staff, and others | 7 |
Total | 54 |
Each group received the same instructions i.e., to identify the conditions essential to successful implementation of large scale online staff development programs. They were asked to select a facilitator and a recorder. Prior to the large group reporting sessions each group was alerted to the amount of time remaining prior to reporting. They were also asked to edit their flip chart notes before reporting. Reports were made verbally to the group with time allowed for questioning to ensure clarification of statements. Two eDL staff members took notes during the discussions. The flip chart reports were collected as archival data and the recorder agreed to be available following the retreat to respond to questions from the staff to ensure accuracy in compiling the notes. The same process was repeated with the focus groups being reconfigured by state teams in contrast to the organization by roles in the first focus group process.
Reporting sessions were held following each set of focus groups. The reporting sessions were facilitated by an eDL staff member. These were extensive sessions with the goal being to ensure that each suggested condition was clearly communicated and that there was consensus that the condition should be retained in the inventory.
Agreement was reached on the items in the inventory of conditions they were reviewed to identify categories in which they appeared to cluster. The results of the reporting sessions at the conclusion of the two focus groups sessions during the first retreat was an inventory of statements describing conditions important to successful implantation of online staff development that were grouped by categories.
Within a week following the retreat the notes from the two focus group sessions were edited as statements describing the conditions identified by the Focus Groups. Each item was edited to achieve some consistency in format and structure of the wording. The intent of the statements was not changed nor were additional ones added. Redundancies were eliminated and the sorting by categories was reviewed to assure reasonable independence of each category.
The inventory of statements, clustered by the original categories, was posted on the Supplemental Project web site. Participants from the Focus Groups were asked to review the items and categories. They were also allowed to add items. Submit forms for responses were included one the web site.
The final instrument consisted of 115 items divided into six categories. Following is a list of six categories.
Conditions that influence how online staff development might be delivered.
Requirements to be met by online delivery models.
Way to engage teachers.
Ways to sustain teachers’ involvement.
Ways to manage online staff development.
Ideas for implementation of online staff development
While the category topics overlapped, it was decided that they were sufficiently independent to serve as a framework. Besides, it was important to ensure comprehensive coverage of the individual conditions that the group judged to be important.
A submit form was developed and placed on the web site, and participants were notified by email that they were invited to respond to the instrument. The instrument was preceded by a brief review of the discussions at the first retreat. Respondents were asked to provide demographic information and to rank each item on a 5-point scale of importance, ranging for responses from Not Important to Very High Importance.
The results of the responses were summarized and presented at the second planning retreat. The focus of this discussion was on how the preliminary results of the responses to the instrument informed the process of framing recommendations for the overall project recommendations on deliver models for the implementation of large scale staff development projects. (Meyen, 2003) A decision was made to do a follow-up to allow participants who had not responded to respond to the instrument. A second copy of the survey instrument was posted and those who had not responded the first time were asked to respond.
Focus groups followed by a total-group discussion session took place at both retreats. However, the first retreat was most productive in generating statements. In the second session the pilot results were reviewed and minor changes were made in items within the six categories. The composition of the two groups was almost identical since the membership of the teams did not change, but some participants were unable to attend the second session. Besides, some participants in the second session were new to the process. Fifty-four participated in the first session and thirty-four in the second session.
The most significant results of the study were derived from the focus groups that generated the items. The ranking process served to confirm the importance of the statements and to rank them in relative importance. While respondents differentiated among statements of high importance and low importance, no item received a mean score of less than 3.52. Table 2 provides a comparison of the mean scores based on rankings of 4.5 and higher as 4.0 and higher across the six categories. The overall mean score across the 113 items was 4.22. Only 20 items received mean scores below 4.0.
Category | # of Items | Mean Scores 4.5+ | Mean Scores 4.0+ |
Range |
1.1 Conditions | 32 | 9 | 27 | 4.76-3.71 |
1.2 Requirements | 12 | 3 | 10 | 4.76-3.71 |
1.3 Teacher Engagement | 25 | 0 | 19 | 4.48-3.52 |
1.4 Sustaining Involvement | 14 | 1 | 11 | 4.52-3.67 |
1.5 Module Management | 10 | 1 | 7 | 4.86-3.67 |
1.6 Implementation | 22 | 4 | 21 | 4.86-4.00 |
TOTAL | 115 | 18 | 95 |
|
Table 3 contains the mean scores and ranking of each item within the six categories. The number in the left column identifies the number of each item as it appeared in the instrument. Items with the same mean score have been assigned the same ranking; thus there are duplicate rankings in each category.
The lessons learned were directly influenced by the intense participation of the state team members in the focus and large-group sessions as described earlier. Their vested interest in the process derived from the high probability that each of them would ultimately have some responsibility for implementing online staff development through their professional roles.
Involving individuals with different professional responsibilities in discussions of online staff development creates an environment that is productive in identifying realistic conditions for effective implementation.
Engaging planning participants in the beta testing of online instruction results in more substantive contributions to planning for subsequent implementation of online staff development.
Once the conditions are described that influence online instruction, educational representatives with different professional roles can reach consensus on the relative importance of each of the specific factors that influence successful implementation of online staff development.
Online staff development, because of easy access and convenience for practicing professional as well as its unique capacity to maintain currency of content, is perceived as an important option in the delivery of staff development.
Item | Items Category | Mean | Rank | ||||
1.1 | Conditions that influence how online staff development might be delivered |
|
| ||||
1.1.1 | Need to access connectivity. | 4.67 | 3 | ||||
1.1.2 | Incentives appropriate to the demands on teachers should be available. | 4.33 | 8 | ||||
1.1.3 | Administrative support essential. | 4.62 | 4 | ||||
1.1.4 | Commitment from the staff development leadership. | 4.29 | 9 | ||||
1.1.5 | Teachers should be given option in selecting the programs they complete. | 4.43 | 7 | ||||
1.1.6 | CEUs and college credit important. | 4.29 | 9 | ||||
1.1.7 | Easy access to technical assistance. | 4.76 | 1 | ||||
1.1.8 | Clear communication on the instruction to be offered and expected outcomes. | 4.57 | 5 | ||||
1.1.9 | System for reporting and managing records of participating teachers. | 4.19 | 10 | ||||
1.1.10 | Effective program of marketing the staff development to teachers. | 4.33 | 8 | ||||
1.1.11 | Established system of cohorts or study groups to support teacher participation. | 3.76 | 15 | ||||
1.1.12 | Minimize the technical skills required to participate in the online staff development. | 4.10 | 11 | ||||
1.1.13 | Reinforcement for an online culture of staff development. | 3.76 | 15 | ||||
1.1.14 | Alternative modes, e.g. online and CD formats. | 3.95 | 13 | ||||
1.1.15 | If fees are passed on to teachers, fees need to be very affordable. | 4.62 | 4 | ||||
1.1.16 | If a licensing fee is employed, it needs to take into consideration the current decline in fiscal support for education. | 4.71 | 2 | ||||
1.1.17 | System in place to provide immediate feedback to teachers. | 4.52 | 6 | ||||
1.1.18 | Feedback on assessment should be immediate and instructional. | 4.62 | 4 | ||||
1.1.19 | Give high visibility to the program in an attempt to establish acceptance of the approach. | 4.29 | 9 | ||||
1.1.19 | Give high visibility to the program in an attempt to establish acceptance of the approach. | 4.29 | 9 | ||||
1.1.20 | Appropriate support at the local, regional, and state levels | 4.57 | 5 | ||||
1.1.21 | Establish relationships with institutions of higher education to facilitate the integration of staff development with institutional offerings. | 4.43 | 7 | ||||
1.1.22 | Resources related to the staff development topics should be accessible. | 4.10 | 11 | ||||
1.1.23 | Licensing requirements should be comprehensive and clearly stated. | 4.05 | 12 | ||||
1.1.24 | Staff development on how to participate in online staff development should be available. | 4.05 | 12 | ||||
1.1.25 | Staff development should be tied to application and ongoing systems of support, e.g. peer coaching, mentoring, etc. | 4.33 | 8 | ||||
1.1.26 | An evaluation system to evaluate the program needs to be in place. | 4.33 | 8 | ||||
1.1.27 | Teachers need to participate in selection of online staff development offerings. | 4.29 | 9 | ||||
1.1.28 | There should be a relationship to certification. | 4.10 | 11 | ||||
1.1.29 | Relate to salary increments. | 3.71 | 16 | ||||
1.1.30 | Engagement of staff development leadership locally. | 4.10 | 11 | ||||
1.1.31 | There should be a capacity in place to maintain currency of offerings and to expand offerings. | 4.05 | 12 | ||||
1.1.32 | The infrastructure should not only be stable but should be perceived as a permanent arrangement. | 3.90 | 14 | ||||
Item | Items Category | Mean | Rank | ||||
1.2 | Requirements online delivery models should meet |
|
| ||||
1.2.1 | A system of quality control needs to be in place to ensure all programs are of high quality. | 4.76 | 1 | ||||
1.2.2 | A strong accountability system to monitor all aspects of the program and to convene the value placed on participation. | 4.48 | 3 | ||||
1.2.3 | When feasible, instructors should participate in the online staff development. | 4.05 | 6 | ||||
1.2.4 | The technical infrastructure should be stable resulting in a minimum of special requirements for participation. | 4.29 | 5 | ||||
1.2.5 | The online staff development program should be compatible with available resources for chats, threaded discussions, FAQs, forums, etc. | 4.05 | 6 | ||||
1.2.6 | The fiscal commitment should be in place prior to implementation to minimize having to curtail offerings once operational. | 4.43 | 4 | ||||
1.2.7 | There should be a vision as to how the program may evolve in the future. | 4.29 | 5 | ||||
1.2.8 | Policies governing the roles of teachers need to take into consideration the flexibility online staff development offers teachers, e.g., if they complete online staff development they should be excused from the scheduled staff development assignments. | 3.90 | 8 | ||||
1.2.9 | A plan should be in place as to how offerings will be expanded. | 4.00 | 7 | ||||
1.2.10 | Establish a practice of ensuring that all offerings will be expanded. | 4.76 | 1 | ||||
1.2.11 | The offerings should be available 24/7/365. | 4.62 | 2 | ||||
1.2.12 | When appropriate content should apply to all teachers. | 3.76 | 9 | ||||
Item | Items Category | Mean | Rank | ||||
1.3 | Models for engaging teachers |
|
| ||||
1.3.1 | Tie offerings to national standards such as the CEC standards. | 4.48 | 1 | ||||
1.3.2 | Broaden focus to attract the interest of teachers in general education. | 4.43 | 2 | ||||
1.3.3 | Offer in-depth offerings not just introductory topics. | 4.29 | 5 | ||||
1.3.4 | Create a career ladder that allows a teacher to gain recognition for completing offerings that enhance their expertise, e.g., a certificate might be awarded for completing a series of offerings on a topic that includes advanced knowledge. | 4.10 | 9 | ||||
1.3.5 | Employment of cohort or study groups as a way of building community around topics of mutual interest. | 3.95 | 11 | ||||
1.3.6 | Create a mechanism that facilitates teachers in deciding which offering or part of an offering will add to their knowledge and skill base. | 4.05 | 10 | ||||
1.3.7 | Involvement of principal in a leadership role that clearly conveys value placed on participation. | 4.38 | 3 | ||||
1.3.8 | Obtain endorsement of professional associations for the online offerings and where appropriate related the topics to the agenda of association. | 3.95 | 11 | ||||
1.3.9 | Provide resources that enhance their applying what they learn. | 4.33 | 4 | ||||
1.3.10 | Relate incentives to successful completion of online offerings. | 4.33 | 4 | ||||
1.3.11 | Create opportunities for teachers to contribute to the development of online staff development offerings. | 4.24 | 6 | ||||
1.3.12 | Establish a portfolio system that allows teachers to manage a dossier of what they have learned via the online staff development offerings. | 3.76 | 13 | ||||
1.3.13 | Establish relationships with IHEs that results in consideration of credit for the work done by teachers. | 4.33 | 4 | ||||
1.3.14 | Coordinate offerings where appropriate with National Board Certification. | 4.29 | 5 | ||||
1.3.15 | Build a comprehensive set of offerings to maximize the probabilities of all teachers being able to match offerings with personal needs. | 4.24 | 6 | ||||
1.3.16 | Provide strategies for teachers to focus on specific instruction for remediation without having to complete an entire course. | 4.33 | 4 | ||||
1.3.17 | Provide released time for teachers in groups to construct their own staff development program from the online resources made available. | 3.71 | 14 | ||||
1.3.18 | Teachers should have opportunities to influence the availability of topics. | 4.10 | 9 | ||||
1.3.19 | Principals should fulfill a leadership role in promoting online staff development and in ensuring the flexible features are exercised. | 4.05 | 10 | ||||
1.3.20 | Engage faculty from nearby IHEs in the staff development. | 4.14 | 8 | ||||
1.3.21 | Include pre and post test assessments. | 4.19 | 7 | ||||
1.3.22 | Maximize flexibility features. | 4.29 | 5 | ||||
1.3.23 | Participation in the program should result in training roles for teachers. | 3.52 | 15 | ||||
1.3.24 | The role of the principal should be central to implementation and maintenance of the program. | 3.86 | 12 | ||||
1.3.25 | Offerings should relate to the priorities of the SIG. | 4.05 | 10 | ||||
Item | Items Category | Mean | Rank | ||||
1.4 | Models for sustaining teacher involvement |
|
| ||||
1.4.1 | Released timed for mentors. | 4.14 | 6 | ||||
1.4.2 | Mechanisms for teacher to manage their progress. | 4.14 | 6 | ||||
1.4.3 | Incentives tied to completion. | 4.33 | 3 | ||||
1.4.4 | Professional culture that values online staff development. | 4.19 | 5 | ||||
1.4.5 | Public acknowledgement of continued professional development. | 4.10 | 7 | ||||
1.4.6 | Opportunities to pursue "elective" opportunities. | 4.14 | 6 | ||||
1.4.7 | Recognition for becoming expert in a staff development topic. | 3.95 | 8 | ||||
1.4.8 | Indicators for relating professional growth to student outcomes. | 4.38 | 2 | ||||
1.4.9 | Opportunities to develop online staff development. | 3.67 | 10 | ||||
1.4.10 | Clear evidence that offerings will be expanded and system sustained. | 3.90 | 9 | ||||
1.4.11 | Application of successful staff development to degrees/ certification/salary increments. | 4.52 | 1 | ||||
1.4.12 | Link to other sources of information important to teachers. | 4.10 | 7 | ||||
1.4.13 | Relate to school improvement plans. | 4.38 | 2 | ||||
1.4.14 | Participation of teachers in planning and refining the online staff development program. | 4.29 | 4 | ||||
Item | Items Category | Mean | Rank | ||||
1.5 | Models for managing online staff development |
|
| ||||
1.5.1 | Establish policies governing the management on online staff development to enhance continuity and to communicate value. | 4.05 | 4 | ||||
1.5.2 | Engage broad participation in designing a management system that tracks the progress of all teachers. | 3.90 | 7 | ||||
1.5.3 | Integrate the communication of teacher progress to appropriate administrators and policy makers into the management system. | 3.95 | 6 | ||||
1.5.4 | Coordinate the features of the management system with others that may be available at the local, regional or state levels. | 4.00 | 5 | ||||
1.5.5 | Tie the management system to individualized portfolios or professional development plans that may be operational at the local or state levels. | 4.00 | 5 | ||||
1.5.6 | Centralize responsibility for the maintenance of the management system. | 3.67 | 8 | ||||
1.5.7 | Teachers should have access to the management system to monitor the accuracy of the system in recording their progress. | 4.24 | 3 | ||||
1.5.8 | The entry of data into the system should be user friendly and when possible automatic as a result of completing an offering. | 4.33 | 2 | ||||
1.5.9 | The management system should be in place as part of the infrastructure prior to implementation. | 4.33 | 2 | ||||
1.5.10 | The system should be affordable in terms of fiscal costs and labor requirements to ensure its sustainability. | 4.86 | 1 | ||||
Item | Items Category | Mean | Rank | ||||
1.6 | Ideas for implementation of online staff development |
|
| ||||
1.6.1 | Open participation to all teachers. | 4.86 | 1 | ||||
1.6.2 | Initiate a communications (public relations) process targeted attracting the interests of teachers. | 4.48 | 4 | ||||
1.6.3 | Phase in the offerings with topics that have the highest probability of being successful. | 4.38 | 6 | ||||
1.6.4 | Present the program as being of high priority to the district and the state. | 4.29 | 8 | ||||
1.6.5 | Obtain endorsement and affiliation is desirable with related professional associations. | 4.43 | 5 | ||||
1.6.6 | Do not start until infrastructure is in place. | 4.29 | 8 | ||||
1.6.7 | Establish communities of practice for those completing offerings. | 4.19 | 10 | ||||
1.6.8 | Have incentives in place along with credit options. | 4.52 | 3 | ||||
1.6.9 | Integrate with professional development or school improvement plans that the SEA or LEA may have in place. | 4.33 | 7 | ||||
1.6.10 | Operate a very visible feedback system to retrieve information from users on how to improve the offerings. | 4.33 | 7 | ||||
1.6.11 | If part of a statewide or regional system a monitoring process needs to be in place to ensure accountability and quality control. | 4.19 | 10 | ||||
1.6.12 | Create options for online teaching that provide immediate feedback to teachers. | 4.05 | 11 | ||||
1.6.13 | Create and value cohort approaches to participating in staff development. | 4.00 | 12 | ||||
1.6.14 | Provide options such a chats, threaded discussions and forums for those enrolled in similar offerings. | 4.00 | 12 | ||||
1.6.15 | Establish a user friendly enrollment process. | 4.57 | 2 | ||||
1.6.16 | Ensure that a management system is operational that maintains records of teacher progress and communicates that progress to appropriate administrators and policymakers. | 4.24 | 9 | ||||
1.6.17 | Ensure that a management system is operational that allows teachers to manage their own progress and build a portfolio that can be integrated with other professional achievements. | 4.52 | 3 | ||||
1.6.18 | Establish policies that allow teachers to meet their staff development obligations online without having to also participate in scheduled activities. | 3.95 | 13 | ||||
1.6.19 | Conduct a marketing strategy that engages a variety of spokespersons that are supportive of online staff development. | 4.19 | 10 | ||||
1.6.20 | Generalize the same value placed on other forms of staff development to online staff development. | 4.19 | 10 | ||||
1.6.21 | Establish partnerships with IHEs. | 4.33 | 7 | ||||
1.6.22 | Involve Regional Resource Centers. | 4.24 | 9 | ||||
This study was part of a planning project to develop guidelines for the implementation of large scale online staff development programs. Fifty-four educators from nine states participated in the planning process. The study was designed to help inform the planning decisions. Two additional studies were conducted for the same purpose, but not reported in this paper. They addressed the beta testing of online staff development modules and the identification of barriers to the implementation of large scale online staff development programs. The conditions/parameters study was conducted prior to the development of implementation recommendations by the planning group. The focus of this study was to identify conditions essential to successful implementation of large scale online staff development programs. Focus groups were utilized to develop statements describing conditions and circumstances that need to be in place to enhance successful implementation of online staff development. Once consensus was reached on the statements they were placed in categories and embedded in an instrument and a ranking process employed to determine their relative importance.
Guskey, T. (2003). What makes professional development effective. Phi Delta Kappan, 84(10), 748-750.
Hirsh, S. (2001). We’re growing and changing. Journal of Staff Development, 22(3), 10-17.
Killion, J. (2000). Log on to learn: To reap the benefits of online staff development, ask the right questions. Journal of Staff Development, 21(3), 48-53.
Killion, J. (2002). Loading the e-learning shopping cart. Journal of Staff Development, 23(1), 12-16.
Meyen, E.L. (2002). Final Report: The Online Academy – Office of Special Education Programs, U.S. Department of Education. Lawrence, Kansas: Center for Research on Learning – University of Kansas.
Meyen, E.L. (2003). Final Report: Online Delivery Model Project – Office of Special Education Programs, U.S. Department of Education. Lawrence, Kansas: Center for Research on Learning – University of Kansas.
Meyen, E.L., Aust, R.J., Bui, Y.N., Ramp, E., & Smith, S.J. (2002). The online academy formative evaluation approach to evaluating online instruction. The Internet and Higher Education, 5, 89-108.
Meyen, E.L., Ramp, E., Harrod, C., & Bui, Y.N. (in press). A national assessment of staff development needs related to the education of students with disabilities. Focus on Exceptional Children.
National Staff Development Council and National Institute for Community Innovations (2001). E-learning for educators: Implementing the standards for staff development. Retrieved January 30, 2003 from www.nsdc.org/e-learning.pdf
Reilly, R. (2002). Barriers to reengineering learning environments. Multimedia School, 9(6), 62, 64.
Richardson, J. (2001). Online staff development has great possibilities and pitfalls. Results. Retrieved July 10, 2003, from http://www.nsdc.org/library/results/res9-01rich.html
Strother, J.B. (2002). An assessment of the effectiveness of e-learning in corporate training programs. International Review of Research in Open and Distance Learning, 3 (1). Retrieved July 10, 2003, from http://www.irrodl.org/content/v3.1/strother.html
Treacy, B., Kleiman, G., & Peterson, K. (2002). Successful online professional development. Learning and Leading with Technology, 30(1), 42-47.
Wolinsky, A. (1999). What works in staff development. Multimedia Schools, 6(2), 36-40.
Edward L. Meyen, Ph.D., is Co-Director of the e-Learning Design Lab and Budig Teaching Professor for the Department of Special Education at the University of Kansas. Contact Dr. Meyen at phone: 785-864-0675 email: meyen@ku.edu
Chien-Hui Yang is a Ph.D. student in early childhood special education and a graduate research assistant with the e-Learning Design Lab at the University of Kansas.
The authors can be contacted at:
The University of Kansas, e-Learning Design Lab
3061 Dole Human Development Center
1000 Sunnyside Drive, Lawrence, Kansas 66045