Editor’s Note: What works and what does not work is the basis of empirical research studies. By testing ideas in different places and contexts we find universal truths and others that are specific to defined content and populations. Sometimes the results are inconclusive. Comparative studies and basic research are providing a mass of data, some for application, and some for further research.

Critical Assessment of the impact of Web Technology on
Total Quality of Educational and Student Support Services from the Open and Distance Learning System

Manoj Killedar

India

Abstract

e-Learning on the World Wide Web is a globally distributed, highly personalized medium for cost-effective delivery of interactive multimedia information and services. The web is expected to have a strong impact on almost every aspect of how we learn. Total Quality is the totality of features perceived by customers of a product or service. Totality of features includes the stated and implied needs and expectations of all types of customers. Total Quality of the learning experience in an Open and Distance Education System (ODES) must be expressed in observable and measurable terms that include teaching and learning and a complete spectrum of support and educational services for students.

A model for open and distance education system was used to measure impact of the Web Technology on the Total Quality of the learning experience in an Open and Distance Education System. Evidence indicates that application of the proposed model can simultaneously optimize quality, access and cost. Thus, a quality learning experience can be provided in an open and distance education system which is comparable to the best in traditional education.

Keywords: open education, distance education, online education, elearning, total quality management, quality feedback

Introduction

Impact of any new technology cannot be fully explored unless system is totally redesigned to exploit the benefits of technology. The new design requires full understanding of all quality parameters, their relative effectiveness, and their importance in formulating the perception of total quality for each type of customer. This research hopes to provide this insight for the designers and developers of an Open and Distance Learning System.

Total Quality is the totality of features desired and perceived by the customers for a product or service. Totality of features includes stated as well as implied needs and expectations for all types of customers. No quality improvement is possible without unambiguous measurement. Total Quality of the learning experience in Open and Distance Education Systems must be expressed in observable and measurable terms which include a complete spectrum of student support and educational services. A model for Total Quality of an open and distance education system was used that provides clear guidelines for how to use and integrate various components of web technology to improve and optimize the quality of the learning experience.

Open and Distance Learning Systems are often perceived as a cheap and low quality alternatives to the traditional educational system. Rich multimedia capability, easy interaction and distributed access of web technology have the potential to reverse this perception. This research may help developers to explore appropriate systems and procedures, for world class, highly flexible and cost effective, virtual Open and Distance Learning System of the future.

Total Quality is a subjective term and it is perceived differently by everyone. A number of respondents were asked about their perception of Total Quality in order to measure the perception of the total group. Evidence indicates that application of the proposed model for Total Quality and Web Technology can simultaneously optimize quality, access and cost.

Review of Literature

The Internet is impacting every aspect of how we learn and how we communicate. Quality and efficiency of academic and administrative services can be significantly better when compared with present status. (Killedar M., 2001). It is flexibile so that teaching and learning can take place at any time and place convenient to both course instructors and participants. Online learning and teaching are definitely the future direction (Cheng & Myles, 2003). Open and distance learning (ODL) gives learners control of time, place, and pace of learning. It is often characterized as Flexible Learning. However, this flexibility goes hand‐in‐hand with the opportunity for procrastination and non‐completion. (Tattersall, Waterink, Höppener, & Koper, 2006).

Prior research shows that equivalent learning activities can be equally effective for online and face-to-face learners (Neuhauser, 2002). Results of this study demonstrate that students can learn equally well in either delivery format, regardless of learning style, provided the course is developed around adult learning theory and sound instructional guidelines (Aragon, Johnson, & Shaik, 2002). Another study conducted two experiments to assess effectiveness of interactive e-learning. Students in a fully interactive multimedia-based e-learning environment achieved better performance and higher levels of satisfaction than those in a traditional classroom and those in a less interactive e-learning environment (Zhang D., 2005).

Managing education effectively requires optimizing of tensions between three vectors of the Eternal Triangle of Education so that all three aspects, access, quality and cost, are improved simultaneously. Traditional methods of teaching and learning cannot produce the changes required because, if we put more students in each class, cost may go down, but quality will deteriorate. Conventional ways of improving quality tend to reduce access (class size) and raise costs. The challenge is clear. Can technology simultaneously increase access, improve quality and lower cost? The evidence shows that it can simultaneously optimize all three parameters (Daniel J. , 13 Nov 2003).

TQM is systematic way to guarantee that all activities within an organization happen as planned. It is a management tool that prevents or corrects problems at source, rather than allowing problems to occur and then correcting them afterwards. The essence of TQM is the simple but extremely powerful belief that it is better and hence cheaper, to do every process right at first time, rather than not to do it right and then correct it afterwards. Doing things right at first time requires no money. Doing things wrong is what only costs money. Thus, longer it takes to identify problem, more will be the cost incurred to correct it  (Eriksen, 1995)(Killedar M., 2007)

‘Any Where, Any Time’ access to formative feedback about Self-Study, allows the distance learner to concentrate his Self-Study precisely on those content areas where his/her understanding is weak. Immediate recognition of Self-Study achievements of a distance learner in comparison with other fellow students is a strong motivation for further Self-Study. (Killedar M., 2002).

Research Design

Aim

The main aim of the study was to measure the extent to which Web technology has influenced quality of education and student support services through ‘Open and Distant Learning System (ODLS)’.

Scope and Limitations of the Problem

This research focused on “Open and Distance Learning System” of India, with the specific example of electronics and mechanical engineering programmes, offered by the School of Science and Technology, Yashwantrao Chavan Maharashtra Open University (YCMOU), India.

Hypothesis

Assuming that the other factors are kept constant, it is hypothesized that,

• Communication with other ‘students and counselors at different study centers’ is significantly more easy through ‘Discussion Forum’ than through ‘Traditional Classroom’.

• It is significantly difficult to select questions of required difficulty in ‘Traditional Class-Test’ based continuous assessment’.

• Examinee could be evaluated for significantly better outcome by using adaptive algorithm, as high achievers get challenging exam and low achievers get simpler exam, which encourages high as well as low achievers to put more and better learning efforts.

• It is significantly difficult to use adaptive algorithm in ‘Traditional Class-Test’ based continuous assessment.

• Complete evaluation of the understanding of the discipline could not be done by ‘Self-Test Center’ based testing.

• ‘Self-Test Center’ could not measure the understanding of examinee about various terms and concepts.

• ‘Self-Test Center’ based testing could provide significantly faster and better feedback about learning to each examinee than in traditional examination.

• Use of computers for a long period of time hurts eyes in web technology based distance education system.

• Web technology based distance education system could not be used in each and every academic programme of the university.

• Basic training is required in workshop for ‘hands on experience’ to each study center staff and students, in web technology based distance education system.

• ‘Discussion Forum’ could significantly improve writing skills by providing easily accessible anonymous space for writing short assignments, which can be fully edited after receiving counselor’s feedback on it.

• Significantly more number of respondents will prefer to have ‘Discussion Forum’ based ‘Online Counseling’ in addition to regular face-to-face counseling offered at the study centers.

Sample of the Study

Sample selection was not a difficult task because the study is restricted to electronics and mechanical engineering programmes of the “Yashwantrao Chavan Maharashtra Open University Nashik”. Complete admission data was available with the researcher. During July–December 2004, total 2532 students were enrolled for the first batch, which includes 2305 male students and 227 female students. 2156 students (about 85.15% of total enrolled students) were from 4 major urban and industrialized regions of the Maharashtra state.

Both of these academic programmes were technology programmes and thus, it is natural that they attract maximum student enrolment in urban regions having good number of industries. This assumption was strongly supported by the fact that total 85.12 % enrolment was just in 4 regions, that is, Mumbai, Nagpur, Nashik and Pune. All these regions are mostly urban having much higher number of industries.

Hence, researcher decided to use “Purposive Sampling” of only these 4 regions (that is, Mumbai, Nagpur, Nashik and Pune), which are mostly urban with large number of industries as representative samples of the target student population for these academic programmes.

Hence, all students and counselors from all study centers from only these 4 regions were invited for providing their feedback about quality, during the face-to-face contact sessions. But, only 287 male students  (about 12.45% of the total 2305 male students during 2004) and 58 female students  (about 25.55% of the total 227 female students during 2004), 49 (18.99 % of total 258 Counselors during 2004) counselors, 21 distance education experts provided their feedback willingly. Thus, the total effective sample comprised of those 415 subjects only.

Questionnaire Design

It was decided to prepare a questionnaire following the guidelines given by Likert (1932). Considering variables under study, a scale was constructed and standardized by using psychometric techniques such as item analysis and reliability, and it was administered on the sample of the study.

The researcher was careful to phrase questions clearly and unambiguously so that the respondent has no doubt which answer to give. The researcher purposefully decided to use a four-point scale; which is a forced-choice method where the middle option of "Neither agree nor disagree" or “Undecided” is not available. This minimizes central tendency bias. The researcher carefully organized items in the questionnaire in an unpredictable mixture of positive and negative statements about the attitude object to minimize acquiescence bias.

Relationships of 60 items with major factors were examined and their relevance to the study objectives of was verified. It was found that 48 items best fitted with the 10 major quality factors, and 12 items were closely related to the objectives of study. It was difficult to categorize some items because the statements fitted more than one factor; they were retained because they had high relevance. Analysis of these 12 items was done separately; whereas 48 items were used for determining the reliability of the scale. Split half reliability of the scale was 0.81. Since, no other scale was available, validity could not be determined. The 48 items were given to 5 judges who were experts in this field and considering their opinion the face validity of the scale was finalized, it was 0.69. Because both reliability and validity were high, the scale could be used for collecting data. In the final form of the scale there were 60 items and each item was provided with 4 point scale, ranging from strongly agrees to strongly disagree.

Procedure for Data Collection

For collecting the data, the scale was transmitted in print form during face-to-face contact sessions with all the students and counselors from all study centers in four major geographic regions of the state of Maharashtra, where about 85.15 % students were enrolled. Along with the instructions, the subjects were provided with one or two examples of how to put a tick mark on the answer they have selected. Sufficient time was given to respond to the scale.

Variables under Study

The four variables - male students, female students, counselors, and distant education experts - were associated with the respondent’s category. Factors that were treated as dependent variables were as follows: (1) effectiveness in learning process, (2) economy regarding time and money, (3) usefulness in understanding, (4) acquisition of more knowledge, (5) organized approaches, (6) easy access to communication, (7) objectivity, quality and standard, (8) effective examination system, (9) human appeal and (10) attractiveness and interesting. In addition, there were 12 items that were treated independently because they were related to more than one factor. Analysis of only these 12 independent items is the topic of this research paper.

Research Design

A multifactor survey process was used in which between the groups research design was employed. In fact, in one way, this is a kind of attitude survey in which both students and other authorities are involved.

Statistical Treatment of Data

Those statements which were treated independently were treated by frequency counts, percentages, and Chi Square Test of significance. But when each dependent factor of scale was treated then parametric statistical technique was used. Means and standard deviations were computed, One Way Analysis of Variance was used, and finally Duncan's New Multiple Range Test was employed.

Analysis and Interpretation of Data

The study was designed to examine the impact of web technology on total quality of educational and students services in open and distance education system. In such topics it is necessary to have a reference point and in present study, the reference point was conventional educational system. During the last decades through web technology it is possible to impart knowledge and education. It is believed that the total quality could be better through web technology in education system.

The following 12 statements which were treated independently were statistically analyzed by using Chi Square Technique. Since, four different groups of subjects were used the appropriate Chi Square Technique was employed. Summary of the responses is as shown below in graphical format. Detail tables of responses were not included in this research paper only due to space and length constraint.

Discussions

1. The first hypothesis of the study was “Communication with other ‘students and counselors at different study centers’ is significantly more easy through ‘Discussion Forum’ than through ‘Traditional Classroom’.” As significant majority of groups agree with this, there is enough evidence to strongly support this hypothesis.

2. The second hypothesis of the study was “‘It is significantly difficult to select questions of required difficulty in ‘Traditional Class-Test’ based continuous assessment’.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

3. The third hypothesis of the study was “Examinee could be evaluated for significantly better outcome by using adaptive algorithm, as high achievers get challenging exam and low achievers get simpler exam, which encourages high as well as low achievers to put more and better learning efforts”. As significant majority of groups agree with this, there is enough evidence to strongly support this hypothesis.

4. The forth hypothesis of the study was “‘It is significantly difficult to use adaptive algorithm in ‘Traditional Class-Test’ based continuous assessment.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

5. The fifth hypothesis of the study was “‘Complete evaluation of the understanding of the discipline cannot be done by ‘Self-Test Center’ based testing” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

6. The sixth hypothesis of the study was “Self-Test Center’ cannot measure the understanding of examinee about various terms and concepts” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

7. The seventh hypothesis of the study was “‘Self-Test Center’ based testing could provide significantly faster and better feedback about learning to each examinee.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

8. The eight hypothesis of the study was “Use of computers for a long period of time hurts eyes in web technology based distance education system.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

9. The ninth hypothesis of the study was “Web technology based distance education system could not be used in each and every academic programme of the university.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

10. The tenth hypothesis of the study was “‘Basic training is required in workshop for ‘hands on experience’ to each study center staff and students, in web technology based distance education system.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

11. The eleventh hypothesis of the study was “‘Discussion Forum’ could significantly improve writing skills by providing easily accessible anonymous space for writing short assignments, which can be fully edited after receiving counselor’s feedback on it.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

12. The twelfth hypothesis of the study was “Significantly more number of respondents will prefer to have ‘Discussion Forum’ based ‘Online Counseling’ in addition to regular face-to-face counseling offered at the study centers.” Groups differ significantly from each other and thus, evidence does not support this hypothesis.

Web technology is useful in many ways and it should be introduced in most of the fields of knowledge. However, in a country like India it is not possible to rely totally on the web technology, both traditional classroom techniques and virtual classroom must function in collaboration with each other.

Conclusions

On the basis of the result of the study, regarding 12 independent quality factors, the following conclusions were drawn:

1.      Compared to traditional classroom, discussion forum was reported as more effective for direct access to communication by significantly large number of respondents.

2.  The four groups of subjects failed to agree upon the assertion that selecting questions of required difficulty in traditional class-test is difficult.

3.  Adaptive algorithm could be used more efficiently and effectively with web technology than with traditional class-tests.

4.  Although it is possible to measure the understanding of various concepts, significantly large number of subjects believed that evaluation of understanding of the discipline is not possible in self-test based testing.

5.  Immediate feedback could be provided to the examinee through self-test center based testing and not through traditional examination.

6.  Significantly large number of subjects believed that exposure to computer for longer period hurt eyes.

7.  Imparting all kinds of education through web technology could be done, but basic training about it must be provided to them.

8.  Four classified groups of subjects failed to agree upon the point that discussion forums provide an easily accessible anonymous space to develop writing skills.

9.  Four classified groups of subjects failed to agree upon the assertion that counseling through web technology should be used in addition to traditional face to face counseling.

Suggestions for Future Research

On the basis of results of the study the following suggestions are made:

1.  In order to make ‘Virtual Classroom’ approach more acceptable among students and teachers, extra efforts are necessary. Future research can indicate appropriate strategic approaches regarding this.

2.  Relevance and utility of ‘Virtual Classroom’ approach need further investigation in rural area with severe shortage of electrical power and other disciplines like social sciences or humanities. There is a need of more research in these areas.

References

Aragon, S. R., Johnson, S. D., & Shaik, N. (2002). The Influence of Learning Style Preferences on Student Success in Online Versus Face-to-Face Environments. American Journal of Distance Education , 16 (4), 227-244.

Cheng, L., & Myles, J. (2003). Managing the Change From On-Site to Online: transforming ESL courses for teachers. Open Learning , 18 (1), 29-38.

Daniel, J. (13 Nov 2003). Education For All and The Role of Open and Diatance Learning, The Global Scenario. Food, Health and Education For All: The Role of Diatance Learning. Nashik, India: Yashwantrao Chavan Maharashtra Open University.

Eriksen, S. D. (1995). TQM and the transformation from an elite to a mass system of higher education in the UK. Quality Assurance in Education , 3 (1), 14-29.

Killedar, M. (2001). Distance Education through Internet Based ELearning. Indian Journal of Open Learning (IJOL) , 10 (1), 68-79.

Killedar, M. (2007). Model for ‘Total Quality’ of the Open and Distance Education System. International Journal of Instructional Technology and Distance Learning , 4 (7), 51-68.

Killedar, M. (2002). Online Self-Tests: A Powerful Tool for Self-Study. Indian Journal of Open Learning (IJOL) , 11 (1), 135-146.

Neuhauser, C. (2002). Learning Style and Effectiveness of Online and Face-to-Face Instruction. American Journal of Distance Education , 16 (2), 99-113.

Tattersall, C., Waterink, W., Höppener, P., & Koper, R. (2006). A Case Study in the Measurement of Educational Efficiency in Open and Distance Learning. Distance Education , 27 (3), 391-404.

Zhang, D. (2005). Interactive Multimedia-Based E-Learning: A Study of Effectiveness. American Journal of Distance Education , 19 (3), 149-162.

Acknowledgments

The researcher wishes to express deep gratitude and sincerely acknowledge the detailed guidance, motivation, help and support, received from
Dr Pandit Palande for the present research study.

About the Author

Mr Manoj Killedar has, since 1993, been Director, School of Science and Technology at Yashwantrao Chavan Maharashtra Open University, Nashik, India. He completed graduate work in “Electric Drives and Machines” from the Indian Institute of Technology Mumbai, in 1984. His 23 years of professional experience includes 8 years in industry and 15 years in distance education. His present research interests are eLearning and its applications in Open and Distance education systems of India.

Email: manoj.killedar@gmail.com