Editor’s Note: Relating perception theory and practice to learning provides additional keys to effective design of web-based lessons. This research provides specific data on English and Turkish websites for consideration by the reader.

An Evaluation of Educational Web-Sites from the Perspective of Perception-Oriented Design Principles

Suzan Duygu Eristi, Ozden Sahin-Izmirli, Serkan Izmirli, Mehmet Firat

Turkey

Abstract

The research was conducted in two steps by means of evaluating educational websites as to visual perception theories. The first step involved developing an evaluation form in concert with designated visual perception theories. These theories are namely as; Gestalt Theory, Brunswik’s Probabilistic Functionalism, Marr’s Computational Model of Vision, Neuropsychological Theory, Constructivist Visual Perception Theory, Theory of Direct Perception and Ecological Optics. The second step involved assessments of websites with educational contents by the use of that evaluation form. The second step saw the analysis of 10 educational websites for primary school level, with the use of “Web based Evaluation Form for Educational Contexts”, 5 of which were developed in Turkish and another 5 in English. Assessment of the websites was carried out with the participation of 16 field experts. Research exposed that analyzed websites had major deficiencies in terms of their conformity to design principles. However, English websites received higher scores from design principles as compared to Turkish sites. The research yielded some critical conclusions through assessment of design principles employed up in the analyzed websites as to selected dimensions, i.e. page layout, color, typography, visuals, animated graphics, navigation and browsing, and content and functionality.

Keywords: perception, visual perception, web design, theories of visual perception.

Introduction

We acquire all knowledge pertaining to events, phenomena and concepts through perceptions. Perceptions are created through sensations and they are basically processes of knowledge attainment. Perceptual process is composed of sensing and recognizing of a stimulus (Demirel and Ün, 1987). It is about the actual environment that literally surrounds the individual and the virtual environment that is called the “socio-cultural environment” formed by information coming from the past and new supplementary information continuously added thereupon (Erinç, 1998). The interplays between the two environments and their manipulations by the individual is what causes perception to take place.

Despite being a widely used concept, primarily in psychology and educational sciences and to a lesser degree in many different others, visual perception lacks a strong research and theoretical ground. Visual perception is the sensory awareness of a perceived concept and selecting, organizing and describing of each particular stimuli holistically as parts of that concept by means of understanding its visual features (Behrens, 1984). Individuals demonstrate certain selectivity through perceptual process. At this point, personal interests, attention and characteristics play a critical role.

The quality of the content, which is intended to create a specific visually attained perception, as well as developing a successful transmission system with an effective message system, eventually enhances the quality of educational designwork. If the visual perception research is to be based on target community’s characteristics, then it may allow the receptor to better use the design since it would resolve the perception-related questions. During the design process, utilizing most significant features of concepts, objects or events, which are intended to create perception, as access points is vital for effectiveness in perception (Smeulders, Worring, Santini, Gupta and Jain, 2000). Perceptions pertaining to visuals are determined by a set of factors. These are visual features that can be placed in three groups, i.e. color, texture and shape, and motion (Eidenberger, 2004).

Quality features related to perception and specifically visual perception have to be considered in many design settings, with a particular emphasis in educational settings and design work. Perceptual characteristics may vary among the individuals, nevertheless from a broader view it can also be suggested that perception has several requirements.

Visual perception determines many design settings and quality aspects, especially for those that are visual content-intensive. One of those environments is educational design. Quality of educational design work is commensurate with how much it meets the perceptual expectations of the target community. In this juncture, the matter of how educational designs’ perceptual quality is produced and developed gains a special importance. In educational design, especially the very issue of what type of qualities it has to possess and how much it conforms to perceptual criteria is highly neglected albeit for a few fundamental design principles. Focusing on perception’s cognitive structure and its reflections on process of educational design is in fact fundamentally essential to produce high quality educational designs.

Theories of Visual Perception and Design Principles

The visual perception theories chosen to develop an evaluation form to assess the educational websites are ones that have been picked up from the perspective of available theories that are can make a direct impact on developing a new design in perceptual process. These theories are: Gestalt Theory, Brunswik’s Probablistic Functionalism, Marr’s Computational Model of Vision, Neuropsychological Theory, Constructivist visual perception theory, and Theory of Direct Perception and Ecological Optics.

Gestalt has been the first of general approaches to perception. It has left a substantial mark on perceptual psychology and it is referred to by a large number of researchers. The Gestalt approach is distinguished with principles of figure-ground relationship, proximity, continuity, similarity and closure. The Probabilistic Functionalism approach, based on the work of Egon Brunswik (1903-1955), advocates that perception of objects depends on intrinsic characteristics of those very objects, the physical context the objects are in, the intermediary between the object and the perceiver, the perceiver him/herself and his/her own characteristics. According to Marr’s Computational Approach, perception can be explained by such context whereof it takes place, and integrating, interlinking and reinterpreting of features pertaining to the perceived concept (Gordon, 2004).

As the Neuropsychological approach constitutes the basis of the “Information Processing Model”, which has a considerable presence in Cognitive Psychology. This suggests that humans’ cognitive activities are made up of 4 parts, i.e. planning, attention, simultaneous and consecutive cognitive processing. Constructivist Visual Perception approach based on the work of Richard L. Gregory, which he produced during 1980s, advocates that such distinct personal characteristics as preliminary information and spatial skills play a key role in visual perception. However, Gibson’s Direct Perception approach is built upon ecological analysis of perceptual problems. Thus, this approach claims that perception happens directly. This is because perception is not a process of indirect inference and sensations are not constructed by sensory organs (Gordon, 2004).  Nevertheless, yje Direct Perception approach fails to provide a profound explanation for illusions.

Literature Review

A search through the literature regarding visual perception approaches employed by educational websites and their evaluation, according to design principles obtained, reveal that very little scientific work is to be found. Evaluation of educational websites is an area where little work is conducted and published (Boklaschuk & Caisse, 2001). Yet, most of those works are not geared towards evaluating websites’ visual design principles alone but rather evaluating them from various perspectives.

Another research by Ivory and Hearst (2002) was carried out with the intention to conduct a quantitative rating of informational, graphical and navigational features of those websites; and the outputs of such research was supposed to help non-professional web-designers advance their skills to a competent level. By the very end of the research, indicators, such as text components, graphic components, hyperlink components, text format, graphic format and hyperlink format, were identified in order to rate quality and user-friendliness of the websites designs. For every indicator, different corresponding explanations were made to show what they mean and different corresponding gauging rules were identified.

In one of research in which Taverna (2003) attempted to evaluate website design and functionality, he collected feedback from 11 participants. Participating students were asked to rate website components regarding their repetition, refresh, adjustment and closure features. At the end of the research, it was found that there were more negative comments associated with color and contrast features than the others. A research carried out by Boklaschuk & Caisse (2001) tried to identify the different applicable criteria that enable evaluation of content and technical aspects of online educational resources. They were experienced difficulties due to a misconception that the necessary indicators were not in place to evaluate educational websites. Research set out the aesthetical, visual attractiveness, navigational and accessibility criteria. It also set out criteria for accountability, truthfulness, objectivity, scope and validity. The research report included an example of the evaluation tool and evaluation of chat sessions.

Adojaan and Sarapuu (2000) suggested a scale for educational websites when they were working out on the criteria to be referred to while developing effective educational websites during one of their researches. The used 5-point Likert-type scale consisted of 48 questions. These sections were about websites’ composition, general educational value and its position as compared to the curriculum. The scale’s convenience was tested in comparison of three different educational sources. The scale can be accessed in template form from http://www.eformular.com/scale/scale.html as it is also open to be developed by everyone.

Dursun (2004) analyzed the conformity of educational websites to the criteria listed in Web Design Scale. The study showed that the Websites meet the visual design criteria with a ratio of 55%. Erişti (2005) carried out a research with the participation of 18 primary school students in which interactive educational CDs, developed as per the graphic design principles, were examined to reveal their effectiveness in elementary art staff education. Based on the research results, it was concluded that interactive CDs ensure a lasting learning for the students. In addition to this, students expressed positive remarks about the prepared material.

At another research conducted by Cengizhan and Ates (2006), visual design’s written and figure elements were analyzed; and principles to be applied to these elements were identified. Visual objects were shown to the Computer Education and Instructional Technology Department students and they asked to comment on the principles used to develop those objects. From the research results, it can be deduced that learners are familiar with the visual design elements and they are knowledgeable about them at the level of practice.

Yigit, Butuner and Dertlioglu (2008), during their research for developing a website evaluation scale, emphasized the importance of evaluating the positive and negative aspects of educational websites and they also indicated that adhering to the criteria to be identified will help web developers compose hi-quality websites. Keser, Esgi and Şimşek (2003) highlighted in one of their academic papers, web design principles that was developed by 36 web design experts in Ankara in the year 2002. Fifty-six (56) out of seventy-five (75) nominated principles, indicated in the literature, were approved. This research aims to contribute to the efforts to develop the web based education by providing objective evaluation standards. These principles, in particular, will benefit the future educational designers who are now at the stage of learning by practicing under the supervision of their educators.

As shown by the search through the related literature, there is only a small amount of academic work to mention both in and outside Turkey. Previous studies have not focused on many aspects of online learning, but not the visual design of websites alone. No academic study could be found that examines visual design of educational websites and relates design principles to visual perception theories. There is a gap in academic literature to evaluate educational websites in accordance with various visual perception theories. The authors of this research study evaluated design principles of educational websites according to six (6) theories of visual perception.

Within the scope of visual perception theories:

1. Is there a significant level of difference among average scores of these websites?,

2. Does it make a difference among the average scores of these websites with regards to whether a website was designed in Turkish or English?,

3. Is there a significant level of difference among average scores obtained from evaluational dimensions of these websites?

Method

Research was patterned with the survey method. This section presents information about research participants, data collection tool and data analyses.

Table 1

Evaluated Websites during the Research

Participants

Website selection was based on the criterion that the website in question was fully accessible (no user or password required). Features of the analyzed websites are given in Table 1. Five (5) out of ten (10) selected websites were in Turkish; the other five (5) were in English.

The evaluation form developed by the researchers was used to evaluate 10 websites with educational contents. The research was conducted in two steps. The first step was to develop an evaluation form based on alternative visual perception theories; the second step was to use this instrument to assess selected websites with educational contents.

Research participants were 16 field experts (one in graphic design, two in educational game design, two in graphics and programming, and eleven in education technology). Each of these 16 field experts assessed 10 websites by using gauging tools developed by the researchers.

Data Collection Tool

Within the frame of the research, the “Web-based Educational Context Evaluation Form”, was developed by the researchers to assess web-based educational contexts according to visual perception principles. The Evaluation Form was developed with reference to academic literature and related to visual perception theories.

The evaluation form, which was developed from visual perception theories for assessing an interactive context design, had taken its items from what came out of focus group meetings. While developing the Evaluation Form, 8 dimensions of website evaluation procedure were identified as a result of focus group meetings conducted by 8 field experts in 6 sessions. These identified dimensions are namely: page layout, color, typography, visuals (pictures, graphics and photographs), animated images/graphics (video, animation, simulation), navigation and browsing, content and functionality. Dimensions identified during the focus group meetings were re-discussed within the scope of visual perception theories; and sub-items were set out by associating with them.

A host of theories, i.e. Gestalt Theory, Probabilistic Functionalism, Computational Theory, Neuropsychological Theory, Constructivist Visual Perception Theory and Direct Perceptional Theory, came into play both in identfying 8 main dimensions by associating them with interactive context design and in setting out sub-items pertaining to these dimensions. Some of these theories were associated with all the dimensions while some others were associated with just one or a few specific dimensions, in consideration of their distinguishing aspects.

In general, research to evaluate educational websites focus on figure-ground relationship, proximity, continuity, similarity and closure principles, within the scope of Gestalt visual perception theories (Mirielli, 2007; Puustinen, Baker & Lund, 2006; Hsiaoa & Choub, 2006).

The reason why other visual perception theories were taken into account during the research rather than what is available in the related literature is the fact that the selected perceptional theories are more likely to make an in-depth analysis. In this respect, focus group meetings and discussions among field experts generated eight main dimensions that can be associated with perception theories that are conducive to analysis of the initial interactive context designs. In the next step, dimensions associated with perception theories and distinguishing aspects of these theories were interlinked to create a pool of items. At this point, since the Gestalt Perception Theory provides a sound evaluation framework for perception in the entire design, it was decided to put items based on that theory into each of the eight dimensions. As per Egon Brunswick’s functionality approach, the dimension that came to the fore from focus group meetings was functionality. A relationship was finely established between functionality and perceivability in relation to characteristics of the target community and content presentation according to probabilistic functionality theory.

Items based on probabilistic functionality were set out with reference to this relationship. Marr’s Computational Theory highlights integrating characteristics of the perceived context and perceiving community. Hence, this approach underlines the relationship between the target community and each dimension that are generated during the research. In other words, Marr’s Computational Theory involves interlinking of many design principles, ranging from visuals to animated graphics as well as from typography to color configuration separately with the target community and in turn studying each of them one by one, and thereafter re-interpreting them as a whole. Therefore, the choice was made to place the items that were associated with Marr’s theory in every dimension. The neuropsychological approach, on the other hand, was associated with the main dimension of composition in order to identify a set of items that can be linked to several dimensions such as planning the total, setting out the activities, and presentational hierarchy of the content: hence the items were generated to this end. Constructivist Visual Perception Theory focuses on individual differences. From this point of view, both the functionality, and developing the content, and also the items that are contained within the aspect of defining and interlinking of design components, were all together associated with the Constructivist Theory. Gibson’s Direct Visual Perception Theory focuses on meanings of design components within the content’s presentation, target community’s impression as to them along with the impact that was made by involving characteristics of the target community as a factor in presentation. So, to this end, the items that were associated with the Theory of Direct Perception put forth other items, which were oriented to studying the target community together with its environment and designing the content accordingly.

By composing an evaluation form, it was intended to demonstrate websites’ features through rating them with respect to a scale of specific main dimensions and sub-items. The four point Likert-Type evaluation form contained five items for the dimension of page layout, six for color, five for typography, ten for visuals, eight for animated graphics, five for navigation and browsing, five for content and six for functionality, a total of 50 items. Three experts with relevant competencies in the field of educational technology were consulted to ensure that the developed form meets the scope and appearance criteria. The final version of the form was made ready for the application as per feedback collected from the field experts.

Data Analysis

The scales of items in the Web-based Educational Context Evaluation Form were keyed as “completely suitable”, “fairly suitable”, “slightly suitable” and “not suitable at all/completely unsuitable”. Separate comparisons among the average scores that the websites received for each of the dimensions of page layout, color, typography, visuals (pictures, graphics and photographs), animated graphics (video, animation, simulation), navigation and browsing, content and functionality as well as a single comparison among the overall average scores of each website was done by using one way analysis of variance (one way ANOVA test). On the other hand, to detect whether websites’ overall scores differ with regards to being designed in Turkish or in English, an independent samples t-test was used.

Findings

Comparison of websites with respect to their average scores

At this section, websites’ average scores were calculated. To this end, 10 websites’ calculated average scores were compared to each other by using a one way ANOVA test. In conclusion, a significant difference was detected in mean scores of websites (F(9, 118)=10.412, p<,05). So to find out which group(s) was/were causing this difference, a Tukey test, which is one of multiple comparison tests, was conducted. The values produced by the test are given in Table 2. WS7significantly outscored all the other websites, except WS2.

Table 2

Analysis of variance to show whether websites’ overall average scores differ or not

*p<.05

** Numbers represent the evaluated websites.

Comparison of websites with regards to whether a website was designed
in Turkish or English

The hypothesis of the existence of a significant difference between the average scores of websites in Turkish and those in English was tested by using an independent samples t-test. The obtained values are given in Table 3. As a result of the analysis, a significant difference was detected between Turkish and English websites, placing English websites above the Turkish ones (t(126) = -2.536, p= .012). In other words, average scores of English websites ( = 2.57) are higher than those of Turkish websites ( = 2.57).

Table 3

The t-test conducted to show whether websites’ overall average scores differ
with respect to being designed in Turkish or English

Comparison of websites with respect to evaluational dimensions

During the research, five (5) educational websites in Turkish and five (5) in English were evaluated with respect to dimensions of page layout, color, typography, visuals, animated graphics, navigation and browsing, and content and functionality. The scores that the websites received for evaluational dimensions were compared by using the one way ANOVA test.

The analysis revealed a significant difference among the scores that the websites received for all dimensions. The Tukey test, which is one of the multiple comparison tests, was conducted to find out which group(s) was/were causing this difference (Table 4).

As given in Table 4, significant differences were detected among websites average scores received for page layout (F(9.147)=4.292, p<.05), color (F(9.146)=8.530, p<.05), typography (F(9.146)=5.462, p<.05), visuals (F(9.126)=12.157, p<.05), animated graphics (F(9.143)=17.521, p<.05), visuals and browsing (F(9.147)=2.494, p<.05), content (F(9.144)=8.452, p<.05) and functionality (F(9.146)=3.722, p<.05) dimensions. It was seen that the differences among websites’ average scores received for each of every dimension was mostly caused by the difference between WS7 and other websites. The lowest difference was received for the dimension of navigation and browsing. Considering the navigation and browsing dimension, it was found that a significant difference only exists between WS7 and WS10.

Table 4

The analysis of variance to show whether websites’ average scores differ with respect to dimensions of page layout, color, typography, visuals, animated graphics, navigation and browsing, and content and functionality.

*p<.05                           
** Numbers represent the evaluated websites.

Discussion and Conclusion

Perception oriented design principles received a higher priority in making educational designs. Expectations regarding the design aspect of websites developed for educational purposes soared up with the rapid spread of internet services over the past decade and the increase in the availability of learning contents in that digital environment.

During the research, 10 educational websites for primary school level, 5 of which were designed in Turkish and another 5 in English, were analyzed by using “Web based Evaluation Form for Educational Contexts”, in consideration of perception oriented design principles. Website assessments were carried out with the participation of 16 field experts.

First, the averages values were calculated out of research participants’ scorings for the analyzed websites’ designs. Then, the websites were compared with each other in terms of average scores. One-to-one comparisons yielded 14 statistically significant differentiations.

Having all the combinations considered, this ratio fell way below the par. That is to say, there does not exist a big difference among websites’ average scores with regards to how much they conform to design principles. However, the English website number 7 received higher scores above all the other websites except Turkish website number 2. In addition to that, it was observed that experts gave significantly higher scores to English websites with educational purposes (=2.57) as compared to Turkish ones (=2.32), in terms of perception oriented design principles. This evidence can be interpreted in a way that the design principles are considered less while developing Turkish websites with educational contents. Yet, it can be said that both the English and Turkish websites fail to meet the design principles, after making a general evaluation by taking all the average scores into account. Both type of websites failed to go beyond the level of “Slightly suitable”.

The research yielded some critical conclusions by the assessment of design principles employed in the analyzed websites as to selected dimensions, i.e. page layout, color, typography, visuals, animated graphics, navigation and browsing, and content and functionality. The most significant differences were found to exist among websites with “animated imagery/graphics” whereas the least significant differences were “navigation and browsing”. This finding demonstrates that websites differ a lot with regard to the way animated graphics are used whereas they bear similar features with regard to their navigation and browsing tools.

When the average scores received for analyzed eight dimensions of perception-oriented design principles are considered, it is seen that all the average scores are from 2.27 to 2.98. This finding appears to show that the 10 analyzed educational websites do not conform to different dimensions of design principles. Along with this finding, the highest average was received by “navigation and browsing” dimension while the lowest was received by “content and functionality” dimension. Within the constraint of the sample of 10 analyzed websites, this situation points to the fact that educational websites’ navigation and browsing tools show better performance in conforming to design principles while their content and functionality is sub-par.

It can be said that the evaluation form which was used to conduct the research is an effective tool to thoroughly assess websites with educational content, based on feedback from participants who actually used and made a scrutiny of that evaluation form. In addition to that, research participants firmly concur the evaluation form contains dimensions and items that makes it possible to assess much thoroughly the qualities born by the websites.

Relying on the evidence from this study, it is thought that, henceforth, greater attention must be paid by the educational designers and educators to all the design principles in general and to design principles of content and functionality in particular while developing websites with educational contents for primary education. Along with this, it is thought that greater attention must be paid to visual design principles while developing websites designed in Turkish for educational purposes. By doing so, it is believed that progress can be made in improving the overall quality of Turkish websites for primary education and thus studies in this area may proliferate.

Studies from now on may concentrate on the integration of perception oriented design principles in the process of developing educational websites at the primary education level. Furthermore, the subject of how different dimensions of visual design principles affect the students’ success in learning from the educational websites can be studied.

References

Adojaan, K. & Sarapuu, T. (2000). Template Scale for the Evaluation of Educational Web Sites. In Proceedings of WebNet World Conference on the WWW and Internet 2000 (pp. 854-856). Chesapeake, VA: AACE.

Boklaschuk, K. & Caisse, K. (2001). Evaluation of educational websites. Accessed from http://www.usask.ca/education/coursework/802papers/bokcaisse/bokcaisse.htm on14.12.2008.

Behrens, R. (1984). Design in the visual arts. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Cengizhan, C. & Ates, D. (2006). Görsel tasarım ilkelerinin BÖTE bölümü öğrencileri tarafından değerlendirilmesi [Evaluation of Visual design principles by CEIT (Computer Education and Instructional Technology) department students]. 15. Ulusal Eğitim Bilimleri Kongresi, 13-15 Eylül, Muğla.

Demirel, Ö. & Ün, K. (1987). Eğitim terimleri sözlüğü [Dictionary of educational terms]. Ankara: Şafak Matbaası.

Dursun, Ö.Ö. (2004). Eğitsel Web Sitelerinin Görsel Tasarım Kriterlerine ve Kullanıcılara Sunulan Hizmetlere Göre Değerlendirilmesi [Evaluation of Educational Web-Sites Regarding Visual Design Criteria and Services Offered the Users]. Unpublished thesis. Anadolu Üniversitesi: Eğitim Bilimleri Enstitüsü.

Eidenberger, H. (2004). A new perspective on visual information retrieval. Accessed from http://www.ims.tuwien.ac.at/media/documents/publications/ei2004-vir.pdf on 01.10.2009.

Erinç, M.S. (1998). Sanatın boyutları [Dimensions of art]. İstanbul: Çınar Yayınları.

Erişti, S.D. (2005). Grafik Tasarım İlkelerine Dayalı Olarak Geliştirilmiş Etkileşimli Eğitim CD’lerinin İlköğretimde Temel Sanat Elemanlarının Öğretiminde Etkililiği [Effectiveness of Interactive Education CD Designed Through the Graphic Design Principles on Teaching the Basic Art Elements in Primary Schools]. Anadolu Üniversitesi: Eğitim Bilimleri Enstitüsü.

Hsiaoa, S.W. & Choub J.R. (2006). A Gestalt-like perceptual measure for home page design using a fuzzy entropy approach. International Journal of Human-Computer Studies, 64(2), 137-156.

Gordon, I.E. (2004). Theories of visual perception. New York: Psychology Press.

Ivory, M.Y., & Hearst, M.A. (2002). Improving web site design. IEEE Internet Computing, 6(2), 56-63.

Keser, H., Esgi, N. & Simsek, A. (2003). Principles of Educational Web Design. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2003 (pp. 2011-2014). Chesapeake, VA: AACE.

Mirielli, E.J. (2007). Using peer-evaluation in a website design course, Journal of Computing Sciences in Colleges, 22(4), 14 – 21.  

Puustinen, M., Baker, M. & Lund, K. (2006). Gestalt: A framework for redesign of educational software. Journal of Computer Assisted Learning, 22(1), 34-46.

Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A. & Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1349-1380. 

Taverna, M.A. (2003). Formative evaluation of web site design. Accessed from http://www.designsinlearning.com/portfolio/portfoliohome.html on 01.01.2009

Yigit, N., Butuner, S.O. & Dertlioglu, K. (2008). Öğretim amaçlı örütbağ sitesi değerlendirme ölçeği geliştirme [Developing scale for evaluation of educational website]. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi (EFMED), 2(2), 38-51.

About the Authors

Suzan Duygu Eristi is an assistant professor at the Department of Computer Education and Instructional Technology, Anadolu University in Turkey.

e-mail: sdbedir@anadolu.edu.tr

Ozden Sahin-Izmirli is a research assistant at the Department of Computer Education and Instructional Technology, Eskisehir Osmangazi University in Turkey.

e-mail: sizmirli@ogu.edu.tr

Serkan Izmirli is a research assistant at the Department of Computer Education and Instructional Technology, Anadolu University in Turkey.

e-mail: sizmirli@anadolu.edu.tr

Mehmet Firat is a research assistant at the Department of Distance Education, Anadolu University in Turkey.

e-mail: mfirat@anadolu.edu.tr