Web-Assisted Education: From Evaluation to Learning

Exams have traditionally been given to evaluate students but in recent years, with the appearance of freely accessible on-line tutoring, quizzes can also be used as a learning tool. In systems where students can request quiz items until a satisfactory grade is obtained, new probabilistic approaches are required for dealing items to students and new statistical approaches are required for analysis, including student evaluation. This paper applies various models to a case study to quantify the actual learning involved in using such a dynamic system and demonstrates the need for new statistical approaches. Responses to repeatedly assigned questions, some with random numbers, are analyzed to distinguish rote learning from learning with understanding. It is seen that models designed for evaluation are inadequate for dynamic systems which are a part of a learning experience in contrast to static systems which merely provide data storage and snapshot evaluation. Keywords: interactive learning, computer-assisted education, logistic regression, rote learning, item response theory, mixed effects models ********** Across the world reading material in the form of printed textbooks dominate in schools, sometimes with handouts on paper, except for the many parts where this is too expensive. Instructors still use mainly blackboard and chalk with recent moves in some parts to whiteboards, pens and (electronic) slide presentations. Homework, quizzes and exams are used to monitor progress, sometimes on the move to hypermedia (Dillon and Gabbard, 1998), with considerable debates on standards (Friesen, 2004). Most uses of electronic media merely make electronic slides available (see references in Stefansson, 2004). These, and storing handouts on-line, are petty uses of the web’s potential and no real improvement over paper. To use the possibilities of the web a system should at least interlink the material and add options for testing. On-line tests are mostly computerized paper tests with fixed questions to be solved in a fixed time slot. Reducing instructor-time, this gives no credit to the interactive nature of the web. However, drilling systems for learning have been developed, from spelling and arithmetic (Frantz, 1982; Thompson and Cloer, 1982; Chang et al., 2008) through generic memorization drills (Feng et al., 2006; Maya et al., 2007; Wozniak, 2007). Most are task-specific, proprietary and not freely available, not specifically designed to enhance exchange of content, nor do they provide a single platform for multiple topics, student groups and instructors. Results in this paper are based on the “tutor-web’, a system for web-assisted education and research on education, both for in-class use and for remote learning. The system is free and includes the possibility of a student taking exams. Instructors anywhere can obtain free access to exchange and use teaching material. Students have free access to the system, including self-evaluation. The system provides a basis for research on web-assisted education such as item selection and uses in low-income areas. A goal of the “tutor-web” project is to set up a repository of educational material for a BSc degree in mathematics and and MSc degree in statistics, complete with slides, handouts and on-line quizzes. This is being done by a consortium of faculty and researchers in some 20 countries. The approach taken is to insert material in various stages of development, from simple handouts through complete sets of interlinked material. Given the interactivity of the WWW and a student’s obvious desire to obtain a high grade, on-line quizzes have the potential to permit the student to work towards a goal, with the quiz becoming a learning experience. The “tutor-web” has been designed with this in mind, as have a few other systems, some of which have been used to evaluate the effects on student performance (Goldberg & McKhann, 2000; Griffin & Gudlaugsdottir, 2006; Feng et al.