Page 1 of 5
Case Study: An Analysis of the Status and Key Technology Design of Digital Examinations*
Liu Yuxia*, Wang Ying
The OUC Examination Centre
The OUC Research Institute of Open and Distance Education
*This email address is being protected from spambots. You need JavaScript enabled to view it.
Abstract: With the rapid development of information technology and the growing popularity of online learning, traditional paper-based examinations are becoming outdated and there is an inevitable shift towards digital examinations. Based on the educational practices of The Open University of China (OUC, the number of students currently represents two thirds of the students in higher distance education), this study analyzes the background, current situation and problems of digital examinations, and looks at the design of key technologies for digital examinations. This study proposes that digital examinations comply with six main design principles: the intercommunication between databases; the importance of scientific examinations; the integration of information technology; the integration of teaching, learning and assessment; the implementation of a range of assessment methods; and the development of different combinations of modules. It also analyzes the efficacy of examination information, the integration of information technology and examinations, and people-first examination design, with an emphasis on the overall design of digital examinations. This study hopes to further the reform of the examination system and the development of distance teaching and learning as a whole.
Keywords: digital examination, technical design, distance education, case study
*This paper is one of the achievements of the OUC’s 2012 key research project “ Quality Factor Analysis of the OUC Examination Database and Research on Technical Model”.
I. Preface
1. Problems
Digital examinations are an extension of paperless examinations. Following the development of Internet technology and its increased application in education and exanimations, paperless examinations became a major direction for reform in 2000 and 2010. There has been a large amount of research on the design of paperless examination systems under different computer database environments. The development environment surrounding of digital examinations includes C/S, B/S, WEB, NET, ASP, SQL Server, and Windows AP. The majority of the research (such as Huang Wenhan, 2011; Yang Lijuan, 2011; Cao Shaojun, 2010) focuses on the design and functional model of paperless examinations. Research on the subjects and types of digital examinations (such as Jing Qingshan, 2009; Zhu Yuelai, 2010, Wang Xiaobin, 2005) mainly focuses on Professional English, Computer Applications, Computerized Accounting, Digital Network Language Lab Systems, Motor Vehicle Drivers, and College English Levels A and B. However, there is a lack of research from the perspective of examination principles, and that faces the real needs of the wide range of learners across the open university system. With the development of international open and distance education, many open universities, such as the Open University UK (OUUK) and the University of Phoenix, USA, are increasingly facing the need to support examinations for large numbers of students with varied needs and interests across a range of geographical locations. It is not enough to achieve paperless testing; it is also necessary to replicate aspects of formative assessment, such as group discussions, evaluation of learning behavior, cultivation of learning interests, and practical operation within the Internet environment. Research on digital examinations within the Internet environment is an unavoidable part of this process.
The rapid development and networked, multimedia nature of information technology has nurtured the study environment of digitalization and made digital examinations possible. The characteristics of digital examinations better meet the needs of distance learners, as well as making it easier to hold unified examinations for large numbers of widely geographically dispersed learners. Digital examinations are also cheaper. Furthermore, by making use of a common bank of test resources, digital examinations help to ensure consistency and confidentiality.
After the strategic transition made in 2012, as a distance mega-university, the OUC focuses on using modern information technology to provide open and distance education for all of society. It is necessary for the OUC to make good use of modern information technology, in order to solve the myriad issues facing teaching locations with a large number of students from different geographical locations and educational backgrounds, and with un-balanced development caused by different economy, system and mechanism. Examination reform is an important breakthrough in this regard, particularly in terms of how to create digital examinations that represent the characteristics of the OUC within the information technology environment.
2. Research methods
The research is based on an understanding of distance higher education and digital examination. Using the OUC’s examinations as a case study, it presents a digital examination design plan based on relevant examination theories and collected materials, in order to further advance examination reform and distance teaching practice.
The paper is based on examination documents and records, analysis reports, interviews with examinators and teachers, and field observations. These materials establish a relevant case study database and provide a basis for reliable and effective questions. Thus, this forms an “evidence chain” for the final digital examination solution.
II. Research process
1. Evidence collection
Examinations are a key part of the education system. The OUC examinations are a form of national unified examination. This not only explains the strictures and standardization of the examinations, it also indicates that they are closely related to teaching requirements of curriculum. The supervision, motivation and evaluation of examinations stimulates the development of education and teaching.
The OUC’s digital examination system was born in 1990s with the development of information technology and the computer network. The OUC’s summative examination system, which was developed at the beginning of the 21st century, was not widely promoted for technical reasons. Before 2005, it received some degree of promotion and application under the acceleration of the “Internet-based curricular examination pilot project”. At present, there are three types of the OUC examinations. First are the curricular examinations of the “Internet-based curricular examination pilot project”, which includes 161 courses. Second are four public computer network courses under the administration of the Ministry of Education (MOE): Computer Basics, College English A, College English B and College Chinese. Third are the OUC’s unified national written examinations, of which there are 856 courses, the distribution of which can be seen in Figure 1. Of these, online examinations represent 16%. The national unified courses make up 84% of the total courses. (Excluding the proportion of the four courses under the administration of the MOE.)
Figure 1 OUC examinations