3. The OUC test bank
The OUC was formally unveiled in July 31, 2012 in the Great Hall of the People, Beijing. The Response to the Establishment of the OUC on the Basis of the CCRTVU (Education Letter No. 103 (2012)) clearly states: The OUC is under the direct administration of the MOE. It is supported by modern information technology and is a new type of higher education institution providing open and distance education to adults. The OUC is different from normal universities. Although it was established on the basis of the CCRTVU, it is different from radio and television universities in its schooling principles, goals and responsibilities. The reform of its assessment model is one of the four major reforms featured in the Strategic Plan for the OUC, which points out that it is necessary to “vigorously advance the construction of a digital test bank, support 24/7 study and testing, and provide fast, convenient, individualized learning assessment services to learners, all with the support of digital study materials.” The National Outline for Medium and Long-term Educational Reform and Development also puts forward the following requirements for examination reform: improve the national examination test bank subject by subject; explore multiple examinations for some subjects; and explore social examinations. The construction of a test bank has become a necessary process as part of the construction of the OUC.
The OUC test bank should be a multi-functional test bank with integrated self-testing, and formative and summative assessments. Its major characteristics are broad application, multiple standards, and complicated composition strategies. Firstly, the OUC’s test bank is different from ordinary test banks in that it can be applied to self-testing, formative assessments and summative testing. According to the requirements of course teaching reform, a test bank can be used for the appraisal of academic aptitude and equivalence. Secondly, the OUC test bank has multiple construction standards, which encourages the development of a number of different assessment methods according to the nature of each course and the study characteristics of the adult cohort, such as 100% formative assessment and 100% summative examination. Thirdly, the OUC test bank has a more complicated composition strategy. On one hand, the composition strategy for a single test cannot be based on difficulty and coverage like a normal test. On the other hand, it should nevertheless ensure that it does meet requirements for difficulty and coverage.
II. Quality factors
Liu Bo from the MOE National Education Examination Authority believes that, “exam questions are the basic elements that make up the test bank. The management system supports the test bank, and safety and confidentiality is an important guarantee for the test bank.” He believes that the qualities that define the test bank are: scientific test content, precise test standard, reasonable parameters, stable difficulty control, standardized test descriptions, and fair usage of materials. Li Shuqing from Jiangxi Normal University and his fellow professors have proposed that the quality of the test bank may be improved by classic measurement theory and project reaction theory, which can be carried out in four aspects. First, updating the guidelines for test compilation and highlighting the precise assessment of the pass level. Second, adopting parameter definition from project reaction theory, and establishing a larger, more reliable test bank with rational content and test types. Third, a two-step composition procedure comprised of “general” and “optimized” so that the computer system can be applied more widely. Fourth, exploring excellent teaching methods to ensure the full realization of theoretical propositions. The author believes that test bank construction covers four factors: objectives, test questions, test papers, and composition strategy. Each of these factors contributes to the quality of the test bank. On the basis of the above, this paper will focus on the practical utility of the construction of the OUC, and put forward that the construction factors of test bank quality should include clearly aligned goals, a suitable structure, an efficient strategy, and timely content
1. Clearly aligned goals. Any test has specific assessment objectives. Assessment objectives are formed out of teaching objectives, thus the two should be highly aligned, otherwise, the credibility of the tests will be affected. A two-way breakdown can be used to show the transformation from teaching to assessment. The two-way breakdown for the OUC course assessment is more complicated than an ordinary breakdown. It not only shows the relationship between the teaching content and the summative examination, but also the relationship between teaching and self-testing, formative assessment and summative assessment, and the intrinsic relationship between the three testing methods. The assessment objectives of the OUC focus on the advancement of teaching reform, encouraging students to study, and guiding and supervising study by means of assessment.
2. A suitable structure. This is a requirement for a quality test bank. The test bank is composed of a large number of individual test questions. In terms of content structure, it should present the reasonable layout, the coherent content and the complete attributes . This can be demonstrated as a proposition table. In order to ensure the efficiency of composition, the proposition table should use the smallest knowledge point as a unit of composition and clearly mark the type, quantity, and score of every knowledge point during the composition of a test paper. In terms of general layout, we should consider the teaching requirements of the teaching outline and evenly distribute the knowledge points of each chapter. We should ensure overall coverage as well as highlight the key points. During the composition of each test item, we should strictly and coherently follow the requirements of the two-way breakdown and the proposition table. In the process of handling the relationship between different knowledge points in the test, we should clearly indicate exclusiveness, inclusiveness and association and make sure that all attributes are complete.
3. Effective composition. The basic purpose of the OUC’s test bank is to meet the need for 24/7, 365 days a year studying and testing, which requires an intelligent, randomized composition. The current composition method uses a computer to randomly compose papers. The main disadvantage of this is that it constrains the overall use of the test bank, and limits the frequency of updates. After the test bank is updated or the test items are modified, the test papers need to be regroups, which wastes data and does not favor dynamically updated test bank content. In addition, composition should consider factors such as the exposure-rate of the test questions, the repetition rate of the test content, and the dormancy order of the test bank content, and rationally distribute assessment cycles, so as to enhance the overall performance of the test bank.
4. Timely content. Firstly, a test bank may be used for up to five to 10 years. Since teaching content is updated according to new social developments, such as legal courses, a dynamic update system should be established for the test bank content. Secondly, students use a lot of raw materials to answer a test paper, which might be better than the materials in the test bank. Including excellent materials is an important factor in maintaining the timeliness of the test bank. Thirdly, it is necessary to eliminate test questions that are rarely used. Otherwise, they will reduce the overall efficiency of the test bank.
III. Technical model
The quality of the OUC test bank is reinforced by the technology. In the early stages of test bank construction, we should carefully review the requirements and objectives of the teaching outline, and get a firm grasp on overall teaching standards. We should formulate the two-way breakdown according to the teaching outline and transform the teaching content and language into the assessment contents and language. Furthermore, we should design and position the test bank structure and formulate a proposition table according to the assessment objectives. In the middle stages of the construction of the test bank, we should strictly control various aspects including the content of the examination, the assessment capacity, the difficulty, the differentiation, the scores, and the time limit. In the latter stages of test bank construction, we should carry out statistical analysis, and make amendments and additions or cancellation accordingly. Therefore, the quality of the OUC test bank is a complete entity with complex linkages. Any part of test bank construction may affect the overall quality of the test bank. We chose to elaborate on four key points: model proposition, equivalent storage, dynamic links and exposure warnings.
Figure 1 Technical diagram of OUC test bank quality