Quizbank/Cost-benefit analysis/Undergraduate prelims

Many undergraduate courses could be taught at reduced by cutting the student-instructor contact time in half, using a system of prelims and study-guides that are an OER resource. Students who select this option would be allowed to acquire fundamental background skills online. If the bank is sufficiently large, individual instructors could maintain some degree of "surprise" regarding exactly what is on the exam. Individual institutions would also set their own standards for the "prelim", and presumably design alternative assessments for students who are not good test takers.

The fact that students can see the questions in advance is a major drawback for a public exam bank. For that reason it should focus on basic "facts", include historical events, definitions, and basic computational skills. Another compensation for for the inferiority of this assessment is that prelims are typically only on a pass-fail basis. This reduce the temptation for students to memorize information beyond the point of diminishing returns as they prepare for a prelim. For these reasons, it is essential that the question bank offer students to self-evaluate for these prelims by practicing online at zero cost.

To ensure diversity and academic freedom, there would be no unique set of questions or standards. The bank would be large enough so individual instructors could select a subset of the questions to be posed. And these same instructors or institutions could openly publish the standards so that students could self-evaluate before taking the time and effort to attempt the prelim associated with that course. And of course, colleges would be expected to offer alternative assessments for those who need it.

The need for flexible evolution

This proposal seems radical, and history tells us that social progress generally evolves in small steps. The first step might be to offer full-semester courses, but give students the option of attempting to "test out" of the first half of the course. Colleges and universities might promote online study guides and undergraduate prelims as recruitment tools, since high school students who take the initiative to prepare for such prelims are likely to be viewed as an asset by most colleges.

Those who are concerned with the social and mental health of future generations are likely to view the prospect of undergraduate prelims with a mixture of hope and concern. Perhaps the most alarming is the prospect of education becoming so oriented towards student assessments based on low-quality exams. It is essential that we resist the temptation to reduce the cost of higher education by excluding important segments of our population. Alternative means for "testing into" the reduced-hours course is not only essential in order to serve the broader community, it offers intriguing opportunities for educators. For example, consider the case of students who are poor standardized test-takers, but are skilled at communication and comprehension. The extra effort to recognize and reward these students by allowing them into the reduced-hours course would result in a socially dynamic student population. Humans are social creatures, and it is impossible to imagine the merits of an emphasis on education that is 100% online. On the other hand, preliminary online activities before the first day of class would allow instructors fascinating opportunities to mix and match people with a variety of strengths (e.g. verbally versus mathematically skilled.)

This reform has another advantage that might not seem obvious. When we change education without full understanding of the consequences, we are essentially experimenting on human subjects. This raises potentially serious ethical issues. Fortunately, this reform targets students who volunteer to "test out". And, since these students perceive themselves capable of learning on their own, we are more likely to "experiment" on the stronger students and therefore less likely to do harm.

Unintended consequences

It is worth pointing out that since standardized tests administrated in high school are not very good predictors of college success,^[1] there is no reason to attach too much weight to college tests taken from an open source quizbank. On the other hand, a wise application of an open source bank can permit educators to focus more effort on the mediocre test takers. It is essential that open exams remain one of several paths towards the more advanced courses. Preliminary exams should only be used to motivate students to review the fundamentals before the course begins, never to screen students out of a course unless it is certain that they do not belong in that course.

Another unintended consequence will be the tendency to reduce education to the lowest (fact-based) domain of Bloom's taxonomy.

The open bank needs to be so highly developed that the prelims and their study-guides are well established. Otherwise instructors will spend an inordinate amount of time developing these materials. To make matters worse, it will probably be necessary for individual instructors to choose from a list of possible preliminary exam systems (just as instructors choose from a list of possible textbooks.) In other words, the routine use of preliminary exams in undergraduate education is probably several years in the future.
Provisions must be met for educating students who fail the prelims. Otherwise we violate the (implied) commitment that higher education not be reserved for only those who are skilled at taking such tests. Fortunately, this is not such a serious constraint as one might think. The prelims could count for only a portion of the grade. Or perhaps the prelims (or some alternative assessment) could also be offered after the course is completed.

Some courses will never be compatible with an open exam bank. And virtually all courses require additional assessment beyond what an exam bank offers. On the other hand, in the future it should be possible to automate certain assessments of writing skills using advanced AI methods, or even something as simple as Calibrated Peer Review.

Footnotes

Quizbank/Creating a bank so students won't ''break the bank''
This pageview analysis suggests there was some interest in my effort on Wikiversity. This site is is based on Quizbank, which is now depricated as all these questions are being moved to MyOpenMath.

Footnotes

↑ Kurlaender, Michal, and Kramer Cohen. "Predicting College Success: How Do Different High School Assessments Measure Up?." Policy Analysis for California Education, PACE (2019).

[1] Kurlaender, Michal, and Kramer Cohen. "Predicting College Success: How Do Different High School Assessments Measure Up?." Policy Analysis for California Education, PACE (2019).

[1]

Quizbank/Cost-benefit analysis/Undergraduate prelims

Contents

The need for flexible evolution

Unintended consequences

Footnotes

See also

Footnotes