Education research: partial credit for multiple-choice questions (PCMC) bibliography

Fall semester 2008 has partial credit for multiple-choice questions (PCMC) being piloted for Astronomy 210 (introductory one-semester astronomy) at Cuesta College (San Luis Obispo, CA). The intent is to get students to spend more time critically selecting their answers, especially with regards to eliminating incorrect answers. Observations of students in previous semesters is that students spend approximately one minute per question before turning in their completed quizzes, which could be considered much too little time, given that there are two minutes per question allotted.

Quizzes have 10 multiple-choice questions, each with four (A)-(D) choices.

Sample quiz questions from Fall semester 2008:
http://tinyurl.com/3tbhnz

From the course policy:

"Partial credit (0.5 points) will be given for an incorrectly circled answer to a multiple-choice question if a single "x" is also made on one incorrect response to that question. No other circumstance other than that previously described will result in partial credit."

A briefer statement on the quiz itself:

"Closed book, closed notes. Clearly circle ("O") the one choice that you think is most definitely correct. Cross out ("×") only one choice that you think is definitely incorrect."

A correct circled answer is 4.0 points, and 0.5 points is awarded for a successfully identified incorrect answer only if the circled answer is incorrect. Thus the partial-credit option is 1/8th of the full-credit possible.

Brief bibliography, with abstracts/excerpts:

"Humanizing the multiple-choice test with partial credit"
Barry M. Cherkas, Joseph Roitberg, International Journal of Mathematical Education in Science and Technology, Volume 24, Issue 6 November 1993, pages 799-812.

"We describe an innovative method for machine-grading precalculus students: the partial-credit, multiple-choice test (PCMCT). Our scheme is to assign partial credit to any wrong multiple-choice answer for which we can ascribe partially correct reasoning. This permits us to accumulate additional information about the extent of a student's incomplete understanding. We add the assigned partial credit to arrive at our final measure of a student's total knowledge—the student's grade. In comparison with the standard multiple-choice test (MCT), the PCMCT judiciously avoids the necessity to scale scores—an educationally unsound but common practice with MCTs. In addition, the PCMCT turns out to be better at distinguishing uneducated guessing than the MCT. We conducted student surveys in our pre-calculus classes and found that most students believed the PCMCT is a more accurate measure of their knowledge than the MCT. We also uncovered the remarkable result that most students are less math-anxious when taking the PCMCT than when taking either the MCT or a standard essay test."

"Comparative Review: Partial-Credit Scoring Methods for Multiple-Choice Tests"
Robert B. Frary, Applied Measurement in Education, Volume 2, Issue 1 January 1989, pages 79-96.

"This review covers multiple-choice response and scoring methods that attempt to capture information about an examinee's degree or level of knowledge with respect to each item and use this information to produce a total test score. The period covered is mainly from the early 1970s onward; earlier reviews are summarized. It is concluded that there is little to be gained from the complex responding and scoring schemes that have been investigated. Although some of them have confirmed potential to increase internal-consistency reliability, this outcome is often obtained only at the expense of validity. Also, the extra responding time required by some methods would permit lengthening a conventional multiple-choice test sufficiently to obtain the same reliability improvement. Partial-credit response and scoring methods that continue to be used will probably earn this status due to secondary characteristics such as providing feedback to enhance learning."

"A Multiple Choice Test that Rewards Partial Knowledge"
M. Bush, Journal of Further and Higher Education, Volume 25, Number 2, 1 June 2001, pp. 157-163(7).

"A new multiple choice test format is presented that allows examinees to select more than one answer to a question if they are uncertain of the correct one. Negative marking is used to penalise incorrect selections. The aim is to explicitly reward examinees who possess partial knowledge as compared with those who are simply guessing. The result is a test method that forces examinees to think more carefully about their answers, and that yields results of a higher resolution than standard multiple choice tests. After describing the new format, the paper presents and critiques several existing methods which have the same or similar aims. The paper ends with a discussion of the feedback and experience gained to date in using the new format."

"A singular choice for multiple choice"
Gudmund S. Frandsen, Michael I. Schwartzbach, ACM SIGCSE Bulletin Volume 38, Issue 4 (December 2006), p. 34-38.

"How should multiple choice tests be scored and graded, in particular when students are allowed to check several boxes to convey partial knowledge? Many strategies may seem reasonable, but we demonstrate that five self-evident axioms are sufficient to determine completely the correct strategy. We also discuss how to measure robustness of the obtained grades. Our results have practical advantages and also suggest criteria for designing multiple choice questions."