Effects of a relative-frequency elicitation question on likelihood judgment accuracy: the case of external correspondence

In three experiments, college students judged the likelihood that they chose the correct alternative for each of 40 two-alternative, general-knowledge items. They responded either to a relative-frequency elicitation question (“Out of 100 questions for which you felt this certain of the answer, how many would you answer correctly?”) or to a probability elicitation question (“What is the probability that you chose the correct answer?”). Judgments in response to the relative-frequency elicitation question tended to be lower, exhibit less scatter, and express complete certainty less often than judgments in response to the probability elicitation question. Two types of explanation for these effects are considered. First, the effect of the relative-frequency elicitation question may be to reduce random response error in participants' likelihood judgments. Second, the relative-frequency elicitation question may encourage the use of frequency information and simpler algorithms for making likelihood judgments.