The Angoff Method is a widely used standard-setting approach in test development.
In plain English, it is a kind of study that test developers use to determine the passing percentage (cutscore) for a test. The passing grade of a test can’t be decided arbitrarily; it must be justified with empirical data. The Angoff method relies on subject-matter experts (SMEs) who examine the content of each test question (item) and then predict how many minimally-qualified candidates would answer the item correctly. The average of the judges’ predictions for a test question becomes its predicted difficulty. The sum of the predicted difficulty values for each item averaged across the judges and items on a test is the recommended Angoff cut score. Here is a real world example that illustrates the process:
Let’s say a test developer needed to determine the passing grade for a language exam that tested a person’s ability to read Arabic. Using the Angoff Method, the developer would employ a number of SMEs (in this case, Arabic-language experts) and ensure that they were properly trained on how to use the Angoff Method, as well as informed on the test’s purpose.
The Arabic-Angoff Panel would then rate each test item based on whether or not a minimally-qualified candidate would answer the item correctly or incorrectly. Once the first round of ratings had been conducted, everyone on the panel would be given access to the ratings of the other SMEs so that they could compare what they determined about a particular item. Then, the SMEs would be asked to rate the items again for a second round. The second round of rating would give the SMEs the opportunity to review their initial rating of an item and decide whether or not they might like to change their decision based on the expert judgments of the other panelists. This second round of ratings would be averaged across the SMEs to determine the final cutscore for the test.
In the end, using the Angoff Method ensures that the passing grade of a test is determined empirically, which is necessary for a test to be legally defensible and meet the Standards for Educational and Psychological Testing. Other standard-setting methods that testing professionals sometimes employ include the Ebel, Nedelsky, and Bookmark methods.
___________________________________________________________________________________________
ALTA is a leader in Language Testing for government agencies and corporate clients.