#NAEMSE09 Designing Critical Thinking Exams
To learn more about EMS test design I attended a four hour pre-conference session at the National Association of EMS Educators symposium. The presenter, Phillip Dickinson, presented best practices and outstanding tips on writing questions that are reliable and valid.
Measuring student performance is critical for knowing a student’s knowledge, practice, competency, and likelihood to pass certification and licensing exams. According to the presenter, “the more measurements we do the more likely it is our assessment of a students performance will be right.” He described different evaluation types, purposes, and best practices. He also discussed the difference between reliability and validity and kept coming back to those concepts throughout the presentation which was useful for understanding the best practices for question types.
His basic test preparation instructions were:
1) relate the test to the objectives
2) develop from a blue print (a determination of how many questions are needed)
3) match the test to the desired domain of learning
4) since the test length increases reliability choose an appropriate number of questions
His specific tips for true false or yes no questions, which are great for gauging what a student understands or recalls about a recently taught topic included:
1) avoid broad and general statements, trivia, and negative statements
2) keep the sentence short, < 10 words
3) focus question on one concept of idea
4) attribute opinions in the question to a source
5) keep items of an equal length
6) write an equal number of true answers as false answers
We also reviewed best practices for multiple choice items, which I write most frequently. The two BEST tips I got from the portion of this presentation were:
1) Write questions as “What should you do … ” instead of “What would you do…” The latter, what would yo do, is always correct because it is what the student would do.
2) Paint a picture of the patient instead of telling about the patient. For example, give a patient profile of vital signs and physical exam findings in the question stem that reflect a hypotensive patient instead of saying “the patient has low blood pressure.”
He shared some several important rules for Multiple Choice item construction that I will be incorporating into future assessments that I write.
Related posts:
