Development of two tier diagnostic items based on ontology in the topic of the particulate nature of matter

The purpose of this study was to develop an instrument assessing student understanding on the particulate nature of matter by utilizing ontological categories as theoretical lens. We particularly focused on: 1) portraying the ontological basis of related misconceptions 2) the way student feedback during pilot testing aided item development, 3) student thinking as revealed through their provided feedback. Aligned with content in the middle school curricula, an assessment instrument of 15 distractor-driven, multiple-choice, two-tier items were constructed. Then, revisions were made to the items according to expert suggestions, ensuring content validity. Subsequent to content validity work, to utilize student feedback for the improvement of the items and the validity of the inferences, we pilot tested our instrument in the second semester of 2007-2008 with 178 students attending a middle school and a private tutoring center in Istanbul. In the pilot test, the students were asked 14 questions about each item. After the pilot testing, most of the items were revised and improved according to the students' feedback. In this article we discuss the development process and exemplify the revisions made according to student feedback and results from the pilot testing.