The Statistics Concept Inventory: Development and Analysis of a Cognitive Assessment Instrument in Statistics

The Statistics Concept Inventory (SCI) is a multiple choice test designed to assess students’ conceptual understanding of topics typically encountered in an introductory statistics course. This dissertation documents the development of the SCI from Fall 2002 up to Spring 2006. The first phase of the project essentially sought to answer the question: “Can you write a test to assess topics typically encountered in introductory statistics?” Book One presents the results utilized in answering this question in the affirmative. The bulk of the results present the development and evolution of the items, primarily relying on objective metrics to gauge effectiveness but also incorporating student feedback. The second phase boils down to: “Now that you have the test, what else can you do with it?” This includes an exploration of Cronbach’s alpha, the most commonly-used measure of test reliability in the literature. An online version of the SCI was designed, and its equivalency to the paper version is assessed. Adding an extra wrinkle to the online SCI, subjects rated their answer confidence. These results show a general positive trend between confidence and correct responses. However, some items buck this trend, revealing potential sources of misunderstandings, with comparisons offered to the extant statistics and probability educational research. The third phase is a re-assessment of the SCI: “Are you sure?” A factor analytic study favored a unidimensional structure for the SCI, although maintaining the likelihood of a deeper structure if more items can be written to tap similar topics. A shortened version of the instrument is proposed, demonstrated to be able to maintain a reliability nearly identical to that of the full instrument. Incorporating student feedback and a faculty topics survey, improvements to the items and recommendations for further research are proposed. The state of the concept inventory movement is assessed, to offer a comparison to the work presented on the SCI. Finally, the dissertation concludes with a summary of the four years’ progress, acknowledging that work is never complete but that the results thus far place the SCI in a strong position to grow for years to come.