TWO INSTRUMENTS FOR ASSESSING DESIGN OUTCOMES OF CAPSTONE PROJECTS

A “good” design process is perhaps best defined by its output—good design processes produce good design outcomes. As part of an NSF-funded research effort to better understand student design processes, we developed two assessment instruments to measure the “goodness” of a design outcome. This paper describes the development and validation of the two instruments, presents the instruments and their implementation, and reports validation statistics on the initial data collected. 1. Introduction A common goal of many engineering design capstone courses in the US and elsewhere is to teach the students a “good” design process. In fact, many course instructors evaluate student design teams in these courses primarily by how well they define and/or follow a prescribed process. The underlying assumption is that good design processes lead to good design outcomes—but is that true? And if so, how can we know whether a design process is good? We propose that the goodness of a design process should be measured by the quality of its outcomes. As part of a larger to study to better understand student engineering design processes, we needed a way to measure design outcomes. Development and successful implementation of a versatile capstone course assessment and evaluation system is potentially useful. Lack of effective assessment and evaluation tools can lead to false or inaccurate conclusions about the goodness of design processes. Yet, considering the ubiquitous presence of capstone design courses in almost every engineering curriculum, outcomes assessment of these courses is perhaps among the most under-researched topics in engineering education. Cost, time and quality are the three basic performance measures attached to any process. In the capstone design projects we studied, time can measured in terms of number of weeks of total design time, e.g., one 15-week semester. The cost can be measured by the number of person hours devoted to the project. This paper, however, focuses on quality measurement, specifically the development of two distinct instruments designed to measure the quality of a design outcome, the Client Satisfaction Questionnaire (CSQ) and the Design Quality Rubric (DQR). After a review of the common assessment strategies currently in use for senior capstone courses, the ensuing sections review the development, validation and data collection of the two