The Quest for Clarity in Engineering Judgment
The Accreditation Board for Engineering and Technology (ABET) states that undergraduate students must possess sound engineering judgment skills upon graduation. However, these skills have never been given a concrete definition. How can students obtain a skill that instructors are unsure how to define, teach and assess?
Two engineers at the University of Pittsburgh Swanson School of Engineering are blazing trails in an exploratory study aiming to develop an expert-based definition and instructional tools for engineering judgment.
Renee Clark, principal investigator and associate professor of industrial engineering, and Samuel Dickerson, associate professor and vice chair for education in the computer engineering program, received $349,551 from the National Science Foundation (NSF) for their proposal, “Defining and Assessing Engineering Judgment.” In addition to creating an expert-driven definition of engineering judgment, investigators are building assessment tools and instructional resources to support the formation of judgment skills for today’s engineers.
“We’ve found that students are uncomfortable when placed in situations with no clear-cut solution and asked to use their judgment to make the best decision,” Dickerson said. “Let’s say Design A, Design B and Design C all have clear advantages and disadvantages; students aren’t sure how to judge what the best way forward is. Renee and I see this happen in the context of design and analysis all the time.”
Turning an Elusive Concept into an Indisputable Definition
“Engineering judgment is critical whether you are an engineering professional, researcher or educator,” Clark said. “It’s currently a common term but an elusive notion. This research starts the conversation we need to have about how to support the formation of today’s engineers through strong judgment skills.”
The study begins with a fifteen-person Delphi panel, which consists of engineering experts who have either previously written about aspects of engineering judgment; taught design, data analysis or computational methods courses; or have served as ABET evaluators. With the panel’s expertise, the researchers aim to create a holistic definition of engineering judgment by incorporating a variety of perspectives.
Clark and Dickerson hand-picked experts by combing through engineering literature, networking at various conferences and engineering events, and receiving support from colleagues in the Swanson School.
How to Define the Future of Judgment Skills
With engineering judgment defined, investigators can transition into the assessment process. Clark and Dickerson will create a rubric highlighting the most important aspects of engineering judgment based on the definition provided by the Delphi panel. Project work in three courses covering competencies related to judgment (i.e., design and data analysis) will be assessed using the rubric to determine the extent of which judgment is present. The investigators will also create a coding scheme to conduct qualitative analysis of student interviews, written reflections and survey responses to identify where aspects of judgment are present in student work.
The study culminates in the creation of educational modules that provide explicit instruction on engineering judgment for use by educators and students. The modules are intended to be an asset for instructors applying engineering judgment to the context of their courses. Clark and Dickerson will promote the adoption of assessment tools and educational modules through a website, workshops and presentations at national conferences, collaborating engineering schools, and industry settings.
“We need to figure out how we can best support students in developing engineering judgment, which is an aspect of the ABET accreditation process,” Clark said. “This also includes considering demographic variables such as gender or ethnicity in the creation of educational materials.”