2020 ASEE Virtual Annual Conference Content Access

In design, reflection is a central practice that helps designers evaluate past strategies, synthesis knowledge gained and plan future actions. However, the design process is fluid as distinct design stages may happen in a different order and repeat or cycle in a sequence unique to the design context and designers involved. Reflection is a key part of the this process, and similarly can happen in different patterns. While past work in design reflection has investigated areas such as reflection content, relationships between reflection and design outcomes and methods to incorporate meaningful reflection in design education, less work has looked at how reflection is distributed over the design process. We seek to address this gap by proposing to analyze designers’ reflection process through a visual analytics approach to generate time plots of students’ reflection activities in context with other key associated design activities. Said time plots will aim to visualize several features of students’ reflection process, such as the timing and duration of a reflection episode, the intensity of the episode as measured by the size of reflection notes and transitions between reflection and other design activities. Identifying common reflection modes student designers employ, such as long intense episodes clustered near the beginning, will help uncover more and less effective reflection strategies and can serve as insight for educators to support students to reflect during design.

Data for this paper comes from 51 high school students from a New England public school who participated in the Net-Zero Energy design project. In the Net-Zero Energy project students are challenged to design a home that conserves or generates enough energy to compensate for all of the home’s energy consumption, while remaining under a set budget and meeting several aesthetic criteria. Students completed the challenge in Energy3D, a computer-aided-design (CAD) platform that has a simulation engine for estimating energy production and consumption, an integrated design journal and the ability to automatically log students design and journal activities. Students were asked to document and reflect on their progress throughout the design challenge but given flexibility to decide when and how to use the journal.

From this data we will produce time plots for each student depicting their design modeling, analysis and reflection processes on a multiplot visualization. The first round of analysis of the time plots and their features will seek to discover the variety of ways students reflect over their time designing. A secondary analysis will aim to cluster students’ time plots into unique reflection modes (i.e. patterns) in how and when students reflect. After identifying common reflection modes, two raters will independently apply these categories to students’ time plots and inter-rater reliability will be calculated to ensure the categories are robust. The emergent reflection modes hold promise to advance our understanding of students’ varying reflection processes and may also provide a foundation for developing new scaffolds to promote student reflection at critical junctures in design, particularly for novice designers.