2017 ASEE Annual Conference & Exposition

Participation in research during the process of undergraduate education in engineering and science disciplines has been shown to boost the retention of students into graduate studies in STEM fields. As a result, significant funding is being allocated for undergraduate research programs such as the Research Experience for Undergraduates (REU) program funded by the National Science Foundation (NSF). For program sites supporting a number of students, it is often suggested or required that the program administrators evaluate the effectiveness of the program. This can result in a significant amount of time for the administrator in developing evaluation tools. This paper reports on the development of surveys to track the career aspirations and changes in research self-efficacy that participants experience during a 10-week research experience.

Self-efficacy is one’s belief that they can produce desired results that are either imposed by themselves or set as expectations. Albert Bandura developed the theory pertaining to self-efficacy, and determined that four sources could influence self-efficacy including mastery experiences, vicarious experiences, social persuasion, and induction of positive physiological responses. We initially developed twelve survey items to assess perceived capabilities to successfully complete various research-related tasks including documenting laboratory work, exploring prior literature and presenting their research work through technical presentations.

Participants were asked to use a six-item Likert response scale (“pretty certain, mostly certain, completely certain” and “somewhat agree, agree, strongly agree”) to rank their agreement with the engineering self-efficacy items at two time points during the summer research program: the first week (Time Point 1) and last week of the program (Time Point 2). In the pilot study, the mean pre-assessment self-efficacy score was lower than the post-assessment score, indicating that students felt more capable to do research at the completion of the program. Specifically, students’ self-efficacy increased for 10 of the 12 tasks. The largest gains in self-efficacy across the program were for conducting literature reviews, identifying opportunities for external funding (scholarships/fellowships), applying to graduate school, and communicating scientific findings through oral presentation. Focus group interview data confirmed these findings and pointed to specific REU programming that boosted students’ self-efficacy in these areas.

Based on these initial findings and comparison with other research self-efficacy scales developed at the Colorado School of Mines and University of North Carolina, we further improved the scale. For example, we modified the survey to include items that more clearly communicated the research context. We now utilize the item ‘I can document my research in a laboratory notebook’ instead of ‘I can document my work within a laboratory notebook’. We also changed “I can perform a literature review” to “I am comfortable with reviewing papers relevant to my research” to ensure that “literature review” is not being confused with English literature or general scientific literature.

This survey will be of interest to other program administrators to help assess their own undergraduate materials research programs. Review of the current awards from the National Science Foundation, show that there currently are 68 REU sites that could utilize this survey and currently only three of these discuss their plan to utilize an evaluation tool other than student artifacts either in the abstract of their awards or their REU website.