2018 ASEE Annual Conference & Exposition

Assessing Faculty and Organizational Change in a Professional Development Program with Workshops and Disciplinary Communities of Practice

Presented at Supporting Faculty in Course Development and Pedagogy

To achieve more effective teaching and learning in undergraduate engineering education, professional development needs to promote change toward innovation of use of evidence-based instructional strategies (EBIS) in faculty beliefs and practice as well as change in organizational attitudes and norms. To develop an effective program, a strategy for change needs to be utilized and assessed at the individual level and the organizational level. Rogers Diffusion of Innovation (DOI) is an individual change model which consists of five steps of a progression towards any type of adoption of innovation change such as sports, business or teaching. The steps for adoption of change consist of awareness, interest, evaluation, trial, and adoption. Coburn’s organizational innovation (COI) change model for organizations consists of four components necessary to support broader change in an educational organization, which include the following. Depth is deep change in faculty beliefs and practice. Sustainability of Innovation refers to flexibility of innovation in differing contexts or disciplines. Spread of Innovation refers to change in beliefs, norms, and principles of individuals across an organization. Shift of Ownership refers to shift in ownership from external facilitators to internal ownership by individuals and their organization. These change theories have been used to develop and assess a large scale NSF IUSE project at a large southwestern university called Just-in-Time-Teaching with Two Way Formative Feedback for Multiple Disciplines (JTFD). The first year of the project has trained 43 faculty in four disciplines using a train-the-trainer model with one semester of 8 biweekly workshops followed by a second semester of 6 disciplinary community of practice (CoP) discussion sessions.

Assessment of elements of the two change theories was done with pre-post measurements of participants with surveys, questions and classroom observations. For the DOI model, Awareness of evidence-based instructional strategies (EBIS) increased an average of 31% for topics such as instructional design and active learning. For DOI Interest and Evaluation a survey was created using expectancy-value theory called Value, Expectancy, and Cost of Testing Educational Reforms Survey (VECTERS). For three EBIS strategies of real-world applications, student-to-student discussions, and formative feedback there were 4% to 12% gains for motivation (expectancy, value, and lowered cost). For DOI Trial 91% of faculty agreed or strongly agreed that “the topics discussed provided me with new ideas for implementation and/or reaffirmed strategies I am currently implementing.” For DOI Adoption, a classroom observation tool, RTOP, (Reformed Teaching Observational Protocol) showed a gain of 22% of classroom practice toward EBIS. For the COI model, Depth of Faculty Beliefs there was an average gain of 26% in faculty who said that they were sometimes or frequently using EBIS strategies of active learning, cooperative learning, objectives and Blooms’s taxonomy. For Sustainability 96% of faculty agreed or strongly agreed that the JTFD project has been successful in creating CoPs which support innovation, implementation, and open dialogue between colleagues.” For COI Spread of Innovation, 91% of faculty agreed or strongly agreed that “the topics discussed provided me with new ideas for implementation and/or reaffirmed strategies I am currently implementing.” For Shift of Ownership 100% of faculty agreed or strongly agreed that “the tools, strategies, and interactions in the JTFD project would be of value to their future instructional practice and career success.” Overall, assessment using the DOI and COI change models demonstrate faculty change and organizational change toward innovation through use of EBIS in teaching and learning.

Authors

Prof. Stephen J. Krause Arizona State University [biography]

Stephen Krause is professor in the Materials Science Program in the Fulton Schools of Engineering at Arizona State University. He teaches in the areas of introductory materials engineering, polymers and composites, and capstone design. His research interests include faculty development, evaluating conceptual knowledge change, misconceptions, and technologies to promote conceptual change. He has co-developed a Materials Concept Inventory and a Chemistry Concept Inventory for assessing conceptual knowledge and change for introductory materials science and chemistry classes. He is currently conducting research on a large scale NSF faculty development project. His team is studying how workshops on strategies of engagement and feedback with support from internet tools and resources affect faculty beliefs, classroom practice, and development of disciplinary communities of practice and associated student achievement. He was a coauthor for the best paper award in the Journal of Engineering Education in 2013 and this year has received the Michael Ashby Outstanding Materials Educator Award from the Materials Division of ASEE.
Prof. Keith D. Hjelmstad Arizona State University [biography]

Keith D. Hjelmstad is President's Professor of Civil Engineering in the School of Sustainable Engineering and the Built Environment at Arizona State University.
Dr. Eugene Judson http://orcid.org/https://0000-0002-0124-8476 Arizona State University [biography]

Eugene Judson is an Associate Professor of for the Mary Lou Fulton Teachers College at Arizona State University. He also serves as an Extension Services Consultant for the National Center for Women and Information Technology (NCWIT). His past experiences include having been a middle school science teacher, Director of Academic and Instructional Support for the Arizona Department of Education, a research scientist for the Center for Research on Education in Science, Mathematics, Engineering and Technology (CRESMET), and an evaluator for several NSF projects. His first research strand concentrates on the relationship between educational policy and STEM education. His second research strand focuses on studying STEM classroom interactions and subsequent effects on student understanding. He is a co-developer of the Reformed Teaching Observation Protocol (RTOP) and his work has been cited more than 2200 times and he has been published in multiple peer-reviewed journals such as Science Education and the Journal of Research in Science Teaching.
Prof. James A. Middleton Arizona State University [biography]

James A. Middleton is Professor of Mechanical and Aerospace Engineering and Director of the Center for Research on Education in Science, Mathematics, Engineering, and Technology at Arizona State University. For the last three years he also held the Elmhurst Energy Chair in STEM education at the University of Birmingham in the UK. Previously, Dr. Middleton was Associate Dean for Research in the Mary Lou Fulton College of Education at Arizona State University, and Director of the Division of Curriculum and Instruction. He received his Ph.D. in Educational Psychology from the University of Wisconsin-Madison in 1992, where he also served in the National Center for Research on Mathematical Sciences Education as a postdoctoral scholar.
Prof. Robert J. Culbertson Arizona State University [biography]

Robert J. Culbertson is an Associate Professor of Physics. Currently, he teaches introductory mechanics and electrodynamics for physics majors and a course in musical acoustics, which was specifically designed for elementary education majors. He is director of the ASU Physics Teacher Education Coalition (PhysTEC) Project, which strives to produce more and better high school physics teachers. He is also director of Master of Natural Science degree program, a graduate program designed for in-service science teachers. He works on improving persistence of students in STEM majors, especially under-prepared students and students from under-represented groups.
Dr. Casey Jane Ankeny Northwestern University [biography]

Casey J. Ankeny, PhD is an Assistant Professor of Instruction in Biomedical Engineering at Northwestern University. Casey received her bachelor’s degree in Biomedical Engineering from the University of Virginia in 2006 and her doctorate degree in Biomedical Engineering from Georgia Institute of Technology and Emory University in 2012 where she studied the role of shear stress in aortic valve disease. Currently, she is investigating cyber-based student engagement strategies in flipped and traditional biomedical engineering courses. She aspires to understand and improve student attitude, achievement, and persistence in student-centered courses.
Dr. Ying-Chih Chen Arizona State University [biography]

Ying-Chih Chen is an assistant professor in the Division of Teacher Preparation at Mary Lou Fulton Teachers College at Arizona State University in Tempe, Arizona.

His research takes two distinct but interrelated paths focused on elementary students’ learning in science and engineering as well as in-service science teachers’ professional development. The first focus involves how language as a learning tool improves students’ conceptual understandings, literacy, and representation competencies in science. His second research focus is on how in-service teachers develop their knowledge for teaching science and engineering in argument-based inquiry classrooms. This research is aimed at developing measures of teachers’ Pedagogical Content Knowledge (PCK) for adopting the argument-based inquiry approach, as well as developing tools to capture the interactive nature of PCK.
Lydia Ross Arizona State University [biography]

Dr. Lydia Ross is a clinical assistant professor in the Mary Lou Fulton Teachers College. She also serves as the executive director of the Association for Education Finance & Policy. She holds a PhD in Educational Policy and Evaluation from Arizona State University. Her research focuses on equity and access and in higher education, with a focus on STEM.
Dr. Lindy Hamilton Mayled Arizona State University [biography]

Lindy Hamilton Mayled is the Director of Instructional Effectiveness for the Fulton Schools of Engineering at Arizona State University. She has a PhD in Psychology of Learning, Education, and Technology from Grand Canyon University. Her research and areas of interest are in improving educational outcomes for STEM students through the integration of active learning and technology-enabled frequent feedback. Prior to her role and Director of Instructional Effectiveness, she worked as the Education Project Manager for the NSF-funded JTFD Engineering faculty development program, as a high school math and science teacher, and as an Assistant Principal and Instructional & Curriculum Coach.
Kara L. Hjelmstad Arizona State University [biography]

Kara Hjelmstad has currently worked as a faculty associate and student teacher supervisor for Mary Lou Fulton Teachers College at Arizona State University. After earning a BA degree in elementary education and an M.Ed. degree in curriculum and instruction, she spent twelve years teaching K-5 and enrichment at the elementary level.

In 2010, Kara began teaching courses and supervising student teachers at ASU. Kara is TAP certified, an evaluation system designed to improve teaching effectiveness and student achievement. The TAP evaluation involves classroom observations, coaching, and feedback/reflection for professional growth. Kara has worked with 60+ student teachers in various subjects at the pre-K through 12th grade level, and conducted over 100 TAP classroom observations.

Since the fall of 2016, Kara has been working with the JTFD Project, an NSF grant working to improve active learning in engineering education. She has completed 300 RTOP classroom observations in ASU engineering courses (civil, environmental, construction, chemical, aero/mechanical, materials, transportation, and biomedical engineering). The RTOP or Reformed Teaching Observation Protocol, is a rubric designed to assess student centered instruction in math and science. Kara also provided instructional coaching for 37 engineering faculty grant participants, after their teaching observations.

Download paper (394 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session