MENU
  • ASEE Logo
  • Give
    Give
    ASEE Donations...
    Classified Volunteer
    Login Join
  • Join Login Volunteer Classified Give
    Give
    ASEE Donations...



About
  • Overview
    • Mission, Vision, Goals
    • Public Policy Statements
    • Constitution
    • Bylaws
    • Organizational Structure
    • Investment Policy
    • Financial Policy
  • Our History
  • Staff Contacts
  • Leadership
    • Board Of Directors
    • Academy Of Fellows
    • Past Board Members
    • Advisory Committees
    • Representatives to External Organizations
    • Executive Director's Message
    • Meeting Minutes
  • Volunteer
  • Careers at ASEE
  • Privacy Statement
I Am A...
  • Member
      Login Required
    • Your Member Page
    • Membership Directory
    • Financials
    • No Login Required
    • Awards
    • Divisions, Fellows, and Campus Reps
    • Sections and Zones
    • Resources
  • Prospective Individual/Organizational Member
    • About ASEE
    • Individual Membership
    • Institutional Membership
    • Major Activities
  • Donor
  • Prospective Partner or Sponsor
  • Advertiser
  • Fellowship Seeker
    • About Fellowships
    • High School
    • Undergraduate
    • Graduate
    • Post-Doctoral
    • Other Programs
Events
  • Conferences and Meetings
    • 2021 Annual Conference & Exposition
    • 2020 Virtual Annual Conference & Exposition
    • Section & Zone Meetings
  • Council Events
    • Conference for Industry and
      Education Collaboration (CIEC)
    • Engineering Deans Institute (EDI)
    • Research Leadership Institute (RLI) (Formerly ERC)
    • Engineering Technology Leaders Institute (ETLI)
    • EDC Public Policy Colloquium (PPC)
  • Featured Events
    • Frontiers in Education
    • NETI
    • CoNECD
    • First Year Engineering Experience
    • Workforce Summit
  • Future Conference Dates
Publications
  • News
    • Newsletters
    • eGFI
    • Division Publications
  • Journals and Conference Papers
    • Overview
    • Journal of Engineering Education
    • Advances in Engineering Education
    • Conference Proceedings
    • Section Proceedings
    • Zone Proceedings
    • PEER
    • Plagiarism
  • Monographs and Reports
  • Prism Magazine
  • Data
    • Profiles of E&ET Colleges
    • Case Study Series: Engineering-Enhanced Liberal Education
Impact
  • Public Policy Statements
  • Data Analysis
  • Annual Reports
  • Diversity
Education & Careers
  • Academic Job Opportunities
  • Course Catalog
  • Engineering Education Research and Innovation
    • Engineering Education Community Resource
  • PreK-12
    • eGFI Teachers
    • eGFI Students
  • Engineering Teacher PD Endorsement
Calendar
2020 Annual Conference
The ASEE 2020 Virtual Annual Conference content is available.
See More....
  • International Forum
    • Past Forums
  • 2020 Research Leadership Institute(RLI) (Formerly ERC)
    • Overview
    • Registration
    • Housing
    • Program Schedule
    • ERC Past Conferences
  • 2018 Engineering Technology Leaders Institute (ETLI)
    • Engineering Technology Leaders Institute (ETLI)
    • Registration
    • Housing
    • Program Schedule
    • ETLI Sponsorship Options
  • 2017 Global Colloquium
  • Pre K-12 Workshop
    • Call for Proposals
    • Registration
    • Housing
    • Sponsors
    • Program Schedule
    • Past Conferences
  • STEP Grantees Meeting
  • Annual Conference
    • Past Conferences
  • International Forum
    • Past Forums
  • 2020 Research Leadership Institute(RLI) (Formerly ERC)
    • Overview
    • Registration
    • Housing
    • Program Schedule
    • ERC Past Conferences
  • 2018 Engineering Technology Leaders Institute (ETLI)
    • Engineering Technology Leaders Institute (ETLI)
    • Registration
    • Housing
    • Program Schedule
    • ETLI Sponsorship Options
  • 2017 Global Colloquium
  • Pre K-12 Workshop
    • Call for Proposals
    • Registration
    • Housing
    • Sponsors
    • Program Schedule
    • Past Conferences
  • STEP Grantees Meeting
  • Annual Conference
    • Past Conferences

2018 ASEE Annual Conference & Exposition

The Effects of Professional Development and Coaching on Teaching Practices

Presented at First-year Programs Division: Best Papers

Engineering faculty, across six disciplines, at a large university in the Southwest participated in professional development supported by an NSF Improving Undergraduate STEM Education grant during the 2016-17 academic year. The two-semester sequence involved faculty participating in bi-weekly discipline-based workshops during Fall 2016. These workshops promoted student-centered classrooms, using formative feedback to refine teaching and learning, and emphasizing real-world connections. During Spring 2017, faculty were to focus on classroom implementation while also participating in discipline-based communities-of-practice. The communities-of-practice sessions focused on themes featured in the workshops, but allowed for more give-and-take, flexibility of topics, and sharing of instructional ideas.

Throughout the academic year, classroom practices of the faculty were evaluated by trained observers using the Reformed Teaching Observation Protocol (RTOP). The RTOP is a 25-item validated observation protocol that has been utilized in numerous middle school through postsecondary projects. The RTOP focuses on gauging the degree to which learning environments are student-centered in science, engineering, and mathematics. Complete observation data were available for 26 faculty members who were observed twice early during the fall semester (pre-observations) and twice late during the spring semester (post-observations). The two pre-observations occurred during the fall semester and post-observations occurred during the last six weeks of the spring semester. It is noted that faculty were also observed twice early during the spring semester, but this study focuses on the changes between the first and last sets of observations.

Because many faculty members became interested in the use of the RTOP and were eager to improve their instructional approaches, an unplanned strategy to support faculty emerged. Faculty members (n = 21) requested to receive one-on-one feedback with an observer who was also an experienced K-12 instructional coach. These coaching sessions typically honed in on a few RTOP items, provided the instructor with insight about student engagement, and offered concrete suggestions for improving instructional practices.

The purpose of this study was to determine the effects of (1) participation in the professional development, and (2) receiving coaching, on instructional practices, as measured by the RTOP. Analysis indicated RTOP mean scores increased significantly (n = 26, p = .014) from pre-observations (x̄ = 57.1) to post-observations (x̄ = 68.6).

The comparison of the small group of five faculty members, who opted to not receive coaching, to the 21 faculty members, who opted for coaching, did not allow for robust comparison. However, although not statistically significant, the coached group had notably greater mean RTOP gains from pre-to-post (Δ = 12.8, SD = 12.8) than the non-coached group (Δ = 6.2, SD = 8.8). Normalized gain scores (aka Hake scores) were calculated to provide comparisons independent of pre-RTOP scores. The normalized gain scores are calculated as a proportion of change in RTOP scores compared to possible change. Here too, although not statistically significant, there were indications of the benefits of being coached. The coached group had conspicuously greater normalized gain scores (g = 0.20) than the non-coached group (g = 0.13).

Authors
  1. Dr. Eugene Judson Orcid 16x16http://orcid.org/https://0000-0002-0124-8476 Arizona State University [biography]

    Eugene Judson is an Associate Professor of for the Mary Lou Fulton Teachers College at Arizona State University. He also serves as an Extension Services Consultant for the National Center for Women and Information Technology (NCWIT). His past experiences include having been a middle school science teacher, Director of Academic and Instructional Support for the Arizona Department of Education, a research scientist for the Center for Research on Education in Science, Mathematics, Engineering and Technology (CRESMET), and an evaluator for several NSF projects. His first research strand concentrates on the relationship between educational policy and STEM education. His second research strand focuses on studying STEM classroom interactions and subsequent effects on student understanding. His work has been cited more than 2200 times and he has been published in multiple peer-reviewed journals such as Science Education and the Journal of Research in Science Teaching.

  2. Lydia Ross Arizona State University [biography]

    Lydia Ross is a doctoral student and graduate research assistant at Arizona State University. She is a third year student in the Educational Policy and Evaluation program. Her research interests focus on higher education equity and access, particularly within STEM.

  3. Kara L. Hjelmstad Arizona State University [biography]

    Kara Hjelmstad has currently worked as a faculty associate and student teacher supervisor for Mary Lou Fulton Teachers College at Arizona State University. After earning a BA degree in elementary education and an M.Ed. degree in curriculum and instruction, she spent twelve years teaching K-5 and enrichment at the elementary level.

    In 2010, Kara began teaching courses and supervising student teachers at ASU. Kara is TAP certified, an evaluation system designed to improve teaching effectiveness and student achievement. The TAP evaluation involves classroom observations, coaching, and feedback/reflection for professional growth. Kara has worked with 60+ student teachers in various subjects at the pre-K through 12th grade level, and conducted over 100 TAP classroom observations.

    Since the fall of 2016, Kara has been working with the JTFD Project, an NSF grant working to improve active learning in engineering education. She has completed 300 RTOP classroom observations in ASU engineering courses (civil, environmental, construction, chemical, aero/mechanical, materials, transportation, and biomedical engineering). The RTOP or Reformed Teaching Observation Protocol, is a rubric designed to assess student centered instruction in math and science. Kara also provided instructional coaching for 37 engineering faculty grant participants, after their teaching observations.

  4. Prof. Stephen J. Krause Arizona State University [biography]

    Stephen Krause is professor in the Materials Science Program in the Fulton School of Engineering at Arizona State University. He teaches in the areas of introductory materials engineering, polymers and composites, and capstone design. His research interests include evaluating conceptual knowledge, misconceptions and technologies to promote conceptual change. He has co-developed a Materials Concept Inventory and a Chemistry Concept Inventory for assessing conceptual knowledge and change for introductory materials science and chemistry classes. He is currently conducting research on NSF projects in two areas. One is studying how strategies of engagement and feedback with support from internet tools and resources affect conceptual change and associated impact on students' attitude, achievement, and persistence. The other is on the factors that promote persistence and success in retention of undergraduate students in engineering. He was a coauthor for best paper award in the Journal of Engineering Education in 2013.

  5. Prof. Robert J. Culbertson Arizona State University [biography]

    Robert J. Culbertson is an Associate Professor of Physics. Currently, he teaches introductory mechanics and electrodynamics for physics majors and a course in musical acoustics, which was specifically designed for elementary education majors. He is director of the ASU Physics Teacher Education Coalition (PhysTEC) Project, which strives to produce more and better high school physics teachers. He is also director of Master of Natural Science degree program, a graduate program designed for in-service science teachers. He works on improving persistence of students in STEM majors, especially under-prepared students and students from under-represented groups.

  6. Dr. Keith D. Hjelmstad Arizona State University [biography]

    Keith D. Hjelmstad is President's Professor of Civil Engineering in the School of Sustainable Engineering and the Built Environment at Arizona State University.

  7. Dr. Lindy Hamilton Mayled Arizona State University [biography]

    Lindy Hamilton Mayled is the Director of Instructional Effectiveness for the Fulton Schools of Engineering at Arizona State University. She has a PhD in Psychology of Learning, Education, and Technology from Grand Canyon University. Her research and areas of interest are in improving educational outcomes for STEM students through the integration of active learning and technology-enabled frequent feedback. Prior to her role and Director of Instructional Effectiveness, she worked as the Education Project Manager for the NSF-funded JTFD Engineering faculty development program, as a high school math and science teacher, and as an Assistant Principal and Instructional & Curriculum Coach.

  8. Prof. James A. Middleton Arizona State University [biography]

    James A. Middleton is Professor of Mechanical and Aerospace Engineering and Director of the Center for Research on Education in Science, Mathematics, Engineering, and Technology at Arizona State University. For the last three years he also held the Elmhurst Energy Chair in STEM education at the University of Birmingham in the UK. Previously, Dr. Middleton was Associate Dean for Research in the Mary Lou Fulton College of Education at Arizona State University, and Director of the Division of Curriculum and Instruction. He received his Ph.D. in Educational Psychology from the University of Wisconsin-Madison in 1992, where he also served in the National Center for Research on Mathematical Sciences Education as a postdoctoral scholar.

Download paper (335 KB)

Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.

» Download paper

« View session


  • Follow Us
  • twitter
  • facebook
  • youtube
  • instagram
  • linkedin
  • 1818 N Street N.W. Suite 600, Washington DC 20036

  • Telephone: 202.331.3500 | Fax: 202.265.8504

  • © 2020 Copyright: ASEE.org All rights reserved. Privacy Policy.