This Complete Research paper investigates if the amount and type of preparation for the ACT is related to the amount of variability in a first year GPA of engineering students that can be explained by ACT scores. Looking specifically at engineering student college success, many studies examining engineering student’s first-year performance have used scores on either the ACT or the Scholastic Aptitude Test (SAT) as either a predictor or covariate (DeBerard, Spielmans, & Julka, 2004; French, Immekus, & Oakes, 2005). In some studies of general college student performance, ACT/SAT scores have been shown to not be statistically significant when related to college GPA (Pritchard & Wilson, 2003). Other research has shown a decreasing trend in the amount of variability in first-year engineering student’s GPA that can be explained by ACT scores (Maruyama, 2012). Since 2012 research at our institution has also shown a decreasing trend in the amount of variability in first year GPA of engineering students that can be explained by ACT scores from nearly 20% in 2012 to 5% in 2017. While some universities have stopped using standardized entrance exams, due to the belief that they do not accurately predict success (PBS, 2019), other universities use scores on these exams as a criteria for entrance to the university and honors programs, as well as for scholarship awards.
There are multiple methods of ACT/SAT test preparation available to students with costs ranging from free to thousands of dollars. In 2017 after analyzing scores from over 200,000 students, the College Board concluded the students who spent at least 20 hours of practice and student with the free official SAT practice on Khan Academy increased their score on average 115 points, which was twice the increase of students who did not use the free service (College Board, 2017). Research looking at other methods of preparation have also shown an increase in test scores related to test preparation (Moore et al, 2018).
The purpose of this research is start to understand why students’ ACT scores might be losing predictive ability. More specifically we seek to understand if the ACT scores of engineering students who complete more hours or different types of test preparation might explain less variability in their first year GPA, than scores earned by students who did less or no preparation. This study is relevant not only due to the high stakes impact of ACT scores at many universities, but also the potential to disadvantage students who do not have the means or are not encouraged to participate in costly and time consuming ACT/SAT preparation; student, who might have equal probability of success as someone who received a higher score due to their method of test preparation.
Data on ACT/SAT preparation methods that students used and estimates of time spent in each method of preparation were self-reported on a survey administered to first-year engineering students the first week of class in the fall semester of 2017. Students were asked for each of the following methods of preparation: self- study, courses, one on one tutoring: When preparing for your standardized College Entrance Exam (e.g., SAT, ACT), please estimate how many hours total you spent in each activity listed. Possible responses were 0, 1-5 hours, over 5 hours.
The same survey questions were administered at two large metropolitan universities. At one institution, class time was given for students to complete the survey. At the other institution students were given the survey as a part of a homework assignment. The resulting sample size was over 1,500 and the response rate was over 90%. Initial analysis of the responses to survey questions show variability in the methods and the time students spent preparing for the ACT or SAT. Demographic and performance data were extracted from university records. The data will be analyzed using regression analysis. If the subsamples are large enough to ensure statistical validity, gender, race and Pell grant eligibility will be considered in the analysis.
If the results of this study show a significant difference in the predictive ability of ACT scores based on student’s preparation methods and time, this could have a major impact on engineering education research. Specifically researchers trying to explain variability in engineering student performance using ACT scores would have to also consider the variability in preparation for the test.
Are you a researcher? Would you like to cite this paper?
Visit the ASEE document repository at
for more tools and easy citations.