This is a work in progress paper. Classroom response systems (CRS) are educational technology tools that when paired with an appropriate pedagogy, provide increased classroom engagement in support improved teaching and learning. They do this by leveraging technology to allow every student to respond to instructor posed questions. Many of these systems, such as Learning Catalytics and clickers, collect and store a wealth of individual student response data that is aggregated to provide instructors with real time student response data. One open challenge in this setting is how to glean insights from all of the the collected response data to identify activities, specific questions, and combinations of questions that associate with student performance. A data driven analysis of student response data collected by a CRS combined with student performance data will enable individual instructors to refine and adapt their use of a CRS, thus closing the loop from an instructional design perspective. This paper presents a data science methodology and preliminary results of analyzing CRS data accumulated from daily activities in two sections of an engineering mathematics course. The data is collected from the CRS Learning Catalytics where students respond to questions in two rounds following the team-based learning model. In the first round, students answer questions individual; in the second round, they answer the same questions as a team after reviewing each other’s answers from round 1. The CRS stores each student’s responses from both rounds along with a timestamp. This study serves two purposes, 1) examine the effect of disagreement between individual responses on student performance and 2) identify which activity question scores, individual versus team-based, are associated with better exam performance, thus allowing the reduction of the number of questions. Using a methodology based on data mining and data exploration, we found three trends between round 1 and round 2 scores: students who have no score change in both rounds, students who increase their scores, and student who decrease their scores after the team-based collaboration. Our plan is to cluster class activity questions depending on the predicted performance, thus paving the way toward an improved data driven design of in class activities.
Asuman Cagla Acun Sener holds B.S. and M.S. degrees in Computer Science and Computer Engineering. She is currently pursuing a doctoral degree in Computer Science at Knowledge Discovery & Web Mining Lab, Department of Computer Science and Computer Engineering, University of Louisville. She is also working as a graduate assistant. Her research interests are educational data mining, visualization, predictive modeling and recommender systems.
Olfa Nasraoui is Professor of Computer Engineering and Computer Science, Endowed Chair of e-commerce, and the founding director of the Knowledge Discovery and Web Mining Lab at the University of Louisville. She received her Ph.D. in Computer Engineering and Computer Science from the University of Missouri-Columbia in 1999. From 2000 to 2004, she was an Assistant Professor at the University of Memphis. Her research activities include Data Mining/ Machine Learning, Web Mining, Information Retrieval and Personalization, in particular in problems involving large multiple domain, high dimensional data, such as text, transactions, and social network data. She is the recipient of the National Science Foundation CAREER Award, and the winner of two Best Paper Awards, a Best Paper Award in theoretical developments in computational intelligence at the Artificial Neural Networks In Engineering conference (ANNIE 2001) and a Best Paper Award at the Knowledge Discovery and Information Retrieval conference in Seville, Spain (KDIR 2018). She has more than 200 refereed publications, including over 47 journal papers and book chapters and 12 edited volumes. Her research has been funded notably by NSF and NASA. Between 2004 and 2008, she has co-organized the yearly WebKDD workshops on User Profiling and Web Usage Mining at the ACM KDD conference. She has served on the program committee member, track chair, or senior program committee of several Data mining, Big Data, and Artificial Intelligence conferences, including ACM KDD, WWW, RecSys, IEEE Big Data, ICDM, SDM, AAAI, etc. In summer 2015, she served as Technical Mentor/Project Lead at the Data Science for Social Good Fellowship, in the Center for Data Science and Public Policy at the University of Chicago. She is a member of ACM, ACM SigKDD, senior member of IEEE and IEEE-WIE. She is also on the leadership team of the Kentucky Girls STEM collaborative network.
Jeffrey L. Hieb is an Associate Professor in the Department of Engineering Fundamentals at the University of Louisville. He graduated from Furman University in 1992 with degrees in Computer Science and Philosophy. After 10 years working in industry, he returned to school, completing his Ph.D. in Computer Science Engineering at the University of Louisville’s Speed School of Engineering in 2008. Since completing his degree, he has been teaching engineering mathematics courses and continuing his dissertation research in cyber security for industrial control systems. In his teaching, Dr. Hieb focuses on innovative and effective use of tablets, digital ink, and other technology and is currently investigating the use of the flipped classroom model and collaborative learning. His research in cyber security for industrial control systems is focused on high assurance field devices using microkernel architectures.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.