Best Practice and Course Reconceptualization in Online Undergraduate Quantitative Reasoning
Brief Abstract
We reduced QR attrition rates from 18% to 5% by implementing theory-based practices, including: Rationale for theory identification, construction of a philosophical framework for course implementation, collection of stakeholder input, implementation, evaluation of impact on attrition, post-implementation maintenance and communication, and institutional socialization of the new paradigmatic shift.
Presenters
Extended Abstract
“I found myself dreading the class too but I take it all back. I have learned a lot and understand that I use quantitative reasoning without even knowing it.”
-Student who just finished a QR course that used theory-based practices
For decades, student attrition has been a challenge in undergraduate general education math (UGEM). Research finds a major contributing factor is the gap in translating theory to practice. Some literature reports that studied and effective practices that were shared as long as 40 years ago are still not common practice in schools. More so, some of these best practices are not even part of teacher education programs. Higher education faces an even greater challenge in addressing this theory to practice gap since most professors and teachers of higher education are specialists in specific subject matter without formal training or knowledge of the findings from education research. In many cases, there is little motivation for faculty to explore knowledge in education research since it is very rarely acknowledged as part of the rigorous and time requirements to obtain institutional tenure. While there have been recent changes in some institutions to reward faculty for their pursuits in teaching, these initiatives are recent and have not closed the multi-decade knowledge gap between education research and practice.
This session describes University of Phoenix’s process to reduce student attrition by closing the theory-to-practice gap. In particular, the purpose of this session is to disseminate the implementation of theory-based practices in UGEM toward reducing student attrition, including: Rationale for theory identification, the construction of a philosophical framework for course implementation, collection of stakeholder input, implementation, evaluation of impact on attrition, post-implementation maintenance and communication, and institutional socialization of the new paradigmatic shift.
The first step identified the leading research-based education theories. Much research has advanced pedagogical theory in recent decades. Many such theories are now well-validated and well-accepted. We sought such modern, validated theories toward closing the gap in UGEM courses, and settled on 7 such theories: Metacognition and Affect [Dole1998], Conceptual Change [Strike1992][Chinn1993], Social Constructivism [Vygotsky1986], Academic Self-concept [Marsh1985][Bong2003], Holism [Dewey1997][Mahmoudi2012], Systemic Functional Linguistics [Halliday1992][Holliday1994], and 21st Century Knowledge Framework [Kereluik2013][Mishra2019]. These theories were selected because not only are they seminal works, but they are also in alignment with the mission and vision of the University.
The second step was to adjust each course feature to apply as many theories as possible. There were 9 course features: Discussion questions, late work policy, grade pass back to students, reading assignments, homework assignments, exams, advisory language, remediation, and rigor. Before adjusting, 0.8 theories were implemented on average per the course feature (i.e., less than 1 theory per feature), and 5.3 theories per course feature after adjusting. The point is that each course feature changed dramatically. For example, each week, course instructors post a question on the class forum for discussion questions. A student earns course points for responding to the question. Before adjusting, questions asked students to interpret the content, such as identifying and discussing some part of an expression. After adjusting, questions asked students to apply recently learned concepts to everyday experiences, requiring prediction, transfer, and application of knowledge; many questions also required students to reflect about their thinking and learning of the content to promote a self-regulative process in assessing and directing their own learning.
The third step was to run focus groups with other Associate Deans, faculty, and academic councilors; and then conduct a pilot study. The intention was to quickly evaluate the efficacy of the changes for rapid iterative improvements. We used the agile development framework, as our university had recently adopted it. Key takeaways from the focus group included: The math courses’ learning materials were not consistent with what students needed to know, students tended to be fearful of math and did not find math relevant to their goals, student dissatisfaction with the course, and segmented learning that may have hindered the ability for students to develop functional models of mathematical knowledge. The pilot study included a control group with 520 students and intervention group with 152 students during a term in October 2019. The control offering’s attrition was 14.6%, whereas the intervention’s was 5.3%. Student surveys found the intervention’s students were much less stressed and had a more positive perspective of the course than the control’s students. One student quote from the intervention’s group was, “The concepts in this class made me realize that I use math more than I thought. I used to say that I hate math and scared of it. I am no longer scared of it. I understand it more and can relate to the topics and understanding.”
The fourth step was to decide on next steps. A critical reflection was conducted to determine the next steps at both a course level and then programmatic level. The pilot data validated the feature changes, so implementation across all sections of QR1 and QR2 was pursued. The process of coming to these conclusions was achieved by engaging a programmatic evaluation lens to make sense of and interpret the data collected from the pilot study. The inputs to the decision were: Historical data, pilot data, stakeholder feedback, the University’s strategic direction, best practice from the scholarly field, and intended learning outcomes for students. The outputs associated with making the decision for moving forward were: Changes for future iterations, implementation plan, and communication plan. The pilot data showed that student attitudes and outcomes improved. This contrasted strongly with the historical data, which supported a decision to move forward with changes. However, before moving forward, data was presented to a variety of institutional stakeholders to secure buy-in and limit any potential barriers to implementation. Additional information considered was the alignment of proposed changes to University strategy, mission, and vision. The newer materials were more strongly aligned to the ability for adult learners to have meaningful experiences and aligned to supporting career advancement. Math education literature was referenced to ensure anecdotal and limited quantitative evidence was consistent with trends reported and generalized by the scholarly field. A subsequent cross check of student learning outcomes was completed to ensure proposed changes would support students through subsequent programmatic coursework. Once each of the constructs described was assessed, the decision was made to implement the changes to QR1 and QR2. The input information helped to guide decisions about improvements prior to implementation, the actual implementation plan, as well as the communication/training plans required for successful roll out of large-scale implementation.
The fifth step began with a full implementation in QR1 and QR2 courses, and then full implementation in the other UGEM courses. We compared student attrition rates (W+F) in course offerings before the feature changes and after the feature changes, using the same terms of the respective years. QR1 had 5,851 students before changes and 7,716 after changes; the attrition rate dropped from 17.5% to 4.7%. QR2 had 6,619 students before changes and 8,083 students after changes; the attrition rate dropped from 13.9% to 4.0%. Interestingly, post-term faculty surveys found an almost bimodal distribution in the faculty’s perspectives of the feature changes, i.e., while many faculty had positive perspectives of the feature changes, many other faculty felt strongly that the feature changes were worse for students. The other UGEM courses included varying degrees of theory-to-practice translation, and each had significant decreases in attrition rates.
References
Dole, J. A. and G. M. Sinatra (1998). "Reconceptualizing change in the cognitive construction of knowledge." Educational Psychologist 33(2-3): 109-128.
Strike, K. A., & Posner, G. J. (1992). A revisionist theory of conceptual change. In R. Duschl & R. Hamilton (Eds.), Philosophy of Science, Cognitive Psychology and Educational Theory and Practice (pp. 147-176). Albany, NY: State University of New York Press.
Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction. Review of Educational Research, 63, 1-49.
Thought and Language - Revised Edition (Paperback) by Lev S. Vygotsky (Author), Alex Kozulin (Editor) Publisher: The MIT Press; Revised edition (August 28, 1986) ISBN-13: 978-0262720106
Marsh, H.W., and Shavelson, R. J. (1985). Self-concept: Its multifaceted, hierarchical structure. Educ. Psychol. 20: 107–123.
Bong, M., and Skaalvik, E.M. (2003). Academic Self-Concept and Self-Efficacy: How Different Are They Really? Educational Psychology Review 15(1).
Dewey, J. (1997). Experience and Education. Publisher : Free Press; Reprint edition (July 1, 1997) ISBN-10 : 0684838281
Mahmoudi, S.; Jafari, E.; Nasrabadi, H.A.; Liaghatdar, M.J. (2012). Holistic Education: An Approach for 21 Century. International Education Studies, 5(2).
Halliday, M. K. (1992). Towards probabilistic interpretations. In E. Ventola (Ed.), Functional and systematic linguistics (pp. 39-63). Mouton.
Holliday, W. G., Yore, L. D., & Alvermann, D. E. (1994). The reading–science learning–writing connection: breakthroughs, barriers, and promises. Journal of Research in Science Teaching, 31, 877-893.
Kereluik, Kristen, et al. "What knowledge is of most worth: Teacher knowledge for 21st century learning." Journal of digital learning in teacher education 29.4 (2013): 127-140.
Mishra, Punya, et al. “Developing the future substance of STEM Education: A Concept Paper”. December 2019. https://education.asu.edu/sites/default/files/substance-of-stem-education-concept-paper.pdf