In Spring of 2009, researchers attempted to assess the impact of the tool array on student affect through the use of 5-point Likert scale surveys. Three sections of pre-algebra students, enrolled in classes where they were encouraged to use the tool array, were asked to complete a survey at the beginning of the semester, and were asked to answer the same questions shortly before the end of the semester. The surveys had five questions in common.
1. On a scale of 1 to 5, how would you rate your level of anxiety about taking this pre-algebra math course? (1 = No Anxiety to 5 = Extreme Anxiety)
2. On a scale of 1 to 5, how would you rate your level of motivation to succeed in this pre-algebra course? (1 = No Motivation to 5 = Extreme Motivation)
3. On a scale from 1 to 5, rate the likelihood that you will enroll in an online math course here at the college. (1 = Not Likely to 5 = Very Likely)
4. On a scale from 1 to 5, rate the likelihood that you will transfer to a four year school after leaving the college. (1 = Not Likely to 5 = Very Likely)
5. On a scale from 1 to 5, rate the likelihood that you will pursue a career that involves a large amount of mathematics. (1 = No Way!!!! to 5 = Very Likely)
Seventy-nine students completed both surveys. The responses for the pre and post surveys were paired and tested for significant change using a two-tailed paired t-test with α = .05. The students expressed a significant reduction in anxiety over the course of the semester (0.399 point average decrease in score; p-value = 0.0089), as well as a significant improvement in motivational level (0.184 point average decrease in score; p-value = 0.0421). These were the results that the tool array was intended to produce. The greatest change was in the willingness to enroll in an online math class (2.867 point average increase in score; p-value < 0.0001). This suggested to the researchers that online tool arrays of this nature might serve as a means to help students transition from traditional classroom settings to classes in the online environment. The last two questions were intended to assess any long-term effects of using the tool array. Unsurprisingly, these did not show significant changes. The researchers did not expect these materials to radically change the career paths of large numbers of students. However the shifts, though small, were in the desired direction. Students expressed slightly more willingness to transfer to a four year school (0.019 point average increase in score; p-value: 0.8603) or pursue a math-intensive career (0.013 point average increase in score; p-value: 0.9142).
Ideally, these results would be compared to surveys submitted to a similar group of pre-algebra students who did not have access to the tool array. The researchers were unable to find such classes onsite since the website is available to everyone and has been well-publicized at their college. Future studies in collaboration with other schools is a possibility.
In addition to the Likert scale questions, the post-survey also included open-ended questions regarding the students’ usage of and reaction to the tool array. One question asked, “If you watch the videos, how do you use them? Do you watch it straight through, pause it, fast forward it, do you watch it with another student, do you watch them at home, etc.” The majority of students viewed the videos at home by themselves using this resource as a supplemental study tool. Several students watched the videos straight through racing the instructor and student character to the solution. In a few cases, students indicated that they preferred working alone rather than with another student. Darlene wrote, “I watch them alone because I don’t like the distraction of study groups or partners.”
Another open-ended question asked students, “If you use the website, how has it affected your attitude towards learning math on a computer?” Many students who used the website found the resources easier to use than expected relieving some of the anxiety they were experiencing. Eric wrote, “I was nervous coming into math for the first time in 12 years. With the addition of the site and its tools, it has helped me tremendously.” Sharron responded with, “The website is so helpful, especially to the student who works fulltime like me. It boosts up my interest in learning math.” These responses suggest to the research team, that due to the unrestricted nature of the tool array, the website might serve to help build self-confidence in mathematics with adult learners who are returning to school after several years.
Students were also asked to describe their feelings regarding the researcher’s portrayal of himself as a student character within the video presentations. Mathew responded, “It makes me feel more relaxed seeing you as the student for some reason.” David response was, “I can relate to Charlie. He reminds me of myself sitting in the classroom.” Sharron simply stated, “It made me laugh.” Karen’s response was, “What? I’m going to watch them now!”
Generally, student responses appeared to be positive with self-directed learners that carried with them some level of fear or math anxiety. Several top performing self-directed learners found the videos to be a bit annoying. Jackie responded with, “Charlie is a little distracting in an annoying way like some of the kids in the class.” Louise’s response was, “Charlie was ok at first, but it got a little old.” The fact that students with low levels of math anxiety did not find this video dynamic useful while those who were fearful of math did find it engaging suggested this video format was successfully addressing student affect.