Faculty face a monumental task in leading educational programs. Days are spent designing curriculum, maintaining teaching responsibilities, and supporting students while juggling committee assignments, assessments, and professional development. As expectations grow, so do reporting responsibilities and the need to balance work and personal time. The best intentions can be sidetracked by emerging issues and unending emails.
To add to the complexity of higher education, programs are being offered through multiple modes of instruction to meet the demands of students and their work schedules. The same program can be offered through a traditional face-to-face offering on campus, multiple off-site locations, or 100% online, and any combination of hybrid offering in between. This diversification of instructional modes provides flexibility for the program to reach students where they are and maintain enrollment. However, it also further stretches faculty in assessment and planning.
To help balance these demands, there are some steps faculty can take to better identify and target the areas where programs need strengthening. These steps can help faculty save valuable time and energy by leading to more intentional and effective improvement efforts. Time can then be devoted to more critical areas.
What is intentional assessment?
Regional accreditors have increased the assessment expectations for institutions and programs. There are variances between accreditors, but institutions and programs are generally expected to demonstrate how results are used to make improvements. Gaston (2018) states that that over the last two decades, expectations have grown from simply showing evidence of ongoing assessment and results to demonstrating that the program is actively implementing improvements. Therefore, assessment does not end at analysis, but continues through improvements based on results. Faculty are expected to take the results and develop a plan for improvement, implement it, and evaluate the impact of the change. This work is the hardest part of program assessment, requiring significant oversight and effort as confirmed by Kuh et al. (2015).
Programs have an obligation to assess the effectiveness of each modality offered. The decision to add an online modality or deliver programs offsite should not be taken lightly. Each modality of a program should offer comparable quality, content, and student outcomes. Intentional assessment design can aid faculty in documenting the evidence of program effectiveness and in decision-making on areas to strengthen.
But how can faculty assess learning and equivalence across multiple modalities? Faculty own the curriculum. Faculty develop course activities and decide on expected learning outcomes at program and course levels. It can be challenging to assess outcomes achievement even when a program has a singular mode of instruction. Adding off-site and online course sections or program delivery further extends the complexity of the assessment system to be designed. Before expanding modalities, it is important to have an assessment plan in place. There are a few factors than can make the plan successful.
Measure learning outcomes with assessments, not course grades
Reviewing course grades will not provide the data needed to make improvements. Faculty spend significant time grading their students’ work, but course grades do not clearly define what students have learned in a course. Suskie (2009) explains that grading and assessment criteria may differ between faculty and course sections; standards may be imprecise; and grades may not be clearly linked to an individual learning competency. Course grades are derived through a multitude of assignments and activities, some of which may not directly relate to one specific outcome. The extent to which a student demonstrates mastery of a specific learning objective may not be reflected in a final grade. Further, Suskie (2009) points out that “grades focus on individual students, while assessment focuses on entire cohorts of students and how effectively everyone, not an individual faculty member, is helping them learn” (p. 10).
Faculty Buy-In and Communication
Offering a program through multiple modalities may increase the number of faculty teaching courses. Online faculty may not maintain office hours on campus while faculty teaching at off-site locations including high schools may not be present on campus regularly or at all. Program planning and assessment work is most effective when it is performed in collaboration with program faculty. It may be difficult to find time to meet to decide on learning outcomes, to agree on the measures and criteria used to assess learning, to analyze results across modalities, and to implement improvements; however, these decisions are difficult to make in a vacuum and the future success of the assessment plan relies on cooperation and understanding between faculty members and assessment staff.
Gaining faculty buy-in for the assessment plan is critical to assure that the assessments given to students are directly related to program outcomes. Cooperation is also important in collecting assessment data and implementing improvements systematically. Transparency and consistent communication with faculty members can maintain the focus on assessment and the pursuit to improve student outcomes of the program.
Data collection and analysis of all modalities
Once the assessment results are collected across all modalities, the analysis should include an evaluation of the extent to which students demonstrated competency in achieving individual learning outcomes. As a best practice, the evaluation should analyze the results of students overall and then differentiate results by modality. Faculty can examine differences between online students and those being taught face-to-face. Differences in performance between students enrolled through fully online or hybrid versions of the program should be examined to inform decisions toward programmatic improvements.
Make time to reflect and discuss
The work doesn’t stop once the assessment results are collected. The analysis requires time for reflection and discussion. Some common points to consider include: Are the assessment methods reliable and providing relevant data for the measurement of student learning? Are all students learning at the same level regardless of modality? Can differences be explained? If a variety of assessments are being used across the program by different faculty or locations, are they assessing the same competencies at the same level?
Assessing across modalities can strengthen the intentionality of curricular design and program improvements. These improvements can enhance the quality and efficiency of educational programs while fostering student success. Differentiating the analysis of assessment results between modalities can provide useful information on the skills and knowledge of students and program outcomes regardless of instructional delivery method employed. As a result, improvements are targeted more directly to the areas that need strengthening. This focus can concentrate faculty efforts and resources to provide more effective results in the long term.
I would like to share your experiences and successes in assessing across modalities with faculty on my campus. Feel free to share with me at Kimberly.Faris@unt.edu.
References
Gaston, P. (2018, April). Assessment and accreditation: An imperiled symbiosis. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). http://learningoutcomesassessment.org
Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence of student learning to improve higher education. Jossey-Bass.
Suskie, L. (2009) Assessing student learning: A common sense guide (2nd ed.). Jossey-Bass.
 |
Kimberly Faris is the director of assessment and accreditation at the University of North Texas where she collaborates with faculty from 14 colleges offering 230 programs. UNT has a long history with online education and currently offers 80 online program options. Kimberly has worked in higher education for over 25 years in enrollment management, teaching and accreditation in public two-year and four-year institutions. She is passionate about student success and enhancing innovation and effectiveness. You can connect with Kimberly on LinkedIn at www.linkedin.com/in/kimfaris |