Overview of Findings from the EdTech Decision Making in Higher Education Study

Concurrent Session 1
Streamed Session

Watch This Session

Session Materials

Brief Abstract

This study investigates the decision-making inputs, processes, and practices around the acquisition of technology to facilitate teaching and learning at over 43 colleges and universities in the United States. This study was part of the EdTech Efficacy Symposium and a result of Working Group B.

Presenters

Lead Presenter: Kristin Palmer, University of Virginia

Director of Online Learning Programs at U.Va. She is responsible for managing production for U.Va.'s partnership with Coursera. Her passion project is an outreach program with Africa that has provided over 24,000 scholarships and reached learners in every country on the continent. Before U.Va., she was successfully delivering enterprise technology projects for various companies including Walt Disney, Hewlett-Packard, eBay and Intuit. She joined U.Va. in 2011.

Co-presenter: Whitney Kilgore, iDesign

Whitney Kilgore, Ph.D. is co-founder and chief academic officer of iDesign, a partner to universities who wish to build, grow, and support online and blended course and program offerings. iDesign provides concierge, white-glove instructional design support to faculty partners. Their designers bring expertise, service, and project structure to bear and ensuring that faculty feel comfortable, informed, and in control throughout the process of creating online learning experiences for their students. Dr. Kilgore has led the development of programs across the U.S., Spain, the Philippines, China, Australia, Latin America, and the U.K. As an academic, Dr. Kilgore has received recognitions for her work, including a research award from UNT and inclusion in the Top 10 Research Articles (co-authored with Aras Bozkurt of Turkey and Matt Crosslin from UTA) for 2017 list produced by Dublin City University for their work on Bot-Teachers in Hybrid Massive Open Online Courses (MOOCs): A Post-Humanist Experience. Whitney has been a part of the EdTech efficacy research project conducted in conjunction with Columbia University and the University of Virginia on EdTech Decision Making in Higher Education. She is currently working on research related to care theory in online learning and the impact to practice of humanizing online teaching and learning. She edited and published the book Humanizing Online Teaching and Learning and Connecting the Dots: Book: Improving Student Outcomes with Exceptional Instructional Design.

Tags

OLC Accelerate 2017 Research Cross-institutional Decision making Methodology Models Multi-institutional Online and blended learning Paradigms Planning

Extended Abstract

This study investigates the decision-making inputs, processes, and practices around the acquisition of technology to facilitate teaching and learning at over 43 colleges and universities in the United States. This study was part of the EdTech Efficacy Symposium and a result of Working Group B.  More information on this study is available at http://www.edtechdecisionmakinginhighered.org.

Research questions:

  1. What sources of information are higher education leaders and faculty currently using to make education technology acquisition decisions?

  2. How is research used in their decision-making processes?

  3. Do institutions of higher education (IHEs) conduct their own investigations or research into how well EdTech products currently being used work?            

Methodology:

We interviewed 52 EdTech decision-makers (Presidents, CIOs, Directors of IT or eLearning, Deans, Faculty etc.) from 43 IHEs between September 2016 and April 2017. Participation was solicited from both 2-year and 4-year IHEs, for-profits and non-profits, and public and private IHEs.    

Findings:                   

  1. The most common source of ongoing information about EdTech products and trends for EdTech decision-makers is colleagues at their own or other IHEs (mentioned in 96% of interviews), followed by vendors (80%), professional associations or consortia (67%), and consultants (53%).

  2. Research organizations and institutes, technical assistance centers, and think tanks were mentioned in less than 10% of the 45 interviews as a source of EdTech information.

  3. The most common medium for gathering information on EdTech products and trends is network events such as conferences and consortium meetings (mentioned in 93% of interviews).

  4. Written publications are also a common source of EdTech information: newspapers or newsletters (mentioned in 62% of interviews), partially or non-peer reviewed journals or papers (mentioned in 56% of interviews), and trade magazines or practitioner publications (mentioned in 44% of interviews) were more often mentioned than peer-reviewed academic journals, which were listed in only 9% of interviews.

  5. Social media and other online sources such as blogs, websites, and Twitter are increasingly replacing more traditional sources of EdTech information.

  6. Opinion leaders, change makers, or innovation leaders in EdTech are most likely to be individuals at IHEs (mentioned in 49% of interviews), business or organizational leaders (31%), and universities (29%).                                                         

The most common goals mentioned for the acquisition of EdTech were:

  • Supporting a variety of pedagogical and assessment strategies including collaboration among students and faculty, interactivity of content, authentic assessment, active learning, individualization of instruction, competency-based education, and virtual reality.

  • Improving operational efficiency and reducing costs.

  • Improving user experience by modernizing or upgrading functionality of existing systems.

  • Increasing capacity to serve students online.                

Decision-making processes

  • At public and non-profit institutions, decision-making for major EdTech acquisitions tends to be protracted, inclusive, and “consultative” with faculty having a strong voice in decisions in addition to professional staff and, in some cases, students. The length of deliberations can be an impediment to change in a fast moving field.

  • For-profit IHEs tend to have swifter, more centralized decision-making processes with faculty and student buy-in often sought only after a decision is made. But inadequate consideration of user needs and preferences can lead to suboptimal implementation.

  • User input is primarily gathered via scheduled meetings, participation in committees or taskforces, surveys, and during vendor demos.

  • Smaller EdTech purchases are increasingly being made by individual faculty members or departments, often leading to redundancy in tool functionalities, to multiple licenses with the same vendor, and to click-through agreements being signed without due care being given to regulations and student data privacy.

Decision-makers each listed a median of 6 decision criteria against which they evaluated the EdTech options discussed in the interview. Criteria commonly used to select EdTech products and services fall into 5 main categories:

  • Features and functionality, mentioned in 95% of interviews.

  • Feasibility of implementation, mentioned in 82% of interviews.

  • Cost or ROI considerations, mentioned in 82% of interviews.

  • User experience or usability, mentioned in 61% of interviews.

  • Vendor capacity and relationship, mentioned in 41% of interviews.

This session will review this study and discuss next steps.  Audience members will be asked to answer poll questions throughout presentation.  Audience members will also be encouraged to brainstorm ways to move forward.  It is assumed this will include working together to develop a template for evaluating edtech pilots and possible repository solutions for gathering such pilot evaluation reports.