Designing High-Quality Surveys for Online Education in The Digital Age
Concurrent Session 1
This hands-on workshop will cover the ins and outs of survey design for online education programs, including question types, response formats, layouts, pilot testing procedures, methods of delivery (social media/smartphones/other digital devices), and sampling methods with specific examples. The workshop will also introduce various survey development platforms and free resources.
Lead Presenter: Kadriye Lewis, Childrens Mercy Hospital - Department of Pediatrics - UMKC School of Medicine
Surveys or questionnaires have been used for some time in all spectrums of online education from course development to program evaluation, including research purposes on many diverse topics. Recently, we have witnessed the rapid spread of surveys delivered through social media, smartphones, and numerous other digital devices that enabled us to collect/process data about human behaviors on visual scales or multi-channel formats that have never been imaginable before. Online surveys are usually structured questionnaires that students/target audience complete over a Web platform generally by a filling out a form. The main benefit of online surveys for online educators and researchers is that they increase productivity by saving time. Data is instantly available and can easily be transferred into specialized statistical software or spreadsheets when more detailed analysis is needed. While the length and format of online surveys can vary based on the purpose, developing questions that accurately assess respondents' opinions, experiences, and behaviors is a critical aspect of survey methodology.
Online survey tools have even increased their value further because online courses have shown significant growth with multiple applications to aid learning since the coronavirus outbreak. However, applying effective design principles in the survey development process is often overlooked by the online faculty although the key to a good survey is its design. Poorly designed surveys confuse respondents and yield inaccurate information, unreliable feedback, and low response rates. At this point, online educators/faculty should be cognizant of the multistage survey development process and effective design principles. Careful planning at the start of survey development will help them create accurate measures and improve data quality that will contribute to all levels of evaluation efforts of online education from teaching to research. This hands-on workshop will cover how to design and develop online surveys tailored to specific objectives for online education programs, including question types, response formats, layout, pilot testing, and sampling with specific examples. The workshop will cover the ins and outs of survey design and the best practices of survey question development with question types, response formats, layouts, pilot testing procedures, and sampling methods. In small groups, participants will be asked to critique sample surveys. They will also discuss the merits and flaws of questions and methods of delivery, e.g., social media, smartphones, and other digital devices. The workshop will also introduce various survey development platforms and free resources.
Level of Participation:
This workshop is structured to be a mutual learning experience with a combination of interactive lectures, and small group activities along with a thought-provoking discussion. The workshop will be a great refresher for an experienced faculty in survey development and for those who have little or no previous survey development experience will gain tremendous knowledge of the best approach to survey development with hands-on practice in small group activities.
The goal of this workshop is to introduce the best design principles for writing survey items that accurately represent what the online educators want to measure or capture data for their program needs.
At the end of this session, participants will be able to:
Identify the best design principles for the development of surveys tailored to specific objectives for online education programs, including question types, response formats, layout, pilot testing, and sampling
Discuss techniques that optimize questionnaire design, the merits and flaws inherent in different types of questions, and additional considerations for surveys delivered through social media, smartphones, and numerous other digital devices
Locate survey development platforms and free resources.
Workshop Agenda, Topics/Activities (90 Minutes)
Step 1 (5 minutes): Opening
Introduction (agenda review)
Step 2 (10 minutes): Icebreaker
Participants are invited to share three facts about surveys
Step 3 (20 minutes): Introduction to Survey Methods (Interactive lecture with questions and answers, and small group activity)
·What is an online survey? How are surveys used in online programs?
Surveys delivered through social media, smartphones, and numerous other digital devices
·Small Group Activity: Stage of development for survey questions
Question structure and wording
Sequencing of questions
Survey question design
Evaluation of survey questions
Step 3 (25 minutes): Survey question design
- Best design principles with specific examples
Types of survey questions (Small Group Activity: Survey critique - Poor question vs. better question and discussion)
- Multiple-choice questions
- Rating scale questions
- Likert scale questions
- Matrix questions
- Dropdown questions
- Open-ended questions
- Demographic questions
- Ranking questions
- Image choice questions
- Click map questions
- Slider questions
- Closed-ended vs open-ended questions
- Merits and flaws inherent in questions
Step 4 (15 minutes): How to test your survey questions to avoid common problems
- Pilot testing
Reliability of survey data
The survey response process
Step 4 (15 minutes): Summary & Key Takeaways
Take-home message and intended application after the workshop
- Questions & Answers