Strategies for Collecting Large-Scale Survey Data

By Jiarui Xie (xie.1196@buckeyemail.osu.edu)

Data collection is often one of the most time-consuming steps in the research process, especially for large-scale studies. Recently, I successfully collected over 700 valid survey responses for my dissertation research within a month, which was much faster than expected. It was an online survey and focused on college students. I summarize some effective strategies I used and hope they will provide helpful insights for other researchers who plan to conduct a large-scale survey study.

Prepare Multiple Recruitment Materials for Different Audiences

Having well-prepared recruitment materials tailored to different audiences can significantly improve your response rate.

  1. For potential individual participants, I would recommend creating an email including your brief introduction (e.g., name, program, and institution), research purpose, participant eligibility, estimated time to complete the survey, incentive (if applicable), survey link (make sure it is clickable), and sincere words of thanks. Highlight key information (e.g., eligibility and incentive) using bold text or color to make it stand out. Here is a template you can use: Potential Participants Recruitment Email.
  2. For people who can help disseminate your survey, the email needs to include your brief introduction, research purpose, what assistance you need from them (e.g., forwarding the email to potential participants, posting an course announcement in the Learning Management System, or allowing you to collect data in their classes), and most importantly, a concise recruitment message they can easily copy and share. Don’t forget to express your gratitude in the end. Here are two templates you can use: Instructor Email, Administration Email.
  3. In addition to email, you can create a SMS or chat app message. Keep the message short, focusing on key information (e.g., eligibility, survey link, and incentive). More detailed information can be included in an attachment (e.g., flyer) that is sent with the message. Adding appropriate emojis is a good idea. Here is a template you can use: Message.
  4. You can also create a recruitment flyer. Highlight key information. Include a QR code linking to the survey. Use eye-catching colors while ensuring good contrast and consistency.

Gather Contact Information from Multiple Sources

Having a solid contact list is essential for effective outreach. Here are some ways to gather email addresses and phone numbers:

  1. Check your Learning Management System (LMS) for courses you are enrolled in—especially large ones like orientation or seminar courses, which may have hundreds of students.
  2. Look through group emails you received. Often, these emails include multiple recipients, some of whom may be potential participants.
  3. Explore the group chats you are in, such as GroupMe, WhatsApp, and Teams.
  4. Browse the university website to find leadership contacts.
  5. Use Excel to organize contact details, creating separate sheets for different groups (e.g., students, faculty, administrators).

Start Reaching Out

Once you have prepared your recruitment materials and gathered contact information, it is time to start reaching out:

  1. Send an invitation email/message to potential participants directly. If you are using Microsoft, the “Mailings” function in Microsoft Word (linked with Outlook Classic for Desktop) can automatically send personalized emails to thousands of recipients in minutes.
  2. Request assistance from the administration, such as college deans and department chairs. They usually have access to student listservs, which include hundreds or even thousands of students. Their support can significantly speed up your data collection process.
  3. Ask instructors for help, especially those teaching large classes.
  4. Leverage your academic and non-academic communities. Share your recruitment message in your communities and ask people to help spread the word.
  5. Use social media carefully. Platforms like Facebook, LinkedIn, Twitter, and Reddit can help you reach a broader audience. However, be cautious! There may be AI bots that can generate fake responses.

Utilize Flyers for Offline Promotion

Print recruitment flyers in different sizes:

  1. Post letter-sized flyers on bulletin boards in permitted areas, such as libraries, classroom buildings, cafeterias, and activity centers.
  2. Distribute smaller flyers at high-traffic locations during peak times.

Incentive Plays an Important Role

If it is possible, provide an incentive for each participant, such as a gift card or entry into a prize drawing. Even a small incentive can make a big difference in motivating participants to complete your survey.

Be Proactive and Explore Every Possible Approach

Don’t hesitate to reach out! This is your research, and every effort counts. Try every possible approach, and your hard work will pay off!

I hope these strategies are helpful and wish all researchers the best of luck with data collection. If you would like to discuss these strategies in more detail or share your experiences, please feel free to contact me at xie.1196@osu.edu.

Insights from Presenting at the 2024 AECT International Convention

By Jiarui Xie (xie.1196@buckeyemail.osu.edu)

Attending the 2024 Association for Educational Communications and Technology (AECT) Convention in Kansas City, Missouri was an inspiring experience!

Jiarui Xie at the 2024 AECT convention.

This year, Jiarui had the privilege of representing the LED research group and presented her collaborative research titled The Application of Artificial Intelligence in Asynchronous Online Discussions in a roundtable session. This ongoing study is a collaboration with her PhD program advisor, Dr. Ana-Paula Correia. She shared four original categories of AI applications in AODs—information classification, content assessment, participation facilitation, and discussant matching—and facilitated an engaging discussion with fellow scholars.

Jiarui Xie (second from right) shared her research in a roundtable session.

A highlight of this experience was the opportunity to interact with experts and peers in the field. The roundtable format allowed for in-depth conversations, where participants exchanged ideas, discussed challenges and explored future applications of AI in education.

The flyer Jiarui shared in the roundtable session

Flyer used in the roundtable session

Presenting at AECT reinforced the potential impact of our research. Thank you to everyone who attended our session and AECT for hosting such an inspiring event. We look forward to the 2025 AECT International Convention in Las Vegas, Nevada!

 

 

LED Research Group at #AERA24

By Mimi Adjei and Ana-Paula Correia

The Learning and Experience Design (LED) research group was reunited at the American Educational Research Association (AERA) 2024 annual meeting in Philadelphia, PA (April 11-14, 2024). We had the opportunity to showcase some of the research we have been working on this past year as well as experience and support others’ research.

Poster Sessions:

Dr. Fan Xu and Dr. Ana-Paula Correia contributed to the SIG-Computer and Internet Application in Education program with their poster, Overcoming Obstacles in Computational Thinking Development: Unveiling Challenges and Solutions for Supporting Middle School Pair Programming. 

Abstract: In the 21st century, computational thinking (CT) is of utmost importance, and pair programming has emerged as a popular method for teaching CT skills. This study aimed to understand the challenges faced by middle school students in acquiring essential CT elements through pair programming. To gather student insights, an open-ended survey was conducted with eighty-four eighth-grade students in the US. Results revealed numerous challenges in cognitive, affective, and social dimensions during the pair programming process, highlighting the need for increased support at the classroom, school, and district levels to ensure successful implementation of pair programming in middle schools. The study provides theoretical, practical, and methodological implications for CT education and pair programming, acknowledges limitations, and discusses opportunities for future research.

Dr. Fan Xu standing next to her poster presentation

Dr. Fan Xu standing next to her poster presentation

Mimi Adjei and Dr. Ana-Paula Correia presented their study The Role of Learning Analytics in Shaping Meaningful K–12 Learning Experiences: A Systematic Review as a poster during the SIG-Instructional Technology program.

Abstract: Learning analytics (LA) is becoming more prevalent in many educational classrooms. It focuses on analyzing student log data and academic performance to provide valuable insights into their learning processes. This systematic literature review assesses the relationship between learning analytics, learning design, and experiences within K-12 contexts in the past sixteen years (2006-2022). It examines how LA impacts academic achievement, engagement, satisfaction, persistence, and educational outcomes. It also sheds light on students’ learning behaviors and interactions with educational content. Challenges and considerations of its use are also addressed. This review promotes the responsible implementation of LA in K-12 education. The implications also extend to practitioners, researchers, and policymakers, providing insights into integrating learning analytics successfully in education and building educational possibilities.

Mimi Adjei (right) interacting with a visitor at her poster station

Mimi Adjei (right) interacting with a visitor at her poster station

Paper Sessions:

Dr. Fan Xu and Dr. Ana-Paula Correia also presented their study, Pair Programming in Middle School: How Individual Factors Impact the Dyad’s Computational Thinking Performances, as a paper session at the SIG—Technology, Instruction, Cognition & Learning’s programming during #AERA24.

Abstract: Computational thinking (CT) is crucial for 21st-century youth, with pair programming commonly utilized for CT education. This study examines the influence of individual factors on students’ CT performance and their partners’ performance in pair programming. An empirical study was conducted with 84 8th-grade students at a Midwest middle school in the US. The results of dyadic analyses reveal a positive correlation between participants’ CT scores and their attitudes toward programming and collaboration. However, no significant relationship was found between scores and prior computer or programming experience. Additionally, students’ attitudes toward collaboration and programming frequency positively predicted their partners’ CT performance. This study offers implications for educational practices and provides suggestions for future research.

Dr. Fan Xu in the midst of her presentation

Dr. Fan Xu in the midst of her presentation

Jiarui Xie, Mimi Adjei, and Dr. Ana-Paula Correia also presented their paper on their study titled Instructor Participation in Asynchronous Online Discussions: A Comparative Study as part of the SIG-Online Learning SIG program.

Abstract: The way instructors participate in asynchronous online discussions has the potential to affect student engagement. This study aimed to compare the effects of two instructor participation approaches: replying to student posts on discussion boards and commenting on student posts on grade pages. The study examined the impact on both the quantity of student participation and the quality of student posts. The findings revealed that when the instructor replied on discussion boards, there was significantly more interaction between students than when the instructor commented on the grade pages. However, the number of student posts had no significant difference between the two approaches. Notably, both approaches contributed to a gradual improvement in the quality of student posts over time.

Dr. Correia(middle), Jiarui (right), and Mimi Adjei (far right) with other presenters after their presentation

Dr. Correia (middle), Jiarui (right), and Mimi Adjei (far right) with other presenters after their presentation

Round Tables:

Sean Hickey also shared his research on Understanding Instructional Design Decisions as part of the AERA Graduate Student Research-in-Progress Roundtable.

Abstract: As technologies emerge and create new job roles requiring new expertise, workplace learning and the role of “instructional designer” have become increasingly important. This qualitative study seeks to better understand the work of high-performing instructional designers and the ways in which they successfully create training materials to meet specific educational or performance objectives. By utilizing an interview format in which designers share work they have done, combining elements of a stimulated recall methodology with a “think-aloud” strategy, the researcher hopes to gain an understanding of how instructional designers interpret theories related to learning, to what extent those theories are consciously applied in the development of learning experiences, and how designers evaluate and engage with emerging technologies (e.g., generative artificial intelligence).

 

Business meetings:

As the past chair of the SIG Online Teaching and Learning, Dr. Ana-Paula Correia co-led the SIG business meeting on April 14, 2024, with her colleagues, Dr. Mary Rice and Dr. Rebecca Quintana. The purpose of the Online Teaching and Learning SIG is to provide a forum for discussion and reporting on issues and research related to teaching and learning. The business meeting provided opportunities for professional development and networking.

Dr. Rice and Dr. Correia at the SIG Online Teaching and Learning business meeting.

Overall, AERA24 was a fantastic opportunity to reflect on and celebrate our hard work over the past year, as well as to be inspired by the research of others. The event served as a platform to recognize the dedication and efforts of the LED research group and our research projects, highlighting the contributions you are making to the field of educational technology and human-computer interaction. Additionally, AERA24 was a source of inspiration, allowing us to engage with the groundbreaking research conducted by our peers and colleagues. The presentations and discussions sparked new ideas and fostered a sense of community among researchers, further motivating us to continue our pursuit of knowledge and innovation. See you at #AERA25 in Denver, CO!

LED members and some of CETE's staff at AERA24

LED members (past and present) and also some members of Ohio State’s Center on Education and Training for Employment at #AERA24

 

LED research group was highly represented at the AECT 2023 International Convention

By Mimi Adjei (adjei.39@buckeyemail.osu.edu)

AECT2023 banner

#AECT2023

All roads led to Orlando, Florida, this October as the LED research group had the opportunity to present its groundbreaking research and celebrate the accomplishments of its members at the Association for Educational Communications and Technology (AECT) 2023 International Convention. The 2023 convention marked 100 years for the AECT as an organization.

Sean Hickey, Dr. Ana-Paula Correia, Katherine Fergus, and the Center on Education and Training for Employment (CETE) received the 2023 AECT Outstanding Book Chapter Award from the Division of Distance Learning (DDL) for their book chapter, The DACUM Virtual Institute: A Case Study in Designing for Adult Learners*.  This award recognizes a book chapter in a print or digital book published in English with a copyright date within the last three years (2020-2023) that describes important theoretical or practical aspects of distance education that can help others involved in distance education or those researching an important aspect of distance education.

Fan Xu received the DDL award on behalf of Sean Hickey, Ana-Paula Correia and CETE

Fan Xu (middle) received the DDL award on behalf of Sean Hickey, Dr. Ana-Paula Correia, and Kate Fergus.

Jiarui Xie, Mimi Adjei, and Dr. Ana-Paula Correia also presented their poster at the AECT 2023 International Convention on their study titled Teacher Interaction in Asynchronous Online Discussions: A Comparison of Discussion Board Replies and Grade Page Comments as part of the DDL’s program.

Abstract: This study compared the effects of two instructor participation approaches: replying to student posts on discussion boards and commenting on student posts on grade pages. The study examined the impact on both the quantity of student participation and the quality of student posts. The findings revealed that when the instructor replied on discussion boards, there was significantly more interaction between students than when the instructor commented on the grade pages. However, the number of student posts had no significant difference between the two approaches. Notably, both approaches contributed to a gradual improvement in the quality of student posts over time.

Jiarui Xie (left) & Mimi Adjei at their poster presentation at the AECT 2023 convention

Jiarui Xie (left) & Mimi Adjei at their poster presentation during the AECT 2023 convention.

Dr. Chenxi Liu, Yixi Wang, Dr. Marvin Evans, and Dr. Ana-Paula Correia also contributed to the Division Emerging Learning Technologies program with their study, Uncovering the Drivers of Mobile Learning Acceptance: A Meta-Analysis of Critical Influencing Factors. Dr. Evans led the presentation on Mobile Learning Acceptance f on behalf of the rest of the team.

Mobile Learning Acceptance

Liu et al.’s study on mobile learning acceptance.

Abstract: This meta-analysis employs the Technology Acceptance Model to examine the factors influencing mobile learning acceptance and moderation effects of education level and region. Using a one-stage meta-analysis structural equation model, we developed an integrating model revealing the significant impacts of subjective norm, self-efficacy, enjoyment, and system quality on mobile learning acceptance. Our findings show that these impacts are weaker in higher education compared to non-higher education and stronger in Asia-Pacific compared to non-Asia-Pacific regions.

Dr. Evans also successfully led a practice session for the Design and Development program on Creating Online Learning With Vygotsky’s Zone of Proximal Development Using Articulate 360.

Left to right: Fan Xu, Mimi Adjei, Jiarui Xie and Marvin Evans

LED members who were at the AECT convention. Left to right: Dr. Fan Xu, Mimi Adjei, Jiarui Xie, and Dr. Marvin Evans.

The LED research group is excited and looks forward to presenting more of its research at the AECT 2024 International Convention in Kansas City, Missouri. Hope to see you there!

AECT24 - Kansas City

*Cite as: Hickey, S., Correia, A.-P., & Fergus, K. (2022). The DACUM Virtual Institute: A Case Study in Designing for Adult Learners. In P. Sullivan, B. Sullivan, & J. Lantz (Eds.), Cases on innovative and successful uses of digital resources for online learning (pp. 262-284). IGI Global.

Welcome Jessica Bates to the LED research group!

The LED research group welcomes Jessica Bates! Jessica is a doctoral student in Educational Studies specializing in Learning Technologies. She brings such valuable experience and passion for creating virtual learning experiences to our research group.

Jessica Bates

Jessica Bates

Jessica currently holds the position of Program Engagement and Curriculum Coordinator, along with serving as the Interim School Support Coordinator at Virtual Virginia. In this capacity, she collaborates with her team to create high-quality online instructional materials for K-12 curriculum, Career and Technical Education, Fine Arts, World Languages, Advanced Placement, Adult Education, and Dual Enrollment courses for Virginia Public Schools. Additionally, she provides support to school divisions throughout Virginia, offering digital resources and professional development opportunities for educators. Jessica is also a current and active member of the Virtual Learning Leadership Alliance.

Jessica has a B.S.Ed. degree from George Mason University and an M.Ed. in Educational Leadership, Administration and Supervision from the University of Virginia in 2013. Following her graduation, Jessica embarked on a career in teaching and played a pivotal role in developing online curricula and virtual courses for Virginia Public Schools. In 2020, she assumed the role of Elementary and Middle School Curriculum Supervisor for Virtual Virginia, a program under the Virginia Department of Education. Since then, she has contributed to the design and development of over 150 online courses and curricular materials for Virginia Public Schools.

Jessica’s academic and research interests revolve around instructional design and development, the effective integration of technology in teaching and learning, and exploring the factors that drive learner motivation and engagement within digital learning environments.

Jessica also spends her time enjoying exercise and attending her three children’s sporting activities, Holloway, 9, Hattie, 6, and Hartwell, 3. You can find the entire family at the Ohio State’s Men’s Volleyball matches in the Spring supporting her husband, Hudson Bates, the Associate Head Coach for OSU Men’s Volleyball.

Welcome Batzaya (Zack) Batsaikhan to the LED research group!

The LED research group welcomes Batzaya (Zack) Batsaikhan! Zack is a doctoral student in Educational Studies specializing in Learning Technologies. With a rich background in product design, Human-Computer Interaction, and a passion for creating meaningful change, Zack brings a wealth of experience and expertise to our research group.

Batzaya (Zack) Batsaikhan

Batzaya (Zack) Batsaikhan

Zack is originally from Mongolia, and he holds an M.Sc. in Human-Computer Interaction from Iowa State University and a B.I.T in Information Technology from the Mongolian University of Science and Technology. Over the span of a decade, Zack has built and managed numerous successful innovative startups as well as designed and implemented human-centered digital products and services. His interests expand to developing an interdisciplinary approach to tackle complex social problems through design, technology, education, and entrepreneurship.

Zack has been featured in Forbes Mongolia’s 30 Under 30 list, was a Fulbright Scholar, Rumsfeld Fellow, LEAD Mongolia & Bhutan Fellow, and World Economic Forum’s Global Shapers alumnus. During U.S. Secretary of State John Kerry’s visit to Mongolia, Zack led his interview and hosted his visit with other Mongolian young entrepreneurs.

His academic and research interests include entrepreneurship in learning technologies, learner experience design, online learning design and development, applications of emerging technologies in education (AI/AR/VR), and human-computer interaction (HCI/UX/UI/IxD).

When Zack is not working, he enjoys traveling, climbing mountains, taking photographs, and spending time with his family.

Congratulations to Fan Xu and Chenxi Liu, our Newest Ph.D. Graduates🎓

By Mimi Adjei (adjei.39@buckeyemail.osu.edu)

Join us in celebrating the remarkable achievements of Fan Xu, Ph.D., and Chenxi Liu, Ph.D. of our Learning and Experience Design (LED) Research Group, whose recent graduations serve as a testament to their incredible journey. Throughout their time with the LED research group, Fan and Chenxi have consistently exhibited brilliance and unwavering passion in their work, leaving a lasting impact on all of us.

Graduates Dr. Fan Xu, and Dr. Chenxi Lui posed in their doctoral regalia with Dr. Ana-Paula Correia.

Left to right: Dr. Fan Xu, Dr. Ana-Paula Correia (Advisor), and Dr. Chenxi Liu.

Fan’s journey has been characterized by transformative discoveries and projects. Through her research, she has not only opened doors to new possibilities but also served as an inspiration to countless colleagues and peers. Fan’s research and insights on Pair Programming in Middle-School Computational Thinking Education will inspire new ideas and advance our understanding of how technology can be used to enhance education and learning. She will continue to be at the forefront of shaping the future of educational technology through her work as a Senior Learning Designer at the Center on Education and Learning for Employment (CETE), The Ohio State University.

Chenxi has brightened our path with her exceptional intellect and insatiable curiosity. Her new Mobile Learning Acceptance Determination (mLAD) Model explains learners’ acceptance of m-learning outside the classroom and constitutes a novel contribution to educational technology. Chenxi’s research and insights will inspire new ideas and advance our understanding of how technology can be used to enhance education. She is currently a full-time Social Sciences Researcher at Stanford University’s School of Medicine, where she continues her pioneering work in exploring the intersection of technology and education and makes contributions to cutting-edge advancements in the field.

We raise a toast as they both embark on new chapters of their lives, ready to make their mark on the world beyond academia. We have no doubt that their success will continue to shine brightly as they apply their expertise and passion to other real-world challenges.

Congratulations Dr. Xu and Dr. Liu!

New Ph.D. graduates Fan Xu and Chenxi Liu posed for a photo in their doctoral regalia.

Left to right: Fan Xu, Ph.D. and Chenxi Liu, Ph.D.

Welcome Mimi Adjei to the LED research group!

The LED research group welcomes Mimi Adjei! Mimi is a doctoral student in Educational Studies specializing in Learning Technologies. She is originally from Ghana where she had her elementary education through to college. She completed her B.S in Actuarial Science from the Kwame Nkrumah University of Science & Technology, Ghana, and holds an M.Sc. in Mathematics and an M.Ed. in Education & Teaching from Ohio University in Athens, Ohio.

Mimi Adjei

Mimi Adjei

After spending time as a teaching assistant during her undergraduate studies, Mimi began her teaching career as a sixth-grade teacher. She taught mathematics and assisted with the day-to-day needs of the newly established department of Educational Technology. She has since taught mathematics and computer science classes at the high school and collegiate levels. Her focus is to help students develop a sense of independence in their learning through self-reflection and meaningful feedback. As a RALLY for STEM teaching fellow, Mimi champions STEM education in rural Southeastern Ohio and provides children with greater access to STEM careers. 

Mimi likes to explore new recipes and is currently developing a collection of fusion cuisines from around the world.  She also recently picked up hand embroidery as her creative outlet. 

Mimi’s current research interests include learning design & analytics, human-computer interaction (HCI), and artificial intelligence applications in education. 

Fan Xu elected as a new Officer for AECT Graduate Student Assembly

Fan Xu has recently been elected the new Social Media Officer for the Association for Educational Communications and Technology Graduate Student Assembly (GSA). Fan run for this office because of her passion for serving her peers and contributing to the GSA community.

After years of study and research in learning technologies, she is pretty familiar with various social media platforms. Fan worked as a social media manager for a student association while attending the University of Hong Kong as a graduate student.

The GSA mission is to provide its members with opportunities to engage in professional leadership roles, improve networks and develop the skills and resources for solving complex problems that may arise within instructional technology. GSA members engage with research and practice-driven topics, extend their professional networks, and evolve as scholars.

Fan Xu and Deborah Hewlett presented study at #AERA2022 Computers and Internet Applications in Education SIG

Fan Xu, Deborah Hewlett, and Dr. Ana-Paula Correia presented at the American Educational Research Association 2022 Annual Meeting on their study titled A Socialized Knowledge Community on Computational Thinking – Teachers-Pay-Teachers.

Fan Xu led the virtual presentation on Teachers-Pay-Teachers and Computational Thinking as part of the Computers and Internet Applications in Education SIG program.

Abstract: To understand teachers’ attitudes toward the current Computational Thinking-related resources in socialized knowledge communities and further explore their need for Computational Thinking (CT) education, this study collected a large amount of data from the Teachers-Pay-Teachers (TPT) portal and used text mining techniques for comprehensive analysis. This work reveals that most CT resources on TPT focused on math education in primary school. Teachers hold a relatively positive attitude towards these resources, even though some online resources might be too difficult to use. The resource itself, the subject, and students at a specific grade level are the three main concerns when teachers seek supportive resources online.

A Socialized Knowledge Community on Computational Thinking: Teachers-Pay-Teachers presentation slides.