Making Sense of Even More College Student COVID Surveys: A Meta-Analysis Update

Many public pools and some beaches are closed due to COVID-19, but we are now swimming in data from college student COVID surveys. These surveys have asked students about their experiences last spring; their feelings about remote/online learning and their institutions; their plans for the fall and the future; and their personal well-being, resource needs and life circumstances. It’s time for an update.

Since my previous meta-analysis of 11 surveys, we have added two more columns to our growing Summary of Student Surveys on COVID table on the MindWires website. These two new columns capture the month of survey administration and the broad survey topics that the student surveys covered. Now we have 22 student surveys and there are more on the way.

Double the data, double the fun, as they say, but do we have double the findings? As I did with my review of the Ithaka S+R report, this post will update the meta-analysis by confirming what we’ve been learning from earlier surveys, adding new details and findings, and exploring what we still don’t know yet.

Common Areas of Focus

Over half of the surveys have asked students questions about a number of topics:

• At least 12 of the 22 surveys asked students to share their feelings about their institution itself, its response to COVID-19, its support for students, its communications, its ability to foster community, and so on.

• At least 14 of the 22 surveys asked students about their fall plans. Some of those surveys provided different scenarios to get students’ reactions. The challenge remains that many students still don’t know how their campus plans to conduct classes this fall.

• At least 12 of the 22 surveys asked students to share their feelings about learning online or remotely, some of which broke it down to specific aspects like engagement

• At least 13 of the 22 surveys asked students about the impact COVID has had on their lives, what challenges, worries or stressors they face, and related personal experiences.

Together these topics support multiple audiences – campus leaders, faculty developers and instructional designers, and student services staff all can gain awareness for decision making and preparing to meet students’ needs in the fall. Now let’s look at the surveys in more depth.

What’s Known: Confirming Earlier Findings

As you might imagine, with this many common areas of focus, this new batch of surveys does confirm some earlier findings.

Students’ fall plans have grown increasingly less certain: The Simpson Scarborough follow-up survey in April confirmed and built upon their March findings: “In late March, 14% of college students said they were unlikely to return to their current college or university in the Fall, or it was ‘too soon to tell.’ Exactly three weeks later, in mid-April, that figure had gone up to 26%” (p. 7). A month later, the California Student Aid Commission (CSAC) found that “over 80% have either changed some aspect of their plans for Fall 2020 or are still uncertain of their plans.” CSAC broke it down further, identifying why those plans are less certain:

• Finances: 34% think they need to work more, 21% think they need to attend a less expensive college

• Location, course delivery method, and fatigue: 25% want to stay closer to family, 22% do not want to take online classes, 15% want a break from college due to the pandemic

Recommendation for campuses: Put together a multi-unit team of advisors who can talk through a given student’s barriers and suggest solutions that would enable them to stay in school.

Almost 6 in 10 students faced basic needs insecurity this spring: The Hope Center’s results confirmed what we learned from the Ithaka S+R survey and added more details: during the spring 58% of students experienced basic needs insecurity – food insecurity, housing insecurity and/or homelessness. Also echoing Ithaka’s findings, The Hope Center found that basic needs insecurity was 19% more prevalent for black students (71%) than for white students (52%). Recommendation for campuses: Identify and create partnerships with local and distant community partners – e.g., churches, food banks – that support students with basic needs. Then broadcast that information far and wide to the students. If it’s true that hunger affects learning in K-12 environments, then it’s equally true for higher ed learners, even if the classrooms are virtual. Similarly, if your campus housing will be closed or reduced this fall, do whatever you can to help students find somewhere to live.

Lack of connection and community: College Pulse shared that “a majority (55%) of students say that a shift to online instruction would make it much more difficult to feel like they were part of a campus community” (p. 9). Recommendation for campuses: Start building, growing and/or promoting virtual community spaces and virtual campus events. Even if your institution will open or partially open, it will be difficult for students to gather and interact with physical distancing and masks.

What’s New: Discerning Unique Details and Findings

Some students who need aid don’t know they are eligible to apply or just don’t apply: The Hope Center found that “many students did not apply for [basic needs] supports [e.g., unemployment insurance, Supplemental Nutrition Assistance Program, emergency aid] because they did not know they were eligible to do so.” CSAC found similar results regarding living expenses: only 43% of students were aware of aid for living expenses, and then only 39% of those who were aware applied. CSAC also found similar results related to technology aid: only 56% of students were aware of free or loaner laptops, and then only 12% of those who were aware applied for one. Recommendation for campuses: Start awareness campaigns before the fall begins, so students have time to apply for emergency aid or other support.

How often students want communication: Many surveys identified students’ dissatisfaction with communication from their institution. SurveyMonkey went further and asked how often the students would want to be contacted by or to have contact with faculty or someone from campus:

• Surprisingly, over half of the students responded that they would like updates from the institution every one to three days – daily (25%), every two or three days (29%). Another quarter (27%) would like updates “whenever information becomes available.” Recommendation for campuses: Create a schedule of updates about different aspects of fall preparations. Even once a week would be a start.

• In its second round of surveys, SurveyMonkey found that almost three quarters of students (72%) expect an email response from instructors within a day. Recommendation for faculty: Schedule time every day to answer student emails.

Less experienced online learners see less value in support mechanisms: SurveyMonkey also asked students to rate the benefit of instructors holding virtual office hours each week. Fewer than 15% thought this would not be beneficial. The cross-tabulation of responses by how prepared the students were for online learning showed that the “extremely/very prepared” students ranked virtual office hours more highly than students who were “somewhat prepared” or “not at all prepared” (see SurveyMonkey’s results for this question below). This may be an indication that less prepared students do not understand the value of virtual office hours or other support mechanisms that would help them succeed. Recommendation for faculty: Make the benefits of office hours and other support options clear to your students, especially those who may need it most!

What students feel are priorities: A number surveys identified the mounting financial pressures and resultant stress for students due to high unemployment rates and other factors. College Pulse found that “Six in ten students say that working on new ways of helping students identify internships and job opportunities (60%) and no longer requiring the purchase of expensive textbooks (57%) should be among the highest priorities for their college or university.” Recommendation for campuses: If you are not informing faculty about Open Educational Resources (OER), it’s time to start. Just before COVID-19 hit, Achieving the Dream found that introducing OER courses at scale – across degree, certificate and credential programs – saved students a lot of money. What’s more, “students taking multiple OER courses earned more credits and similar grades than students who took no OER courses.”

How campuses might hold onto existing students: Niche Partners found that “30% of students reported that they would be likely to transfer if their school continued online learning in the fall.” They followed that grim number with a ray of hope: “The best scenarios to mitigate undergraduate attrition are either living on campus while learning online (the model used by Minerva Schools at KGI) or a hybrid flexible model (in-person and online simultaneously). These scenarios proved to be appealing to students who were otherwise strongly considering transferring.” As a telling sign, each week more campuses are claiming to adopt some form of hybrid or hybrid flexible courses, even if they don’t stick to the official definitions of those two course delivery models. Recommendation for campuses: Follow the lead of campuses like Tarleton State University in the Texas A&M system. They have been scheduling departmental workshops on HyFlex Course Design for Faculty and have been training a cohort of 50 faculty to act as peer mentors for other faculty within their schools and disciplines.

How campuses can help existing students: In my post about helping students to be successful (resilient) learners this fall and other posts, I have mentioned that we are putting a lot of effort into preparing faculty over the late spring and summer. However, it’s not as clear how much we are doing to prepare students for whatever comes their way. Adding credence to this point, in their follow-up survey SurveyMonkey found that “58% feel unprepared to continue learning in [a virtual environment] for an unknown period of time.” Recommendation for campuses: Start building comprehensive, modular orientations that students can complete before classes begin. Support them with online learning skills, as well as addressing needs for finding adequate study spaces and resources.

How effective teaching affects student satisfaction with online learning: Every Learner Everywhere is one of the few surveys that asked students if their teachers used evidence-based practices. They provided a list of eight known practices, such as “using real-world examples to illustrate course content,” and asked students whether or not an online course used them. Ratings of student satisfaction with online learning increased as their instructors incorporated more practices. “Net satisfaction for courses employing 0-2 of the recommended online instructional practices was 43 percent compared to 61 percent for courses using 3-5 of the practices, and 74 percent for courses using 6-8 of the practices” (see Every Learner Everywhere’s Figure 8 below). Recommendation for faculty: Incorporate these instructional practices that had the largest individual effects:

• Use personal messages on how each student is doing or to make sure they can access course materials

• Use real-world examples to illustrate course content

• Require students to express what they learned and what they still need to learn as part of each assignment

What’s Missing?: Identifying What We Still Don’t Know

The surveys are not really asking students what they need: These surveys – individually and collectively – tell an important story. However, despite participation by thousands of students, that story feels like it’s told in the third person. In the common areas of focus, I mentioned the campus stakeholders who can learn from the survey results to address students’ needs. Above I also show how some surveys broke down different academic, organizational, social and personal factors to get to the root of specific issues. On the flip side, very few of the 22 surveys directly asked students to state what they need. For example, the Student Senate for the California Community Colleges (SSCCC) specifically asked “How can the SSCCC help you?” This means that we may be relying on our analyses of survey results instead of going straight to the source.

We need more surveys that ask about online teaching and learning practices: Every Learner Everywhere asked students to note when effective online teaching practices were used. Some of the surveys asked about access to adequate study spaces (around a quarter do not have them), access to technology, and other resource related questions. I have not seen enough attention to generating student awareness about practicing evidence-based online learning skills – e.g., time and task management, self-directed learning – and how to practice those skills in suboptimal conditions and/or while balancing other priorities like work and family.

It’s not enough to read the survey summaries: Having read through the results of each survey multiple times, I’ve noticed that the summaries are best seen as helpful guides. However, in most cases, you get a much more complete picture when you start looking at the results in detail. In its one-page summary of key findings, College Pulse shared that “More than six in ten (63%) students say online learning could be improved by using a better technology platform” (p. 1). However, it’s not clear what those students mean from this one statistic. Moreover, just a couple of pages later, a student quote callout painted a slightly different picture: “Three out of four of my professors had never taught in an online setting. One transitioned very well, but not the other two. The biggest obstacle is the technology. One professor had us email our assignments because they didn’t know how to create assignments on Canvas” (p. 3). So are those 63% of students thinking a better technology platform would make it easier for them to use, or easier for new online faculty to use? Do they really mean that faculty should have more experience with the technology platform? Or is it something else entirely?

What’s Next: Moving Beyond the Surveys

With 22 surveys and counting, we have plenty of data to mine. My first meta-analysis and this update certainly don’t cover everything there is to know. Plus, there are more surveys on the way. Several organizations are going to conduct another round as we get closer to the fall academic term. Keep letting us know about new ones that emerge or your institution’s plans to survey students. There are also a growing number of faculty surveys, which I intend to analyze as well.

With so little time before the fall begins, though, we also need to move beyond the surveys. Over the past two weeks, I have been pleased to learn about a number of institutions that are engaging students more directly. On different listserv conversations, campuses are exploring how to engage students as partners. Concepts range from conducting local student focus groups to creating student advisory boards. Bryn Mawr facilitates a Students as Learners and Teachers Program which has run for about 14 years now. More recently this group has generated very useful resources that support creating your own student partnership or pedagogical partnership programs. In June Alison Cook-Sather at Bryn Mawr and a student partner, Nicole Litvitskiy, produced and shared a two-page document, “Student Partners’ Recommendations Regarding Remote Teaching and Learning.” It’s full of ideas and quotes by students that offer insight into their side of the teaching and learning process. If you want to take a deeper dive into student partnerships, I recommend that campus staff and faculty download or purchase the print version of the book, Pedagogical Partnerships: A How-To Guide for Faculty, Students, and Academic Developers in Higher Education, by Alison Cook-Sather, Melanie Bahti and Anita Ntem. Please share your stories of how you are engaging students at your campus.

The post Making Sense of Even More College Student COVID Surveys: A Meta-Analysis Update appeared first on Phil Hill & Associates.