The State of Online Learning

Was this forwarded to you by a friend? Sign up, and get your own copy of the news that matters sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.

The annual CHLOE survey on online learning was released this week. This year, Educause joined Quality Matters and Eduventures in conducting the survey, titled “Strategy Shift: Institutions Respond to Sustained Online Demand.” The result is an interesting snapshot of the state of online learning but one that leaves me with more questions.

For the uninitiated, since 2017 the CHLOE report has surveyed Chief Online Learning Officers (COLOs) at higher education institutions in the US to map the changing landscape of online education. It has become an invaluable resource in understanding the topography of online learning in the US as well as how it is changing as more institutions move online and that environment becomes more regulated and more competitive.

I found this year's survey much more engaging than in recent years. Many of the questions asked and the way they were framed address the issues of concern to people managing online learning. There is a lot in the survey, and I am going to break my coverage into two parts. In this first post, I want to cover the more general findings of the survey as well as some of my concerns with it. In my next post, I want to do a deeper dive into what the CHLOE survey reports about outsourcing and OPMs, a topic that we have covered extensively in this newsletter.

The focus of online strategy

One of the fascinating insights from the survey is the extent to which the emphasis in online learning currently is on developing online versions of on-campus courses. 69% of COLOs report that the top priority was creating online versions of on-campus courses. This goes against a lot of conventional wisdom about the fears of online programs cannibalizing on-campus students.

This is an interesting finding, and I would love to see some analysis of what this means. The optimist in me wants to see this as an example of schools getting it right, of doing what Phil recently spoke of: the most successful online programs using online education as a strategic way to meet student needs. Instead of chasing trends or revenue for its own sake, this move could signify expanding access and flexibility. However, the pessimistic side of this could be that institutions are working with what they know and what they have on hand rather than exploring new and emerging needs in their communities and ways of addressing them.

Increasing enrollment is the primary driver for most online initiatives, but demand from existing students is clearly also important.

Online learning and revenue

While online programs are in demand from students, they are also an important source of revenue for institutions. The 52% of COLOs identifying online learning as a net revenue generator is up from 47% in 2020, and those who see online learning as a cost center are down from 26% that year.

This view is marginally stronger among four-year public institutions than private ones, although the question wording is not helpful here. The fact that revenue varies across programs does not mean that programs aren't an important source of revenue, so it is hard to tell what the number is.

I was surprised by the survey findings about revenue distribution within the institution. Close to half of COLOs report that revenue from online programs is combined centrally with all other revenue and then distributed. I would have expected that number to be lower than 48%. I see a lot of institutions where most online revenue goes directly to colleges and departments, but this is likely a result of my spending more time looking at larger research-intensive institutions. It would be interesting to get a breakdown of revenue distribution by type of school.

At institutions where revenue does go to the central administration, it would be interesting to learn about how they are incentivizing departments and instructors to participate in online learning. This can be a challenge when more departments do not directly see some of the revenue from online learning. Naturally, having revenue flow directly to departments or colleges has its drawbacks, especially as online learning numbers grow. However, the predominance of centralized budget models may explain some of the barriers identified to online learning growth.

Some thoughts on method

This year's CHLOE survey is a vast improvement on last year's (and not only because the word "HyFlex" was only mentioned once). The questions covered topics for which we need answers to better understand how online learning is changing. But I do have some complaints.

In part because the questions and question wording have changed so much over the years, it is hard (but not impossible) to get a real sense of change from the data. The general title for the CHLOE project is "The Changing Landscape of Online Education," and we get a stronger sense of a snapshot than change, per se. I realize the need to bring in new topics, such as AI in this year's survey. I have also run big surveys and understand the frustration that comes from not being able to fix a poorly worded or outdated question because of a concern with longitudinal data. But more ability to get that sense of change over time would strengthen future CHLOE surveys.

The sample is small. They received 324 responses for an overall response rate of 13.5%. That is not bad as a rate, but the overall number is small when you consider the variety of different types of institutions and the widely divergent places that institutions are when it comes to online learning. Given the involvement of Quality Matters and Educause in the survey, surely they can get more of their members to participate.

The sampling also gives us some odd results. For example, in a question about the relative strength of student support, we could walk away with the impression that library support is an area of greatest strength. This does not sound accurate to me, especially having just read through many university proposals for new online graduate programs where the argument was made that adding (sometimes hundreds) of new graduate students would have no impact on the library. The library may have a different perspective. If this is true of library support, I wonder how strong support is in other areas identified in the question.

Similarly, in a question about having a strategic approach to online learning, you could come away with the impression that online learning is fully a part of at least a third of most institutional strategies if not more.

Again, I think this is sampling bias reflecting the kinds of COLOs more likely to respond to this sort of survey. Having read a dizzying number of institutional strategies, online learning does not often feature, though I expect that will change over time.

Parting thoughts

Despite these issues, the latest CHLOE survey is a welcome contribution to our understanding of online learning in the US. I look forward to digging into their coverage of OPMs and online learning regulations, which will be the topic of my next post.

Update 14 Aug: Fixed the reference to Quality Matters and not OLC.

The main On EdTech newsletter is free to share in part or in whole. All we ask is attribution.

Thanks for being a subscriber.