Friday Follow Up, Part One

Such a backlog of news and responses after the BBQ-fest of Educause in San Antonio

As a reminder, this will be the last Friday Follow Up post in the series to be shared with all subscribers (free and premium). Next week the series will be premium only. You can continue reading with a $10/month or $100/year premium subscription along with all of our other content.

After four days in San Antonio for Educause with little time for writing (someone has to see if whiskey or mezcal is a better complement to brisket), there was a lot of stuff that built up this week. To keep the emails a reasonable length, this week’s entry will be broken into two parts - this one on post follow ups, and the next one on news events.

The Lockhart pilgrimage with my wife Emily and Kevin Kelly

The Research Angle on the ‘Fully Online is Bad’ Kerfuffle

Two and a half weeks ago Morgan and I co-wrote a post calling out the bad faith PR campaign from the Arnold Ventures-funded coalition (this time Third Way), regurgitating a flawed research article from nearly a year ago titled "The Role and Influence of Exclusively Online Degree Programs in Higher Education". The core of the research and accompanying op-eds by the researches claimed that fully-online programs lead to “students enrolled in exclusively online degree programs were 8.3 percentage points less likely to complete a bachelor’s degree than their non-exclusively online peers.” Our post focused on the coordinated campaign, the misleading regurgitation-as-new-data, and the gullible media amplification.

Morgan and I briefly noted that we have problems with the research itself but didn’t go into details. Well, thanks to noted researcher Al Essa, we don’t need to. After Essa made some initial comments to our article in LinkedIn comments, he subsequently wrote a tour de force article titled “Does Online Education Suck? A Research Perspective.” The article examines the research and along the way providing an incredibly useful education on research methods. I know what you might be thinking - don’t invite Essa to a cocktail party. But you would be wrong - the post is informative with great examples and a compelling narrative.

Essa described the research and study design from a researcher’s perspective, and then he clearly called out his own approach to his critique. In clear terms, Essa explains the role and limitations of quasi-experimental designs and how the OHA (based on the last names of the three researchers) research implemented their design. Essa calls out where the techniques were thorough and well-conceived, but he also calls out the fundamental problems. There is a ton of useful information, and I strongly recommended reading Essa’s piece in full.

What I’d like to highlight for now is one key issue that Essa identified - the OHA researchers on one hand developed an overly-complex method while on the other hand left out key confounding variables such as institution type (i.e., “a factor that influences both whether students choose online education AND their likelihood of graduating). Other factors include support levels for students, etc.

If institution type is a major confounder that isn't properly specified in the main analysis, the sensitivity analysis can tell us how strong such unmeasured confounding would need to be to explain away the results, but cannot directly tell us whether institution type specifically is that confounder. OHA’s statement of sensitivity analysis is also misleading: “Our sensitivity analysis checked all outcomes and model specifications, revealing that it would be unlikely (emphasis mine) that an observed treatment effect associated with enrolling in an exclusively online degree program was caused by an unmeasured or hidden confounder.” It is misleading because it implies the improbability of a hidden confounder, which is not what Rosenbaum’s sensitivity analysis can assess. Rosenbaum’s method is designed to measure how strong a potential unmeasured confounder would need to be to nullify or significantly weaken the observed treatment effect, not to estimate the likelihood of such a confounder existing.

There were some excellent contributions by the OHA researchers in their approach, but overall it was fundamentally flawed by ignoring alternative explanations, not including key confounding variables, and making potentially harmful policy recommendations. The problem was not just in the PR campaign, there were also key issues with the research itself.

To Essa’s analysis, I would add the factor that the cohort studied in the OHA report started in the 2011-12 academic year, when online education was in a far different state. This variability between data sources from a decade ago and conclusions / recommendations for today is another unexamined aspect in the research.

I’ll leave this section with Essa’s conclusions on the policy traps.

Without properly accounting for institution type as a confounder, we risk implementing policies that address symptoms rather than causes, potentially making it harder for non-traditional students to access higher education while failing to improve their chances of success.

The Black Knight of EdTech Investment Activity

Last week I noted that something was in the air around EdTech investment activity - both M&A and new investment. Not a turnaround, per se, but some change is happening. I received a thoughtful reply by email from Christopher Nyren, founder of Educated Ventures, and he gave me permission to quote as needed. In short, Nyren also sees a change, but more in the K-12 space.

The bottom fell out of the market in 2022 and buyers were afraid of the end of the world or at least under strict instructions to cut spending and conserve cash. Many of the newish EdTech start-ups / founders were a bit insulated from that fear mongering or still believed in the 2021 pandemic era school-at-home bubble and the employment oriented workforce / HCM start-ups had yet to see their end clients really start to reduce staff and, with such, seats for their underlying SaaS or marketplace solutions.

Buyers were marshalling cash and for those that did want to buy, there were no market clearing prices.

This changed in 2023, at least from my perspective and portfolio. Initially the ice was thawed for AI education start-up with real technical chops and actual client revenue traction fresh off the attention grabbing launch of Chat GPT 3.5 in Q3-4 2022, but by mid 2023, I was seeing plenty of demand for my K-12 curriculum companies at historical SaaS multiples (6-12x ARR). I guess buyers got the all-clear from their investors, especially in K-12 which has remained relatively insulated and increasingly confident that the expiration of ESSER funding was not going to crash their business. This continues through today.

That being said, employment levels and HR budgets have remained challenged thus making it hard for me to get most of my Workforce / Professional Education / HR deals over the finish line. With a challenged outlook for end markets as well as the new demographic realities and eternal product-market fit issues of traditional college programs, Higher Ed M&A has also remained difficult for me and my companies.

Again, at least from my perspective, Education M&A is really a Tale of Two Markets: K-12 and everything else.

Add to this Matthew Tower’s comment on LinkedIn.

I agree, there is something in the air. I believe these deals show there is plenty of appetite for education businesses with healthy unit economics and a strong grip on cashflow. Less appetite for growth at all costs.

Something is in the air, but the extent and full nature of the changes are not clear yet.

While We’re At It

This one is not a direct follow up to one of our On EdTech posts, but I’ll highlight it as it is Robert Ubell’s tour de force. On LinkedIn (and X), I strongly recommended that people read Ubell’s article in The Evolllution titled “The Uberization of Higher Ed.” The usage of adjuncts is not a new topic, but Ubell really nails the hidden problems and contradictions inherent in the system (yes, two Monty Python references in one post).

According to the latest government data, higher ed contingent workers now occupy the lion’s share of the nation’s academic workforce at an overwhelming two thirds (68%), while tenured and tenure-track faculty have been reduced to a dismal quarter (24%). These numbers reflect a deeply troubling dependency on perilous academic workers, with some calling them the new faculty majority. In a topsy-turvy transformation, it’s a dramatic reversal from three decades ago, when the nation’s college instructional staff occupied nearly equal proportions of contingent and tenure-stream faculty.

It’s a common belief that professors teach most classes at our colleges, when the fact is that most students in American classrooms receive instruction from others, mostly graduate teaching assistants and adjuncts. If you wander down the halls, poking into classrooms, there’s no obvious, discernable difference between full-time, credentialed faculty and contingent instructors, especially when the two are assigned similar courses, often teaching right next door to each other in the same hallway. As in Shakespeare comedies, you often can’t tell who’s who, but in college it’s no laughing matter. Students mostly aren’t aware of who among their teachers is a tenured professor and who is an adjunct. And they don’t know which one earns a handsome wage and who makes just peanuts.

My contribution was to look at the ability to improve teaching while using adjuncts in the way they are most commonly used.

“Colleges mostly use contingent faculty for teaching and learning,” Phil Hill, publisher of the On EdTech newsletter told me. “With tenure, you’re involved year after year, but adjuncts don’t get access to their courses until the last minute. So, if you care about quality teaching, you’re scrambling to make it happen.”

Robert Kelchen, head of the Department of Educational Leadership and Policy Studies at the University of Tennessee at Knoxville, agrees: “There’s a very real concern about the quality of adjunct teaching.”

Personally, I think that the tenure angle is overused and often enables a caste system in higher education - the bigger issues are part-time and precarious when not by choice. But part of the strength of Ubell’s article is that his core arguments stand, even without the tenure angle. In my LinkedIn post, there was some excellent commentary picking up on this ‘last minute’ problem I noted.

Deb Adair from Quality Matters expounded on this topic.

There are many good points raised by Robert Ubell in this article - many negative consequences for both adjuncts and students - in the ways adjuncts are used and treated in HE. To highlight just one, just-in-time teaching assignments serve no one. How can these instructors be adequately prepared to teach in this situation? These last minute assignments are far too frequent, a tactic rather than an exception, because it keeps instruction costs in alignment with demand and revenue.

For online courses, it can be especially problematic if the instructor has not had adequate training to teach online. This can be as true for full-time faculty who primarily teach F2F and are thrust into teaching online at the 11th hour as it is for adjuncts. If I had a dime for everytime poor student engagement and performance is blamed on the online modality, rather than poor practice in preparing and supporting faculty to teach in this modality, I would be a rich woman.

And there you have it - this section also ties back to the online modality argument at the beginning.

You know, you come from nothing - you're going back to nothing.

What have you lost? Nothing!

Always look on the bright side of life.

- Mr. Cheeky

The premium On EdTech+ newsletter is intended for a single recipient with occasional forwarding allowed.