CourseGateway: They knew the solution all along

Author

Phil Hill
October 14, 2022

I have a feeling that a blog series is developing.

Two weeks ago I wrote a post about “The Gates Foundation Messaging Machine,” where despite claims of following data, the reality of the research release was all about the messaging desired by the foundation. I asked:

Wait – is this a research report trying to understand perspectives of those who chose to not to pursue a degree, or is this an advocacy post with pre-ordained outcomes?

Late yesterday EdSurge published an article about CourseGateway, titled “New Effort Hopes to Make ‘Weed-Out’ Courses More Equitable,” clearly leveraging the interest in the NYU Organic Chemistry debacle as well as the general concern over disadvantaged groups’ performance in these critical gateway courses. And this story is also about the Gates Foundation’s desired messaging. The setup:

Research by the nonprofit Gardner Institute and other groups has found that these typically-freshman-year classes tend to prune some kinds of students from academic tracks more than others. For example, a new study published in the journal PNAS Nexus found that, among students who perform poorly in intro STEM classes, those who are underrepresented minorities are even less likely to end up earning STEM degrees than their white, male counterparts.

And the pandemic may be exacerbating this situation. Research out this year from consulting firm Tyton Partners, the Bill & Melinda Gates Foundation and Every Learner Everywhere found that professors lately are reporting an increase in the DFWI rate—that is, the percent of students in a course who receive a D grade, fail, withdraw or don’t complete the class.

The problem is the increase in DFWI rates of gateway courses, with a particular emphasis on underrepresented minorities. That is the issue at hand – a bad situation that needs to be addressed.

But the article then adds this whiplash-inducing transition from the problem to the proposed solution.

“Statistics show significant inequities in the completion rate in gateway courses,” says Andrea Jones-Davis of Educause. “Research shows students may drop out or change their major based on these gateway courses.”

Jones-Davis is the director of a new effort, CourseGateway, which aims to help more students—and especially Black, Latino, indigenous and low-income students—succeed in the first few classes they encounter in higher ed by promoting the adoption of high-quality courseware, a category of tech tools that packages digital reading materials with assessments and study supports. The theory is that courseware has the potential to improve student outcomes because it tends to offer students personalized instruction and instant feedback on their work, Jones-Davis says. These tools can also help instructors track whether and how students are studying and completing assignments.

Apparently, there was insufficient curiosity to ask the obvious question: what is the basis for your belief that promoting the adoption of high-quality courseware will actually help more students – and especially Black, Latino, indigenous and low-income students? The rest of the article goes into descriptions of how this courseware promotion intitiative will function. The key point is that the CourseGateway initiative is based on a product review site that aims to “provide enough information so that higher ed educators can select products for use in courses.”

Promotion

Promotion is the key word here, both in who is doing the promoting and what is being promoted. Educause received a 30-month grant last year from the Gates Foundation for $4,945,954, with CourseGateway being one of the key deliverables. Who is promoting? The Gates Foundation through Educause with an assist from EdSurge. For those attending the Educause conference in two weeks, I am quite confident that you will hear more about this initiative in Denver. ISTE, the parent of EdSurge, is working on two separate grants from the same foundation for $3,180,033. Those grants are focused on K-12 and not higher education, but they should have been disclosed with the article.

As to what is being promoted, the answer is clear and historically consistent, “the adoption of high-quality courseware.”

Searching for a Connection

Even if the article won’t address the question about the connection of the problem and the solution, we can.

The Gardner ($600,000) and PNAS Nexus cited research directly touch on the question of underrepresented minority performance in gateway courses, as stated, but that is not the issue at hand. The Tyton Partners research ($10,381,410 under three grants, with the two postsecondary ones ending around the time of the stated research release), however, does touch on the solution of courseware.

Tyton, along with Bay View Partners and everylearner everywhere (the Gates Foundation funded initiative under WICHE, which has $2,909,910 funding under three grants) wrote the report “Time for Class: The state of digital learning and courseware adoption,” and it is based on a Spring 2022 survey of “approximately 850 administrators and 3,200 faculty at 1,200 unique postsecondary institutions and interviews with over 15 instructional materials and digital learning providers,” focusing mostly on introductory gateway course usage. Like most of Tyton-led research, it is very well-written, organized, and documented. I’ll admit some jealously in how well they present their reports. So far, so good – let’s hear from educators in an organized fashion.

The quote used in the EdSurge article is accurate (increased in DFWI rates), but that does not show any connection between problem and solution.

What is apparent from reading the report is that it is research in the sense of product management research around courseware. Let’s analyze adoption, look at use cases, look at Net Promoter Scores, align to known evidence-based teaching practices, and finish up with recommendations to improve the product category. And it is honest product management research that does not artificially inflate the value of courseware. In fact, the organized view of educators does not support the claimed connection between high-quality courseware being a solution to inequitable gateway course performance. In current use cases for courseware, the bottom two listings are improving outcomes for students with financial need and improving outcomes for Black, Latinx, and Indigenous students.

Courseware use cases based on survey of educators

A good section in the report is based on comparing non-courseware adopters to courseware adopters in their usage of evidence-based teaching (EBT) practices. In this view – courseware adoption seems to be negatively associated with these practices, with only three out of 10 practices increasing for courseware adopting educators.

Use of detailed evidence-based teaching practices by courseware usage

The report then compared courseware to e-texts to a new product category – advanced e-texts. The idea of this new set of products is to add a few pieces of functionality to a static e-text (auto-grading, enhanced LMS integration, practice questions, etc). When looking at adoption of evidenced-based teaching practices, it is the new advanced e-text product category that shines, not courseware, most likely due to product simplicity.

ebt adoption by courseware usage or e-text usage type

I’ve Got a Fever, and the Only Cure is More Courseware

The CourseGateway initiative, like most of its Gates Foundation funded predecessors, makes nebulous claims without any real basis. On their about page:

When implemented well, digital learning demonstrates the power to close equity gaps in gateway courses and throughout the curriculum. Two crucial components of this process are building awareness and adopting high-quality, equitably designed courseware that improves student outcomes.

Ouch, that’s some slippery logic. Digital learning can solve it all, but that includes many product categories. But they declare that building awareness and adopting courseware are crucial and produce results.

Really? Back up that claim if you want us to believe that this initiative will produce any results. The Tyton report funded by the Gates Foundation suggests that product review of courseware might not be the answer, at least based on the input from real educators teaching real students. And there is no other research or, you know, evidence presented that suggests that courseware adoption as an intervention will improve the DFWI trends of underrepresented students in gateway courses, aka “the problem to be solved.”

Disclosures: Lumen Learning, which has courseware being evaluated through CourseGateway, has been a long-time client of MindWires. MindWires used to be a grantee of the Gates Foundation from 2013 – 2018.