Gates Foundation Response to Posts on Courseware Evidence

I wrote a series of posts in the fall about the Bill & Melinda Gates Foundation’s work in postsecondary education centered on digital courseware, challenging their claims of following the data. You can read the first post about the messaging machine, the second on the push for CourseGateway, and the third on a research background in advance of their response to my questions. It has been roughly two and half months since the last post, but this topic is important, and I promised to share the foundation response. I appreciate the foundation staff and their willingness to cooperate. Without further ado (meaning that I will save commentary for later and let this response stand on its own), here is the reply from Alison Pendergast, Senior Program Officer at the foundation. – Phil [link to full-page audio]

*******

Thank you for the opportunity to share more about the foundation’s work on digital courseware. Ensuring the field knows and can apply what we’re learning from research about what does and doesn’t work for students and institutions is a crucial part of our being able to make progress collectively. It is equally critical to be clear about the why – which is to say that our work in courseware is rooted in the goal of ensuring that many more students can complete their certificates and degrees and that race, ethnicity, and income do not serve as predictors of student success. That drives me in this work; I am sure many of your readers share that aspiration. In that spirit, please find answers to the questions you posted below.

Our work at the foundation is guided by the belief that every person deserves a chance to live a healthy, productive life. This includes the opportunity to achieve an education after high school that has value, which is more critical than ever before. We do this by supporting colleges and universities committed to transforming policies and practices to be more student-centered. Helping students get on a path to completing a certificate or degree and staying on that path is central to transformation efforts.

The first step for many students towards getting on a degree pathway is taking first-year general education “gateway” courses that should serve as an introduction to a particular subject or field of study but too often contribute to pernicious and persistent equity gaps among Black, Latino, Indigenous students, and students from low-income backgrounds. Poor outcomes in these courses lead to student attrition, longer time to a degree, higher cost of a degree, and potential loss of Pell and other financial assistance tied to academic performance, despite the deep commitment and high academic potential of impacted students. So, our “NorthStar” is working with faculty and institutions who share our mission to transform the teaching and learning experience, which includes re-envisioning these large-enrolling gateway courses that function as barriers to success for Black, Latino, and Indigenous students and students from low-income backgrounds.

This goal drives our focus on digital learning curricula and courseware as a critical level in helping to help enact higher quality undergraduate digital learning, particularly as higher education institutions continue to expand blended, hybrid, and fully online course offerings. The impact of COVID also contributed to more institutions and faculty adopting digital learning tools like courseware. We believe that blended learning, in which face-to-face interaction is combined with an interactive curriculum (like courseware), has the potential to improve student outcomes, engagement, and persistence in gateway courses. We see evidence that well-designed courses using blended learning methods incorporating the use of courseware can lower DFW rates (i.e., the percentage of students in a course or program who receive a D or F grade or who withdraw) (House, Means, Peters Hinton, Boyce, Wetzel, & Wang, 2018; van Leusen, Cunningham, & Johnson, 2020), with a more significant effect on the learning outcomes of racially minoritized and historically underserved students (+0.16 vs. +0.09 among all students) (House et al., 2018). While we are focused on supporting innovative, high-quality digital learning solutions like courseware that we believe can make learning more accessible, relevant, engaging, interactive, personalized, and cost-effective for students, we are also intensely focused on supporting faculty implementing evidence-based practices that can be enabled by courseware (such as active and inclusive teaching strategies) to close opportunity gaps. These are the main focus areas in our ongoing work and research efforts related to courseware.

We see a clear role for the foundation in building on and expanding the evidence base around high-quality courseware, particularly in faculty professional development and deepening the field’s understanding of the correlation between implementation practices and improved student outcomes.

To take a step back, the primary research we funded from 2011 – 2018 examined whether 12 courseware products effectively improved outcomes across 42,000 students. From this, we learned that courseware was more likely to improve grades when implemented for specific use cases (e.g., for STEM subjects, at 4-year colleges, and in blended & online delivery compared to face-to-face instruction). However, these studies were not designed to examine how faculty implemented courseware and which factors led to improved outcomes. Therefore, we have dedicated resources to a comprehensive research plan over the next four years to understand better the faculty and student user experience with the courseware and gain insights into implementation best practices and effectiveness. This research will help us understand which courseware and implementation factors lead to the best outcomes for our focus students.

Specifically, as it relates to product/user research, we’re working with courseware partner Lumen Learning to do user research on a new statistics courseware for the general or introductory-level statistics course. Lumen has established partnerships with Minority Serving Institutions across the United States, allowing Lumen to work directly with our focus students and the faculty teaching them to co-create courseware materials through student-led user testing centers. Rockland Community College in New York and Santa Ana College in California has opened user testing centers. Lumen Learning collaborates with Howard University to gain guidance and feedback on the course and platform development, learning from Howard’s rich history of success supporting Black students.

Through this research and similar funded research with ASU and CMU in general chemistry, we aim to understand better how faculty and students (and more specifically – Black, Latino, Indigenous students, and students from low-income backgrounds) use digital learning products to support their teaching and learning. Based on the Time4Class longitudinal research we support from Tyton Partners, we know that students (and faculty) generally have poor experiences with digital learning products. There’s considerable opportunity to improve usability, general product satisfaction, and efficacy. Through this research, we aim to provide the field with deeper insights into how to design and develop courseware that is more relevant, accessible, inclusive, and effective for students from diverse racial, ethnic, and socioeconomic backgrounds.

We’ve also partnered with leading educational researcher Digital Promise to conduct a large-scale implementation and impact study for the Lumen Learning statistics courseware, focusing on understanding how faculty implement the courseware in the classroom and whether the use of courseware and implementation practices correlate with improved course outcomes for our focus populations of students. Through this research, we hope to provide the field with more insights on whether the effectiveness of courseware correlates with the use of evidence-based teaching (EBT) practices and, if so, how many semesters of implementation it takes for instructors to achieve improvements in student course success rates. We believe these data can help institutions with their gateway course redesign initiatives.

In addition to these funded efforts, we work closely with our digital learning partners at Every Learner Everywhere. ELE is a network of leading scholars, educators, education researchers, and institutions working together to understand the systemic barriers, root causes, and obstacles experienced by Black, Latino, and Indigenous students and students experiencing poverty that reinforce and perpetuate academic inequities in gateway courses. ELE partner organizations bring together significant expertise in evaluating, implementing, scaling, and measuring the impact of educational technology, course design strategies, and teaching practices in blended and online gateway course learning environments. Over the last four years, ELE partners Achieving the Dream, APLU, and Digital Promise have been working closely with twelve two-year colleges and four-year universities to implement digital learning courseware in 62 selected gateway courses enrolling nearly 27,000 students. Nearly all faculty participating (96%) believe that adding digital learning courseware has the potential to improve student learning and helps them efficiently monitor student progress and increase student engagement. While these course-level implementations yield troves of data, much more research is still needed to understand how institutions should design and deliver effective and equitable blended / hybrid and online learning programs at scale.

Over the last decade, as digital learning tools, resources, and methodologies have advanced, educators and researchers have endeavored to work more closely to understand their impact on student learning. However, there is still a tremendous gap in the evidence base of what works best in what context and for whom.

Related, we know the growth of blended and online learning will continue to grow as the technology matures and student demand grows. As we think about our research efforts moving forward with this context in mind and based on lessons learned from our own previously funded research efforts, we’re shifting our approach in several ways.

First, we’re actively engaging with and centering the needs of Black, Latino, and Indigenous students and students from low-income backgrounds and the institutions serving these students in a participatory design and development approach to ensure we’re designing with – not for- our focus populations. Second, we’re funding a much more rigorous research agenda to understand courseware’s impact on student learning outcomes, student engagement, and completion rates (e.g., do students complete the course in a manner that sets them up for success in the next step in their educational journey?). The research we’re funding is meant to prove/disprove our hypotheses rigorously and that like previously, we will be publishing and sharing the results of this research publicly to advance the field.

We continue to believe that this moment in time presents both a critical and exciting opportunity for higher ed to understand better how high-quality and affordable digital courseware can help faculty implement active, inclusive pedagogies that lead to more equitable education outcomes for students. We’re proud of the work we’re doing alongside partners and grateful for the opportunity to highlight it.

The post Gates Foundation Response to Posts on Courseware Evidence appeared first on Phil Hill & Associates.