Sometimes a Good Result is Just a Good Result

ASU virtual reality program achieving significant course improvements at scale

Author

Phil Hill
June 01, 2023

Was this forwarded to you by a friend? Sign up, and get your own copy of the news that mattered sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.

To be honest, as the author of “We Don’t Need Vanity-Driven MOOC 4.0 Hype” about a recent Arizona State University (ASU) initiative, I was a little surprised at the invitation to see a demo of the Dreamscape virtual reality (VR) lab at the ASU+GSV conference. And then to be invited to follow-up on the ASU campus with the faculty, staff, and Dreamscape leaders heading up the initiative. I come in with a certain level of cynicism, especially given the Michael Crow taps Steven Spielberg to rethink education narrative.

After the campus visit and extensive demo and tour of the course, however, I have a different perspective:

  1. This is a pedagogy-driven course redesign initiative that happens to use VR, effectively, in a narrative format.

  2. ASU is getting impressive results already, beyond a pilot and at scale.

Sometimes a cigar is just a cigar.

The Basics

Inside Higher Ed described the initiative in a recent article.

Arizona State University had a problem. Many students arrived on campus eager to study science, technology, engineering and mathematics subjects. But by the end of their first year at the university, nearly half switched to non-STEM majors or left the university altogether, according to Annie Hale, executive director of the Action Lab at Arizona State. Though that statistic tracked with the national trend, campus leaders were concerned that these students were no longer on track for high-paying STEM jobs.

“We measure ourselves by our charter of who we include and not who we exclude,” Hale said—the institution has been designated a Hispanic-serving institution by the U.S. Education Department.

To address the problem, Arizona State turned to an unlikely source: Hollywood. Dreamscape Immersive, a virtual reality company co-founded by an individual responsible for blockbuster movies such as WarGames and Men in Black, partnered with the university to create Dreamscape Learn. The new educational company revamped Biology 181—the university’s introductory course. In the new version, which was offered first in the spring of 2022 alongside the original version, immersive VR experiences replaced traditional labs.

In addition to Biology 181 for STEM majors, ASU and Dreamscape redesigned Biology 100 for non-STEM majors, also using the VR-driven labs although at a reduced level.

The popular discussion unsurprisingly focuses on the VR angle, as students sit down to an immersive experience with VR headsets and hand sensors.

What has changed is the lab experience, where the traditional course labs have completely been replaced by a VR design that immerses students in an imaginary world. Some students take this campus immersion version, but online students take a 2D streaming digital immersion version.

In the setting that I experienced on campus, students are virtually placed in pods that move through the environment to observe a problem with dying flora that happen to be near the water distribution system. Each week of the course, the lab design will cover one part of this story - observing the environment, finding diseased plants, investigating what might be causing plants to die, taking direct measurements to further diagnose the cause (spoiler alert - heavy metals leaching from water pipes), trying to cross-pollinate plants to find versions that are resistant to the toxins and can thrive, and then watching the new ecosystem emerge.

At this point, most of the coverage jumps to the results where in Spring 2022 students across the board in the VR lab version outperformed students in the traditional lab setting. (DSL = Dreamscape Learning).

ASU made a big bet in Fall 2022, replacing the traditional lab with the DSL version (both campus immersion and digital immersion) for all students, roughly 6,000 - 8,000 per term. The results from Fall 2022 no longer compared to non-DSL labs, but the results continued to improve (note that not all students consented to participating in the study, which is a limitation described in the report).

There’s a lot to digest here, partially because the ASU team did such a thorough job with the reports, showing results, limitations of the study, withdrawal rates, observational analysis, etc. And there is good coverage at the ASU site, at IHE, and in today’s Hechinger Report newsletter.

Missing Course Redesign Angle

What is not coming through in those articles is the comprehensive nature of what ASU has done to Biology 100 and 181. This is a pedagogically-driven course redesign with the labs enabled by VR and tied together with a narrative structure.

The lab experience was designed alongside the rest of the course instruction, and for the approximately three hours of weekly lab time, the VR sessions are only 15 minutes. After the short VR sessions, student then have nearly three hours of graded lab activities housed in Canvas, the university’s LMS. And Canvas integrates with DreamScape Learning to college the VR-activities (including formative assessments from virtually pushing buttons), to pull the data together.

John Vanden Brooks and Michael Angilletta, both professors and associate deans of immersive learning at ASU, redesigned the courses comprehensively, with their own material (not based on a textbook). The team working with these two included the DSL staff, who handled creating the environment but also with creating a narrative.

The Hechinger Report summarized the results:

Data from an in-house study done last spring indicates that this new technology seems to be working; students in the Dreamscape Learn lab version of introductory biology were 1.7 times more likely to earn an A in the class than those enrolled in the traditional model. And the study found that across gender, race and ethnicity, and socioeconomic status (measured by Pell grant eligibility), students enrolled in the Dreamscape Learn version earned higher median scores on lab assignments than their counterparts who were not. The only subgroup that did not score higher was honors college students, whose scores stayed the same across both groups. 

Engagement, Engagement, Engagement (in Jan’s voice)

Beyond the comprehensive nature of the redesign and the relatively small but important role of VR, I was specifically blown away by the engagement angle. Participating in the VR sessions and observing lab activities, I was struck by how much of my attention was directed toward the story. IHE covered a different storyline of an imaginary creature within the lab environment, with an edgy quote.

“Don’t do that!” another student responded as [Annie Hale, executive director of the Action Lab] and her team looked on. “You’re going to fucking kill the astelars!” That was the moment that Hale understood that VR fostered the students’ empathy for the (imaginary) creatures.

“I’ve been blown away by students’ reactions,” Hale said. “The VR is the glue that’s really engaging the students and inviting them to participate in the curriculum, willingly excited to look at Excel spreadsheets.”

That last part is the key, even though I would term it engagement instead of empathy - I wouldn’t care about the demise of an astelar unless it tasted like brisket, but the story kept my attention. The overall design and immersive nature of the labs, and the narrative structure, pull you into engaging with the material. The concepts make sense, you don’t check out and look at your phone, you don’t google for answers - you pay attention. And you think about what happened earlier and what might happen next.

Poor student engagement is a plague not just upon digital education but increasingly across all types of higher education, especially since the pandemic.

What ASU has demonstrated is that higher education courses can drive up student engagement and therefore student learning outcomes. At scale. This is not an isolated experiment for ASU, it is how they teach introductory level Biology to all students, thousands of them per term.

Extrapolation, Extrapolation, Extrapolation

ASU is a partial owner of Dreamscape Learning, and both organizations plan to spread this course design, through licensing and VR sales. I do not have enough of the financials at hand to comment on the feasibility of other schools adopting this design, other than to say that with ASU’s scale, the numbers work out. This is not an operation that has to run at a loss for ASU, and if you factor in improved retention, it will likely pay for itself.

The problem, however, is that higher education in general is not the best environment for extrapolating results or designs. Morgan covered this in her post about the CUNY corequisite remediation situation. In that case, other systems and states seemed to have isolated on corequisite (as in no remedial courses) and ignored that CUNY also replaced algebra with statistics. Good for CUNY and their results, but bad for those hoping to extrapolate with a partial understanding.

I believe that will be the challenge in this case as well. Other schools will tend to focus on the flashy VR angle, and even if they license the whole course design, they might not include the right kinds of student supports as well as faculty incentives and professional development as is the case at ASU.

But even with those future challenges, I think it is worth appreciating the successful results to date in this bold course redesign and the case study on how to engage students academically.

The premium On EdTech+ newsletter is intended for a single recipient with occasional forwarding allowed.