From Required Data to Useful Action

Was this forwarded to you by a friend? Sign up, and get your own copy of the news that matters sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.

Increasingly, people from across the political spectrum agree that students graduating from US universities should earn a reasonable salary that reflects their investment in education. We see this argument in many current conversations about college, as well as in regulatory moves like Gainful Employment and Financial Value Transparency. More recently we are seeing it in the reconciliation bill from the US Congress. In that bill, earnings data are used in complex equations that calculate how much financial risk an institution should share, alongside graduation rates and program costs.

We touched on the the Return on Investment (ROI) or College Premium of higher education in the most recent Online Learning Across the Atlantic podcast (now featuring episode titles even folks not named Phil can understand!). Yesterday, Phil followed up with a post explaining why using earnings data to evaluate and hold higher education accountable has some real risks. On the podcast, Neil Mosley and I argued that earnings data could be used in additional, useful ways to help institutions improve their operations.

I want to build on that idea because I think there’s a real opportunity to move beyond simply collecting earnings data to satisfy regulatory requirements (both those that exist now and those that may be forthcoming) and to actually improve student outcomes.

Since colleges and universities are likely going to have to collect the data anyway, it would be a shame if it wasn’t put to good use. But there’s a real risk that the data will be collected but not acted on, or worse, that any action taken will be predictable and won’t meaningfully improve student outcomes, at least not anytime soon.

Universities should use the data not just to meet reporting requirements or to pinpoint what isn’t working, but to actively improve the student experience.

Using the Data

My concerns about the usage of earnings data were sparked in part by a recent post from the HEA Group. While the post is largely promotional in advertising HEA Group’s services, it’s also instructive in what data it assumes institutions want, and what they hope (or should hope) to do with the data.

But first, let’s get one thing out of the way: I don’t think earnings data will help students choose majors or careers. While students do seem to be making more instrumental choices, leaning towards business and STEM fields because they believe these lead to better-paying jobs, I just don’t see them poring over program-level earnings data in government websites. Their decisions are made at a higher level of abstraction.

Back to the HEA Group’s suggestions for how colleges could use earnings data, along with other data, to improve the potential for graduates to earn higher wages. They identify three main areas of focus, most of which will already be familiar to higher education institutions and are likely already being collected.

These types of data go beyond what is part of the College Premium regulations, but that is the point.

The biggest danger, however, is one we’ve seen over and over in student success efforts: gaining insight from data is not the same thing as action. Institutions often collect data and then stop there, acting as if gathering the data itself fixes the problem. or they become so focused on the data that they forget to move on to taking actual steps.

This is a huge barrier to progress, and it’s so pervasive that I’ve been trying to come up with a catchy phrase to describe it. (the data delusion, the data dead-end? Suggestions welcome!) The point is: data itself won’t solve the problem, it’s what you do with it that matters.

Moving Beyond Data

Data about program-level earnings and student outcomes are valuable. Even if it wasn’t about to become a federal mandate, universities should absolutely be collecting such data. But that’s just the first step. After that, institutions should dig deeper. They should be doing additional often qualitative research to find out why graduates from certain departments aren’t doing well in the workforce. Are the issues:

• Simply a function of the major itself (e.g., fine arts majors typically earn less)?

• That the department isn’t well-regarded or doesn’t prepare students well?

• Or is it because students aren’t getting the experiences (like internships) or coaching they need to succeed?

Contra HEA, additional research is needed to understand where the problem is and why. Earnings and labor market data alone cannot answer these questions.

If the problem is one of the first two issues above, it’s a tough challenge. As Brian Rosenberg’s recent book highlights, even if there’s a will to change (and usually there isn’t), the realities of faculty governance and curriculum change can make acting quickly nearly impossible. By the time changes are made, the labor market might have moved on and the data might be outdated.

The same holds for aligning skills with jobs. The data can show a mismatch, but by the time universities act, the job market may have shifted again.

Ignoring the Obvious

The kinds of data listed in the HEA Group post, earnings by program, skills alignment, labor market demand etc. are the kinds of things universities have long focused on. But what’s often missing is connecting the data to the student experience outside of the classroom. For example, how well do universities support students in finding internships or navigating their entry into the job market?

Surprisingly, that’s often ignored, even though it’s a much easier fix than revamping a curriculum. Business and engineering schools usually have career offices to help students with internships and job prep, but students in other colleges are often left on their own. Some universities are starting to see the problem. The University of Utah, for example, expanded its business school’s career services to the entire university based on earnings data (disclosure, my spouse works at the University of Utah but is not affiliated with Career Services).

That’s the kind of operational change, using earnings data to support career services, that can make a real difference in the short- to medium-term. More universities should be making these kinds of connections, but I fear most will not.

Parting Thoughts

Collecting and reporting earnings data has become a requirement for colleges and universities. They shouldn’t waste the opportunity to use the data wisely. If they don’t move beyond simply collecting it, they’ll miss a chance to truly improve the student experience and outcomes.

The main On EdTech newsletter is free to share in part or in whole. All we ask is attribution.

Thanks for being a subscriber.