Illustrating the Absurdity
HEA Group "report" illustrates the fundamental flaw in Earnings Premium metrics
Phil Hill
August 07, 2025
Was this forwarded to you by a friend? Sign up, and get your own copy of the news that matters sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.
In “The Fundamental Flaw of Earnings Premium” I described a massive problem with the Earnings Premium metric that is becoming all the rage in institutional accountability research and regulations. The idea is to see if a college credential leads to graduates earning more than they would have if they had not entered and completed a program.
The problem is that the median earnings are calculated for academic programs in a specific location. The comparison data to determine if the program has value is aggregated across an entire state.
This approach is now a law, thanks to the One Big Beautiful Bill Act (OBBBA) and the Senate Do No Harm institutional accountability embedded in that act. I admit to some frustration that more people are not getting this flaw.
Well, thanks to the HEA Group, led by Michael Itzkowitz, I have a simpler way to show this point. I mention Itzkowitz by name because he is very relevant. He headed up the initiative to create the College Scorecard in the Obama Administration (for which I honestly thank him), but he was also active through Third Way in creating the Gainful Employment & Financial Value Transparency regulations and the Earnings Premium metric that Senate Republican staff copied, slightly improved, and embedded into law.
His company’s report released this week is titled “Golden Returns: A Regional Look at the ROI of California’s Community and Career Colleges.” In that report, Itzkowitz and his team make the same fundamental flaw which could have easily been avoided.
From the press release from the report [emphasis added]:
A first-of-its kind regional analysis of California’s community and certificate-granting colleges shows that while some institutions offer a rapid economic return to students, far too many provide little to no ROI, especially in underserved regions of the state. [snip]
“The regional approach we took in this analysis really shows stark differences in ROI depending on where students live,” said Michael Itzkowitz, President and Founder of The HEA Group. “In some regions, learners are unknowingly navigating multiple options—some with very strong outcomes alongside those that are costly and of low economic value. And since learners typically attend one of these institutions close to home, the findings show that geography can shape access to opportunity in profound ways.”
In that report’s usage of Earnings Premium, the earnings are aggregated at the college level but the comparison is still at the state level. And yet the entire premise of the report is that there are stark differences within the state depending on where students live. Yet everything is compared to a statewide metric.
Hot take - location affects both college graduate earnings and high school graduate earnings.
It took me 30 minutes to find a California Department of Finance web site to look at high school graduate earnings for age 25+ by California county. This is not exactly the same as used by HEA Group or the OBBBA, as that is for ages 25-34, but this will clearly show the regional variation in the comparison group.
tl;dr - the comparison metric varies across California counties by roughly 50 percent. From a low of $32,303 in Mariposa County to a high of $48,859 in Mono County. The statewide metric is $40,371.
A college in Mariposa County might have students who increased earnings by $8,000 (from $32,303 to $40,303, a 25% increase) but show up in the HEA Group study has having no value, an Earnings Premium of essentially zero.
A college in Mono County might have students who did not increase earnings at all but show up in the HEA Group study as having a large Earnings Premium.
Yes, those are the two extreme cases, but they should illustrate the point.
Now let’s look at a map of this metric (median earnings for HS grad / no college by county).
That certainly looks regional to me. Bay Area and eastern Sierra counties have much higher median earnings for the comparison group. LA area is much lower. So when HEA Group claims they have discovered a big regional difference between Bay Area and LA area colleges, there should be no surprise because the metric is flawed.
From today’s IHE-sponsored webinar, we see HEA Group comparing Bay Area and LA area colleges and claiming the former have colleges producing great results and the latter having much worse results. Without once taking into account that the comparison earnings are different on a regional basis.
This fundamentally flawed analysis and conception of the Earnings Premium is not just a problem in the HEA Group report that got past multiple sponsors and led to plenty of media coverage. It is the same problem with the Earnings Premium that is now law in OBBBA and will cause a lot of harm for colleges and universities in the lower-income regions of their states.
We already have a problem with educational deserts, with poorer regions not having good higher education options. The Earnings Premium metric as written will make this problem worse. Colleges and universities in these regions risk the same problem as show for Mariposa County.
The Department of Education is taking public comments as they work to draft regulations around the law. Now is the time to give feedback and hope that this flaw is addressed.
The main On EdTech newsletter is free to share in part or in whole. All we ask is attribution.
Thanks for being a subscriber.