Interesting Reads This Week

Was this forwarded to you by a friend? Sign up, and get your own copy of the news that matters sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.

Apparently, it's survey season right now. So, which ones did I read, and what did I learn?

Before diving in, I want to say a few words about surveys. I read and write about them frequently, and I’m often quite critical. You can partly blame that on my graduate school training. In a social science PhD program, you get very good at identifying flaws in others’ work. That said, except in rare cases, my criticism is never meant to be personal or dismissive of the research itself.

I try to avoid covering studies that I think are fundamentally flawed or driven by a hidden agenda. When I do offer critiques, as I do in a few cases below, it’s because I believe the work is worthwhile and deserves to be taken seriously. My goal is to help readers engage with the research more deeply and understand it in context. Critique, in this sense, is about learning. We all make mistakes, and the only way to get better is to learn from them.

This point was underscored for me this week while reading Black Box Thinking by Matthew Syed. He explores the dangers of disciplinary cultures where open feedback is discouraged. In EdTech, we talk a lot about learning, and a critical part of that is being willing to put your work out there, let others poke at it, and then improve it. That’s how we make progress.

Interest and investment

Inside Higher Ed released its latest survey of campus Chief Technology Officers, and it offers a number of interesting insights. While the sample size is quite low and skewed toward smaller institutions, the findings still provide some valuable food for thought.

There’s a lot going on in this chart on priorities for digital transformation. For starters, it’s not entirely clear what “digital transformation” means in this context. The low priority assigned to digital transformation in libraries is especially striking (earning a scornful harrumph from the librarian in my household). I suspect many CTOs don’t fully grasp the extent of the digital work libraries are already doing, in part because so much of it happens behind the scenes and is managed in-house.

What stood out to me even more were the findings on data / student success. CTOs rated data / student success as the highest priority area, with 67% viewing it as either a high priority or essential. In those two categories, it ranked highest among all the topics listed.

This stands in sharp contrast to how CTOs assess the effectiveness of their investments in this area so far.

Anything related to data and student success ranks low on the list when it comes to perceived effectiveness. Only 39% of CTOs rate their investments in data analysis and learning/managerial analytics as very or extremely effective. An even lower 33% say the same about their investments in student success initiatives.

That latter figure rings especially true to me. During my seven-plus years at Gartner, speaking with hundreds of clients each year, I made a habit of asking whether their student success technologies were actually moving the needle. I can count on the fingers of one hand the number of technology leaders who confidently said yes.