The Burden of Misunderstanding

How ED's outdated consumer-protection view of online education could lead to bureaucratic burden on every online course in US higher ed

Author

Phil Hill
August 15, 2024

In yesterday’s post on the CHLOE9 report, we mistakenly referred to the Online Learning Consortium (OLC) as co-authors when we meant Quality Matters. The online post was updated yesterday, and we apologize for this oversight. And thanks to our readers for quickly letting us know about the error.

Based on our research, including participation in conferences and online discussions, I do not believe that most higher ed people in the US understand just how big of a burden they are about to face if the Department of Education’s (ED’s) proposed rules on online attendance taking get finalized. From the Inside Higher Ed article that I referenced in last week’s post on the Arnold Ventures (AV) funded coalition:

Administrators who oversee online education say a federal plan to require colleges to take attendance in virtual courses would impose significant time demands on faculty members and increase cost burdens on institutions.

What is understood by ED and others is that attendance-taking is a misnomer and is much more than checking logins.

Taking attendance would not be as simple as students logging into the learning management system or stating “here” at the beginning of each class session. Every 14 days, students would be expected to turn in an assignment or interact with a professor or fellow students during lectures and course discussions, although the department has yet to define exactly what mechanism or standard it would require colleges to use to align with the new policy.

What is not understood by ED and its allies is how online education works in most colleges and universities and therefore how big of a burden this proposed rule would represent.

Instructor Experience

Put yourself in the position of an instructor teaching an online course. Maybe all activities are run through the one official LMS used by your institution. In that case, someone will have to configure the LMS to report all activity data per student and to integrate the LMS with campus financial aid reporting systems. But even then, the College of Arts and Sciences has completely different guidance and support for faculty teaching online than does the College of Business. And Central Technology groups might not even have permission to look at each and every course and to determine standards.

In perhaps 10 - 20% of institutions, a college or even a department may use a different LMS, or perhaps not even use an LMS. How then will an institution aggregate the reporting into a activity-date for each and every student?

And what about the usage of third-party courseware (think Pearson MyLabs, McGraw-Hill Connect, Cengage MindTap, etc) or separate tools (think Padlet, Piazza, MS Teams, Harmonize, Google Docs, etc, etc)? Those tools that capture the academic engagement activities that actually constitute attendance might be integrated with the official LMS, at least for single sign-in and grade reporting, but very often the daily activity for each student is not passed back to the LMS. Just the end results of an assignment.

And what about course designs that are less formal in nature, not requiring tightly defined assignments? Think authentic assessment or project-based learning. There are just so many variations if you look at all online courses across all of US higher education.

On a daily basis, some process or someone will have to review all available academic engagement activity types and determine which students are participating.

The Assumptions

Theoretically this whole process could be automated, since almost all systems involved (LMS, third-party EdTech tools, publisher courseware, etc) log data on student activity. But automation makes several assumptions:

  • There are one or maybe two LMSs on campus, and all online courses use those official systems to directly or through integration log all academic activity;

  • There is a finite number of approved third-party apps used in online courses that must be approved by the institution;

  • All apps must be integrated with the LMS, including the daily logging of per-student activity;

  • All EdTech is centrally controlled, or at the very least a) the central unit has full visibility into per-course per-student activity or b) all colleges and departments have to report back to financial aid;

  • The central LMS is integrated with financial aid systems for the purposes of reporting attendance and triggering alerts of non-activity; and

  • All faculty use the LMS and third-party apps in an approved manner.

If any of those assumptions are wrong for a specific institution, there would be two choices in order to comply with the new rules:

  • The institution must change policy and IT architecture to meet all of those assumptions above; OR

  • Faculty members themselves or direct support staff must manually check and mark attendance-through-activity daily - and all faculty must be trained on the new requirements.

Faculty and Staff Burden

I have talked to several schools and read second-hand analysis of many schools, and the burden for that second choice ranges roughly from 5 - 20 minutes per instructor per course section per day. And that is after a new training program for all faculty members teaching online courses on how to comply. Consider a 15,000 student university that has ¼ of students fully online and ¼ of students taking at least one but not all online courses (the national averages). Adjusting for a mix of full-time and part-time students and assuming 25 students per course section, that would lead to roughly 540 online course sections at any one time: (3,750 × 2.5 + 3,750 × 1.1) / 25.

Even at the low end of the burden estimates (5 minutes per course section), this institution would have a daily aggregate burden of 45 hours each and every day, just with faculty time. Only institutions that are highly centralized and well-staffed will be able to fully automate the process, and a majority will require manual interventions.

Financial aid and registrar staff would also be placed into a cat-herding role to determine whether all colleges, departments, and faculty are complying with the new process - not just doing it at all, but also that they are doing it consistently and immediately logging activity.

The Notice of Proposed Rulemaking (NPRM) for the rules makes its own estimate of the reporting burden. For the one-time burden of automating the system [emphasis added]:

The Department estimates that each of the institutions would be required to do an initial review of their distance education system to ensure that attendance is being collected and potentially develop or add attendance taking to the system. The Department expects that this would require an average of 10 hours per institution as a one-time burden. 

And the additional, recurring burden is also estimated [emphasis added].

Due to the highly automated delivery of these types of courses, and the availability of such coursework on a daily basis, the Department estimates half of the institutions offering distance education courses would already be performing this task. Therefore, the Department estimates it would take the remaining fifty percent of institutions offering distance education about 10 minutes on a daily basis to capture attendance information for their records.

To be clear, these are estimates per institution, and these estimates vary wildly. On a daily basis, ED estimates 10 minutes for compliance, but the example used above shows how an average institution might need 45 hours per day.

Institutional Feedback

This disconnect between what real colleges and universities believe it will take to comply with the proposed rules (dozens of hours per day in aggregate, including from faculty) and what ED and AV-funded allies believe (10 minutes per day in aggregate, no faculty impact) is the reason for the comments from David Baime, senior vice president for government relations at the American Association of Community Colleges, in the IHE article.

“Despite what the department has said, it can ultimately be quite complicated and difficult and furthermore costly for institutions to provide documentation that the department appears to be requiring as a result of the attendance requirement for online courses,” he said. [snip]

Baime believes the department “wildly underestimated” the costs and time involved for the process.

Caleb Simmons from University of Arizona Online (the in-house program, not UAGC), described the faculty burden.

“We’d be able to figure it out on the financial side and even the workload side, although it would be a pain point,” said Simmons, who oversees Arizona Online. “But there has to be an acknowledgment of the additional workload that’s placed on faculty—in addition to teaching obligations, being there for students, research. It’s an ever-growing amount of work with no additional compensation.”

Baime and Simmons both said implementing the requirement would be difficult; Baime called it “inconceivable” and Simmons, brainstorming, said Arizona would potentially have to create some tool in its learning management software, since “we could not ask every individual faculty [member] to send us a report.”

There are many other reports coming in from college and universities and the associations representing them that ED is wrong in its assumptions. WCET, UPCEA, Quality Matters, OLC, AACC, and DEAC wrote a joint public letter after the negotiated rulemaking sessions but before the NPRM submission describing what they are hearing from member schools.

In talking to numerous institutional personnel and a representative from the National Association of Student Financial Aid Administrators (NASFAA), the “simplification” appears to be true in two cases: if the institution is fully online or if the institution is already an attendance-taking institution. All others reported numerous problems including:

o Having to capture every instance of academic engagement in the LMS or other software.

o Having to create new connections between the LMS and financial aid systems.

o Having to address this requirement across multiple LMS systems at large institutions.

o Developing and disseminating institutional policies and procedures on how to implement this requirement.

o Developing and offering faculty and staff development on how to implement this requirement.

In brief, this “simplification” is much more work to obtain the same piece of evidence for “last date of attendance” as is currently done. Every instance of academic engagement would need to be archived for every student rather than just finding the test, paper, or discussion post for those few students who withdraw without notice.

During the neg reg sessions, negotiator Scott Dolan from Excelsior College pointed out ED’s faulty assumptions on implementation.

And I also am a little bit concerned that there's an assumption that we have a clear definition of what a distance education course is and that those are consistently implemented across institutions in the same way. We are a distance Ed, fully distance Ed institution, we have mechanisms in place to track not only logins but attendance as is required by financial aid. There are pretty sophisticated systems that we use to do this being fully online. The provision here, as stated, is going to require private nonprofits that have distance Ed of any variety to think about and implement similar kinds of systems. [snip]

But there's an assumption, I think, that's made in the language around learning management systems and how those are being used and monitored and how easily it would be to implement this at more traditional institutions and fully online institutions. So I guess, I also get a little concerned that we talk about the technical capability of monitoring distance education in one venue as being relatively straightforward and easy when it comes to this provision.

I should also point out that across the board, the negotiators and associations and colleges and universities that I’ve talked to support the goals of the rule. Simplification of process and making the determination of last date of attendance more accurate. The problem is in the faulty ED assumptions and poorly-conceived details of the rule.

The Other View

Barmak Nassirian from Veterans for Education Success (VES) and a non-federal negotiator during the negotiated rulemaking sessions this year, remarked in the session transcript his incredulity that we do not already have audit trails for academic engagement daily.

And, yes, you know, capabilities vary from program to program, institution to institution. But if you can't, I mean, it's really I struggle to understand how it's possible to offer a distance education program that doesn't have audit trails to know who is in and what they're doing. If you don't have that capability, you're probably not offering a particularly robust academic coursework. So I think there are reasonable grounds for the Department to do this, and I strongly support what they're doing. There needs to be a little redrafting, it seems to me, around the margins.

There is, I should note, a difference between audit trails that can be checked after the fact and real-time tracking of student activity, and during the session Mr. Nassirian argued for the primacy of audit trails over attendance taking.

So, when the Department re-drafts this as I think it probably will have to given the objections that have been raised, it would be good to make sure that only substantive academic interactions may be used, and that they should all be used consistently and tracked in the system with audit trails that can be validated after the fact if there's a dispute.

Despite the incredulity, I think the problem for both (audit trails and real-time attendance) comes in having substantive academic interactions used consistently across all online courses. Only a minority of institutions (typically fully-online and already attendance-taking) can meet this assumption in reality. The scope of the rules goes across innumerable course and program design types, and we cannot just declare from the federal level to consistently deal with academic interactions.

Reaction to Feedback

How does ED react to this conflicting feedback? Thus far ED staff have ignored or rebuffed the complaints made by college and university staff and from the open letter, simply disagreeing and not changing its assumptions. ED also rejected neg reg arguments that this rule makes no sense for Direct Assessment programs, which was the primary cause of the no-consensus outcome of the sessions.

At the same time, ED and its allies simply know better than the complainers. In reaction to my post last week, Wesley Whistle (who just left ED to join New America) argued strongly about his knowledge and against what I described as ED not listening to schools.

Wesley Whistle let me know that he could explain this to me (or to Jeff Selingo).

I accepted his offer.

Despite my acceptance, Wesley Whistle then went silent, but his New America colleague Eddy Conroy jumped in with similar views.

First of all, I completely agree with Eddy Conroy’s point that there are some burdens that schools should bear, assuming that we have a solid understanding of the scope of the problem and evidence that the burden will address the specific problem. But on the online attendance-taking question, Eddy Conroy declares that this is not a significant burden and implicitly rejects the arguments from colleges and universities.

The Offer

I honestly liked Wesley Whistle’s offer, and I’ll modify it. I realize that ED staff cannot publicly discuss the topic during the public comment period, but I would be happy to interview any of those in the coalition who think that there will be no significant burden from compliance. Show me “how hard it isn’t” to report the data. And I’ll write up what is said separate from any of my own analysis, and publish in this newsletter. If it is preferable, you can submit a guest post that I won’t even edit - I’ll just post by itself. All I ask is that the post focus on the question at hand, specifically why the proposed online attendance-taking rules represent a reasonable burden for institutions and not very hard to accomplish.

Carolyn Fast, Barmak Nassirian, Wesley Whistle, Eddy Conroy, or even Kevin Carey - this is an offer to explore this topic openly, in a manner that would benefit the higher ed community. Contact me directly by the website form, or DM me at X or at LinkedIn.

Time to Comment

There are plenty of other points to be made on this proposed rule:

  • the lack of evidence supporting the treatment of online ed differently than f2f or hybrid;

  • the redefinition of regular and substantive interaction;

  • the impact of this simplification rule actually complicating matters for compliance; and

  • the risk of auto-withdrawal for 14-day inactivity periods, etc.

For now, I wanted to be more precise on what I believe is a misunderstood compliance burden of ED’s proposed rule, and ED’s inability to listen to feedback from colleges and universities and associations representing them. And that while the details of this proposed rule might seem arcane, it will have a major impact across higher ed.

It is very important to note that we are in the middle of the public comment period for these proposed rules, and that ED should hear directly from colleges and universities about the impact of the proposed rules. You can comment here through next Friday (August 23rd).

The main On EdTech newsletter is free to share in part or in whole. All we ask is attribution.

Thanks for being a subscriber.