I like frameworks. I like them because they help align conversations. When folks talk about a topic as amorphous as analytics, a framework helps to get everyone on the same page and have them using the same language.
When we talk about analytics in Higher Education, the conversation usually goes something like this:
“So, you want to use analytics at your institution. What do you hope to achieve?”
“Well, I want to use data and analytics to help my students succeed.”
“Got it. What do you mean by ‘student success’?”
“Well…ummm…I mean that they should…ummm. I’m not sure.”
So, here’s the framework I use to address this conversation. It breaks student success down into three orthogonal dimensions:
Progression: This is milestone-based success. Will the student pass this class? Will the student graduate in 5 years? Will the student still be here 12 months after starting? These are very easy to measure since they are all date-based milestones. The drawback of this dimension is that it lacks qualitative substance. Just because a student made it to the 12 month milestone, for example, it doesn’t mean it was a positive thing for the student and/or the institution
Engagement: This has to do with the level to which the student was engaged with the academic process. It could be measured by a variety of things including logins to the LMS, hits to content in the classroom, or even customer satisfaction scores. These metrics may or may not be easy to measure. The challenge is that engagement-based metrics vary in their overall correlation to student success. Customer satisfaction might be highly correlated to success, while logins to the LMS may not be as representative.
Learning:I would argue that ‘learning’ is both the best measure of success and also the hardest dimension to measure. The crux of the argument is this: can one quantitatively measure whether or not a student learned what they were supposed to learn. This is a whole other blog posting on its own…let’s save that for another time.
One of the key takeaways here is that these three dimensions are orthogonal. A student might progress through the program, but not learn what they were supposed to learn. They might be engaged in the process, but end up not making it to the next milestone due to external reasons.
When an institution looks at using analytics to improve student success, think about these three dimensions as a way to organize what it is you really mean by success. This approach will help give clarity as you approach the analytics project.