You’ve heard of the saying “it’s free like kittens, not free like beer”? I love it…it’s a great way of telling someone that “free” may not be the kind of free they are thinking about. In our role with higher ed data and analytics, we come across numerous ways of getting at data and analyzing the results to help improve student success. One core concept that we keep coming back to is that nothing is free (free like beer, that is). There are definitely ways of doing things more efficiently, but in the end, you have to pay the piper in some way (free like kittens).
So, where does the kitten/beer schism come into play? Turns out, there are many misconceptions about where the work is and where the pain points are when it comes to acquiring and analyzing data. Since we have worked with data across different institutions and collaborated with colleagues at conferences about the process, here are three beer myths and their corresponding kitten realities that we have observed in the higher ed academic analytics space:
Beer Myth 1: Once you’ve written a “plugin” to get data from a learning application, you can reuse that plugin freely again and again
Kitten Reality 1: While the reusing the plugin might get you the data, there’s likely to be a heap of quality review you have to do for each integration
Blue Canary works with institutions to aggregate SIS data, LMS data, and other valuable application information so that we can help identify at-risk students based on their activity. We end up interfacing with multiple systems, and there are efficiencies to be gained by reusing the integration code. While this reuse does save some time, the task of data quality review is still a necessity and it’s a non-trivial task. Institutions have different rules about terms, modalities, definitions of attendance, statuses, dates, etc. etc. There are also anomalies like “if the course date is prior to 9/1/2009 then this field is registration date, otherwise it’s course start date”. There’s a reason we call ourselves data janitors.
Beer Myth 2: Analytics standards will remove the cost of having to aggregate learner data
Kitten Reality 2: You need to have someplace to put all of the data
I’m a fan of standards. The IMS LTI standard is free like beer. If D2L and ALEKS are both LTI compliant, then you can add an ALEKS link in a D2L course in 3 minutes. End of story. Analytics standards like IMS Caliper and the Tin Can API are free like kittens. If an application is instrumented with one of these API’s, they’ll spit out data, but you have to do the work of storing the data somewhere (like in an LRS). The point here is to realize that standards solve one part of the analytics equation. There’s still more work to do. Which leads us to…
Beer Myth 3: Once I’ve got the data (through plugins or standards), the hard work is done
Kitten Reality 3: While data acquisition and cleansing does take up the bulk of the time, you then have to make sense out of the data in order to derive value
Sensemaking…converting raw chunks of data into something more meaningful that allows an institution to take action and improve student success. It’s REALLY hard. This requires a combination of understanding the institution’s teaching/learning model, having access to good analysts, and possessing experience in the field. It also takes a fair amount of effort to complete. It’s an iterative process, and the best you can do is to be very efficient operationally so that your iterations take hours/days, not weeks. Getting the data is one thing…understanding what to do about it is another thing.
There you have it. Our three beer/kitten myths in the higher ed data and analytics space. Do you have any other ones? Tweet us @BlueCanaryData with suggestions…we know you’ve got them!