In the predictive analytics space, there is always talk about secret sauce. The roots of it make sense to me. Think about the financial industry…if you built a model that could predict future trends in stock prices, you’d probably want to keep that a secret. In the education space, though, the logic starts to break down.
First of all, education is a highly collaborative space and it represents a social good. Keeping a valuable secret that might help students succeed is antithetical to the nature of education. Second, education is a complex ecosystem of people, processes, policies, content, etc. I would have strong doubts about anyone who claimed to have a formula that worked for a wide variety of institutions. Third, I think it creates an element of distrust. Having recently switched from the institution side to the vendor side, I’m keenly aware of the delicate balance in the partnership between the two. If one side claims to have a valuable secret and asks the other side to pay for it, it’s not going to help bridge any gaps.
As an entrepreneur, I also think it’s a poor business model. If your whole business is predicated on one secret formula, then you’re putting a lot of eggs in one basket. Between reverse engineering and up-and-coming researchers who are way smarter than most of us, I don’t think the formula would have a long shelf life.
All of this talk about secret sauce reminds me of a great yet subtle scene from Fast Times at Ridgemont High. Brad (Judge Reinhold) is training a new guy to prepare burgers at the All American Burger fast food restaurant. The trainee used to work at Bronco Burger, but he just came over to All American Burger:
Brad: Okay, here’s your preparation stuff. You got your sliced tomatoes, shredded lettuce, secret sauce.
Trainee: What’s the secret sauce?
Brad: Thousand Island dressing. What’s the secret sauce at Bronco Burger?
Trainee: Ketchup and mayonnaise.
Brad: Gotcha.
I find it so relevant to this topic because aside from not being secret, the secret sauce isn’t anything that complex. I’m sure Ina Garten has a phenomenally intricate recipe for her secret hamburger sauce, but that’s the exception rather than the rule. Most that I’ve seen tend to be permutations of a smaller set of variables. Here are some examples (these models have different dependent outcomes, so they are not apples-to-apples comparisons):
- American Public University
- What factors influence online student retention?
- 1: No transfer credits
- 2: Number of courses taken
- 3: Student failed a class
- What factors influence online student retention?
- Course Signals (Purdue)
- Course success (risk of failing current course)
- 1: Performance (points earned in course)
- 2: Effort (relative LMS activity)
- 3: Prior academic history (high school GPA, test scores)
- 4: Student demographics
- Course success (risk of failing current course)
- Rio Salado’s PACE model
- Course success
- 1: LMS logins
- 2: Engagement with content
- 3: Assignment scores
- Course success
- University of Phoenix
- Will the student attend class next week?
- 1: Assignment scores
- 2: Content access
- 3: Past absences
- 4: Number of days engaging with the LMS in a week
- Full disclosure: I was one of the principal developers of the University of Phoenix model
- Will the student attend class next week?
You can see that with the last three models where the outcome is some flavor of course success, the variables are very similar. There is no secret sauce…there is no magic bullet. Rarely is there a “tell” where you can look at a simplistic parameter and divine complex findings from it. The dating site OK Cupid has a great blog post where they correlate innocuous questions to match characteristics. For example, you can ask a date “Do you prefer the people in your life to be simple or complex?” If they answer “complex” it’s 2:1 odds that their politics are liberal-leaning (and vice-versa for conservative). I don’t believe this kind of correlative analysis works for learners. There is too much detail and too many environmental variables (the student, the instructor, personal situations, course curricula, financial influences…).
I’m a firm believer that the value is not in the secret sauce…it’s in the ability to execute. Most colleagues I’ve seen in my last 12 years in higher ed technology have been very knowledgeable, especially when it comes to their own institution. One would be hard-pressed to come up with insightful findings that they don’t already know about. I believe that the gap is in the ability to gather the data, do the analysis, and feed the findings to the right parties. The gap is not about knowing which variables to look at…or which jar of sauce to put in your shopping cart.