Top Hat — software to be avoided

As a PhD student having taught the Economics Introductory Statistics at Queen’s a few times, I get the occasional solicitation by publishers, sometimes sending me free textbooks in the hopes that I will assign them to my class.

Far more than all publisher contact combined, however, is the relentless unsolicited attempts to “connect” with me from a company called Top Hat, who has a software service that will solve many, if not all, of the challenges of teaching. Sounds wonderful, right?

Read on for why I think Top Hat in a terrible idea for education, both with objections to the general pedagogical technique, but also with strong objections to how Top Hat conducts its business.

Here’s an e-mail I received two days ago:

Hi Professor Rhinelander,

I hope you’re well, I have been working with at Queens who use our student response platform Top Hat in their classes.

I understand you tend to teach Introduction to Statistics, I would love to learn more about your teaching style to see if Top Hat is something that could benefit you.

The reason why I am reaching out to you personally is I work with a few professors that teach a very similar course and wanted to share some insights on how they are leveraging Top Hat on campus to increase student engagement, boosting participation, quizzing and getting instant feedback.

Would love to connect sometime this week or next to see if this is something that could add any value to your courses or not.

Would Friday or next week work best for you?

It seems pretty unlikely that the “reaching out to you personally” part is actually true. In particular, the e-mail doesn’t have a `User-Agent` header which all personal e-mail clients would have added; it was followed up today with a reply e-mail missing the `References` header (which is how e-mail clients know how to group messages into a single conversation/thread), and had an embedded, invisible image that attempts to track whether I read the e-mail (the image links to a company called YesWare which, surprise-surprise!, is an e-mail marketing company). Another hint that this isn’t personal: the course that I tend to teaching, listed as “Introductory to Statistics”, is the course name as listed in the Queen’s calendar, but not the actual course name used by the department (“Introductory Statistics”). Anyone looking me up to “reach out … personally” would have got the name we use in the department; only by scraping the Queen’s course calendar would the former name have been used.

I also don’t know what “working with at Queens” means. My guess is this e-mail was generated algorithmically and there was supposed to be another word in there (perhaps the name of another Queen’s professor using Top Hat?), but that the substitution was empty. (Given the rest of the indications of their software quality, as I look at in more detail below, it wouldn’t surprise me to learn that their mailing software system is also buggy.) Two years ago I received two nearly identical e-mails from Top Hat: one talking about how Top Hat would be wonderful for my “Statistics lectures”, another identically worded message (except for changing “Statistics” to “Anatomy”) extolling the wonders of using Top Hat for my “Anatomy & Physiology lectures.” Oops.

But okay, lots of companies use automatically generated e-mails to solicit sales. It’s called SPAM, and everyone on the internet is familiar with it. (That this spam, in particular, is aimed at university lecturers doesn’t change the nature of the beast).

Normally I just ignore such messages, but this is the tenth message from Top Hat I’ve received, spread across five different representatives of the company, so I decided to articulate the reasons why I think Top Hat is such a bad idea: hence this blog post.

In no particular order:

Student disengagement

One of Top Hat’s big claims is that it fosters student engagement. The line goes like this, from a testimonial on Top Hat’s main page:

My students already have their phones in their hands, they are probably looking at them in class, so why not turn that into something useful?

To be blunt, this sounds like a comment made by someone who has never actually used a modern smartphone. While we use the phrase “on their cellphones,” that isn’t an activity on its own: they are sending texts, browsing Facebook or Youtube, playing FarmVille, etc. Giving them an additional reason to have a phone in front of them strikes me as a terrible idea. Switching to Top Hat to hit a button, then switching back to Facebook takes all of two seconds—this is just giving students an excuse to pull out their phones without feeling guilty about it.

The very last thing any educator should want is to nudge students towards opening up their phone or tablet: sure, they’ll open them to answer the Top Hat question, but then they’re also going to check their e-mail, Facebook, text messages, etc. before they put the phone down again. Sorry, Top Hat, but this is a horrible idea. While it’s true that some students do this already, the Top Hat solution is going to make the problem worse, not better.

Monitoring attendance

I believe that, as educators, our role is to foster a desire to learn. We do that by being innovative, interesting, receptive in and out of lectures. If our classes aren’t interesting enough to keep students coming, that’s the instructor’s fault. Using tools like Top Hat or clickers to mandate attendance might make us feel better about the number of people coming to class, but at the end of the day it’s a cop-out: a way to make us feel better about our teaching without actually having to improve it.

Aggressive marketing

I find the company’s marketing aggressive to the point of being offensive. The fact that they push it so hard makes me highly hesitant to use it, even if it did fit with my own teaching style. Despite my non-interest (after all, I haven’t requested any additional information on the previous 9 e-mails), they continue to contact me which changes the tactic from merely annoying to offensive.

SPAM

As I mentioned above, the e-mails seem to be a unsolicited marketing campaign, i.e. SPAM. Both Canada and the US have anti-spam laws (“The CAN-SPAM Act” in the US, “CASL” (Canada’s Anti-Spam Legislation) in Canada). The unsolicited message seems to be violating both of them (being Canadian, the latter applies to me). It’s possible that there is some pre-established business relationship between Top Hat and Queen’s that allows companies out of the CAN-SPAM/CASL requirements, but shame on Queen’s if that is true.

Rent extraction from students

Top Hat’s business model is, like most of the education-related business models, based on creating highly inelastic demand. Here’s how that works: a supplier of a good or service actively pursues professors to require the use of that good or service in his or her class at no cost to the professor. This, in turn, creates very strong, highly inelastic demand among students: they really don’t have the option of shopping around for a different textbook or different service because the professor has already established that this textbook and/or this service is the one used for the course. Highly inelastic demand (which comes from a combination of being both a requirement and having no substitutes) is a supplier’s wet dream: the optimal pricing is very high not because of the cost to run the service, but because the supplier is able to extract large rents from the purchasers.

We see this in effect with Top Hat, which starts off their Pricing page with this highly dubious claim:

IT CUTS COSTS FOR STUDENTS

Uh huh. Let’s think about that.

If I required this in a course of 100 students, Top Hat is essentially charging $2800 for the use in that course. (They would probably dispute that figure because they apparently live in a dream world where all students use Top Hat for every course, but only have to buy the subscription once). Is it worth $2800 to me, as a professor, to use Top Hat in the course? There’s a reason the payment side of the software is targeted at students, but kept free for professors: if we had to internalize the cost, we wouldn’t.

As an educator, I care about the welfare of my students: how could I possibly justify them needing to spend an extra $2800 to attend my course? Does anyone think they are delivering a $2800 value? I’m highly skeptical that Top Hat adds anything even remotely close to that value to a course; for me to use it anyway would be ignoring the welfare of my students, and I’m not willing to do that.

As for that all-caps claim that “IT CUTS COSTS FOR STUDENTS,” the reasoning seems to be the dubious logic that, because they charge a flat rate for all courses, the per-course cost of Top Hat declines with every course in which it is used. Student costs still go up, of course: but it cuts costs compared to what Top Hat *could* have charged if they charged per course, so I guess students should be grateful or something. The costs for prepaying for multiple semesters (or even a full 4-year program) are also lower, but students are unlikely to actually know whether future classes will require it (and, in many cases, they won’t since Top Hat is best used in larger classes, which tend to be large introductory classes). I’m sure Top Hat knows this as well, which is why the four-year, 8-semester cost is not even three times the one-semester cost.

Edit: I originally wrote in this section:

I wouldn’t mind so much if universities could purchase a blanket, campus licence for Top Hat use; but I’m certain that would reduce Top Hat’s profits since a university has much more negotiating power than individual students in a course mandating its use who have essentially no negotiating power.

That was wrong in two ways. First, Top Hat’s negotiation threat point is going to be the profit they would earn without such a licence–i.e. with individual professors assigning the tool and students paying for it. In other words, from a campus-wide position, a university is not going to be able to negotiate a lower overall cost than Top Hat would get without such an agreement. On a per-student basis, the cost is going to look much lower, but that doesn’t automatically make it a good deal. (Especially because under such a licence, a substantial number of students are being forced to pay for it even if none of their classes use it).

The second mistake is that Top Hat apparently does offer universities site licences, though they don’t seem particularly common and any details related to costs are apparently secret. I mentioned above, it’s reasonable to expect the total cost to be roughly comparable to what Top Hat expects to earn via individual subscriptions from the university without such a licence.

An instructor at a university with a site licence obviously doesn’t need to consider the extra monetary cost he or she would impose on students. While I would disagree with the value to the university as a whole of such a site licence, I would find it less objectionable as an individual instructor to use the software in such a case. The decision would come down to the rest of the points I bring up in this post.

Poor software experience

As a major proponent of open source software, I’m personally very reluctant to rely on closed source software. Students and educators should be free to examine the source code of the software they use; this allows the possibility (even if unexercised) of seeing how the software works, of verifying that it is secure, and of being free to modify the software as they see fit. Another issue (related to its non-openness) is that the Top Hat presentation software requires Windows or macOS. I don’t use either.

A deeper issue that I think anyone considering Top Hat should consider the magnitude of scathing reviews of the Android app—i.e. the part that students have to use. As of this writing, it has 1703 review, with 598 being 1-star review (the lowest possible score). Now some of those may be from students angry that they have to pay for and use the software, so I delved into some of the reviews. Many are simply that the software is buggy or crashes, without much additional detail—bad enough—but the following recent review I found particular illuminating:

I despise things like this. They squeeze even more money out of already poor college students, when it’s the instructors they should be charging, as they’re the ones the require it in the first place.

I agree completely. But that’s how the education racket works.

That same review continues:

That aside, it’s extremely buggy as well. Seriously? No commas allowed in multiple selection questions? You couldn’t have picked a better separator character? Use something uncommon, like a tilde, or a multicharacter separator. This is basic stuff that any first semester programming student knows, and you’re charging for this junk.

He or she is right: this reeks of an amateur coding mistake, the sort of thing that leads to SQL injections and unencrypted password disclosures. Software that has this sort of limitation should be extremely suspect: it’s a pretty good sign of poor software quality.

Finally this same review mentions that:

… the math rendering especially needs to be much better on this mobile version. As it is, it renders every math tag on a new line which wastes a huge amount of space. Rendering it inline would be so much better.

Yuck. That alone makes it useless for an economics course. It’s also pretty inexcusable: this isn’t 2005, there are wonderful, free browser-based math rendering libraries out there like MathJax, which can render the following in your browser, whether on a computer or a phone (this is *not* a pre-rendered image):

\(\widetilde{W} = \left\{
\begin{array}{ll}
w \left(\frac{1 – \left(\frac{1}{1+r}\right)^{T_r – T_g}}{1 – \left(\frac{1}{1+r}\right)}\right) & \mbox{if } r \neq 0 \\
w \left(T_r – T_g\right) & \mbox{if } r = 0
\end{array}
\right.\)

Here are some other reviews that should worry anyone considering Top Hat:

“… The stupid app crashed while I was taking a quiz, it just wouldn’t let me submit the answers and then it stopped working.”

“I have missed only 2 class periods but my attendance is less than 80% because half the time this app doesn’t record it.”

“It won’t keep the answers I choose. Every five seconds when it syncs up it changes my answers and when I submit it changes my answers. I have an Android phone and it’s causing nothing but a headache and costing me a decent grade.”

So, judging from these reviews, I shouldn’t even use Top Hat as a means for verifying participation or for conducting quizzes because the poor software quality means I might be unfairly penalizing students for bugs in the software which don’t appear, based on the number of reviews, to be isolated incidents.

Software security

The security of software is something of a concern to me, as I mentioned above. Given the multitude of reviews indicating buggy performance, I have little reason to have confidence in the rest of the software, in particular that the software is secure both against internal and external tampering. Top Hat can claim otherwise until blue in the face, but software trust requires either a good, established reputation; or open source code so that, if so inclined, the security can be externally reviewed. Top Hat appears to have neither: it has a reputation for buggy software and is closed source. Sorry, but I can’t trust that.

Privacy concerns

In interviews, the Top Hat CEO has talked about how wonderful it is to have the massive amount of data Top Hat collects. For example:

“It’s certainly very exciting the amount of data we have access to. We know [if] students are showing up to class, do they understand what’s going on in class, are they doing their homework, when are they doing their homework?” said Silagadze. “We have complete information, which is highly predictive data that’s very useful for identifying students that might be at risk.”

Imagine you’re a student in a class requiring Top Hat for your grade, and read that again, carefully. This is basically an admission that Top Hat retains and analyzes student data to the point that they can individually identify students’ private study habits. This should raise massive privacy and ethical concerns for any professor. That Top Hat’s CEO somehow thinks that such a huge invasion of privacy by a third party (i.e. neither the teacher nor the student) is “very exciting” means that not only am I personally opposed to the use of the product for the reasons I’ve described above, but that I will, in the future, actively try to discourage others from using it as well.

I doubt that there’s anything technically illegal here: undoubtedly there’s an incredibly long EULA that students have to agree which allow’s Top Hat to use their personal information to be used however Top Hat wants. The issue, for me, is that by using Top Hat, I would be essentially forcing students to choose between taking my course (or major, if the course is required) and surrendering their right to privacy. I cannot in good conscience agree to subject anyone to that sort of decision.

Top Hat Terms of Service

(Edit: I originally didn’t delve into this, but decided to take a look.)

Top Hat’s Terms of Service have a few objectionable parts:

We may change these Terms of Service, the Acceptable Use Policy or the Privacy Policy from time to time, without prior notice. All changes will take effect within 30 days of being posted on our home page or otherwise notified to users. Your continued use of the THM Services will signify your acceptance of such changes.

Holy shit. That “or otherwise” means they are allowing themselves to change the terms of service without making any effort to notify users beyond a post on their website somewhere. Not even an e-mail, an in-app notification, but just a post on their website. And remember, students don’t actually have any real choice to actually decline this agreement: they have to agree in order to use the software which is a required part of their course.

2.1 Registered Users. … You further agree that you are responsible for the conduct of any party that uses Your Account, whether or not authorized by You, and for any breach of the security of the THM Services related to the use of your user name and/or password.

This is highly problematic. In other words, if someone hacks a student’s account, that student has agreed to be liable for anything that the hacker does with their account, even if the account was hacked through no fault of the student.

10.1 Linking to our Web site. You may link to our Web site’s home page, provided you do so in a way that is fair and legal and does not damage our reputation

So apparently, if I was a Top Hat user, I wouldn’t be allowed to link to Top Hat’s web site in this blog post because this post might be considered to damage their reputation. Should I trust an organization that attempts to restrict open discussion of its products like this? Luckily I’m not a Top Hat user subject to this agreement, and so here are some links to Top Hat’s website: Top Hat sucks. Professors shouldn’t use Top Hat. Universities shouldn’t use Top Hat. Yes, I’m being a bit petty, but attempting to silence criticism through a user agreement is shameful and disgusting.

The Privacy policy is here. As far as private policies go, it’s fairly weak in respecting user privacy: they grant themselves the right to do pretty much anything with personal data (though at least they agree not to sell or rent it to third parties). They don’t give students any way to opt-out of this collection, though, except by opting out of the service entirely:

It is always your choice whether or not you provide us with your personal information; however, a decision to withhold personal information may restrict or prevent us from providing you with a particular product or service including access to any subscriber features of the THM Services.

Some choice: “We get to use your personally-identifying data however we want or else you can drop out of university.”

Top Hat didn’t have to go with these legal terms; they could have built a service that doesn’t blame users for being hacked, that encourages rather than suppresses discussion, and that respects user privacy. Top Hat didn’t because they know two things: most users don’t read terms of service agreements; and even if they do, they don’t have any real option to actually decline. Because instructors make the decision; we should be aware of what we are forcing our students to agree to (and in this case, we should decline on behalf of our students).

Summary

In summary, Top Hat is a company peddling a software service to professors using high pressure sales tactics in the hopes that professors will—at no cost to the professor—require its use, thus generating considerable profits for Top Hat due to the disconnection between buyer (the professor) and payer (the student). While some might find this service valuable in teaching (personally I do not), when you add to this the ethical concerns and poor user experience quoted by numerous reviewers, and dubious terms of service requirements, we’re left with a company whose business strategy consists more of exploiting students than of providing a valuable service.

Professors and universities have no business encouraging this exploitation of our students and should actively reject the use of Top Hat.

In the words of another Android app reviewer:

Fix yo sht

7 thoughts on “Top Hat — software to be avoided”

  1. A) you’re a Grad student and half the stuff your saying makes it clear you don’t know what your talking about. And before you come back talking about your education, remember that professors (let alone grad students) are people who never left school and actually proved themselves in business.
    B) “I wouldn’t mind so much if universities could purchase a blanket, campus licence for Top Hat use; but I’m certain that would reduce Top Hat’s profits since a university has much more negotiating power than individual students in a course mandating its use who have essentially no negotiating power.”

    – clearly, they do offer that to Universities, just like every other company in education/every other company in the world when they have the ability.

    – you claim to be an econ guy but seem very inexperienced in how business works. You talk about is it worth 2800 to you, as if charging 200 for a textbook over 100 students is any better from an economical ethics standpoint.

    Get over yourself, you moron

    1. … remember that professors (let alone grad students) are people who never left school and actually proved themselves in business

      It’s a peculiar belief that making money is the same as proving one’s self. Just for the record, I did leave school for several years, “proved” myself, apparently, by making money, and then returned as I didn’t like the environment where taking advantage of people for a buck proves one’s prowess at… life or something.

      clearly, they do offer that to Universities, just like every other company in education/every other company in the world when they have the ability.

      They apparently do, but it is far from “clear.” (But I’ve updated the post with the information).

      you claim to be an econ guy but seem very inexperienced in how business works

      Err, Economics ≠ business. But that aside, I pretty clearly identify why Top Hat’s strategy makes money in the post; I just called out the actual value of the service and think that professors should strongly weigh that against the supposed benefits before using it.

      You talk about is it worth 2800 to you, as if charging 200 for a textbook over 100 students is any better from an economical ethics standpoint.

      Textbook publishers use exactly the same approach (highly inelastic demand for assigned texts) to charge students very high prices for textbooks. Though in terms of what students get out of it, I actually find $200 for a well-written textbook to be a considerably better value than $28 for a 4-month subscription to the Top Hat service.

      Get over yourself, you moron

      Good one.

      1. It’s pretty clear Top Hat does offer university licenses with honestly the most basic of googling. They even have a webpage set up reaching out to faculty and admins directly. Right from the horses mouth: http://bit.ly/2tEwsM6

        Honestly it was like link #4 when I typed “top hat school license” in to Google, man.

        1. Thank you for your comment. That site doesn’t specifically mention anything about site licences. The closest is a mention of using it “school-wide” which I suppose you could infer means via a site licence. I looked into this a little deeper, and there are some other indications that site licensing is available, however (the Terms of Service mentions it). The University of Calgary appears to be using a site licence, though given the amount of effort on the web site devoted to pricing and shipping of pre-pair licence cards for sale in campus bookstores, and the lack of results for search results for universities with site licences (as opposed to many with instructions on buying a licence), it seems that this is not the main revenue source for Top Hat.

          I have, however, updated the blog post to reflect the existence of such licences.

  2. I’ve been getting a bunch of emails from Top Hat for a course I’ll teach in the fall. Having never heard of the software, I decided to do some research and came across your thoughtful and helpful blog. Thank you for your work.

Leave a Reply

Your email address will not be published. Required fields are marked *