Well, I am not sold. First, a disclosure of potential conflict of interest - I am a professor. Of Chemistry. I have never been rated on the site. Several of my friends and colleagues have. And in the process of watching this happen, I have noticed several things.
First and foremost, remember that the ratings are being done by students. The ratings are voluntary. And they have to remember that the site exists, and be motivated enough to go find it, and type something in. And therein lies the rub. All of the ratings I have read are either incredibly complimentary, or incredibly negative. There is very little in between. This should come as no shock to anyone. Students are no different than any other information consumer. People don't write letters to the editor unless they are ticked off. And students don't write comments on RMP unless they were either thrilled with the class or hated it. And the professor along with it. This leads to some very skewed evaluations. As a veteran of many student evaluation cycles, I can tell you that every evaluation I have ever had has had a very broad distribution, not the all-or-nothing comments you find on sites like RMP.
Second, the sample of students commenting is VERY small. I have a friend who has been evaluated frequently on RMP. She teaches at a small university, but even there, two students out of the 150 or so that she teaches every year MIGHT comment on RMP. This is hardly representative of the opinion of the class.
And the entire question of whether students, in the throes of a challenging (we hope) class, are accurate evaluators of teaching performance is open for debate. Some research has suggested that only 16 to 25% of the distribution of student learning is correlated to student evaluations. A much higher positive correlation exists between students grades and expected grades and the ratings that they give. Students who are doing well in the course love the professor, and those who aren't, don't. What do you think the evaluations of IRS auditors would be if they were done by the people being audited?
So this gives rise to a real question - how much of a service do sites like RMP really provide? If only one or two percent of my enrolled students are commenting, and then only if they are angry with me, or their grades, or in love with my teaching, or their grades, is there anything meaningful at all about the rating? Other than that chili pepper by my name... I want that!
Don’t stop reading this just because I used the term Web 2.0. I know it’s tired. I know it’s overused. It’s tired because everything is user-generated now, everything is social, and nobody’s actually figured out how to make the semantic web work well enough to call themselves Web 3.0. However, it’s the very ubiquity of Web 2.0 that makes what MTV has done with the site ratemyprofessors.com all the more impressive and, ultimately, quite important to its brand.
MTV is a media powerhouse owned by an even bigger media powerhouse (Viacom). While many of us will remember when MTV played music videos, MTV Networks now includes a host of stations and brands, all of which contribute to serious reach, particularly among the under-25 demographics (given that Nickelodeon is an MTV network, when I say under 25, I mean all the way under 25). One particular network is mtvU, which, according to the company, is
Broadcast to more than 750 college campuses and via top cable distributors in 700 college communities nationwide, mtvU reaches nearly 9 million U.S. college students – making it the largest, most comprehensive television network just for college students. Twenty-four hours a day, seven days a week, mtvU can be seen in the dining areas, fitness centers, student lounges and dorm rooms of campuses throughout the U.S., as well as on [most] cable systems…
mtvU also carries extensive Web properties and leans heavily on social sites like Foursquare and its own internal social tools to engage college students. One of the more interesting properties is ratemyprofessors.com, a student-driven site in which users can rate, comment on, and otherwise discuss the instructors at their universities. Unlike the secondary school equivalent, ratemyteacher.com, Rate My Professors is actually trying to add value for students as consumers of university education, rather than merely being a spot to complain about teachers.
I had a chance to talk today with both Carlo DiMarco, Vice President of University Relations for mtvU, and Isaac Hazard, Director, Strategic Consulting at Mzinga about Rate My Professors and how the two companies are working together to ensure that a student-authored site carries high-quality, high-value content that can also be leveraged to build the mtvU and MTV brands.
Rate My Professors is a relatively high-volume site, with 11 million opinions about 1 million instructors at 6000 colleges. Mr. DiMarco considers both the scale and the complete lack of mtvU editorial control to be real strengths of the site, but also noted that maintaining a quality, safe, useful environment required an “independent eye,” in the form of contracted moderation services from Mzinga. With 94k ratings last month, 18% growth year to year, and 3.2 million college kids using the site every month, this is clearly an opportunity to make money, drive traffic, build a brand, and possibly even help people.
So beginning September 1st, Mzinga began providing 18 hours a day of monitoring and moderation for the site. Mzinga uses a combination of user flagging, automated spam and language detection, and human intervention to keep the content on Rate My Professors as focused on instruction as possible, while eliminating personal slurs and attacks. Although Mzinga’s Hazard notes that 18 hours a day is an atypically high number, the volume of comments required a substantial commitment. Interestingly, 7 out of 10 comments on the site are actually positive, meaning that students are turning to the site not just to complain, but to share really positive experiences with their peers in a very authentic way.
mtvU can then take these ratings and data and turn them back into additional content for the network. Whether they are allowing professors with poor ratings to defend themselves or highlighting schools with particularly good ratings for their instructors, mtvU is using Web 2.0 content to create additional popular and compelling content both online and on the network.
This sort of interplay, focused on high-quality content flowing across multiple media suggests that Web 2.0 is far from tired out or passe. Rather, it’s incumbent upon companies to take it to the next level, going beyond simple social content and exploring what it takes to reach and engage an audience.
