When it was reported earlier this month that Facebook had manipulated users’ news feeds to see how the changes affected their moods, people were mad. Now, OkCupid CEO Christian Rudder has admitted that his company has also conducted experiments on users without their consent, by matching them up with people they weren’t necessarily compatible with—and people are, unsurprisingly, furious.
Rudder revealed the experiments in a blog post last Monday, with the not-at-all-ominous-sounding title, “We Experiment on Human Beings!”:
We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.
He then proceeded to list a number of experiments the website had performed on its user base in the name of love and science. First, to celebrate the release of their (failed) Crazy Blind Date app, they deleted all user photos for seven hours as part of “Love Is Blind Day.” While they found that usage plummeted during that seven-hour time period, users had “deeper” conversations with their matches and reported having fun on their blind dates, regardless of whether they were rated more “attractive” than their dates.
OK, so OkCupid tried to teach its users the important lesson that looks don’t matter as much as general compatibility, and unlike the Facebook experiments, they informed their users of what they were doing beforehand. (They also proved the opposite in a second experiment where they deleted the text in user profiles, to determine whether users were ranking potential matches based on looks or personality).
That’s not so bad, right? Well, check out Rudder’s description of OkCupid’s third user experiment. To test the power of suggestion in the match process, and to see if their oft-touted matching algorithm actually worked, OkCupid used the algorithm to intentionally mismatch people on the site:
We took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.
They also performed the experiment the other way around: i.e., they told people who were actually compatible with each other, that they weren’t compatible with each other. Rudder says that basically the same thing happened, and that the odds of a single message turning into a conversation significantly decreased.
Rudder’s blog post was obviously intended as a defense of Facebook’s user experiments, while simultaneously being more transparent about OkCupid’s user matching processes—a laudable goal, in itself. But naturally, people weren’t too pleased by the suggestion that one of the world’s most popular dating sites was using their IRL love lives as a springboard for testing the reliability of their algorithm. Here’s what Andrew Leonard over at Salon had to say about the OkCupid and Facebook experiments:
There’s a big different between straightforward A/B testing — presenting two different versions of a site to different groups of users in order to see what works better — and consciously presenting false information or otherwise skewing emotionally laden data. One is completely acceptable tinkering designed to improve usability, while the other is irresponsible behavior that treats human beings like lab rats and their emotions as play toys.
Dylan Matthews at Vox speculated that the third experiment could have real-world detrimental effects on couples who’d been mismatched through OkCupid:
What if an actual couple, whose relationship began when they thought they were a 90 percent match, were then informed they were a 30 percent match? It’s hardly the biggest emotional blow in the world, and if the relationship was really working, it’d make sense to shrug off the new information. But users can be forgiven for finding the prospect of OkCupid dropping bombs like that on relationships still in their honeymoon phase a tad disturbing.
While I highly doubt that two people basking in the warm glow of their newfound love for each other would care that much about how compatible OkCupid’s algorithm thinks they are, the controversy over the blog post raises an interesting point: Is the revelation that OkCupid has been experimenting on its users actually tantamount to “dropping bombs” on relationships? Should we really find it so surprising that the algorithms and compatibility metrics on sites like OkCupid aren’t as reliable as they claim they are, or even that they’re sometimes being manipulated against our better interest?
As someone who has always had a healthy dose of skepticism toward OkCupid’s algorithm, and other sites that claim to match their users based on complex, mathematically-based compatibility metrics, I don’t necessarily think we should be. Being told that you’re 80 percent or 95 percent or even 35 percent compatible with someone on OkCupid isn’t going to affect how you perceive the outcome of your relationship; maybe it’s a helpful little nudge toward your initiating contact with that person, but other than that it doesn’t serve much purpose.
Maybe tech wonks and privacy advocates are surprised and enraged by the revelation that OkCupid “treats human beings like lab rats,” but considering that OkCupid informed its users about its first two experiments (and considering how little real-world significance the manipulated compatibility scores probably had in the third), people who have use OkCupid on a regular basis probably shouldn’t be. Either way, following the uproar over the Facebook experiment, the fact that yet another giant company also experiments on its users shouldn’t come as a surprise to us anymore.
H/T Vox | Photo by Martin Fisch/Flickr (CC BY SA 2.0)
