Everybody is spamming everybody else on Mechanical Turk
Two months ago Panos Ipeirotis, an NYU professor who studies anthropological aspects of online slavery as well as doing nerdy computer science stuff, pointed out in his blog post that 41 percent of mechanical turk HITs are posted by spammers. These requesters are typically those who ask turkers to click on a link, friend somebody on facebook or trash somebody on facebook. These tasks may not look like spam to us as I personally friend and/or trash people on facebook on regular basis so no big deal, but I never get paid for it:)
Anyways as a curious researcher who doesn’t want to accept anything until he has done his due diligence I posted a spam HIT with a generous 1 dollar reward asking people to write a review about interesting places to visit in the “Shiraz City, France”. Shiraz is known for its ridiculously hot girls and great wine but the problem is that it is not city in France, surprisingly it is in Iran.
I exclusively asked for those turkers who have lived in “Shiraz City, France” and made sure to highlight that if the turker has never lived in that city the work is going to be rejected. (For those who would like to bring up ethical issues for this experiment I would like to point out that turkers have screwed me over and over on countless number of occasions so I reserve the right for myself to mess with them occasionally, additionally you can look at this as the Milgram experiment of the crowdsourcing era. Who doesn’t want to be called Philip Zimbardo of the online slavery market? Now that I am thinking about it I actually kind of like it.
Anyways if the market was fair I assume two things would have never happend
1) I would have never been allowed to submit such an obscene work
2) I would have never received any responses
To my surprise both happened.
Nobody really polices the mturk market so I was able to get my HIT through. but the more interesting thing was that I was actually receiving suggestions for visiting a city that didn’t even exist!
I receive three types of responses:
- 1- Random excerpts from our old friend “the internet” about cities in france (mostly Paris)
- 2- Email addresses of the turkers
- 3- Random user names
We all know that Mechanical Turk challenges the whole “Junk-in, Junk-out” dilemma and makes it more like “Always junk-out, regardless of the input process” but I feel that users posted junk for my junk HIT assuming that there is no quality assurance process working behind the task and I would probably just accept them all (which I have done for many jobs before)
Anyways I just wanted to highlight that the spamming goes both ways. Turkers spam requesters, requesters spam turkers, everybody wins!
(On a side note, I really need some behind the scene data from Amazon, how can I convince amazon to give me some data for research? In return I am willing to give them a high quality travel booklet that I have curated about a city in France, the city is called “Shiraz” : )