According to a new paper from researchers at EPFL, a Swiss university, a significant percentage of crowd workers on Amazon’s Mechanical Turk service have been found to cheat by using tools like ChatGPT to complete assigned tasks. This discovery raises concerns about the integrity of the platform.
Amazon’s Mechanical Turk has been a popular choice for developers seeking human assistance with various tasks. It is an API that connects humans with tasks that computers may struggle with. Examples of such tasks include drawing bounding boxes for computer vision models and creating high-quality datasets. However, the recent revelation of cheating poses risks to data reliability.
It is challenging to differentiate between text generated by humans and by language models. To investigate this issue, the researchers developed a methodology to identify whether content was created by a human or a machine.
The study involved asking workers to summarize research abstracts into 100-word summaries. This particular task aligns with the capabilities of generative AI technologies like ChatGPT.
