Crowdsourcing simply refers to a method of fund sourcing in
which organizations or individuals use contributions from internet users to
achieve a set objective. The word was adopted in 2005 and seems to combine the
word ‘crowd’ and outsourcing. The belief
is that crowdsourcing has to do with outsourcing work to a crowd people.
There’s a difference between crowdsourcing and outsourcing because, with
crowdsourcing, the work can originate from an undefined public (rather than a
predetermined group). Some of the main benefits of using crowdsourcing include
improved speed, adaptability, costs, quality, diversity, or versatility
Crowdsourcing has been highly beneficial in gathering
high-quality gold standard used in making automatic systems in natural language
processing. Promoted by efforts like the ESP game (von Ahn and Dabbish, 2004)
and Peekaboom (von Ahn et al., 2006), Human Computation based games can be a
viable approach to engage users and gather vast quantity of data inexpensively.
Two player games can likewise automate verification of human provided
Amazon Mechanical Turk (MTurk) simply refers to an online
crowdsourcing marketplace that makes it possible for businesses and individuals
to organize the use of human intelligence to carry out tasks that cannot
currently be performed by computers.
It’s a website that is owned by Amazon.
Jobs known as Human Intelligence Tasks can be posted by employers
(HITs), such as writing descriptions, picking the very best among multiple images
of a storefront, or identifying performance in music recordings. So-called
workers can later search through a large collection of existing jobs; they can
complete these jobs in exchange for monetary rewards as fixed by the employer.
The requesting programs place jobs using an API, or the more limited MTurk
Requester site which seems to be more limited.
For a requester to submit an order to be accomplished through the
Mechanical Turk platform, he has to submit a billing address in one of about 30
CrowdFlower refers to a San Francisco based crowdsourcing
and data mining company. The company provides a software solution with which
users can obtain access to an online workforce to label, clean, and enrich
data. CrowdFlower uses an online
workforce to clean up messy and fractional data. The great number of
CrowdFlower users are data scientists who use the solution to build training
models as well as machine learning algorithms.
As soon as data is uploaded into the system, the work is
automatically allocated to contributors and is tested against established
answers which are hidden within the task (this is called “job” in
CrowdFlower). The system trust individuals based on the way they perform on
these hidden tasks. Contributors are allowed to continue working on a
particular job as long as they are still trusted. If they lose that trust, they
lose the job, and their work is disregarded. The judgments of many contributors
are collated and the result is given based on aggregate answers with an
associated confidence score (contributors’ agreement weighed by the trust of
each contributor) (http://en.turkcewiki.org/wiki/Crowdflower).