Assignment 2-2: Testing Out Your HITs (10%)

(This assignment was developed based on Jeff's class project 2.)

In this assignment (Assignment 2 as a whole), you'll take on the role of a requester by creating tasks on Amazon Mechanical Turk, and submitting your first HIT "programmatically" -- that is, without using the GUI provided on the Amazon Mechanical Turk site. This is one of the first steps of really "programming the crowd" because it demonstrates the start of being able to write programs that automatically draw in human intelligence when it's needed.

A HIT (or task page) is just a web page, so we'll start by creating a web page for your task. A web page with that contains a form that will be submitted to Mechanical Turk with a specific format. You might find it useful to include the script "mturk.js" that is also included on that site. Similar to the Javascript that we developed in class, that script will help to set your page up for Amazon Mechanical Turk.

In assignment 2-1, you have created a worker interface and hosted it under HTTPS. Now is time to post it to Amazon Mechanical Turk.

You'll need to prepare your environment to programmatically post your job. We've collected some tools and tutorials to for you to use to post your jobs on Mechanical Turk programatically.

After setting up the tools, you'll want to modify the files to use your Mechanical Turk Requester credentials. There are some great instructions on getting started with Mechanical Turk that overviews this process (see above.) One step that you'll need to do is to get your security credentials and put them into the Python scripts you use. You should be able to modify the example that you downloaded in order to post your own HITs to Mechanical Turk.

You can use this third-party tool to Manage HITs individually: Manage HITs individually

Debugging your HITs on the Mechanical Turk Sandbox

We strongly recommend posting and debugging your HITs on the Mechanical Turk sandbox before posting on the real site and potentially using real money.

Wage

When setting the wage for your work, we strongly encourage you to do your best to make it a living wage. That is a minimum of $7.25/hour. Many workers will not work if the pay works out to less than $10/hour. Since you built your interface, it's probably really easy for you to use, but it may take someone new to it longer to figure out. Please try your interface with someone else (and err on the high side) when setting wages.

We expect that you'll spend between $5 and $10. There's not an easy way for us to reimburse you for this, but Kenneth will buy the class coffee and donuts!

What to turn in

You'll submit four things:

  1. A screenshot of your posted HITs on Mturk (see below)
  2. The URL (HTTPS) of your HIT page that you posted to Mechanical Turk
  3. A PDF file that answers the questions below
  4. A Google Spreadsheet of your data in the format described below
1. A screenshot of your posted HITs on Mturk (see below)

Please upload a screenshot image as follows to show your HITs posted on mturk. Try to screenshot your HIT on real Mturk. If it's taken too quickly to capture, you can screenshot on sandbox mturk.

2. The URL (HTTPS) of your HIT page that you posted to Mechanical Turk

The URL of the page you posted to Mechanical Turk. The web page should be viewable / usable even if not viewed through the Mechanical Turk interface. One way to do this is to only disable page elements if assignmentID==”ASSIGNMENT_ID_NOT_AVAILABLE”

3. A PDF file that answers the questions below

Questions to provide short responses to:

  1. Briefly describe your design. What did you do to encourage high-quality inputs? What did you change from your initial prototype to your final one?
  2. Do you think you got good data back from workers?
  3. How much did it cost you to complete your experiments?
4. A Google Sheet of your data in the format described below

Submit the URL to a Google Sheet containing your data in the following format:

Columns: workerid, text_of_story

There should be more than one rows. 1 row at the top with the column headings, a few more rows containing the data entered by each worker.