psiTurk commands¶
Each of these commands can be run either from an interactive shell, or as
arguments to the psiturk command (e.g., psiturk amt_balance
or
psiturk hit create 1 0.01 1
from a bash prompt).
See also
amt_balance
¶
Displays your current AMT balance, or your worker sandbox balance (always $10,000.00) if you are in sandbox mode.
An example of checking your balance in sandbox mode:
[psiTurk server:off mode:sdbx #HITs:1]$ amt_balance
$10,000.00
config
¶
Used with a variety of subcommands to control the current configuration context.
Commands
config print
¶
Prints the current configuration context.
Example:
[psiTurk server:off mode:sdbx #HITs:0]$ config print
[Server parameters]
threads=auto
...
[Shell parameters]
launch_in_sandbox_mode=true
[psiTurk server:on mode:sdbx #HITs:0]$
config reload
¶
Reloads the current config context (both local and global files). This will cause the server to restart.
Example:
[psiTurk server:on mode:sdbx #HITs:0]$ config reload
Reloading configuration requires the server to restart. Really reload? y or n: y
Shutting down experiment server at pid 82701...
Please wait. This could take a few seconds.
Experiment server launching...
Now serving on http://localhost:22362
[psiTurk server:off mode:sdbx #HITs:0]$
config help
¶
Display a help message concerning the config subcommand.
debug
¶
Makes it possible to locally test your experiment without contacting Mechanical
Turk servers. Type debug
to automatically launch your experiment in a
browser window. The server must be running to debug
your experiment. When debugging, the server feature that prevents participants
from reloading the experiment is disabled, allowing you to make changes to the
experiment on the fly and reload the debugging window to see the results.
debug -p, --print-only
Use the
-p
flag to print a URL to use for debugging the experiment, without attempting to automatically launch a browser. This is particularly useful if your experiment server is running remotely.Example using the
-p
flag to request a debug link:[psiTurk server:on mode:sdbx #HITs:0]$ debug -p Here's your randomized debug link, feel free to request another: http://localhost:22362/ad?assignmentId=debugDKSAAE&hitId=debug2YW8RI&workerId=debugM1QUH4 [psiTurk server:on mode:sdbx #HITs:0]$
download_datafiles
¶
Accesses the current experiment database table (defined in config.txt) and creates a copy of the
experiment data in a csv format. download_datafiles
creates three
files in your current folder:
`eventdata.csv
Contains events such as window-resizing, and is formatted as follows:
=============== =========== ========== ========== ========= column 1 column 2 column 3 column 4 column 5 =============== =========== ========== ========== ========= unique user ID event type interval value time =============== =========== ========== ========== =========
questiondata.csv
Contains data recorded with psiturk.recordUnstructuredData(), and is formatted as follows:
=============== ============== ========== column 1 column 2 column 3 =============== ============== ========== unique user ID question name response =============== ============== ==========
trialdata.csv
Contains data recorded with psiturk.recordTrialData(), and is formatted as follows:
=============== =========== ========== =========== column 1 column 2 column 3 column 4 =============== =========== ========== =========== unique user ID trial # time trial data =============== =========== ========== ===========
Note
More information about how to record different types of data in an experiment can be found here.
help
¶
Usage:
help
help <command>
The help
command displays a list of valid psiturk shell commands.
Entering help
followed by the name of a command brings up information about
that command.
Examples:
List all commands:
[psiTurk server:on mode:sdbx #HITs:0]$ help psiTurk command help: \=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\= amt_balance debug mode server config download_datafiles open setup_example version db hit psiturk_status status worker basic CMD command help: \=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\=\= EOF ed help li py run shortcuts _load edit hi list q save show _relative_load eof history load quit set cmdenvironment exit l pause r shell
psiTurk commands are listed first, followed by commands inherited from the python cmd2 module. More information about cmd2 commands can be found here.
View the help menu for a command and its subcommands
[psiTurk server:on mode:sdbx #HITs:0]$ help server Usage: server on server off server restart server log server help 'server' is used with the following subcommands: on Start server. Will not work if server is already running. off Stop server. May take several seconds. restart Run 'server off', followed by 'server on'. log Open live server log in a separate window. help Display this screen.
Note
With commands with subcommands such as server
,
you can also view the help screen by entering <command> help
. For
example, server help
has the same effect at help server
.
hit
¶
The hit
command is used to create, view, delete, and modify Human Intelligence
Tasks (“HITs”) on Amazon Mechanical Turk.
Commands
hit create
¶
Usage:
hit create [<numWorkers> <reward> <duration>]
Create a HIT with the specified number of assignments, reward amount, and
duration. Will be posted either live to AMT or to the Worker Sandbox depending
upon your current mode. hit create
can also be run interactively by
entering the command without parameters.
The duration
specifies how long a worker can “hold on” to your HIT
(in hours or hours.<fraction_of_hour>). This should be long enough for workers
to actually
complete your HIT, but sometimes workers will “accept” a HIT which is
worth a lot of money but come back and do the work later in the
day. You can specify a shorter duration if you want workers to
complete your HIT immediately.
Example of creating a HIT in the sandbox with three assignments that pays $2.00 and has a 1.5 hour time limit:
[psiTurk server:on mode:sdbx #HITs:0]$ hit create 3 2.00 1.5
*****************************
Creating sandbox HIT
HITid: 2XE40SPW1INMXUF9OJUNDB6BT8W2F4
Max workers: 3
Reward: $2.00
Duration: 1.5 hours
Fee: $0.60
________________________
Total: $6.60
Ad for this HIT now hosted at: https://ad.psiturk.org/view/Q3HWnfqzg3MP9VDbu3kFyn?assignmentId=debugJCI80S&hitId=debug9AWC90
[psiTurk server:on mode:sdbx #HITs:1]$
hit extend
¶
Usage:
hit extend <HITid> [--assignments <number>] [--expiration <time>]
Extend an existing HIT by increasing the amount of time before the HIT expires (and and is no longer available to workers) or by increasing the number of workers who can complete the HIT.
Example adding both time and assignments to a HIT:
psiTurk server:on mode:sdbx #HITs:1]$ hit list --active
Stroop task
Status: Assignable
HITid: 2776AUC26DG6NRIGNVRFN0COYO0B4R
max:3/pending:0/complete:0/remain:3
Created:2014-03-07T21:36:33Z
Expires:2014-03-08T21:36:33Z
[psiTurk server:on mode:sdbx #HITs:1]$ hit extend 2776AUC26DG6NRIGNVRFN0COYO0B4R --assignments 10 --expiration 12
HIT extended.
[psiTurk server:on mode:sdbx #HITs:1]$ hit list --active
Stroop task
Status: Assignable
HITid: 2776AUC26DG6NRIGNVRFN0COYO0B4R
max:13/pending:0/complete:0/remain:13
Created:2014-03-07T21:36:33Z
Expires:2014-03-08T21:48:33Z
Note that both the remaining number of assignments and the expiration time of the HIT have increased. One can also increase the number of assignments or the expiration independently.
hit expire
¶
Usage:
hit expire (--all | <HITid> ...)
Expire one or more existing HITs, or expire all HITs using the --all
flag.
Examples:
Expiring two HITs at once:
[psiTurk server:on mode:sdbx #HITs:4]$ hit expire 2Y0T3HVWAVKIMG42A2S75Z9943NNFG 2RVZXR24SMEZFG314ME9X8P9CPPH0X expiring sandbox HIT 2Y0T3HVWAVKIMG42A2S75Z9943NNFG expiring sandbox HIT 2RVZXR24SMEZFG314ME9X8P9CPPH0X [psiTurk server:on mode:sdbx #HITs:2]$
Expiring all active HITs:
[psiTurk server:on mode:sdbx #HITs:2]$ hit expire --all expiring sandbox HIT 2776AUC26DG6NRIGNVRFN0COYO0B4R expiring sandbox HIT 2VUWA6X3YOCCVET8PKOPWINIWJFPO0 [psiTurk server:on mode:sdbx #HITs:0]$
worker
¶
The worker
command is used to list, approve and reject, and bonus worker
assignments on Amazon mechanical Turk.
worker approve
¶
Usage:
worker approve (--hit <hit_id> | <assignment_id> ...)
Approve worker assignments for one or more assignment ID’s, or use the
--hit
flag to approve all workers for a specific HIT.
Examples:
Approve a single assignment:
[psiTurk server:on mode:sdbx #HITs:0]$ worker approve 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7 approved 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7
Approve all assignments for a given hit:
[psiTurk server:on mode:sdbx #HITs:0]$ worker approve --hit 2QKHECWA6X3Y4QTYKCG5NXPTWYGMLF approving workers for HIT 2QKHECWA6X3Y4QTYKCG5NXPTWYGMLF approved 2MB011K274J7PY7FQ1ZN76UXH0ECED approved 2UO4ZMAZHHRR1T7J8NEVUH1KJCAKBY
worker reject
¶
Usage:
worker reject (--hit <hit_id> | <assignment_id> ...)
Reject worker assignments for one or more assignment ID’s, or use the --hit
flag to reject all workers for a specific HIT.
Example rejecting a single assignment:
[psiTurk server:on mode:sdbx #HITs:0]$ worker reject 2Y9OVR14IXKOIZQL1E3WD6X30CD98U
rejected 2Y9OVR14IXKOIZQL1E3WD6X30CD98U
worker unreject
¶
Usage:
worker unreject (--hit <hit_id> | <assignment_id> ...)
Unreject worker assignments for one or more assignment ID’s, or use the
--hit
flag to unreject all workers for a specific HIT.
Note
Unrejecting an assignment automatically approves that assignment.
Example of unrejecting a single assignment:
[psiTurk server:on mode:sdbx #HITs:0]$ worker unreject 2Y9OVR14IXKOIZQL1E3WD6X30CD98U
unrejected 2Y9OVR14IXKOIZQL1E3WD6X30CD98U
worker bonus
¶
Usage:
worker bonus (--amount <amount> | --auto) (--hit <hit_id> | <assignment_id> ...)
Grant bonuses to workers for one or more assignment ID’s, or use the --hit
flag to bonus all workers for a specific HIT.
Enter the bonus --amount <amount>
in an X.XX format, or use the --auto
flag to bonus each worker according to the ‘bonus’ field of hte database
(requires a custom bonus route in the experiment’s
custom.py file).
Upon running worker bonus
, you will be asked to input a reason for the
bonus. This message will be displayed to workers who receive the bonus.
Note
You must approve the worker assignment before you grant a bonus.
Warning
While it isn’t possible to approve an assignment more than once, it is
possible to grant a bonus repeatedly. When running worker bonus
with the
--hit
flag, only workers who have not yet received a bonus for the
assignment will be bonused. However, when running worker bonus
on
individual assignments the worker will be bonused regardless of whether they
have already received one.
Examples:
Bonusing an individual assignment. The bonus can be granted repeatedly, making this risky:
[psiTurk server:on mode:sdbx #HITs:0]$ worker bonus --amount 2.00 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7 Type the reason for the bonus. Workers will see this message: Here's a bonus! gave bonus of $2.00 to 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7 [psiTurk server:on mode:sdbx #HITs:0]$ worker bonus --amount 2.00 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7 Type the reason for the bonus. Workers will see this message: Here's another one! gave bonus of $2.00 to 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7
Say there are approved assignments for a HIT, one already bonused, one not yet bonused. Bonusing by HIT prevents repeated bonuses:
[psiTurk server:on mode:sdbx #HITs:0]$ worker bonus --amount 2.00 --hit 2ECYT3DHJHP4RRU304P8USX9BCXU1O Type the reason for the bonus. Workers will see this message: you haven't been bonused yet. Here's a bonus! bonusing workers for HIT 2ECYT3DHJHP4RRU304P8USX9BCXU1O gave a bonus of $2.00 to 2MB011K274J7PY7FQ1ZN76UXH0ECED bonus already awarded to 21A8IUB2YU98ZV9C5BUL3FBJB5B8K7
If a compute-bonus route exists in the experiment custom.py, we can also use the
--auto
flag to automatically give each worker the correct bonus:[psiTurk server:on mode:sdbx #HITs:0]$ worker bonus --auto --hit 2ECYT3DHJHP4RRU304P8USX9BCXU1O Type the reason for the bonus. Workers will see this message: Thanks for moving science forward! bonusing workers for HIT 2ZQIUB2YU98JX6A4V3C0IBJ9W0HL9P gave a bonus of $3.00 to 27UQ45UUKQOYW1ZFLNJ8RG012VYDVP gave a bonus of $2.50 to 24IIHPCGJ2D2H2KFPX80MPPSKQM933
Note
Unlike the commands to approve, reject, or unreject workers, the worker
bonus
command requires the psiturk shell to be launched in the same
project as the HIT for which workers are being bonused, since the
information about which workers have been bonused is stored in the
experiment database.
worker list
¶
Usage:
worker list [--submitted | --approved | --rejected] [--hit <hit_id>]
List all worker assignments, or list worker assignments fitting a
given condition using the provided flags. Use the --hit
flag to
list workers for a specific HIT.
Examples:
Listing all submitted workers:
[psiTurk server:on mode:sdbx #HITs:0]$ worker list --submitted [ { "status": "Submitted", "assignmentId": "2VQHVI44OS2K18PW7EQSEAP5DPV5ZY", "workerId": "A2O6BB9HXEUXX1", "submit_time": "2014-03-04T16:14:32Z", "hitId": "2ZRNZW6HEZ6OUI7FRTZ6CGUMGIQPZ0", "accept_time": "2014-03-04T16:14:05Z" }, { "status": "Submitted", "assignmentId": "2XB92NJKM05B2XAD1YN2JTP9TYXAFG", "workerId": "A2O6BB9HXEUXX1", "submit_time": "2014-03-03T23:35:17Z", "hitId": "2RWSCWY2AOO2W03X0OFGTSCMKZZ22I", "accept_time": "2014-03-03T23:34:19Z" } ]
Listing approved workers for a specific HIT:
[psiTurk server:on mode:sdbx #HITs:0]$ worker list --approved --hit 2ECYT3DHJHP4RRU304P8USX9BCXU1O listing workers for HIT 2ECYT3DHJHP4RRU304P8USX9BCXU1O [ { "status": "Approved", "assignmentId": "21A8IUB2YU98ZV9C5BUL3FBJB5B8K7", "workerId": "A2O6BB9HXEUXX1", "submit_time": "2014-02-26T03:26:55Z", "hitId": "2ECYT3DHJHP4RRU304P8USX9BCXU1O", "accept_time": "2014-02-26T03:26:36Z" } ]
psiturk_status
¶
Usage:
psiturk_status
Display startup screen with message from psiturk.org.
Example:
[psiTurk server:off mode:sdbx #HITs:1]$ psiturk_status
http://psiturk.org
______ ______ __ ______ __ __ ______ __ __
/\ == \ /\ ___\ /\ \ /\__ _\ /\ \/\ \ /\ == \ /\ \/ /
\ \ _-/ \ \___ \ \ \ \ \/_/\ \/ \ \ \_\ \ \ \ __< \ \ _"-.
\ \_\ \/\_____\ \ \_\ \ \_\ \ \_____\ \ \_\ \_\ \ \_\ \_\
\/_/ \/_____/ \/_/ \/_/ \/_____/ \/_/ /_/ \/_/\/_/
an open platform for science on Amazon Mechanical Turk
--------------------------------------------------------------------
System status:
Hi all, You need to be running psiTurk version >= 1.0.5dev to use the
Ad Server feature!
Check https://github.com/NYUCCL/psiTurk or http://psiturk.org for
latest info.
psiTurk version 1.0.8dev
Type "help" for more information.
[psiTurk server:off mode:sdbx #HITs:1]$
quit
¶
Usage:
quit
Quits the psiTurk shell. If you have a server running, psiTurk will confirm that you want to quit before exiting, since quitting psiTurk turns off the server.
Example of quitting psiTurk with the server running:
[psiTurk server:on mode:sdbx #HITs:0]$ quit
Quitting shell will shut down experiment server. Really quit? y or n: y
Shutting down experiment server at pid 40182...
Please wait. This could take a few seconds.
$
server
¶
The server
command is used with a variety of subcommands to control the
experiment server.
server on
¶
Start the experiment server.
Example:
[psiTurk server:off mode:sdbx #HITs:0]$ server on
Experiment server launching...
Now serving on http://localhost:22362
[psiTurk server:on mode:sdbx #HITs:0]$
server off
¶
Shut down the experiment server.
Example:
[psiTurk server:on mode:sdbx #HITs:0]$ server off
Shutting down experiment server at pid 32911...
Please wait. This could take a few seconds.
[psiTurk server:off mode:sdbx #HITs:0]$
server restart
¶
Runs server off
, followed by server on
.
server log
¶
Opens the server log in a separate window. Uses Console.app on Max OS X and xterm on other systems.
status
¶
Usage:
status
The status
command updates and displays the server status and
number of HITs available on AMT or in the worker sandbox.
Note
This information is also displayed in the psiTurk shell prompt, but
#HITs is not updated after every command (as every update
requires contacting the AMT server). status
provides a
way to make sure the prompt is up-to-date.
Example of using the status
command in sandbox mode:
[psiTurk server:off mode:sdbx #HITs:1]$ status
Server: currently offline
AMT worker site - sandbox: 1 HITs available
mode
¶
Usage:
mode
mode <which>
The mode
command controls the current mode of the psiTurk shell. Type
mode live
or mode sandbox
to switch to either mode, or simply mode
to switch to the opposite mode. The current mode affects almost every psiturk
shell command. For example, running hit create
while in sandbox mode will
create a HIT in the sandbox, while running it in live mode will create a HIT on
the live AMT site. Similarly, commands like worker list all
or hit list
all
will list assignments and HITs from either the live site or the sandbox,
depending on your mode.
Note
Switching the psiturk shell mode while the server is running requires the server to restart, since at the end of the experiment participants need to be correctly redirected back to either the live AMT site or the sandbox. Therefore, you should not change modes while you are serving a live HIT to workers.
Examples:
Switching mode, with and without
<which>
specifier:[psiTurk server:off mode:sdbx #HITs:0]$ mode Entered live mode [psiTurk server:off mode:live #HITs:0]$ mode sandbox Entered sandbox mode [psiTurk server:off mode:sdbx #HITs:0]$
Switching mode with the server running:
[psiTurk server:on mode:sdbx #HITs:0]$ mode Switching modes requires the server to restart. Really switch modes? y or n: y Entered live mode Shutting down experiment server at pid 33447... Please wait. This could take a few seconds. Experiment server launching... Now serving on http://localhost:22362 [psiTurk server:on mode:live #HITs:0]$
Type
n
instead to abort the mode switch harmlessly.