diff --git a/.gitignore b/.gitignore index f8384286..983fa433 100644 --- a/.gitignore +++ b/.gitignore @@ -29,3 +29,8 @@ nohup.out # testing results are stored here .cache + +# vim +*.swp + +local/docker-compose.yml diff --git a/local/README.md b/local/README.md index 4cc0c5b3..ebc133b7 100644 --- a/local/README.md +++ b/local/README.md @@ -1,10 +1,10 @@ # Running NEXT locally -We provide this structure to run NEXT locally (which could be necessary if data -private). +This document contains instructions on how to run a NEXT experiment on your local machine. + ## Dependencies -To run NEXT locally, you need a machine with the following things installed: +To start the NEXT backend, you need a machine with the following things installed: ``` docker @@ -12,19 +12,25 @@ docker-compose python2.7 ``` -It is assumed you have python2.7 by default on linux. - `docker` can be installed via the [Docker install guide]. `docker-compose` can be installed via `pip install docker-compose`. -### Using macOS -If using macOS/OS X, download [Docker for Mac], not [Docker Toolbox]. It +Optionally, you need the PyPI packages `requests` and `pyyaml` to run the `launch.py` script in this directory: + +`pip install requests yaml` + + +### Using MacOS +If using MacOS/OS X, download [Docker for Mac], not [Docker Toolbox] It provides an easier interface to get started. [Docker for Mac]:https://docs.docker.com/engine/installation/mac/#/docker-for-mac [Docker install guide]:https://docs.docker.com/v1.8/installation/ +[Docker Toolbox]:https://www.docker.com/products/docker-toolbox + + ## Starting the backend First clone this repository and navigate to the `NEXT/local/` directory. @@ -38,51 +44,47 @@ optionally provide a path to the repo if you are running the ./docker_up.sh [host] [/path/to/NEXT] ``` -where the relative path to NEXT is optional and host is not. A typical -command is +The default will assume host is `localhost` and `NEXT` is located at `../../`: ``` ./docker_up.sh ``` -which assumes the directory `NEXT` is located at `../` when cd'd to `local`. +The first time you run this, docker will download and build the images, which will take a few minutes. -## Starting example experiment +Once the backend is launched, you should be able to go to `http://localhost:8000/home` to see the NEXT homepage. -To set up an experiment, you need the target files hosted somewhere, -and to have a manifest file consisting of JSON specifying filename:url -pairs. For an example, see `NEXT/local/strangefruit30.json`, which -assumes the files are served hosted on `localhost:8999`. -Hosting the images on `localhost:8999` can be achieved by running -`python -m SimpleHTTPServer 8999` in the directory that contains the images. +## Starting an experiment -An example is provided in `experiment_triplet.py`. In -the `strangefruit30` example, unzip `strangefruit30.zip` and run -`python -m SimpleHTTPServer 8999` from the `strangefruit30` directory. +Once the backend is running you can launch an experiment. To set up an experiment, you need to create a yaml file specifying all +the required parameters and data. See `NEXT/local/strange_fruit_triplet/init.yaml` +for an example. This `init.yaml` file is nearly identical to the files in `NEXT/examples`, +except that the targets information is specified explicitly (rather than generated when uploaded to the EC2 server). -``` bash -$ python experiment_triplet.py -``` +Additionally, if your targets are images, they need to be hosted somewhere. +The example **strange_fruit_triplet** example assumes the files are hosted on `localhost:8999`, +which can be achieved by running +`python -m SimpleHTTPServer 8999` in the directory that contains the images (e.g. +`NEXT/local/strange_fruit_triplet/images/`) -This `experiment_triplet.py` file is nearly identical to the files in -`examples/` as the dictionaries to initialize an experiment are constant. The -only difference is launching on EC2 vs. launching locally which manifests -itself on the last line which calls `launch_experiment.py`. +You can launch an experiment by clicking on the *Experiment launch* link on the NEXT homepage and uploading the +appropriate `init.yaml` file (no need to specify a targets file). +Alternatively, you can launch the experiment by running: +```bash +python launch.py strange_fruit_triplet/init.yaml +``` -## Starting new experiment -Two things are needed to run all the examples in `examples/`: -1. Copy the files from `examples/` and modify the `launch_experiment` line at - the end to match `local/experiment_triplet.py` -2. (optional) If you're running an experiment with non-numeric targets (which - we use to test), you'll have to host the images on some URL. A process for - hosting them on `localhost` is described above. +## Creating your own experiments +To run any of the remaining examples in `NEXT/examples/`: -Once this is set up, you can start the experiment with +1. Copy the appropriate directory from `NEXT/examples/` +2. Modify the `init.yaml` file to include the appropriate information about the targets. +3. Host the image(s) by running `python -m SimpleHTTPServer 8999` in the appropriate directory. +4. Start the experiment through the *Experiment launch* interface or by running `python launch.py path/to/init.yaml` -``` -python my_experiment.py path/to/my_experiment.json -``` +To create your own experiments, simply create a new `init.yaml` with the appropriate configuration +and follow the same steps. Note that if your targets are just text, you may skip the hosting step +and simply include the data in the `init.yaml` file. -[Docker Toolbox]:https://www.docker.com/products/docker-toolbox diff --git a/local/data/strangefruit30.zip b/local/data/strangefruit30.zip deleted file mode 100644 index a5ac0ceb..00000000 Binary files a/local/data/strangefruit30.zip and /dev/null differ diff --git a/local/experiment_triplet.py b/local/experiment_triplet.py deleted file mode 100644 index 93d1be4e..00000000 --- a/local/experiment_triplet.py +++ /dev/null @@ -1,73 +0,0 @@ -import os, sys - -# import launch_experiment. We assume that it is located in the next-discovery top level directory. -sys.path.append("../") -from launch_experiment import * - -target_file = 'strangefruit30.json' - -experiment_list = [] -alg_ids = ['RandomSampling','RandomSampling','UncertaintySampling','CrowdKernel','STE'] - -# Create common alg_list -alg_list = [] -for idx,alg_id in enumerate(alg_ids): - alg_item = {} - alg_item['alg_id'] = alg_id - if idx==0: - alg_item['alg_label'] = 'Test' - else: - alg_item['alg_label'] = alg_id - alg_item['test_alg_label'] = 'Test' - #alg_item['params'] = {} - alg_list.append(alg_item) - -# Create common algorithm management settings -params = [] -for algorithm in alg_list: - params += [{'alg_label': algorithm['alg_label'], - 'proportion': 1.0 / len(alg_list)}] - -algorithm_management_settings = {} -algorithm_management_settings['mode'] = 'fixed_proportions' -algorithm_management_settings['params'] = params - - -# Create experiment dictionary -initExp = {} -initExp['args'] = {} -#initExp['args']['n'] = 30 -initExp['args']['d'] = 2 -initExp['args']['failure_probability'] = .01 -initExp['args']['participant_to_algorithm_management'] = 'one_to_many' -initExp['args']['algorithm_management_settings'] = algorithm_management_settings -initExp['args']['alg_list'] = alg_list -initExp['args']['instructions'] = 'Test instructions' -initExp['args']['debrief'] = 'Test debrief' -initExp['app_id'] = 'PoolBasedTripletMDS' -#initExp['site_id'] = 'replace this with working site id' -#initExp['site_key'] = 'replace this with working site key' - - -curr_dir = os.path.dirname(os.path.abspath(__file__)) -experiment = {} -experiment['initExp'] = initExp -experiment['primary_type'] = 'json-urls' -experiment['primary_target_file'] = target_file -experiment_list.append(experiment) - -# Launch the experiment -host = "localhost:8000" -print "It's happening" -exp_uid_list = launch_experiment(host, experiment_list) -print "Made experiments {}".format(exp_uid_list) -# Update the cartoon_dueling.html file wit the exp_uid_list and widget_key_list -# with open('strange_fruit_triplet.html','r') as page: -# print "opended file" -# page_string = page.read() -# page_string = page_string.replace("{{exp_uid_list}}", str(exp_uid_list)) -# page_string = page_string.replace("{{widget_key_list}}", str(widget_key_list)) -# with open('../../next_frontend_base/next_frontend_base/templates/strange_fruit_triplet.html','w') as out: -# out.write(page_string) -# out.flush() -# out.close() diff --git a/local/launch.py b/local/launch.py new file mode 100644 index 00000000..e0de5346 --- /dev/null +++ b/local/launch.py @@ -0,0 +1,80 @@ +""" +Usage: + launch.py + launch.py + +Arguments: + initExp_path (required) : path to a YAML file for arguments to launch experiment + targets_path (optional) : yaml file that contains the targets information + +Options: + -h, --help + +Example: + + > cd ~/Developer/NEXT/local/ + > python launch.py strange_fruit_triplet/init.yaml strange_fruit_triplet/targets.yaml + +""" + +import os +import sys +from collections import OrderedDict +import base64 +import yaml +import requests + +sys.path.append('../next/lib') +from docopt import docopt + + +def launch(init_filename, targets_filename=None): + + with open(init_filename, 'r') as f: + init = yaml.load(f) + + if targets_filename: + with open(targets_filename, 'r') as f: + targets = yaml.load(f) + if 'targets' not in init['args']: + init['args'] = {'targetset': {}} + if 'targetset' not in init['args']['targets']: + init['args']['targets']['targetset'] = dict() + init['args']['targets']['targetset'] = targets + + header = "data:application/{};base64," + exp_args = base64.encodestring(yaml.dump(init)) + d = {'args': header.format('x-yaml') + exp_args} + + d = OrderedDict(d) + + host_url = os.environ.get('NEXT_BACKEND_GLOBAL_HOST', 'localhost') + host_port = os.environ.get('NEXT_BACKEND_GLOBAL_PORT', '8000') + host_url = 'http://{}:{}'.format(host_url, host_port) + + header = ['{}:{}'.format(key, len(item)) for key, item in d.items()] + header = ';'.join(header) + '\n' + + to_send = ''.join([item for _, item in d.items()]) + + data = header + to_send + + r = requests.post(host_url + '/assistant/init/experiment', data=data) + response = r.json() + if not response['success']: + print('An error occurred launching the experiment') + print(response['message']) + sys.exit() + + dashboard_url = host_url + '/dashboard/experiment_dashboard/{}/{}' + dashboard_url = dashboard_url.format(response['exp_uid'], init['app_id']) + print('Dashboard URL:\n {}'.format(dashboard_url)) + print('\n') + print('NEXT Home URL:\n {}'.format(host_url + '/home')) + return response + +if __name__ == "__main__": + args = docopt(__doc__) + print(args) + print('') + launch(args[''], targets_filename=args['']) diff --git a/local/launch_experiment.py b/local/launch_experiment.py deleted file mode 100644 index 0e85b0eb..00000000 --- a/local/launch_experiment.py +++ /dev/null @@ -1,216 +0,0 @@ -""" -author: Lalit Jain, lalitkumarjj@gmail.com -modified: Chris Fernandez -last updated: 06/03/2015 - -A module that can be used to create and launch experiments. Can be imported as a module or used on the command line. - -Usage: -As a module: -exp_uid_list, exp_key_list, widget_key_list = launch_experiment(host, experiment_file) - -Command line: -python launch_experiment --experiment_file= -""" -import os -import re -import csv -import imp -import sys -import json -import getopt -import zipfile -import requests -import datetime -from StringIO import StringIO - -def generate_target_blob(prefix, - primary_file, - primary_type, - experiment=None, - alt_file=None, - alt_type='text'): - ''' - Upload targets and return a target blob for upload with the target_manager. - - Inputs: ::\n - file: fully qualified path of a file on the system. - Must be a zipfile with pictures or a text file. - prefix: string to prefix every uploaded file name with - ''' - print "generating blob" - targets = [] - is_primary_zip = ((type(primary_file) is str and primary_file.endswith('.zip')) - or (zipfile.is_zipfile(primary_file))) - - if is_primary_zip: - target_file_dict, target_name_dict = zipfile_to_dictionary(primary_file) - if alt_type != 'text': - assert alt_file != None, 'Need an alt_file.' - alt_file_dict, alt_name_dict = zipfile_to_dictionary(alt_file) - try: - pairs = [(target_name_dict[key], - target_file_dict[key], - alt_name_dict[key], - alt_file_dict[key]) - for key in target_file_dict.keys()] - except: - raise Exception('Primary target names must' - 'match alt target names.') - - for primary_name, primary_file, alt_name, alt_file in pairs: - target = {'target_id': '{}_{}'.format(prefix, primary_name), - 'primary_type': primary_type, - 'alt_type': alt_type, - 'alt_description': alt_url} - targets.append(target) - else: - for i, (key, primary_file) in enumerate(target_file_dict.iteritems()): - primary_file_name = target_name_dict[key] - if i % 100 == 0 and i != 0: - print('percent done = {}'.format(i / 50e3)) - target = {'target_id': '{}_{}'.format(prefix, primary_file_name), - 'primary_type': primary_type, - 'alt_type': 'text', - 'alt_description': primary_file_name} - targets.append(target) - elif primary_type == 'json-urls': - with open(primary_file, 'r') as f: - targets_dict = json.load(f) - targets_next = [] - for i, (name, url) in enumerate(targets_dict.items()): - target_next = {'target_id': str(i), - 'primary_type': 'image', - 'primary_description': '"' + url + '"', - 'alt_type': 'text', - 'alt_description': name} - targets_next += [target_next] - targets = targets_next - else: - if type(primary_file) is str: - f = open(primary_file) - else: - f = primary_file - f.seek(0) - i = 0 - for line in f.read().splitlines(): - line = line.strip() - if line: - i += 1 - target = {'target_id': str(i), - 'primary_type': 'text', - 'primary_description':line, - 'alt_type': 'text', - 'alt_description':line} - targets.append(target) - print "\ntargets formatted like \n{}\n".format(targets[0]) - return targets - -def zipfile_to_dictionary(filename): - """ - Takes in a zip file and returns a dictionary with the filenames - as keys and file objects as values - - Inputs: ::\n - file: the concerned zip file - - Outputs: ::\n - result: the returned dictionary - """ - zf = zipfile.ZipFile(filename,'r') - files_list = zf.namelist() - dictionary = {} - names_dictionary = {} - for i in files_list: - if re.search(r"\.(jpe?g|png|gif|bmp|mp4|mp3)", - i, re.IGNORECASE) and not i.startswith('__MACOSX'): - f = zf.read(i) - name = os.path.basename(i).split('.')[0] - dictionary[name] = f - names_dictionary[name] = os.path.basename(i) - return dictionary, names_dictionary - -def import_experiment_list(file): - # Load experiment file - mod = imp.load_source('experiment', file) - experiment_list = mod.experiment_list - return experiment_list - -def launch_experiment(host, experiment_list): - """ - Initialize experiment from an array in an experiment file. - - Inputs: ::\n - host: hostname of server running next_frontend_base - experiment_file: Fully qualified system name of file containing experiment info. Should contain an array called experiment_list, whose elements are dictionaries containing all the info needed to launch an experiment. The dictionary must contain the key initExp, a qualified experiment initialization dictionary. It can also contain an optional target_file key that should be the fully qualified name of a target_file on the system. The target_file can be either text (must end in .txt) or a zipfile containing images (which must end in .zip). Can also add additional context_type and context keys. If the context_type is an image, the context must be a fully qualified file name. - """ - exp_uid_list = [] - #exp_key_list = [] - #widget_key_list = [] - # establish S3 connection and use boto get_bucket - - # Initialize experiment - for experiment in experiment_list: - # Upload the context if there is one. - # This is a bit sloppy. Try to think of a better way to do this. - if 'context' in experiment.keys() and experiment['context_type']=='image': - print experiment['context'].split("/")[-1], experiment['context'] - experiment['initExp']['args']['context'] = context_url - experiment['initExp']['args']['context_type'] = "image" - elif 'context' in experiment.keys() and experiment['context_type']=='video': - print experiment['context'].split("/")[-1], experiment['context'] - experiment['initExp']['args']['context'] = context_url - experiment['initExp']['args']['context_type'] = "video" - elif 'context' in experiment.keys(): - experiment['initExp']['args']['context'] = experiment['context'] - experiment['initExp']['args']['context_type'] = experiment['context_type'] - - print 'launch:211', experiment['initExp']['args'].keys() - - # Upload targets - if 'primary_target_file' in experiment.keys(): - targets = generate_target_blob(prefix=str(datetime.date.today()), - primary_file=experiment['primary_target_file'], - primary_type=experiment['primary_type'], - alt_file=experiment.get('alt_target_file', None), - experiment=experiment, - alt_type=experiment.get('alt_type','text')) - - experiment['initExp']['args']['targets'] = {'targetset': targets} - else: - experiment['initExp']['args']['targets']['n'] = n - - targets = experiment['initExp']['args']['targets'] - n = targets['n'] if 'n' in targets.keys() else len(targets) - - url = 'http://{}/api/experiment'.format(host) - print 'Initializing experiment', experiment['initExp'].keys() - response = requests.post(url, - json.dumps(experiment['initExp']), - headers={'content-type':'application/json'}) - - initExp_response_dict = json.loads(response.text) - print "initExp_response_dict", initExp_response_dict - exp_uid = initExp_response_dict['exp_uid'] - #exp_key = initExp_response_dict['exp_key'] - #perm_key = initExp_response_dict['perm_key'] - - exp_uid_list.append(str(exp_uid)) - #exp_key_list.append(str(exp_key)) - #widget_key_list.append(str(perm_key)) - - print "\nQuery Url is at: http://"+host+"/query/query_page/query_page/"+exp_uid + "\n" - - print "exp_uid_list:", exp_uid_list - #print "exp_key_list:", exp_key_list - #print "widget_key_list:", widget_key_list - - return exp_uid_list#, exp_key_list, widget_key_list - -if __name__=='__main__': - opts, args = getopt.getopt(sys.argv[1:], None, ["experiment_file="]) - opts = dict(opts) - print "Mapping targets from ", opts['--experiment_file'], "to", opts['--next_host'] - experiment_list = import_experiment_list(opts['--experiment_file']) - - launch_experiment(opts['--next_host'], experiment_list) diff --git a/local/data/strangefruit30/i0022.png b/local/strange_fruit_triplet/images/i0022.png similarity index 100% rename from local/data/strangefruit30/i0022.png rename to local/strange_fruit_triplet/images/i0022.png diff --git a/local/data/strangefruit30/i0036.png b/local/strange_fruit_triplet/images/i0036.png similarity index 100% rename from local/data/strangefruit30/i0036.png rename to local/strange_fruit_triplet/images/i0036.png diff --git a/local/data/strangefruit30/i0050.png b/local/strange_fruit_triplet/images/i0050.png similarity index 100% rename from local/data/strangefruit30/i0050.png rename to local/strange_fruit_triplet/images/i0050.png diff --git a/local/data/strangefruit30/i0074.png b/local/strange_fruit_triplet/images/i0074.png similarity index 100% rename from local/data/strangefruit30/i0074.png rename to local/strange_fruit_triplet/images/i0074.png diff --git a/local/data/strangefruit30/i0076.png b/local/strange_fruit_triplet/images/i0076.png similarity index 100% rename from local/data/strangefruit30/i0076.png rename to local/strange_fruit_triplet/images/i0076.png diff --git a/local/data/strangefruit30/i0112.png b/local/strange_fruit_triplet/images/i0112.png similarity index 100% rename from local/data/strangefruit30/i0112.png rename to local/strange_fruit_triplet/images/i0112.png diff --git a/local/data/strangefruit30/i0114.png b/local/strange_fruit_triplet/images/i0114.png similarity index 100% rename from local/data/strangefruit30/i0114.png rename to local/strange_fruit_triplet/images/i0114.png diff --git a/local/data/strangefruit30/i0126.png b/local/strange_fruit_triplet/images/i0126.png similarity index 100% rename from local/data/strangefruit30/i0126.png rename to local/strange_fruit_triplet/images/i0126.png diff --git a/local/data/strangefruit30/i0142.png b/local/strange_fruit_triplet/images/i0142.png similarity index 100% rename from local/data/strangefruit30/i0142.png rename to local/strange_fruit_triplet/images/i0142.png diff --git a/local/data/strangefruit30/i0152.png b/local/strange_fruit_triplet/images/i0152.png similarity index 100% rename from local/data/strangefruit30/i0152.png rename to local/strange_fruit_triplet/images/i0152.png diff --git a/local/data/strangefruit30/i0184.png b/local/strange_fruit_triplet/images/i0184.png similarity index 100% rename from local/data/strangefruit30/i0184.png rename to local/strange_fruit_triplet/images/i0184.png diff --git a/local/data/strangefruit30/i0194.png b/local/strange_fruit_triplet/images/i0194.png similarity index 100% rename from local/data/strangefruit30/i0194.png rename to local/strange_fruit_triplet/images/i0194.png diff --git a/local/data/strangefruit30/i0200.png b/local/strange_fruit_triplet/images/i0200.png similarity index 100% rename from local/data/strangefruit30/i0200.png rename to local/strange_fruit_triplet/images/i0200.png diff --git a/local/data/strangefruit30/i0208.png b/local/strange_fruit_triplet/images/i0208.png similarity index 100% rename from local/data/strangefruit30/i0208.png rename to local/strange_fruit_triplet/images/i0208.png diff --git a/local/data/strangefruit30/i0220.png b/local/strange_fruit_triplet/images/i0220.png similarity index 100% rename from local/data/strangefruit30/i0220.png rename to local/strange_fruit_triplet/images/i0220.png diff --git a/local/data/strangefruit30/i0254.png b/local/strange_fruit_triplet/images/i0254.png similarity index 100% rename from local/data/strangefruit30/i0254.png rename to local/strange_fruit_triplet/images/i0254.png diff --git a/local/data/strangefruit30/i0256.png b/local/strange_fruit_triplet/images/i0256.png similarity index 100% rename from local/data/strangefruit30/i0256.png rename to local/strange_fruit_triplet/images/i0256.png diff --git a/local/data/strangefruit30/i0312.png b/local/strange_fruit_triplet/images/i0312.png similarity index 100% rename from local/data/strangefruit30/i0312.png rename to local/strange_fruit_triplet/images/i0312.png diff --git a/local/data/strangefruit30/i0322.png b/local/strange_fruit_triplet/images/i0322.png similarity index 100% rename from local/data/strangefruit30/i0322.png rename to local/strange_fruit_triplet/images/i0322.png diff --git a/local/data/strangefruit30/i0326.png b/local/strange_fruit_triplet/images/i0326.png similarity index 100% rename from local/data/strangefruit30/i0326.png rename to local/strange_fruit_triplet/images/i0326.png diff --git a/local/data/strangefruit30/i0414.png b/local/strange_fruit_triplet/images/i0414.png similarity index 100% rename from local/data/strangefruit30/i0414.png rename to local/strange_fruit_triplet/images/i0414.png diff --git a/local/data/strangefruit30/i0420.png b/local/strange_fruit_triplet/images/i0420.png similarity index 100% rename from local/data/strangefruit30/i0420.png rename to local/strange_fruit_triplet/images/i0420.png diff --git a/local/data/strangefruit30/i0430.png b/local/strange_fruit_triplet/images/i0430.png similarity index 100% rename from local/data/strangefruit30/i0430.png rename to local/strange_fruit_triplet/images/i0430.png diff --git a/local/data/strangefruit30/i0438.png b/local/strange_fruit_triplet/images/i0438.png similarity index 100% rename from local/data/strangefruit30/i0438.png rename to local/strange_fruit_triplet/images/i0438.png diff --git a/local/data/strangefruit30/i0454.png b/local/strange_fruit_triplet/images/i0454.png similarity index 100% rename from local/data/strangefruit30/i0454.png rename to local/strange_fruit_triplet/images/i0454.png diff --git a/local/data/strangefruit30/i0470.png b/local/strange_fruit_triplet/images/i0470.png similarity index 100% rename from local/data/strangefruit30/i0470.png rename to local/strange_fruit_triplet/images/i0470.png diff --git a/local/data/strangefruit30/i0494.png b/local/strange_fruit_triplet/images/i0494.png similarity index 100% rename from local/data/strangefruit30/i0494.png rename to local/strange_fruit_triplet/images/i0494.png diff --git a/local/data/strangefruit30/i0524.png b/local/strange_fruit_triplet/images/i0524.png similarity index 100% rename from local/data/strangefruit30/i0524.png rename to local/strange_fruit_triplet/images/i0524.png diff --git a/local/data/strangefruit30/i0526.png b/local/strange_fruit_triplet/images/i0526.png similarity index 100% rename from local/data/strangefruit30/i0526.png rename to local/strange_fruit_triplet/images/i0526.png diff --git a/local/data/strangefruit30/i0572.png b/local/strange_fruit_triplet/images/i0572.png similarity index 100% rename from local/data/strangefruit30/i0572.png rename to local/strange_fruit_triplet/images/i0572.png diff --git a/local/strange_fruit_triplet/init.yaml b/local/strange_fruit_triplet/init.yaml new file mode 100644 index 00000000..88be36cf --- /dev/null +++ b/local/strange_fruit_triplet/init.yaml @@ -0,0 +1,51 @@ +app_id: PoolBasedTripletMDS +args: + alg_list: + - {alg_id: CrowdKernel, alg_label: Test, test_alg_label: Test} + - {alg_id: STE, alg_label: STE, test_alg_label: Test} + - {alg_id: RandomSampling, alg_label: RandomSampling, test_alg_label: Test} + - {alg_id: UncertaintySampling, alg_label: UncertaintySampling, test_alg_label: Test} + algorithm_management_settings: + mode: fixed_proportions + params: + - {alg_label: Test, proportion: 0.25} + - {alg_label: STE, proportion: 0.25} + - {alg_label: RandomSampling, proportion: 0.25} + - {alg_label: UncertaintySampling, proportion: 0.25} + d: 2 + debrief: Test debrief + failure_probability: 0.01 + instructions: Test instructions + participant_to_algorithm_management: one_to_many + targets: + targetset: + - {primary_description: 'http://localhost:8999/i0022.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0036.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0050.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0074.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0076.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0112.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0114.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0126.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0142.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0152.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0184.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0194.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0200.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0208.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0220.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0254.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0256.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0312.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0322.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0326.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0414.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0420.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0430.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0438.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0454.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0470.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0494.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0524.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0526.png', alt_description: '', primary_type: 'image', alt_type: ''} + - {primary_description: 'http://localhost:8999/i0572.png', alt_description: '', primary_type: 'image', alt_type: ''} diff --git a/local/strangefruit30.json b/local/strangefruit30.json deleted file mode 100644 index ff88189f..00000000 --- a/local/strangefruit30.json +++ /dev/null @@ -1,32 +0,0 @@ -{ -"i0022.png":"http://localhost:8999/i0022.png", -"i0036.png":"http://localhost:8999/i0036.png", -"i0050.png":"http://localhost:8999/i0050.png", -"i0074.png":"http://localhost:8999/i0074.png", -"i0076.png":"http://localhost:8999/i0076.png", -"i0112.png":"http://localhost:8999/i0112.png", -"i0114.png":"http://localhost:8999/i0114.png", -"i0126.png":"http://localhost:8999/i0126.png", -"i0142.png":"http://localhost:8999/i0142.png", -"i0152.png":"http://localhost:8999/i0152.png", -"i0184.png":"http://localhost:8999/i0184.png", -"i0194.png":"http://localhost:8999/i0194.png", -"i0200.png":"http://localhost:8999/i0200.png", -"i0208.png":"http://localhost:8999/i0208.png", -"i0220.png":"http://localhost:8999/i0220.png", -"i0254.png":"http://localhost:8999/i0254.png", -"i0256.png":"http://localhost:8999/i0256.png", -"i0312.png":"http://localhost:8999/i0312.png", -"i0322.png":"http://localhost:8999/i0322.png", -"i0326.png":"http://localhost:8999/i0326.png", -"i0414.png":"http://localhost:8999/i0414.png", -"i0420.png":"http://localhost:8999/i0420.png", -"i0430.png":"http://localhost:8999/i0430.png", -"i0438.png":"http://localhost:8999/i0438.png", -"i0454.png":"http://localhost:8999/i0454.png", -"i0470.png":"http://localhost:8999/i0470.png", -"i0494.png":"http://localhost:8999/i0494.png", -"i0524.png":"http://localhost:8999/i0524.png", -"i0526.png":"http://localhost:8999/i0526.png", -"i0572.png":"http://localhost:8001:8999/i0572.png" -} diff --git a/local/strangefruit30.zip b/local/strangefruit30.zip deleted file mode 100644 index a5ac0ceb..00000000 Binary files a/local/strangefruit30.zip and /dev/null differ