Plugin Reference

Plugin Files

  • Plugin Python File: The primary file for implementing the logic contained in the plugin. The name of this file must be <PluginName>.py and be contained in the top level directory, which also has the same name as the plugin. Names are case-sensitive.
  • Configuration Interfaces: There are three different configuration interfaces, all of which are optional. If your plugin does not require any configuration to execute, you do not need to implement all of the following:
    • instance.html: If present, this page appears when a plugin is run from the Manual Launch button on the Run Report page.
    • plan.html: When you create a plan in the Plugin chevron, you have the option of launching a configuration interface for that plan. If not present, you can still select the plugin to run, but there will be no configuration data.
    • config.html: This interface is presented in the plugin configuration interface and sets up the default configuration, which is used if neither of the other two configuration caches are present.
  • Static Files: HTML files included in or generated by a plugin may need to load static assets (JS/CSS/Images). Include these files in a directory called pluginMedia in the root plugin directory. Reference the static files using the following URL pattern: /pluginMedia/<PluginName>/example.css
  • Documentation:
    • about.html: If present, this information is accessible from the plugins manage menu on the plugin page.

Base Class

All plugins must create one and only one class that inherits from the IonPlugin base class. The IonPlugin base class requires that the following attributes and methods be overridden, although some are optional.

Attributes & Properties

Attribute Name Required Type Default Description
name No string Empty Stores the name of the plugin in addition to the name of the class itself for logging.
version Yes string Empty The four number version number for the plugin.
runtypes No list(string) Empty Indicates if this plugin is used for wholechip and/or thumbnails.
features No list(string) Empty Holds a list of current features enabled for this plugin.
runlevels Yes list(string) Empty Holds a list for each run level that this plugin executes during the pipeline execution.
depends No list(string) Empty Lists all plugins which are executed before this one, if present.
major_block No boolean False If true, indicates that this plugin’s output is presented as part of the run report.
requires No list(string) [‘BAM’] Currently unused.
output No dictionary Empty Currently unused.
results No dictionary Empty Currently unused.
exit_status No int Empty Currently unused.
context No dictionary Empty Currently unused.
blockcount No int 0 Currently unused.
plugin No Plugin None Currently unused.
analysis No Result None Currently unused.
pluginresult No PluginResult None Currently unused.
startplugin No dictionary None This returns “an” in the memory dictionary of the startplugin.json.

Methods

Method Name Required Return Type Description
launch_wrapper No None Do not use.
generate_output No None Currently unused.
pre_block No None Currently unused.
post_block No None Currently unused.
pre_launch No boolean Run prior to launch. Return False if you do not want the plugin to execute.
post_launch No None Currently unused.
launch Yes boolean If true, this indicates that this plugin’s output is presented as part of the run report.
getUserInput No None Currently unused.
barcodetable_columns No list This method returns a list of columns to be used to generate the plugin barcodes table ui.
barcodetable_data No dictionary This returns a dictionary to populate the contents of the plugin barcodes table ui.
get_restobj No dictionary This method returns a dictionary based on a REST API function call.

Enums

Import the enumerations from the ion.plugin.constants module. Since enumerations are not supported in python 2, you implement them by creating a type with static members according to the following scheme.

  • Feature
    • EXPORT: Used to indicate that this plugin is run last because it exports data.
  • Run Types
    • COMPOSITE: Block based chip runs.
    • THUMB: A thumbnail result.
    • FULLCHIP: Ion PGM™ System Full chip results.
  • Run Levels
    • PRE
    • DEFAULT
    • BLOCK
    • POST
    • SEPARATOR
    • LAST

Execution Environment

When you execute plugins, they are controlled with a script which is written to the results output directory call ion_plugin_*<Plugin Name>*_launch.sh. The plugin framework creates this file and directs the Grid Engine to execute this as the entry point for the plugin execution. The file handles the following:

  • Updating the status in the database to be reflected in the run results page
  • Setting up environmental variables (these are used in legacy plugins)
  • Setting the umask to 0000
  • Preventing core files from being written from core dumps
  • Implementing the use of the output.json to create output (currently incomplete)

Run Levels

To use the run levels, you must assign them to your plugin class. If none are assigned the plugin will use DEFAULT runlevel. The run levels, block and conventional, are described in the Getting Started section. While it is technically possible, we do not recommend that you assign run levels from both groups concurrently.

Clusters

To accommplish clustering, the execution of the plugins is queued through a grid engine which, for single servers, is executed on the same computer as the one hosting the web site. In a cluster, the plugins are run only on the computer nodes and never on the head node. Consider the following when writing a plugin:

  • The plugin is not run on the head node, so any references to “localhost” are incorrect. For example, if you are making a REST call and you have hard-coded the domain name of the url to be localhost, this attempts to call the REST API from the compute node, which results in an error. Instead, ensure that any REST API calls use the protocol and domain name in the startplugin.json contents runinfo->net_location.
  • To distribute the executable code to the compute nodes, the system piggy-backs on the commonly shared NFS mount “/results” by creating a child folder at “/results/plugins/”. This means that the stability of the plugin framework is going to be intimately tied to the stability of the network. Ensure that the connection to the results folder is as stable and redundant as reasonably possible.

Dependencies

Due to the method of distribution of plugin logic over NFS mounts, there are very few libraries on the compute nodes during run time. To work around this, package all dependencies (beyond the standard python libraries) into the install file with the core logic so they are also installed into the /results/plugins/<Plugin Name> directory on the NFS mount to be redistributed out to the compute nodes.

REST API Extensions

The plugin framework currently gives you an option of extending the REST API with custom endpoints by implementing a python file in the root of the plugin folder, which must have the name “extend.py”. By implementing a method in this module with a single dictionary argument named “bucket”, which is described below, the extension is exposed through the REST API for execution using the following url:

http(s)://{HOSTNAME}/rundb/api/v1/plugin/{PLUGIN}/extend/{METHOD_NAME}

bucket["request_get"] = request.GET
# assume json
if request.method == "POST":
    bucket["request_post"] = json.loads(request.body)
bucket["user"] = request.user
bucket["request_method"] = request.method
bucket["name"] = plugin.name
bucket["version"] = plugin.version
# not sure if we want this or not, keep it for now
bucket["config"] = plugin.config

Configuration

In many situations for plugins, there needs to be some sort of configuration declared before the plugin run time. This means that the plugin needs to implement an interface to allow users to configure it. Users can select from three configuration options in Torrent Suite™ Software: global, plan, and manual. The HTML pages can reference static assets, see Plugin Files.

Global Configuration

Include an HTML file named config.html in your root plugin directory to enable global configuration. This HTML window appears in an iframe on the global plugin configuration page at /configure/plugins/. Click the gear_icon (Settings) next to a plugin, then select “Configure”.

Reading Configuration

Read from the plugin api endpoint. “/rundb/api/v1/plugin/” + TB_plugin.pk + “/” Or Read the window.TB_plugin js variable.

Writing Configuration

Write to the plugin api endpoint with a PUT request. “/rundb/api/v1/plugin/” + TB_plugin.pk + “/”

Plan Configuration

Include an HTML file named plan.html in your root plugin directory to enable plan configuration. This HTML window appears in an iframe on the planning screen plugin configuration page at /plan/page_plan_plugins/. Click the checkbox next to a plugin, then select “Configure”.

Reading Configuration

Read the window.TB_plugin js variable, then wait for window.restoreForm to be called with the last data passed to serializeForm.

Writing Configuration

window.serializeForm is called by the parent frame to gather the current configuration when users click “Save” in the parent frame.

Manual Configuration

Include an HTML file named instance.html in your root plugin directory to enable manual configuration. The HTML window appears in an iframe on the report page at /report/<ID>/. Click “Select Plugins To Run”, then select a plugin.

Reading Configuration

Read the window. TB_plugin js variable.

Writing Configuration

Write to the results endpoint with a POST request. “/rundb/api/v1/results/” + TB_result + “/plugin/” Then call the following to close the iframe. window.parent.$.colorbox.close()

Barcode Table UI

Plugin Barcode Table UI is an optional service provided by the plugin framework. It allows plugins to generate a simple GUI that can be used to select which barcodes to process and specify per-barcode parameters. The table is similar to the barcode sample table in plan screen with one row per barcode and columns specified by the plugin. This UI is provided for manual plugin launch only and is an opt-in service for plugins to use if desired.

Defining table columns

def barcodetable_columns(self):
    # plugin class method to specify which columns to display
    # inputs: none
    # outputs: list of columns and options to show
    columns_list = [
        {
            "field": "selected",
            "editable": True
        },
        {
            "field": "barcode_name",
            "editable": False
        },
        {
            "field": "sample",
            "editable": False
        },
        ...
    ]
    return columns_list

List of available columns can be retrieved from framework by executing the following command line:

python /results/plugins/<myPlugin>/<myPlugin>.py --bctable-columns

Providing default table contents (optional)

Plugin barcode table will be populated on page load from existing samples information entered during run planning. Additionally, the plugin can modify or augment this initial data if it specifies the following function:

def barcodetable_data(self, data, planconfig={}, globalconfig={}):
    # plugin class method to specify default table contents
    # inputs:
    #    data - same structure as in barcodes.json
    #    planconfig - plugin configuration from planning (plan.html)
    #    globalconfig - plugin global configuration (config.html)
    # outputs:
    #    data, modified as needed
    return data

Changing instance.html

The plugin’s instance.html must add the contents of barcodes table to the plugin data before POSTing it to the results API. This data will be written to startplugin.json file at plugin runtime under “pluginconfig” section.

Helper TB_plugin_functions js variable is available to interact with the barcode table UI:

  • TB_plugin_functions.get_plugin_barcodetable()
    returns table data as json object
  • TB_plugin_functions.update_plugin_barcodetable(data)
    can be used to update the table with data json object
  • TB_plugin_functions.plugin_barcodetable_div
    barcode table DIV element

Input Files

The plugin framework creates two different files for general plugin consumption as its inputs. The variables, which are communicated to the plugin from the framework, are spread across two separate JSON files.

barcodes.json

This file has all the references required for iterating through all of the barcodes for a particular run.

Developer Option

By default all the barcodes where the filtered key is true are not included in the barcodes.json file. You can overwrite this behavior by adding “PLUGINS_INCLUDE_FILTERED_BARCODES = True” to the local_settings.py and restarting the ionPlugin service.

nonbarcoded: {
    aligned <bool>: Flags if the reads in bam_file are aligned to the reference genome.
    bam_file <string>: Name of reads file. (May or may not be be aligned to the reference.)
    bam_filepath <string>: Full file path to bam_file on the local torrent server. (File may not exist if read_count is 0.)
    control_sequence_type <string>: Currently either ERCC Mix 1 or ERCC Mix 2 and only defined in plan screen for RNA Sequencing. (Purpose unspecified.)
    filtered <bool>: Flags if the barcode passed the |TS| analysis pipeline filtering criteria.
    hotspot_filepath <string>: Full file path to HotSpot target regions (BED) file on the local torrent server. ("" if not used.)
    genome_urlpath <string>: URL path used to specify the genome for applications like IGV. Typically the path to the FASTA file on the local torrent server.
    nucleotide_type <string>: Currently either DNA or RNA depending on application. Primarily used to distinguish barcodes with AmpliSeq DNA+RNA runs.
    read_count <int>: Total number of barcode-assigned reads in bam_file (prior to alignment).
    reference <string>: Common (short) name of the reference genome used in the pipeline for this barcode, e.g. hg19
    reference_fullpath <string>: Full file path to the to the reference sequences in FASTA format on the local torrent server. (May be "" for unaligned reads.)
    sample <string>: Name of the sample associated with this barcode. (May be associated with multiple barcodes.)
    sample_id <string>: Sample identification code associated with sample.
    target_region_filepath <string>: Full file path to target regions (BED) file on the local torrent server. ("" if not used.)
}
barcode_name: {
    aligned <bool>: Flags if the reads in bam_file are aligned to the reference genome.
    bam_file <string>: Name of reads file. (May or may not be be aligned to the reference.)
    bam_filepath <string>: Full file path to bam_file on the local torrent server.
    barcode_adapter <string>: DNA adapter sequence used to separate barcode_sequence from sequenced read.
    barcode_annotation <string>: User-specified annotation for this barcode.
    barcode_description <string>: Description text associated with this barcode.
    barcode_index <int>: Index of barcode in the barcode set, starting at 1.
    barcode_name <string>: Name of the barcode in the barcode set (barcode_name).
    barcode_sequence <string>: DNA sequence used to identify this barcode.
    barcode_type <string>: User-specified type for this barcode.
    control_sequence_type <string>: Currently either ERCC Mix 1 or ERCC Mix 2 and only defined in plan screen for RNA Sequencing. (Purpose unspecified.)
    filtered <bool>: Flags if the barcode passed the |TS| analysis pipeline filtering criteria.
    hotspot_filepath <string>: Full file path to HotSpot target regions (BED) file on the local torrent server. ("" if not used.)
    genome_urlpath <string>: URL path used to specify the genome for applications like IGV. Typically the path to the FASTA file on the local torrent server.
    nucleotide_type <string>: Currently either DNA or RNA depending on application. Primarily used to distinguish barcodes with AmpliSeq DNA+RNA runs.
    read_count <int>: Total number of barcode-assigned reads in bam_file (prior to alignment).
    reference <string>: Common (short) name of the reference genome used in the pipeline for this barcode, e.g. hg19
    reference_fullpath <string>: Full file path to the to the reference sequences in FASTA format on the local torrent server. (May be "" for unaligned reads.)
    sample <string>: Name of the sample associated with this barcode. (May be associated with multiple barcodes.)
    sample_id <string>: Sample identification code associated with sample.
    target_region_filepath <string>: Full file path to target regions (BED) file on the local torrent server. ("" if not used.)
}

Example barcodes.json for a barcoded run (TSS v5.0.3)
{
   "IonXpress_001":{
      "aligned":true,
      "bam_file":"IonXpress_001_rawlib.bam",
      "bam_filepath":"/results/analysis/output/Local/with_many_samples_017/IonXpress_001_rawlib.bam",
      "barcode_adapter":"GAT",
      "barcode_annotation":"",
      "barcode_description":"",
      "barcode_index":1,
      "barcode_name":"IonXpress_001",
      "barcode_sequence":"CTAAGGTAAC",
      "barcode_type":"",
      "control_sequence_type":"",
      "filtered":false,
      "genome_urlpath":"/auth/output/tmap-f3/hg19/hg19.fasta",
      "hotspot_filepath":"",
      "nucleotide_type":"DNA",
      "read_count":20,
      "reference":"hg19",
      "reference_fullpath":"/results/referenceLibrary/tmap-f3/hg19/hg19.fasta",
      "sample":"First Sample name",
      "sample_id":"",
      "target_region_filepath":""
   },
   "IonXpress_033":{
      "aligned":true,
      "bam_file":"IonXpress_033_rawlib.bam",
      "bam_filepath":"/results/analysis/output/Local/with_many_samples_017/IonXpress_033_rawlib.bam",
      "barcode_adapter":"GAT",
      "barcode_annotation":"",
      "barcode_description":"",
      "barcode_index":33,
      "barcode_name":"IonXpress_033",
      "barcode_sequence":"TTCTCATTGAAC",
      "barcode_type":"",
      "control_sequence_type":"",
      "filtered":false,
      "genome_urlpath":"/auth/output/tmap-f3/hg19/hg19.fasta",
      "hotspot_filepath":"",
      "nucleotide_type":"DNA",
      "read_count":231321,
      "reference":"hg19",
      "reference_fullpath":"/results/referenceLibrary/tmap-f3/hg19/hg19.fasta",
      "sample":"Second Sample Name",
      "sample_id":"",
      "target_region_filepath":""
   },
   "IonXpress_034":{
      "aligned":true,
      "bam_file":"IonXpress_034_rawlib.bam",
      "bam_filepath":"/results/analysis/output/Local/with_many_samples_017/IonXpress_034_rawlib.bam",
      "barcode_adapter":"GAT",
      "barcode_annotation":"",
      "barcode_description":"",
      "barcode_index":34,
      "barcode_name":"IonXpress_034",
      "barcode_sequence":"TCGCATCGTTC",
      "barcode_type":"",
      "control_sequence_type":"",
      "filtered":false,
      "genome_urlpath":"/auth/output/tmap-f3/hg19/hg19.fasta",
      "hotspot_filepath":"",
      "nucleotide_type":"",
      "read_count":267041,
      "reference":"hg19",
      "reference_fullpath":"/results/referenceLibrary/tmap-f3/hg19/hg19.fasta",
      "sample":"",
      "sample_id":"",
      "target_region_filepath":""
   }
}

Usage Notes

  1. For consistency, we recommend that you iterate and present barcodes in order of increasing barcode_index value.
  2. For default plugin configurations, barcodes with filtered == true are not output. (A plugin option to include these may become available soon.)
  3. Barcodes with a sample name provided (i.e. not “”) are represented with filtered == false, regardless of read_count value.
  4. The bam_filepath value is set to the expected location of the bam_file on the Torrent Server. Barcodes with read_count == 0 may not have a bam_file saved, so you can expect a failure to find the bam_file at bam_filepath. If read_count > 0 then a missing bam_file should be treated as an unexpected error. (This would most likely be a result of automated deletion of old files to make space on the server.)
  5. Although control_sequence_type and nucleotide_type appear to be general attributes, at 5.0.3 these are only defined for barcodes that were specified (associated with samples) in the plan. For nonbarcoded elements or barcodes with no sample data that had sufficient reads. these attributes have the value “”.

startplugin.json

This is the primary file to get all of the information regarding the file.

{
    chefSummaary <dictionary> : This optional section will convey information regarding the chef parameters used. {
    }
    datamanagement <dictionary>: Holds information regarding the data management state of the run. {
        Basecalling Input <bool>: This will indicate if the basecalling information is available for use.
        Intermediate Files <bool>: This will indicate if the intermediate files are available for use.
        Output Files <bool>: This will indicate if the output files are available for use.
        Signal Processing Input <bool>: This will indicate if the signal processing information is available for use.
    }
    expmeta <dictionary>: This is an aggregate of data contained in the expMeta.dat file and the ion_params_00.json file. {
        analysis_date <date>: Gets the time of results analysis based on the last modified time stamp on the ion_params_00.json file.
        barcodeId <string>: The barcode kit name from the experiment analysis settings.
        chipBarcode <string>: The barcode of the chip derived from the ion_params_00.json->exp_json->chipBarcode... mostly.
        chiptype <string>: This is the chip which was used to do the run.
        flowOrder <string>: The flow order used to sequence the run.
        instrument <string>: The name (not type) of the instrument which was used to do the sequencing.
        notes <string>: Any notes which may have been included in the experiment.
        project <string>: A list of all of the projects which this result may belong to.
        results_name <string>: The name of the results that will be processed.
        run_date <datetime>: The date/time stamp of the experiment.
        run_flows <int>: The number of flows used in the run.
        run_name <string>: The name of the run as opposed to the name of the result.
        runid <string>: A short identifies for each id.
        sample <string>: This is the name of the first sample which is associated with this run.
        output_file_name_stem <string>: This is a merger of the experiment name and the results name.
    }
    globalconfig <dictionary>: This section is for the global environment of the result. {
        MEM_MAX <string>: Hardcoded to always read "15G".
        debug <int>: Hardcoded to always read 0.
    }
    plan <dictionary>: This section is where all of the elements of the experiment plan are stored. {
        barcodeId <string>: The barcode kit name from the experiment analysis settings.
        barcodedSamples <dictionary>: This is a dictionary of all of the samples information and the barcodes they are associated with. {
            -Sample Name- <dictionary>: The name of the sample {
                barcodeSampleInfo <dictionary>: Contains the information for the barcodes. {
                    -Barcode ID- <dictionary>: {
                        controlSequenceType <string>: The name of the kit used for the controls for specific per Sample applications.
                        controlType <string>: The experimental control used for this sample. eg (No Template Control)
                        description <string>: Free form description field.
                        externalId <string>: Free form id from any external sources
                        hotSpotRegionBedFile <string>: The name of the hotspot data used for this sample.
                        nucleotideType <string>: This will be the nucleotideType used for this barcode (DNA/RNA/Fusions).
                        reference <string>: The name of the reference
                        sseBedFile <string>: The SSE Bed file reference.
                        targetRegionBedFile <string>: The name of the target region data used for this sample.
                    }
                }
                barcodes <list>: A list of strings which should only have one entry equal to the single dictionary key for barcodeSampleInfo.
            }
        }
        bedfile <string>: The name of a bed file used in this plan.
        controlSequencekitname <string, nullable>: The name of the kit used for the controls.
        librarykitname <string>: The name of the library kit used in the plan.
        planName <string>: The name of the plan used in the run.
        regionfile <string>: The file to define regions for this plan.
        reverse_primer <string>: The reverse primer used in the plan.
        runMode <string>: The run mode value of 'SingleRead', 'PairedEnd' or 'Undefined'
        runType <string>: The type of sequencing for this plan, for example "GENS"
        runTypeDescription <string> : A plain english description of the run type, for example "Generic Sequencing".
        sampleGrouping <dictionary>: A representation of the sample group.
        samplePrepKitName <string>: The name of the sample prep kit.
        sampleSet_name <string>: The name of the sample set.
        sampleSet_planIndex <int>: deprecated
        sampleSet_planTotal <int>: deprecated
        sampleSet_uid <string>: deprecated
        sampleTubeLabel <string>: The barcode sample prep label on the sample tube.
        sequencekitname <string>: The name of the kit used for sequencing.
        templatingKitName <string>: The name of the kit used for templating.
        threePrimeAdapter <string>: The sequence of the three prime adapter being used.
        username <string>: The name of the user who created the plan.
    }
    runinfo <dictionary>: Information regarding the sequencing run. {
        alignment_dir <string>: The path of the directory with the alignment data.
        analysis_dir <string>: The path of the directory using the Analysis data.
        api_key <string>: The api key which can be used to access the
        api_url <string>: The base directory url for *most* of the rest api calls.
        barcodeId <string>: The identifier for the barcoding kit.
        basecaller_dir <string>: The path to the directory with the basecaller information.
        chipDescription <string>: The description of the chip used for sequencing.
        chipType <string>: The type of the chip used for sequencing.
        library <string>: The reference library used.
        library_key <string>: The key sequence to the library.
        net_location <string>: The url to the master node used for the run.
        pk <int>: The primary key for this run in the database.
        platform <string>: The type of sequencer being used.
        plugin <dictionary>: This section describes the run parameters for this plugin in this run. {
            depends <list>: The list of dependency plugins for this run.
            features <list>: The list of features for this plugin.
            hold_jid <list>: A list of SGE job id's which this process was asked to hold on.
            id <int>: The database pk for the id of the plugin.
            name <string>: The name of this plugin.
            path <string>: The path to the plugin executable directory.
            pluginconfig <dictionary>: This is a freeform dictionary which contains the global configuration used for this plugin run.
            pluginresult <int>:  The database primary key for the plugin results entry.
            results_dir <string>: The directory path to the plugin result output.
            runlevel <list>: The list of run levels this plugin has been asked to run at.
            runtype <list>: This list of run types that this plugin can be run on.
            userInput <dictionary>: This is a freeform dictionary which contains the run configuration used for this plugin run.
            version <string>: The version of the plugin running.
        }
        plugin_dir <string>: The path to the plugin executable directory.
        plugin_name <string>: The name of this plugin.
        pluginresult <int>: The database primary key for the plugin results entry.
        raw_data_dir <string>: The path to the directory which contains the raw observational data.
        report_root_dir <string>: The path to the directory of the report.
        results_dir <string>: The path to the directory of the plugin results.
        sigproc_dir <string>: The path to the directory of the signal processing data.
        systemType <string>: The type of sequencer being used.
        testfrag_key <string>: The sequence key to the test fragments.
        tmap_version <string>: The version of the tmap program being used.
        url_root <string>: The file path to the directory of the results data. (not a url)
        username <string>: The user who is performing the run.
    {
    runplugin <dictionary>: The exact parameters used for this plugin run. {
        blockId <string>: The id for the block currently being processed.  Blank if not a block process.
        block_dirs <list>: A list of all of the directories of all of the block data.
        numBlocks <int>: The total number of blocks processed.
        run_mode <string>: The run mode that this is being processed in, either 'pipeline' or 'manual'.
        run_type <string>: The type of the run.  Thumbnail, wholechip or composite.
        runlevel <string>: The current run level being run.
    }
    sampleinfo <dictionary>: A dictionary of information used to convey information regarding the samples. {
        SampleName <dictionary>: The name of the sample. {
            attributes <dictionary>: Any attributes {
            }
            description <????>: A free form description of the sample.
            displayedName <string>: The name of the sample.
            externalId <string>: Any remote identifier used for the sample.
            name <string>: The name of the sample without whitespace.
        }
    pluginconfig <dictionary>: This is a freeform dictionary which contains the run configuration used for this plugin run. {
    }
}

Example
{
    "datamanagement": {
        "Basecalling Input": true,
        "Intermediate Files": true,
        "Output Files": true,
        "Signal Processing Input": true
    },
    "expmeta": {
        "analysis_date": "2015-09-02",
        "barcodeId": "",
        "chipBarcode": "AA0026665",
        "chiptype": "\"314R\"",
        "flowOrder": "TACGTACGTCTGAGCATCGATCGATGTACAGC",
        "instrument": "PGM_test",
        "notes": "",
        "project": "SampleData",
        "results_name": "Auto_user_CB1-42-r9723-314wfa-tl_36",
        "run_date": "2011-04-07T12:44:38Z",
        "run_flows": 260,
        "run_name": "R_2011_04_07_12_44_38_user_CB1-42-r9723-314wfa-tl",
        "runid": "ZN2MB",
        "sample": "e5272-wfa-l165"
    },
    "globalconfig": {
        "MEM_MAX": "15G",
        "debug": 0
    },
    "plan": {
        "barcodeId": "",
        "barcodedSamples": {},
        "bedfile": "",
        "controlSequencekitname": null,
        "librarykitname": "Ion Xpress Plus Fragment Library Kit",
        "planName": "CopyOfSystemDefault_R_2011_04_07_12_44_38_user_CB1-42-r9723-314wfa-tl",
        "regionfile": "",
        "reverse_primer": null,
        "runMode": "single",
        "runType": "GENS",
        "runTypeDescription": "",
        "sampleGrouping": null,
        "samplePrepKitName": null,
        "sampleSet_name": null,
        "sampleSet_planIndex": 0,
        "sampleSet_planTotal": 0,
        "sampleSet_uid": null,
        "sampleTubeLabel": null,
        "sequencekitname": "IonPGM200Kit-v2",
        "templatingKitName": "Ion PGM Template OT2 200 Kit",
        "threePrimeAdapter": "ATCACCGACTGCCCATAGAGAGGCTGAGAC",
        "username": string
    },
    "pluginconfig": {},
    "runinfo": {
        "alignment_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001",
        "analysis_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001",
        "api_key": "9516e00c170496012b6df5810431aca7ac558163",
        "api_url": "http://ion-ts-vm/rundb/api",
        "barcodeId": "",
        "basecaller_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001/basecaller_results",
        "chipDescription": "",
        "chipType": "\"314R\"",
        "library": "e_coli_dh10b",
        "library_key": "TCAG",
        "net_location": "http://ion-ts-vm",
        "pk": 1,
        "platform": "pgm",
        "plugin": {
            "depends": [],
            "features": [],
            "hold_jid": [],
            "id": 11,
            "name": "FilterDuplicates",
            "path": "/results/plugins/FilterDuplicates",
            "pluginconfig": {},
            "pluginresult": 5,
            "results_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001/plugin_out/FilterDuplicates_out.5",
            "runlevel": [],
            "runtype": [
                "composite",
                "wholechip",
                "thumbnail"
            ],
            "userInput": "",
            "version": "5.0.0.0"
        },
        "plugin_dir": "/results/plugins/FilterDuplicates",
        "plugin_name": "FilterDuplicates",
        "pluginresult": 5,
        "raw_data_dir": "/results/PGM_test/cropped_CB1-42",
        "report_root_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001",
        "results_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001/plugin_out/FilterDuplicates_out.5",
        "sigproc_dir": "/results/analysis/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001/sigproc_results",
        "systemType": "pgm",
        "testfrag_key": "ATCG",
        "tmap_version": "tmap-f3",
        "url_root": "/output/Disabled/Auto_user_CB1-42-r9723-314wfa-tl_36_001",
        "username": "ionadmin"
    },
    "runplugin": {
        "blockId": "",
        "block_dirs": [
            "."
        ],
        "numBlocks": 1,
        "run_mode": "manual",
        "run_type": "wholechip",
        "runlevel": "default"
    },
    "sampleinfo": {
        "e5272-wfa-l165": {
            "attributes": {},
            "description": null,
            "displayedName": "e5272-wfa-l165",
            "externalId": "",
            "name": "e5272-wfa-l165"
        }
    }
}

Seq Files (BAMs)

The actual sequence information is a critical portion of all of the plugins running information. When you attempt to access them, refer to the barcodes.json file for references to their path in the “bam_filepath” key.

Output Files

The primary output of all of the plugins is the report HTML file, which is produced by the plugin. Name this file *_block.html or *_block.php. There can be any number of them, and they are all displayed in separate iFrames. If plugin output doesn’t contain a _block.html or _block.php file then all HTML/PHP files in the plugin result folder will be shown as links in the plugin section.

Additionally, the SGE produces a log file for recording the standard output of the plugin execution, which is called drmaa_stdout.txt. This contains all the information printed from the controlling script, including the standard output of the plugin itself, and is a primary source of information for debugging.

See Rendering Templates for an example using HTML templates. This usually results in cleaner code than assembling large strings or multiple-file writes.

File Permissions

The SGE executes all of the plugins as the user ‘ionian’ to perform the execution. All files produced have both the owner and group of ionian and full read/write access to the plugin result directory. This also includes the ability to create new directories. The plugins have only read access to all other files, most notably the file in the run results directory.

Upgrades

When upgrading the plugins, after all of the changes have been made to the logic of plugin, all you need to do is to increment the version of the plugin and repackage the plugin for deployment.