IRIDA Uploader

Features

  • Command Line interface for Linux and Windows
  • Single Directory Upload
  • MiSeq/MiniSeq/NextSeq Sequencing Run Parser
  • Directory Parsing
  • Batch Uploads
  • Assemblies Uploads
  • Fast5 Uploads
  • Can be used with cron/windows task scheduler for automated uploads
  • GUI

Upcoming Features

  • File upload checksum validation
  • Post-processing tasks
  • Pause and resume uploads

Getting Started

GUI Tutorial

You can find a tutorial and walkthrough on the phac-nml github https://github.com/phac-nml/irida-uploader-tutorial

Install / Setup

Installation

Windows

You can download pre-built packages for Windows from our GitHub releases page.

Run an installer (links above) and follow along with the install wizard.

You will need to configure your uploader before running. See Configuration for details

If you would prefer to build a windows installer from source code, please see the README on GitHub

Linux

The IRIDA Uploader requires Python version 3.5 or newer

$ python3 --version

Ensure that pip, and setuptools are up to date

$ pip install pip -U
$ pip install setuptools -U

Install the latest release with pip

$ pip3 install iridauploader

Run the uploader

$ irida-uploader

You can also install and run the GUI in the same way

$ pip3 install iridauploader[GUI]
# irida-uploader-gui

If you would prefer to build the uploader from source code, please see the README on GitHub

Please Note: You will need to configure your uploader to connect to IRIDA before running. See Configuration section below.

Upload Modes

By default, the IRIDA Uploader will upload fastq sequence files, but it can also upload Assemblies (fasta), and Fast5 data

When uploading from the command line, use the --upload_mode=<mode>, with assemblies or fast5 to upload those file types.

See the --help command for more details.

You can also select an upload mode when using the GUI via a drop down menu on the main screen.

Configuration

You will need to configure IRIDA and the uploader to upload files.

How to configure

If you do not create a configuration file, IRIDA uploader will create one for you with default values the first time it try's to upload.

You will need to edit this file with your IRIDA credentials, and the parser that matches your data.

Choose a Parser

The config file has a parser field that you can use to parse different directory structures

We currently support the following:

directory : Generic Directory

miseq / miseq_v26 : Miseq (Control Software Version 2.6)

miseq_v31 : Miseq (Control Software Version 3.1)

miniseq : MiniSeq

iseq : iSeq

nextseq : NextSeq

Starting an upload

You can upload with the following commands

Windows:

Open a Command Prompt terminal and use the iridauploader command to upload

C:\Users\username> iridauploader -d \path\to\my\samples\

Linux:

Use the irida-uploader.sh script included with the source code to upload.

./irida-uploader.sh -d /path/to/the/sequencing/run/

Note:

After uploading, an irida_uploader_status.info file will be created which indicates if a run is complete, or has failed

You can delete this file to make it ready for reupload, or use the --force option when running the uploader to ignore the status of a run directory.

Batch Uploading

You can upload a directory containing run directories by using the --batch option when running the uploader.

Example batch upload directory:

./irida-uploader.sh --batch /path/to/BatchDirectoryToUpload/

.
└── BatchDirectoryToUpload
    ├── Run_1
    │   ├── SampleSheet.csv
    │   └── <other run files>
    ├── Run_2
    │   ├── SampleSheet.csv
    │   └── <other run files>
    └── Run_3
        ├── SampleSheet.csv
        └── <other run files>

The --force option can be used with the --batch option

WARNING! When uploading nextseq data and using --batch upload with an auto-upload script, incomplete fastq files could be uploaded if bcl2fastq has not finished when the upload begins.

Logging

Logs about individual runs are written to the sequencing run directory that they are uploaded from.

Full debug logs are written to your system default logging directory

Linux

/home/<user>/.cache/irida-uploader/log/

Windows

C:\Users\<username>\AppData\Local\irida-uploader\irida-uploader\Logs

Read Only Mode

You can upload in read-only mode with the config option, the --readonly / -r command line option, or the checkbox on the GUI.

In this mode, the status file and the logging file will not be created in the sequencing run directory during upload.

Problems?

Problems uploading?

Check the Errors Page for help with common errors.

Found a bug or have an idea for a feature?

Create an issue on our GitHub Issues Board

Developers

The modules of the IRIDA Uploader can be used through pip/pypi

pip install iridauploader

Example to get you started:

import iridauploader.api as api
import iridauploader.parsers as parsers

api_instance = api.ApiCalls(client_id, client_secret, base_url, username, password, max_wait_time)
parser_instance = parsers.parser_factory("miseq")

Want to create a parser for a sequencer that we don't yet support or have an idea for an IRIDA related project?

Requirements for new parsers

Information on the IRIDA python API

Object Model Reference

Cloud Deployment