IRIDA Uploader
Features
- Command Line interface for Linux and Windows
- Single Directory Upload
- MiSeq/MiniSeq/NextSeq Sequencing Run Parser
- Directory Parsing
- Batch Uploads
- Assemblies Uploads
- Fast5 Uploads
- Can be used with cron/windows task scheduler for automated uploads
- GUI
- Resume Uploads
Upcoming Features
- File upload checksum validation
- Post-processing tasks
Version Compatibility
IRIDA Uploader Version 0.9.1 and later require IRIDA to be at least IRIDA Version 23.01
If you are using an older version of IRIDA, you can use IRIDA Uploader Version 0.9.0, but we recommend that you speak with your IRIDA administrator to upgrade to the latest version.
Getting Started
GUI Tutorial
You can find a tutorial and walkthrough on the phac-nml github https://github.com/phac-nml/irida-uploader-tutorial
Install / Setup
Installation
Windows
You can download pre-built packages for Windows from our GitHub releases page.
Run an installer (links above) and follow along with the install wizard.
You will need to configure your uploader before running. See Configuration for details
If you prefer to build a windows installer from source code, please see the README on GitHub
Linux
If you prefer to build the uploader from source code, please see the README on GitHub
Command Line
The IRIDA Uploader requires Python version 3.7 or newer
$ python3 --version
Ensure that pip
, and setuptools
are up to date
$ pip install pip -U
$ pip install setuptools -U
Install the latest release with pip
$ pip3 install iridauploader
Run the uploader
$ irida-uploader
GUI
You can also install the GUI by specifying the GUI
extra in your pip install command
$ pip3 install 'iridauploader[GUI]'
$ irida-uploader-gui
Please Note: You will need to configure your uploader to connect to IRIDA before running. See Configuration section below.
Upload Modes
By default, the IRIDA Uploader will upload fastq sequence files, but it can also upload Assemblies (fasta), and Fast5 data
When uploading from the command line, use the --upload_mode=<mode>
, with assemblies
or fast5
to upload those file types.
See the --help
command for more details.
You can also select an upload mode when using the GUI via a drop down menu on the main screen.
Configuration
You will need to configure IRIDA and the uploader to upload files.
If you do not create a configuration file, IRIDA uploader will create one for you with default values the first time it try's to upload.
You will need to edit this file with your IRIDA credentials, and the parser that matches your data.
Choose a Parser
The config file has a parser
field that you can use to parse different directory structures
We currently support the following:
directory
: Generic Directory
miseq
/ miseq_v26
: Miseq (Control Software Version 2.6)
miseq_v31
: Miseq (Control Software Version 3.1)
miniseq
: MiniSeq
iseq
: iSeq
nextseq
: NextSeq
Starting an upload
You can upload with the following commands
Windows:
Open a Command Prompt terminal and use the iridauploader
command to upload
C:\Users\username> iridauploader -d \path\to\my\samples\
Linux:
Use the irida-uploader.sh
script included with the source code to upload.
./irida-uploader.sh -d /path/to/the/sequencing/run/
Note:
After uploading, an irida_uploader_status.info
file will be created which indicates if a run is complete, or has failed
You can delete this file to make it ready for reupload, or use the --force
option when running the uploader to ignore the status of a run directory.
Batch Uploading
You can upload a directory containing run directories by using the --batch
option when running the uploader.
Example batch upload directory:
./irida-uploader.sh --batch /path/to/BatchDirectoryToUpload/
.
└── BatchDirectoryToUpload
├── Run_1
│ ├── SampleSheet.csv
│ └── <other run files>
├── Run_2
│ ├── SampleSheet.csv
│ └── <other run files>
└── Run_3
├── SampleSheet.csv
└── <other run files>
The --force
option can be used with the --batch
option
WARNING! When uploading nextseq
data and using --batch
upload with an auto-upload script, incomplete fastq files could be uploaded if bcl2fastq
has not finished when the upload begins.
Logging
Logs about individual runs are written to the sequencing run directory that they are uploaded from.
Full debug logs are written to your system default logging directory
Linux
/home/<user>/.cache/irida-uploader/log/
Windows
C:\Users\<username>\AppData\Local\irida-uploader\irida-uploader\Logs
Read Only Mode
You can upload in read-only mode with the config option, the --readonly
/ -r
command line option, or the checkbox on the GUI.
In this mode, the status file and the logging file will not be created in the sequencing run directory during upload.
Continue Partial/Failed Uploads
If an upload fails to complete upload for any reason, the upload can be continued from either the GUI (by selecting continue upload) or with the command line by using the --confingue_partial
argument.
You can see details about which files have been uploaded by viewing the status file generated in your sequencing run directory.
Note: --continue_partial
and --force
are mutually exclusive, as --force
indicates that a run should be restarted
Problems?
Problems uploading?
Check the Errors Page for help with common errors.
Found a bug or have an idea for a feature?
Create an issue on our GitHub Issues Board
Developers
The modules of the IRIDA Uploader can be used through pip/pypi
pip install iridauploader
Example to get you started:
import iridauploader.api as api
import iridauploader.parsers as parsers
api_instance = api.ApiCalls(client_id, client_secret, base_url, username, password, timeout_multiplier)
parser_instance = parsers.parser_factory("miseq")
Want to create a parser for a sequencer that we don't yet support or have an idea for an IRIDA related project?