Software

open nebula

The Open Nebula cluster is currently being used to run a variety of infrastructure services for UK Med-Bio including the help-desk software (RT) and web access gateways. In addition, it enables us to provide VMs to serve specific services on behalf of UK Med-Bio, both internally and externally available. It is also used to offer sand-boxed development and pre-production environments for our software developers.

Examples of current internal-use hosted VMs include:

• Quality Assurance workflows for the National Phenome Centre.
• An interactive login VM, supporting bioinformatics practical teaching sessions for the MSc course in Genomic Medicine.

 Externally available resources hosted on our Open Nebula cluster include:

• CorrMapper, written by Dr Daniel Homola in the Department of Surgery and Cancer, helps to explore, integrate and visualise the data of complex biological studies. It requires one or two OMICS datasets, along with a metadata table, which holds clinically or biologically relevant information about these samples.

A full list of available services is available from the Tools and services webpage.

Galaxy

We will shortly be making available our own local Galaxy server VM. Galaxy is a widely used workflow and pipeline system, developed for bioinformatics applications. Workflows can be created, re-run and shared via a web interface, and the open source project has a very active user community. You can find out more information about the Galaxy project itself from its site. Initially, we will focus on common NGS workflows (e.g. for primary RNA-Seq analyses) but can add specific software and additional workflows from the Galaxy ToolShed as requested.

This has been configured to allow large-scale analyses as follows:

• support for multi-threading jobs
• a large selection of centrally-maintained reference databases available
• use of Pulsar to push jobs requiring large compute resources (RAM, number of cores) out to our nodes on the HPC cluster
• mounting of additional scratch working space

If you are interested in developing and running your own Galaxy workflows, or want a private instance for a specific project, please get in touch.  We can offer a ‘vanilla’ Galaxy VM suitable for development of specific project workflows, particularly if they require other systems such as Docker.

other servers

Other servers are currently in use as standalone interactive work nodes for guaranteed fast turnaround processing of specific jobs, and for profiling large jobs (cores, RAM, runtime),  before resources are allocated for repeated runs on HPC resources. 

Many types of analysis require access to reference databases and curated datasets which frequently are large, requiring considerable download times and disk utilisation, and typically require indexing using a method appropriate for the desired analysis method. The indexing process itself will also frequently take over 12 hours for a mammalian genome. Frequently, the whole process can take 2-3 days, and is generally carried out by every researcher needing to carry out such analysis requiring reference datasets.

To help reduce the overhead associated with obtaining and preparing reference datasets, we are establishing a centralised collection of reference datasets which are available to researchers across the college. These are accessible on UK Med-Bio, HPC and BDSG systems, and remotely via NFS and HTTP.

Databases are automatically updated weekly where appropriate.

The following datasets are currently available:

•     Illumina iGenomes (reference database indexed for BWA, bowtie and bowtie2 searches; fasta format chromosome files; annotations as e.g. GTF files). A full list of the datasets available through iGenomes is available at http://support.illumina.com/sequencing/sequencing_software/igenome.html. Of these, we currently hold:
  • Human GRCh37 (Ensembl annotation)
  • Human GRCh38 (NCBI annotation)
  • Mouse GRCm38 (Ensembl annotation)
  • C. elegans WS220 (Ensembl annotation)
  • E. coli K12 DH10B (Ensembl annotation)
• Human genome resources: Additional reference datasets for related to the Human genome
  • 1000 genomes variant calls
  • GATK resource bundle (including dbSNP, 1000 genomes, hapmap, gold standard indels preformatted for use with the Genome Analysis ToolKit (GATK)
• Kegg pathway databases - available to registered users only
• Blast databases:
  • NCBI NR
  • NCBI NT
  • RefSeq Genomic
  • Refseq RNA
  • Refseq protein
  • Uniprot Swissprot
  • Uniprot Trembl

The current holdings are a minimal set we believe to be frequently used at present. If you require access to a reference datasets not listed here, please contact medbio-help@imperial.ac.uk and let us know of your requirements.