About DEE2

The compendium is designed to bring biologists closer to large scale gene expression data sets. We have processed thousands of public RNA-seq data sets from a veriety of organisms with open-source bioinformatics tools and make them freely accessible.


About us

The compendium is brought to you by Mark Ziemann (Deakin University) with support from the Epigenetics in Human Health and Disease Laboratory and Monash eResearch Centre. We value your feedback, so feel free to contact us by email (mark.ziemann[at]gmail.com) or raise an issue on our GitHub Repo.


Acknowledgements

A project of this size cannot be done by a single person or team, so we acknowledge support from the following:

Nectar Research Cloud, a collaborative Australian research platform supported by the National Collaborative Research Infrastructure Strategy (NCRIS).

The Multi-modal Australian ScienceS Imaging and Visualisation Environment (MASSIVE).

Deakin eResearch, Monash eResearch Centre and the IT department of Baker Heart and Diabetes Institute.

This research would not have been possible without facilities and help from NCBI. In particular, we acknowledge support from SRA and GEO for curating and hosting these data.



Latest News

5th November 2019 - the webserver has had a few hiccups recently, but all is restored to normal operation. In the next few weeks we will be trying to complete the rest of the outstanding mouse and human datasets and then moving to a new source of metadata.

We suffered an unscheduled outage 16-18th Oct 2019 due to human error (sorry!) which was compounded by hardware issues on the Nectar cloud. All seems to be working now.

19th June 2019 - The June 2019 update to DEE2 has new datasets, new features and new challenges. Highlights include new integration with Degust and iDEP and >200k new runs added for human and mouse. SRAdbV2 is now permanently dead, so DEE2 wil not be adding any new SRA runs until an alternative is found. For the complete story, read on here.

8th April 2019 - Very proud to have DEE2 published in Gigascience. Thanks to everyone who has contributed, including reviewers who made several important suggestions. Link here.

11th January 2019 - In response to feedback we are unveiling several new features:

28th November 2018 - DEE2 was presented at the ABACBS2018 conference in Melbourne. See the Poster.

28th November 2018 - A guide to loading in bulk dumps had been added to my Blog.

More news

Data processing

Our data processing procedure entails:

  1. -Download from NCBI SRA

  2. -Diagnose sequence format

  3. -Sequence quality trimming and adapter clipping

  4. -Alignment to genome and transcriptome

  5. -Assignment of reads to genes and transcripts

More information regarding the data processing method is available at the GitHub repo. Below are the versions and major parameters used in the pipeline.

Software versions and parameters used in the pipeline.
Software, version Purpose Parameter
SE PE
Aspera client, v3.5.4 Rapid download of sequence data ascp -l 500m -O 33001 -T -i $ID $URL .
SRA toolkit, v2.8.2 Validate downloaded SRA files vdb-validate $SRA
diagnose single or paired end fastq-dump -X 4000 --split-files $SRA
dump fastq (see parallel-fastq-dump below)
FastQC, v0.11.5 Diagnose basespace / colorspace, quality encoding, read length from 4000 reads fastqc $FQ1 fastqc $FQ2
parallel-fastq-dump, 0.6.3 Rapid decompression of sequence data from .sra files parallel-fastq-dump --threads $THREADS --outdir . --split-files --defline-qual + -s ${SRR}.sra
Skewer, v0.2.2 3’ quality trimming skewer -l 18 -q 10 -k inf -t $THREADS -o $SRR $FQ1 skewer -l 18 -q 10 -k inf -t $THREADS -o $SRR $FQ1 $FQ2
Adapter clipping skewer -l 18 -t $THREADS -x $ADAPTER -o $SRR $FQ1 skewer -l 18 -t $THREADS -x $ADAPTER1 -y $ADAPTER2 -o $SRR $FQ1 $FQ2
5’ trimming skewer -m ap --cut $CLIP_NUM,$CLIP_NUM -l 18 -k inf -t $THREADS $FQ1 skewer -m ap --cut $R1_CLIP_NUM,$R2_CLIP_NUM -l 18 -k inf -t $THREADS $FQ1 $FQ2
Minion, v13-100 3’ adapter detection minion search-adapter -i $FQ1 minion search-adapter -i $FQ2
Bowtie2, v2.3.2 Adapter contamination detection bowtie2 -f -x $BT2_REF -S /dev/stdout $ADAPTER
FASTX-Toolkit, v0.0.14 Progressive 5’ trimming fastx_trimmer -f {5,9,13,21} -m 18 -Q 33 -i $FQ1 fastx_trimmer -f {5,9,13,21} -m 18 -Q 33 -i $FQ2
STAR v020201 Gene-level mapping, Diagnose strandedness STAR --runThreadN $THREADS --quantMode GeneCounts \
--genomeLoad LoadAndKeep --outSAMtype None \
--genomeDir $STAR_DIR --readFilesIn=$FQ1
STAR --runThreadN $THREADS --quantMode GeneCounts \
--genomeLoad LoadAndKeep --outSAMtype None \
--genomeDir $STAR_DIR --readFilesIn=$FQ1 $FQ2
Kallisto, v0.43.1 Transcript-level mapping kallisto quant $KALLISTO_STRAND_PARAMETER \
--single -l 100 -s 20 -t $THREADS -o . \
-i $KAL_REF $FQ1
kallisto quant $KALLISTO_STRAND_PARAMETER \
-t $THREADS -o . -i $KAL_REF $FQ1 $FQ2


Reference genome information

The compendium relies on reference genome sequence and annotation information provided by Ensembl Genomes .

Species Genome Reference Sequence and Annotation
Arabidopsis thaliana Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Caenorhabditis elegans
Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Drosophila melanogaster Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Danio rerio Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Escherichia coli Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Homo sapiens Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Mus musculus Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Rattus norvegicus Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Saccharomyces cerevisiae Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)


About the quality metrics

A description of each of the quality metrics is provided on the Gitub page here.


Update schedule

We are updating the compendium fortnightly. Upon release of an updated genome build, we intend to update the data for that organism within a year, keeping a previously archived version for bulk download only. Gene annotation sets will not be updated windependent of the genome build.