About DEE2

The compendium is designed to bring biologists closer to large scale gene expression data sets. We have processed thousands of public RNA-seq data sets from a veriety of organisms with open-source bioinformatics tools and make them freely accessible.


About us

The compendium is brought to you by Mark Ziemann (Deakin University) with support from the Epigenetics in Human Health and Disease Laboratory and Monash eResearch Centre. We value your feedback, so feel free to contact us by email (mark.ziemann[at]gmail.com) or raise an issue on our GitHub Repo.


Acknowledgements

A project of this size cannot be done by a single person or team, so we acknowledge support from the following:

Nectar Research Cloud, a collaborative Australian research platform supported by the National Collaborative Research Infrastructure Strategy (NCRIS).

The Multi-modal Australian ScienceS Imaging and Visualisation Environment (MASSIVE).

Deakin eResearch, Monash eResearch Centre and the IT department of Baker Heart and Diabetes Institute.

This research would not have been possible without facilities and help from NCBI. In particular, we acknowledge support from SRA and GEO for curating and hosting these data.



Latest News

11th January 2019 - In response to feedback we are unveiling several new features:

28th November 2018 - DEE2 was presented at the ABACBS2018 conference in Melbourne. See the Poster.

28th November 2018 - A guide to loading in bulk dumps had been added to my Blog.

12th November 2018 - Ever wondered how accurate DEE2 data actually is? Well we have just undertaken a simulation study and comparison to GEO deposited data to demonstrate the quality of DEE2 data. More details here.

26th October 2018 - Recently there have been many projects with ≥500 SRA runs, meaning that they have not been readily available from the webserver as one file. In order to address this, I've packaged these SRA projects with ≥200 SRA run numbers into zip files. Only projects with EVERY run processed successfully by DEE2 are included. These are now available here.

More news

Data processing

Our data processing procedure entails:

  1. -Download from NCBI SRA

  2. -Diagnose sequence format

  3. -Sequence quality trimming and adapter clipping

  4. -Alignment to genome and transcriptome

  5. -Assignment of reads to genes and transcripts

More information regarding the data processing method is available at the GitHub repo. Below are the versions and major parameters used in the pipeline.

Software versions and parameters used in the pipeline.
Software, version Purpose Parameter
SE PE
Aspera client, v3.5.4 Rapid download of sequence data ascp -l 500m -O 33001 -T -i $ID $URL .
SRA toolkit, v2.8.2 Validate downloaded SRA files vdb-validate $SRA
diagnose single or paired end fastq-dump -X 4000 --split-files $SRA
dump fastq (see parallel-fastq-dump below)
FastQC, v0.11.5 Diagnose basespace / colorspace, quality encoding, read length from 4000 reads fastqc $FQ1 fastqc $FQ2
parallel-fastq-dump, 0.6.3 Rapid decompression of sequence data from .sra files parallel-fastq-dump --threads $THREADS --outdir . --split-files --defline-qual + -s ${SRR}.sra
Skewer, v0.2.2 3’ quality trimming skewer -l 18 -q 10 -k inf -t $THREADS -o $SRR $FQ1 skewer -l 18 -q 10 -k inf -t $THREADS -o $SRR $FQ1 $FQ2
Adapter clipping skewer -l 18 -t $THREADS -x $ADAPTER -o $SRR $FQ1 skewer -l 18 -t $THREADS -x $ADAPTER1 -y $ADAPTER2 -o $SRR $FQ1 $FQ2
5’ trimming skewer -m ap --cut $CLIP_NUM,$CLIP_NUM -l 18 -k inf -t $THREADS $FQ1 skewer -m ap --cut $R1_CLIP_NUM,$R2_CLIP_NUM -l 18 -k inf -t $THREADS $FQ1 $FQ2
Minion, v13-100 3’ adapter detection minion search-adapter -i $FQ1 minion search-adapter -i $FQ2
Bowtie2, v2.3.2 Adapter contamination detection bowtie2 -f -x $BT2_REF -S /dev/stdout $ADAPTER
FASTX-Toolkit, v0.0.14 Progressive 5’ trimming fastx_trimmer -f {5,9,13,21} -m 18 -Q 33 -i $FQ1 fastx_trimmer -f {5,9,13,21} -m 18 -Q 33 -i $FQ2
STAR v020201 Gene-level mapping, Diagnose strandedness STAR --runThreadN $THREADS --quantMode GeneCounts \
--genomeLoad LoadAndKeep --outSAMtype None \
--genomeDir $STAR_DIR --readFilesIn=$FQ1
STAR --runThreadN $THREADS --quantMode GeneCounts \
--genomeLoad LoadAndKeep --outSAMtype None \
--genomeDir $STAR_DIR --readFilesIn=$FQ1 $FQ2
Kallisto, v0.43.1 Transcript-level mapping kallisto quant $KALLISTO_STRAND_PARAMETER \
--single -l 100 -s 20 -t $THREADS -o . \
-i $KAL_REF $FQ1
kallisto quant $KALLISTO_STRAND_PARAMETER \
-t $THREADS -o . -i $KAL_REF $FQ1 $FQ2


Reference genome information

The compendium relies on reference genome sequence and annotation information provided by Ensembl Genomes .

Species Genome Reference Sequence and Annotation
Arabidopsis thaliana Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Caenorhabditis elegans
Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Drosophila melanogaster Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Danio rerio Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Escherichia coli Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Homo sapiens Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Mus musculus Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Rattus norvegicus Ensembl release 90
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)
Saccharomyces cerevisiae Ensembl release 36
Genome sequence (fasta)
Gene annotation set (GTF)
cDNA sequences (fasta)


About the quality metrics

A description of each of the quality metrics is provided on the Gitub page here.


Update schedule

We are updating the compendium fortnightly. Upon release of an updated genome build, we intend to update the data for that organism within a year, keeping a previously archived version for bulk download only. Gene annotation sets will not be updated windependent of the genome build.