Nextflow Data Processing & Configuration
Our pipeline configuration is still in-development, and the contents of this document are subject to change.
Summary of Processing
Datatype | Method | Output | Tried yet? |
---|---|---|---|
WES or WGS | DeepVariant | Yes | |
WES or WGS | Strelka, Mutect2 | Yes | |
WES or WGS | TBD | Germline and Somatic Structural Variants | No |
WES or WGS | TBD | Germline and Somatic CNV | No |
WES or WGS | TBD | Tumor MSI | No |
SNV, INDEL variants | TBD | No |
Datatype | Method | Output | Tried yet? |
---|---|---|---|
RNA-Seq | STAR with Salmon in alignment-based mode | Yes |
More information about the workflows is available here: https://sagebionetworks.jira.com/wiki/spaces/WF/pages/2363359776
bam/cram to fastq conversion
When fastq files are not available, cram/bam files are converted to fastq using this pipeline: GitHub - nf-core/bamtofastq: Converts bam or cram files to fastq format and does quality control. (v1.2.0).
If unaligned bam files are available instead of fastq files, we recommend providing u-bam files for direct input to sarek 3.0.
WES and WGS Variant Calling (SNV & INDEL)
Germline SNV + INDEL
This involves transformation of WES fastq
or cram
files to variant call files in VCF format (.vcf
files).
As of Jan 2022, the reference genome
used is Homo_sapiens/GATK/GRCh38
(sarek/conf/igenomes.config at 2.7.1 · nf-core/sarek)
The processing steps include the following:
Raw fastq files uploaded to Synapse by researcher in a folder with name format
experiment_name_rnaseq_fastq_date
. No white space should be present in the filenames (all filenames should have_
for whitespaces.All experiment and sample related annotations need to be added on Synapse before processing can start. This is a required step so that a sample sheet can be generated to trigger the processing workflow
The sample sheet should contain the following information in a comma-separated file (
.csv
) with at least 3 columns, and a header row as shown below. (More information here)
sample | subject | status | sex | file_1 | file_2 | lane | parentId | bed_file | output_parent_Id |
Synapse specimenID | Synapse individualID | 1 (Tumor = 1, Normal 0) | XX or XY |
|
| Lane information | SynapseID of parent folder | Synapse ID of BED file (if WES sata) | Synapse ID of folder where all processed files will be indexed |
The files are pulled into NextFlow workflow setup and processed using the following versions of software:
nf-core/sarek v2.7.1
Nextflow v21.10.5
BWA 0.7.17
GATK v4.1.7.0
FreeBayes v1.3.2
samtools v1.9
Strelka v2.9.10
Manta v1.6.0
TIDDIT v2.7.1
AlleleCount v4.0.2
ASCAT v2.5.2
Control-FREEC vv11.6
msisensor v0.5
SnpEff v4.3t
VEP v99.2
MultiQC v1.8
FastQC v0.11.9
bcftools v1.9
CNVkit v0.9.6
htslib v1.9
QualiMap v2.2.2-dev
Trim Galore v0.6.4_dev
vcftools v0.1.16
R v4.0.2
Commands used for running JHU samples on DeepVariant:
All files and sample sheets are first staged in S3 buckets linked to NFTower. then the following command are used to launch the processing pipeline.
Params
:
input: s3://jhu-biobank-nf-project-tower-bucket/jobs/02-sage-sarek-2.7.1-deepvariant/inputs/sample-sheet.tsv
outdir: s3://jhu-biobank-nf-project-tower-bucket/jobs/02-sage-sarek-2.7.1-deepvariant/outputs/
genome: GRCh38
igenomes_base: s3://sage-igenomes/igenomes
model_type: WES
tools: "deepvariant"
Config
:
process {
errorStrategy = 'retry'
maxRetries = 3
withLabel:deepvariant {
container = "google/deepvariant:1.1.0"
cpus = 24
}
}
Pre-run Script
:
Profiles
:
Somatic SNV + INDEL
TBD
Annotated Variants
Currently, germline variant calls in VCF format are being processed manually using VEP and vcf2maf
RNA Sequencing Data Quantification
Processing RNA-seq files involve transformation of raw data (fastq
files) to transcript counts (quants.sf
files).
The quantification software of choice is Salmon
.
As of Jan 2022, the reference genome
used is Homo_sapiens/NCBI/GRCh38
.
Processing involves the following steps:
Raw fastq files uploaded to Synapse by researcher in a folder with name format
experiment_name_rnaseq_fastq_date
. No white space should be present in the filenames (all filenames should have_
for whitespaces. While the naming convention is a best practices recommendation and not a strict rule, the exclusion of whitespaces is required.All experiment and sample related annotations need to be added on Synapse before processing can start. This is a required step so that a sample sheet can be generated to trigger the processing workflow
The sample sheet should contain the following information in the following format (saved as a
.csv
file) (More information here) :
sample | single_end | fastq_1 | fastq_2 | strandedness |
Synapse specimenID | 0 (1 if paired-end) | synID | synID | auto |
The files are pulled into NextFlow workflow setup and processed using the following versions of software:
Command used to process JHU Biobank samples:
Params
:
Config
:
Pre-run script
:
Profile
: