Welcome to the GDC Documentation!

This project provides tools and pipelines for interacting with the Genomic Data Commons (GDC). It is designed to run efficiently on high-performance clusters like the UMN MSI Agate cluster.

Contents:

• Installation
  • Quick Start (MSI HPC or Sandbox)
  • Automatic Software Installation
  • Prerequisite Software
  • Software Loading Methods
  • Software Environment Summary
  • External Dependencies
  • Software Environment Files
  • Container Images
  • Troubleshooting
• New HPC Setup Guide
  • Option 1: Clone and Use Directly
  • Option 2: Create a Custom Module
  • Quick Reference: New HPC Setup Checklist
  • Creating a Custom SLURM Script
  • Troubleshooting
  • See Also
• Genomics
  • Standard Procedure
  • Internal PCA Methods
  • File Naming Conventions
  • Technical Implementation
• Usage
  • Software Environment Setup
  • Workflow Overview
  • Configuration
  • Available Profiles
  • Running the Pipeline
  • Advanced Options
  • Master SLURM Job
  • Custom SLURM Script
• Tutorial: Quality Control Pipeline in Practice
  • Prerequisites
  • Lab Exercise: Running QC Pipeline
  • Interpreting Pipeline Outputs
  • Discussion Points: Multi-Ancestry and Admixed Study Designs
  • Data Types Reference
  • Next Steps
  • Lab Materials
• Tutorial: Ancestry Classification in Practice
  • Prerequisites
  • Lab Exercise: Running Ancestry Classification
  • Interpreting Pipeline Outputs
  • Discussion Points
  • Next Steps
  • Lab Materials
• Tutorial: Assembling 1000 Genomes Reference Data
  • Prerequisites
  • Lab Exercise: Assembling 1kG Reference Panel
  • Understanding the Processing Steps
  • Pipeline Outputs
  • Discussion Points
  • Next Steps
• Tutorial: SNP Heritability Estimation
  • Prerequisites
  • Lab Exercise: SNP Heritability Estimation
  • Interpreting Results
  • Exploration Exercises
  • Discussion Points
  • Next Steps
• Tutorial: Phenotype Simulation and Heritability Estimation
  • Prerequisites
  • Phenotype Simulation Overview
  • Lab Exercise: Phenotype Simulation
  • Simulation Parameters
  • Interpreting Results
  • Exploration Exercises
  • Discussion Points
  • Next Steps
• Developer Guide: Computing Environment Abstraction
  • Overview
  • Current Environments
  • Wrapper Script Structure
  • Modifying for Different Snakemake Versions
  • Profile Configuration
  • Creating a New Environment
  • Module File Integration
  • Common Snakemake Arguments
  • Testing Changes
  • Troubleshooting
  • See Also
• Linux / vi cheatsheet
  • vi commands
  • Bash terminal
• Overview
• Genome formats
  • Relatedness formats
  • Ancestry formats
  • Local ancestry (RFMix) formats
  • Association testing formats
  • Heritability formats
  • Phenotype & simulation formats
  • PRS intermediate formats
  • R object formats
  • Other useful text formats
  • Completion sentinel files
  • See also
• Overview
• Table legend
• Data format conversion
• Sample and variant selection
• Missingness
• Allele frequency
• Hardy-Weinberg equilibrium
• Heterozygosity / inbreeding
• Sex check
• LD pruning
• Relatedness
• PLINK2 flag quick-reference
• See also

Getting Started

Recommended Learning Pathway:

1. Installation - Set up software environment

2. Usage - Learn how to run the pipeline

3. Tutorial: Assembling 1000 Genomes Reference Data - Download reference data

4. Tutorial: Quality Control Pipeline in Practice - Run quality control

5. Tutorial: Ancestry Classification in Practice - Classify ancestry

6. Tutorial: Phenotype Simulation and Heritability Estimation - Simulate phenotypes for method testing

7. Tutorial: SNP Heritability Estimation - Estimate heritability (with real phenotypes)

Quick Setup (MSI/UMN HPC):

module use /path/to/GDCGenomicsQC/envs
module load gdcgenomicsMSI
conda activate snakemake
cd GDCGenomicsQC
snakemake --version

See also: Usage for detailed instructions on running the pipeline with module load or local snakemake.

Indices and tables

• Index

• Module Index

• Search Page