Gene Expression Data
Count Matrix
A count matrix $X \in \mathbb{N}^{n \times p}$ has $X_{ij}$ as the number of observed RNA molecules from gene $j$ in sample $i$.
- rows are cells or spots
- columns are genes
For spatial transcriptomics, the rows are often spots isntead of cells, so $X \in \mathbb{N}^{\text{spots} \times \text{genes}}$.
Each spot has a physical coordinate on tissue slide $(i,j) \mapsto (x_i,y_i)$. We can view spatial transcriptomics as $(X,C,I)$ where
\[X = \text{spot} \times \text{gene count matrix}\] \[C = \text{spot coordinates}\] \[I = \text{histology image (H\&E)}\]UMIs
UMI means unique molecular identifier.
UMI is a short random barcode attached to an RNA molecule before amplification. This is to distinguish biological molecules from PCR duplicates.
PCR
When a RNA gets amplified into many sequencing reads, all reads share the same UMI so it will still maintain a single molecule count.
- read is a sequenced fragment. Each cell/spot produces many reads which are aligned or pseudoaligned to genes/transcripts.
- Read counts are more raw, whereas UMI counts are more molecule-like
Gene expression matrices are extremely sparse as
- gene is truly not expressed in cell
- RNA molecule existed but was not captured
- sequencing depth, or the total number of reads/UMIs collected for a sample, cell, or spot. Low sequencing depth means fewer molecules are observed, which could leave out a gene.
- mapping/counting pipeline failed to assign it confidently
Dropout means a gene was actually expressed is observed as zero due to technical limitations.
Library size is the total number of counts observed in a cell/spot. Motivates normalization as it gives us the relative abundance of a gene relative to the cell/spot.
Bulk RNA-seq vs single-cell RNA-seq vs Spatial Transcriptomics
Bulk RNA-seq takes many cells together and measure average expression. A row is usually a biological sample
\[X \in \mathbb{N}^{\text{samples}\times\text{genes}}\]Bulk RNA-seq gives strong, relatively stable expression measurements, but loses cell-type composition.
Single-cell RNA-seq takes per cell, so a row is
\[X \in \mathbb{N}^{\text{cells}\times\text{genes}}\]Single-cell gives cellular resolution but is noisier and sparse. It also loses spatial location, so we don't know exactly where it sat in the tissue.
and each spot has coordinates $c_i = (x_i, y_i)$ and an associated histology image patch. This preserves tissue organization, however, a spot may contain multiple cells so it could be information that is at the multicellular level.
In general, an independent unit is usually at the patient/slide/biological replicate for statistics. Without this, this could lead to pseudoreplication (statistical error).
Normalization and Transformations
Our counts matrix has per entry an integer count (how many UMIs or reads were observed per gene in a cell/spot).
Normalization separates technical scale effects from real biological differences.
Library-size normalization: CPM / CP10K
[TODO: CP10K / CPM: counts per 10k or million library-size normalization]
[TODO: log1p transform: why it compresses high-expression genes]
[TODO: Size-factor normalization: DESeq2-style median-ratio idea]
[TODO: Z-scoring genes: when useful for ML, when dangerous biologically]
[TODO: Variance stabilization and highly variable gene selection]
[TODO: What information each normalization preserves or destroys]
Differential Expression
[TODO: Goal: find genes whose expression differs across conditions]
[TODO: DESeq2: negative binomial model, size factors, dispersion]
[TODO: edgeR and limma-voom: what assumptions they make]
[TODO: Wilcoxon/Mann--Whitney: why common in single-cell, why often flawed]
[TODO: Multiple testing correction: FDR, Benjamini--Hochberg]
[TODO: Effect size vs p-value]
[TODO: Why DE can be confounded by batch, cell type, slide, patient, or library size]
Pseudobulk Analysis
[TODO: Define pseudobulk: aggregate counts by sample/patient/condition/cell type]
[TODO: Why pseudobulk fixes pseudoreplication]
[TODO: When to aggregate by patient, slide, region, cell type, or spot group]
[TODO: How pseudobulk connects single-cell data back to bulk RNA-seq tools]
[TODO: Compare pseudobulk DESeq2 vs cell-level Wilcoxon]
[TODO: Failure mode: too few biological replicates]
Batch Effects and Integration
[TODO: Define batch effects: chemistry, lab, slide, sequencing run, patient, scanner]
[TODO: Harmony: what it does conceptually]
[TODO: scVI: latent variable model for single-cell integration]
[TODO: ComBat, MNN, Scanorama: rough intuition]
[TODO: Correction for visualization/clustering vs correction for differential expression]
[TODO: Overcorrection: removing real biology]
[TODO: Undercorrection: model learns batch instead of biology]
Dimensionality Reduction and Clustering
[TODO: PCA: linear compression and denoising]
[TODO: UMAP/t-SNE: visualization, not proof of biology]
[TODO: Leiden/Louvain clustering]
[TODO: Highly variable genes and marker genes]
[TODO: Cluster annotation using known markers]
[TODO: Failure mode: clusters can reflect batch, cell cycle, or quality]
Spatial Transcriptomics
[TODO: Define spatial transcriptomics and Visium-style spots]
[TODO: Spot resolution: each spot may contain multiple cells]
[TODO: Spatial coordinates and tissue image registration]
[TODO: Spatial autocorrelation: nearby spots are not independent]
[TODO: Tissue regions, neighborhoods, gradients, and boundaries]
[TODO: Deconvolution: estimating cell-type composition per spot]
[TODO: Difference between predicting genes and predicting spatial tissue state]
Cell-Type Annotation and Deconvolution
[TODO: Marker-gene based annotation]
[TODO: Reference mapping and label transfer]
[TODO: RCTD, cell2location, Tangram: high-level purpose]
[TODO: Cell-type composition as a major driver of expression]
[TODO: Distinguish cell-intrinsic expression from cell-composition effects]
[TODO: Why H&E may predict cell types more easily than individual genes]
Pathology Image Representations
[TODO: Basic H&E stain biology: hematoxylin, eosin, nuclei, cytoplasm]
[TODO: Morphological structures: tumor, stroma, immune infiltrate, necrosis, lumen, adipose]
[TODO: Artifacts: folds, blur, pen marks, bubbles, scanner issues]
[TODO: Magnification, MPP, patch size, and physical scale]
[TODO: Tissue masking: background vs tissue vs holes/lumen/adipose]
[TODO: Color normalization and stain variation]
[TODO: What visual information might correlate with gene expression]
H&E--Gene Expression Alignment
[TODO: Image-to-expression prediction: spot-level supervision]
[TODO: Contrastive H&E--ST alignment]
[TODO: Coordinate-based baselines and spatial leakage]
[TODO: Cell-density and tissue-region shortcuts]
[TODO: Whether models learn gene regulation, cell composition, or morphology]
[TODO: How masking/reconstruction objectives may fail to learn biology]
[TODO: What a biologically meaningful representation should capture]
Evaluation Metrics for Biology Models
[TODO: Pearson and Spearman correlation for gene prediction]
[TODO: $R^2$: calibrated vs uncalibrated]
[TODO: Top-$K$ gene metrics]
[TODO: Genes beating KNN or coordinate baseline]
[TODO: Differential-expression recovery]
[TODO: Pathway enrichment recovery]
[TODO: Cell-type prediction accuracy]
[TODO: Spatial localization metrics]
[TODO: Segmentation Dice and IoU]
[TODO: Why good reconstruction does not imply good biology]
Controls, Baselines, and Leakage
[TODO: KNN baseline using spatial coordinates]
[TODO: Mean-expression and tissue-type baselines]
[TODO: Cell-density baseline]
[TODO: Color/stain-only baseline]
[TODO: Train/test split by spot, slide, patient, cancer type]
[TODO: Same-slide leakage and patient leakage]
[TODO: Batch leakage and scanner/site leakage]
[TODO: Ablations that test whether the model learned biology or shortcuts]
Biological Interpretation
[TODO: Marker genes and canonical pathways]
[TODO: GO, KEGG, Reactome pathway analysis]
[TODO: Tumor microenvironment: immune, stromal, endothelial signals]
[TODO: Ligand-receptor analysis]
[TODO: Immune infiltration and inflammation signatures]
[TODO: Necrosis, hypoxia, proliferation, EMT, angiogenesis]
[TODO: Difference between predictive features and causal biology]
Common Failure Modes
[TODO: Pseudoreplication: too many cells/spots but too few patients]
[TODO: Batch effects mistaken for biology]
[TODO: Overcorrecting away real signal]
[TODO: Learning coordinates instead of morphology]
[TODO: Learning stain/color shortcuts]
[TODO: Optimizing MAE reconstruction while failing downstream biology]
[TODO: Gene metrics dominated by easy cell-composition signals]
[TODO: Spatial autocorrelation inflating validation performance]
[TODO: Weak or noisy biological ground truth]
others
merfish maybe
illumina sequencing
To uniquely determine a gene on a genome, you need a specific sequence of about $30$ to $50$ contiguous base pairs (nucleotides).