Skip to content

DATA.md

Latest commit

 

History

History
79 lines (59 loc) · 2.82 KB

DATA.md

File metadata and controls

79 lines (59 loc) · 2.82 KB
Raw

Data

What is and is not in this repository
Included per-cell average-precision tables, gzipped, under data/aggregated/
Not included — rebuildable the GTEx whole-blood expression matrix used by the real-TF stages (see below)
Not included — not redistributable all cohort application data (LLFS, FHS, HRS, WHI) and the analysis code specific to it. Access is governed by dbGaP / the individual studies.

Nothing in this repository contains individual-level human data.

data/aggregated/ — the shipped results

256 gzipped TSVs, one per benchmark cell, named exactly as the cell: stage{1..4}_map_{method}[_{form}][_v2][_{variant}].txt.gz. Columns:

column meaning
runid grid row (4-digit, zero-padded)
repid replicate, 0-indexed
AP average precision for that replicate
MAP mean AP over the replicates of that runid
samples, betas, noise the design point, joined from the grid file

These are what every figure script reads, so the published figures can be regenerated without re-running the grid:

source config.sh
mkdir -p $CIDER_OUT/aggregated
for f in data/aggregated/*.gz; do gunzip -c "$f" > "$CIDER_OUT/aggregated/$(basename "${f%.gz}")"; done

sim/grids/cells.tsv maps each cell name back to the command that produced it.

Rebuilding the GTEx whole-blood input

The real-TF stages (2 and 4) draw their TF expression from GTEx v10 whole blood. Two files are needed, both under $CIDER_GTEX_DIR:

file contents
gtex_v10_wholeblood_3cpm_25perc_log1tmmcpm.tsv genes × 803 donors; index Name = versioned Ensembl gene id; log1p(TMM-CPM)
..._annt.tsv gene_id, gene_id_stable, HGNC, is TF

Inputs (all public, none redistributed here)

source where
GENCODE v39 comprehensive annotation GTF https://www.gencodegenes.org/human/release_39.html
GTEx v10 whole-blood gene reads GCT https://gtexportal.org/home/downloads/adult-gtex/bulk_tissue_expression
Lambert et al. (2018) human TF list http://humantfs.ccbr.utoronto.ca/

Pipeline

  1. GENCODE v39 protein-coding gene models; exclude chrM and PAR; deduplicate by gene name keeping the longest.
  2. GTEx v10 whole-blood read counts, subset to those gene ids.
  3. Filter: ≥ 3 CPM (raw CPM) in ≥ 25 % of samples.
  4. TMM-CPM (edgeR), then log1p.
  5. Annotation: HGNC symbol from GENCODE; is TF = stable Ensembl id present in the Lambert list (834 TFs).
source config.sh
python sim/python/build_gtex_wholeblood_input.py \
    --gtf gencode.v39.annotation.gtf.gz \
    --counts gene_reads_v10_whole_blood.gct \
    --lambert Lamberts.csv \
    --outdir $CIDER_GTEX_DIR

Requires pyranges and the qtl package (qtl.io, qtl.norm) in addition to the base Python dependencies.