Notes on modeling, computation, and biology.
camdl is a domain-specific language and inference framework for stochastic compartmental models. The compiler does the bookkeeping — units, dimensions, stratification, symbolic gradients — so the modeller writes only the model. A tour of the design and the alpha release.
The Great Obsession of population geneticists, to borrow John Gillespie's words, is genetic variation. As an evolutionary biologist, it's rather hard to not be obsessed with genetic variation, for it's the ultimate source of the two most striking features of life on earth: the mind-boggling diversity of species, and adaptations so utterly clever they look as though they were assembled by a designer. Both life's dizzying diversity and cunning adaptations are the result of evolutionary processes like natural selection and numerous historical accidents, overlaid on one another...
Darwin thought natural selection was slow, but we now know it can act rapidly. The challenge is detecting this selection at the DNA level—especially when adaptation involves many genes of small effect acting in concert.
The last chapter of my dissertation with Graham Coop was recently published in PNAS last week. In an effort to communicate my research to a broader audience, I have written two blog posts on our work. The first post, is meant to introduce the historical context and concepts like linked selection and polygenic adaptation to a non-scientist, and the second post, below, describes our work on temporal covariance as a signature of polygenic linked selection and its application to four evolve-and-reseqeunce data sets.
Snakemake automates the tedious task of re-running data processing pipelines when upstream files change. This tutorial explains how Snakemake works by first covering its predecessor, Make, and the core idea of dependency graphs.
The X chromosome has a peculiar inheritance pattern: it's passed from mothers to all children, but fathers only pass it to daughters. This creates an embedded genealogy of X ancestors that affects how segments of DNA are shared between relatives.
R's apply and table functions are flexible but slow for counting genotypes in large matrices. A simple Rcpp function provides a 10x speedup by specializing for the 0/1/2/NA genotype encoding.
The MD tag in BAM files encodes information about the reference sequence at aligned positions, allowing SNP and indel calling without the reference. These notes work through several examples of how MD tags interact with CIGAR strings.
Unix pipes are elegant, but they don't work when programs have multiple inputs or outputs. Named pipes and process substitution solve this problem, and the performance gains over writing intermediate files to disk are substantial.
Bioinformatics involves a tremendous amount of interfacing between programs, yet this receives surprisingly little design attention. Better interfaces—through foreign function interfaces, shared libraries, and cleaner design patterns—could save us from the current mess of file parsing and system calls.
Science is increasingly data-driven, yet there's a troubling asymmetry in how we're taught to handle data versus how we actually should. The rise of open data offers tremendous opportunities, but realizing its potential requires rethinking how we train scientists.
If you can visualize data quickly, problems look different. Learning to make plots without friction means you'll use visualization as a debugging tool, catch confounders early, and notice mistakes that would otherwise go undetected.
A practical design pattern for parallelizing bioinformatics file processing using find, basename, and xargs. This approach scales well for data-parallel tasks across multiple cores on a single server.
A tutorial on loading 23andme genotype data into R and using Bioconductor's gwascat and GenomicRanges packages to explore GWAS associations, find SNPs in genes like APOE, and visualize risk alleles.
Notes on Git internals and workflows, covering remote tracking branches, the relationship between fetch and pull, and common use cases with illustrative examples.
Bioconductor solves several hard problems in reproducible bioinformatics: versioned software, packaged data, and strict coding standards. As genomics data grows, these problems will only get worse—Bioconductor offers a way forward.
Julia is a fast, promising new language for technical computing. But challenging R means replicating not just the language, but the entire ecosystem of CRAN and Bioconductor packages—a massive undertaking.
Machine learning is powerful, but it's easy to let its pattern-finding abilities bias our understanding of complex systems. Coming from the social sciences, where data mining was frowned upon, I've had to recalibrate—but the old caution about confusing prediction with understanding still holds.
When sequencing data is severely contaminated, finding the culprit can be tricky. Using k-mer frequency analysis and Kullback-Leibler divergence, we can identify contamination patterns that would otherwise be invisible.
Open source software benefits from a large, diverse user base, but this growth brings growing pains. Being rude to users doesn't help anyone—it hurts OSS adoption, poisons collaboration, and ultimately damages science.