Geneformer Atlas
Interactive SAE mechanistic interpretability exploration for Geneformer, focused on feature-level biological semantics and circuit inspection.
Open atlasBiological foundation models are the best model organisms mechanistic interpretability has, they carry capability their outputs never expose, and their internals hold biology nobody has written down yet. Biodyn acts on all three, and publishes what survives the controls.
Better virtual cells, fundamental interpretability, biosecurity, and automating the work itself. Each track feeds the others.
Read biological knowledge out of model internals and make the models themselves better stand-ins for real cells. Gene regulatory network inference, perturbation response, and cell-state structure, with the extracted result checked against CRISPRi and Perturb-seq rather than against the model that produced it. This is also where we publish the cases in which a foundation model adds nothing over a simple baseline.
Treat biological models as model organisms for interpretability itself. They are small enough to trace exhaustively and arrive with ground truth nobody constructed for a method's benefit, so a technique that survives donor-disjoint splits, covariate-matched nulls and perturbation screens here has survived something a synthetic task cannot supply.
Measure what a model knows, not only what it says. The distance between the sanctioned output and the best internal layer, and how much of that distance a modest fine-tuning budget recovers, are the numbers a capability audit actually needs. A released checkpoint is a starting point, not a fixed capability.
Automate the research loop, from data ingestion and quality control through experiment design, execution, evaluation and reporting. Coordinated agents run the repetitive work and screen hypotheses at a scale nobody would attempt by hand; people supply the scientific steering.
Five stages, and the fifth is what pays for the first.
Reading a biological model from the inside produces different things for different people. Pick the one you are, and the answer is the object beside it.
Interactive atlas modules for sparse autoencoder (SAE) feature analysis across Geneformer, scGPT, Novae, C2S-Scale, and MaxToki.
Interactive SAE mechanistic interpretability exploration for Geneformer, focused on feature-level biological semantics and circuit inspection.
Open atlasInteractive SAE mechanistic interpretability exploration for scGPT, including atlas views for feature behavior across biological contexts.
Open atlasInteractive SAE mechanistic interpretability exploration for Novae, with atlas views for feature structure and biological program organization.
Open atlasInteractive SAE mechanistic interpretability exploration for C2S-Scale-Gemma-2-2B, a Cell2Sentence language model that reads each cell as a rank-ordered sentence of gene names, across a 13-layer subset.
Open atlasInteractive SAE mechanistic interpretability exploration for MaxToki-217M, decomposing twelve layers into features and grouping them into co-activation modules.
Open atlasPreprints, papers, and public research outputs from the Biodyn pipeline.
We collaborate with academic labs and also maintain technical exchanges around models developed externally. Where noted as a research engagement, this reflects discussion and input rather than formal co-development.
Academic groups with whom we collaborate directly on biological foundation models and adjacent interpretability questions.
Collaborative research around biological foundation models, network biology, and mechanistic interpretability in the Geneformer ecosystem.
Collaborative research around spatial foundation models and interpretable analysis for spatial transcriptomics and tissue organization.
Independent mechanistic interpretability work informed by direct discussion and feedback from the teams behind the models.
We are applying our mechanistic interpretability toolkit to GenBio-PathFM, a histopathology foundation model by GenBio AI. The work benefits from discussion and input from the team while remaining an independent interpretability effort.
We are applying our mechanistic interpretability toolkit to Nucleotide Transformer. This work has been informed by direct exchange with the team while remaining separate from model development.
