Skip to content
Lotfollahi-labPublic

About

[NeurIPS 2026] Generative model for de novo design of regulatory DNA sequences, using property-guided generation to reach out-of-distribution objectives within a lab-in-the-loop design cycle

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

LocoGen

Locality-controlled guidance that steers a frozen masked diffusion language model toward high-activity enhancer DNA β€” no fine-tuning, no reward model training.

NeurIPS 2026 License: CC BY-NC-SA 4.0 Checkpoints

LocoGen overview: the separator identifies invariant regions, property guidance and an off-distribution push are applied locally, and the generated sequence preserves invariant regions while diversifying the background. Right: three-axis radar plot and balanced score against MDLM, MDLM+CG, DNA-Diff, DRAKES and SVDD.


πŸ“– Table of Contents


πŸš€ About the Project

Locality-controlled guidance for masked discrete diffusion of regulatory DNA. LocoGen samples enhancer sequences from a frozen MDLM (Sahoo et al., 2024) backbone β€” no fine-tuning β€” using a V-REx-trained masked separator to localize the guidance gradient to invariant regulatory positions.

The separator learns where a sequence must be left alone: positions whose contribution to measured activity is invariant across environments. Guidance and the off-distribution push apply only outside them, so sequences keep their regulatory grammar while the background diversifies β€” which is what lets property, novelty and plausibility improve together instead of trading off.

This repository trains the separator (Stage 1), samples with LocoGen on a DRAKES MDLM backbone (Stage 2), and reproduces the three-axis benchmark (Stage 3).


✨ Key Features

  • No fine-tuning β€” the generative backbone stays frozen; nothing in it is retrained.
  • V-REx masked separator β€” finds environment-invariant positions and confines the guidance gradient to them.
  • Localized OOD push β€” --ood_sil_localized damps only separator-background positions.
  • Cell-type aware β€” HepG2, K562 and SK-N-SH separators, checkpoints on the Hugging Face Hub.
  • Reproducible benchmark β€” property (Pred-activity, ATAC_acc), novelty (Authenticity, Rarity) and plausibility (|Ξ”Gini|, |Ξ”hits|) under scripts/eval/, with a manifest of the runs behind the reported numbers.

πŸ›  Getting Started

Prerequisites

  • Python β‰₯ 3.11, < 3.15
  • Poetry
  • A CUDA GPU β€” sampling and both reward oracles run on GPU
  • ~6 GB free disk β€” the DRAKES archive is ~5.3 GB zipped
  • Git β€” the DRAKES backbone is cloned, not vendored here

Installation

1. Install the package

pip install poetry
poetry install
eval $(poetry env activate)

2. Clone the DRAKES code β€” the backbone definition (diffusion_gosai_update.py), its hydra config (configs_gosai/config_gosai.yaml) and the oracle loader (oracle.py):

git clone https://github.com/ChenyuWang-Monica/DRAKES.git
export LOCOGEN_DRAKES_REPO=$(pwd)/DRAKES/drakes_dna

3. Download the DRAKES weights and data β€” one ~5.3 GB archive holding the backbone, both reward oracles and the Gosai dataset:

# DRAKES_data.zip β€” "All data and model weights" in the DRAKES README
wget -O DRAKES_data.zip "https://www.dropbox.com/scl/fi/zi6egfppp0o78gr0tmbb1/DRAKES_data.zip?rlkey=yf7w0pm64tlypwsewqc01wmfq&dl=1"
unzip DRAKES_data.zip          # extracts ./data_and_model/
export LOCOGEN_DRAKES_DATA=$(pwd)/data_and_model/

Note

A Dropbox share link, so it may be rotated or expire. If it fails, take the current one from the DRAKES README.

LocoGen reads four files from it. Only mdlm/ is needed β€” pmpnn/ and protein_*/ belong to the DRAKES protein experiments.

File, relative to $LOCOGEN_DRAKES_DATA Size Used for
mdlm/outputs_gosai/pretrained.ckpt 54 MB frozen MDLM backbone
mdlm/outputs_gosai/lightning_logs/reward_oracle_ft.ckpt 2.8 GB guidance oracle P_Ο†
mdlm/outputs_gosai/lightning_logs/reward_oracle_eval.ckpt 2.8 GB held-out scoring only
mdlm/gosai_data/processed_data/gosai_all.csv 194 MB Stage 1 training data

Important

Mind the lightning_logs/ level on the oracle paths. The two oracles are separately trained models, not copies: guidance uses reward_oracle_ft.ckpt, reported Pred-activity uses reward_oracle_eval.ckpt. Keeping them apart is what makes that metric non-circular.

4. Link the dataset. The Stage 1 configs default to ./data/gosai_all.csv, which ships in the same archive:

mkdir -p data
ln -s $LOCOGEN_DRAKES_DATA/mdlm/gosai_data/processed_data/gosai_all.csv data/gosai_all.csv

Warning

Both env vars (LOCOGEN_DRAKES_REPO, LOCOGEN_DRAKES_DATA) must be set before any scripts/drakes_backbone/* entry point.

About gosai_all.csv (data provenance)

DRAKES' repackaging of the Gosai et al. MPRA data: 735,156 rows, each a 200 nt seq plus measured activity in hepg2, k562 and sknsh. The original release is Supplementary Table 2, "Processed MPRA data used to train Malinois", of Gosai et al., Machine-guided design of cell-type-targeting cis-regulatory elements, Nature 2024; raw data and model weights are at Zenodo 10698014, the RNA-seq reads under NCBI BioProject PRJNA1075667. LocoGen reads it only via the DRAKES archive; the upstream links are provenance, not a second download.


πŸ’» Usage

Train a separator, sample with it, score the samples. Stage 1 is optional β€” the released checkpoints go straight to Stage 2.

Stage 1 β€” Train the V-REx masked separator

The separator learns a binary mask over positions such that the masked sequence stays predictive of activity invariantly across environments (K-means clusters in its x_en embedding). Three loss terms: invariance (V-REx penalty), L_reg (sparsity), L_smooth (mask smoothness).

python -m scripts.masked_separator_trainer \
    --config ./configs/enhancer_gosai_masked_separator_hepg2.yaml \
    --out_dir ./runs/masked_separator

Swap the config for cell-aware separators (*_k562.yaml, *_sknsh.yaml). The trainer writes masked_separator_total_epoch{N}.ckpt each epoch, plus masked_separator_best.ckpt.

The checkpoints behind the reported numbers are released at DabinJeong/LocoGen β€” download one to sample without retraining.

Stage 2 β€” Sample with LocoGen

The main entry point. Ξ» controls the OOD push and --alpha_ratio the guidance strength; the reference setting is Ξ» = 0.7, r = 3.0. Fetch the separator for the cell line you want:

CKPT=$(python -c "from huggingface_hub import hf_hub_download; \
print(hf_hub_download('DabinJeong/LocoGen', 'hepg2/masked_separator.ckpt'))")

Full reference settings β€” --lam and --alpha_ratio alone do not reproduce the reported numbers:

python -m scripts.drakes_backbone.mood_guided_sample_mdlm \
    --name locogen_hepg2 \
    --cell_type hepg2 \
    --ms_config ./configs/enhancer_gosai_masked_separator_hepg2.yaml \
    --ms_ckpt "$CKPT" \
    --lam 0.7 \
    --alpha_ratio 3.0 \
    --alpha_cap 20.0 \
    --ood_sil_localized true \
    --ood_sil_threshold 0.5 \
    --sil_t_threshold 0.3 \
    --num_samples 640 \
    --batch_size 64 \
    --seed 0

--num_steps stays at its default of 128; reported results average over seeds 0, 1 and 2. Outputs are sequences.csv (one row per generated 200 nt sequence) and summary.json with the run config.

--ood_sil_localized confines the OOD damping to separator-background positions. That is a different mechanism from --use_masked_separator, which instead rewrites the reward as P_Ο†(xΒ·M + bkgΒ·(1-M)) and stays at its default false in the reported runs.

Caution

Swap cell type, config and checkpoint together. A --cell_type / --ms_config mismatch aborts at startup β€” the config's masked_separator.target_idx is checked against the oracle head. --ms_ckpt is not validated, so a separator trained for another cell line loads silently and scores against the wrong oracle head.

    --cell_type k562 \
    --ms_config ./configs/enhancer_gosai_masked_separator_k562.yaml \
    --ms_ckpt "$(python -c "from huggingface_hub import hf_hub_download; \
print(hf_hub_download('DabinJeong/LocoGen', 'k562/masked_separator.ckpt'))")"

Stage 3 β€” Three-axis benchmark

Every measurement script takes an explicit path, so any sequence CSV β€” a LocoGen run, your own baseline, a sweep point, an ablation β€” scores directly:

# Property axis
python -m scripts.eval.property.pred_activity      --generated <csv> --cell_type hepg2
python -m scripts.eval.property.atac_accessibility --cell hepg2 --label locogen --csv <csv> \
                                                   --out_csv runs/atac_hepg2.csv
# Novelty axis
python -m scripts.eval.novelty.authenticity_rarity --generated <csv> --cell hepg2
# Plausibility axis
python -m scripts.eval.plausibility.motif_gini_hits --label locogen --csv <csv> \
                                                    --out_dir runs/motif_hepg2

Each writes an eval_metrics.json next to the sequences. Aggregate across seeds into the reported mean Β± std table:

python -m scripts.eval.aggregate_seeds --cell_type hepg2
# β†’ runs/eval_aggregate_hepg2/eval_mean_std_hepg2.csv

scripts/eval/manifest.py is the roster aggregate_seeds reads, and the single source of truth for the runs behind the reported numbers β€” Ξ»=0.7, r=3.0 over 3 cell lines Γ— 3 seeds:

python -m scripts.eval.manifest --count
python -m scripts.eval.manifest --list
python -m scripts.eval.manifest --index 5    # prints KEY=val lines

Not in this repository: the third-party baselines LocoGen is compared against (MDLM, SVDD, DRAKES, DNA-Diffusion), whose samplers live upstream and whose sequences therefore cannot be regenerated here; sweeps and ablations; and the LSF dispatcher and per-cell table assembly, which live in the authors' research tree.


βš™οΈ Dependencies

  • Backbone: frozen MDLM (Sahoo et al., 2024) via the DRAKES release β€” pretrained.ckpt
  • Oracles: DRAKES reward oracles (reward_oracle_ft for guidance, reward_oracle_eval for scoring) and the 7-head ATAC classifier
  • Deep learning: PyTorch 2.3.1, PyTorch Lightning, hydra-core, ml-collections, timm
  • Genomics: gReLU, pyjaspar + Biopython (JASPAR PSSM scanning), selene-sdk
  • Data / tracking: pandas, scipy, wandb, huggingface-hub

Pinned ranges live in pyproject.toml; poetry install is the only supported install path.


πŸ—‚ Repository Layout

Expand the directory tree
LocoGen/
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ locogen/                     # core package
β”‚   β”œβ”€β”€ model/
β”‚   β”‚   β”œβ”€β”€ masked_separator.py   # V-REx separator + Y-head (CNN over soft one-hot)
β”‚   β”‚   β”œβ”€β”€ property_scorer.py    # K-member ensemble: mu_hat / sigma2_hat
β”‚   β”‚   └── property_regressor.py # single ensemble member (SeqRegressor)
β”‚   β”œβ”€β”€ utils/
β”‚   β”‚   β”œβ”€β”€ train_utils.py
β”‚   β”‚   β”œβ”€β”€ grelu_compat.py       # loads DRAKES-era grelu checkpoints
β”‚   β”‚   β”œβ”€β”€ seq_io.py             # sequence loading / encoding
β”‚   β”‚   β”œβ”€β”€ seq_stats.py          # k-mer counts, Jensen-Shannon
β”‚   β”‚   └── jaspar.py             # JASPAR PSSM scanning
β”‚   β”œβ”€β”€ metrics/                  # one file per reported metric
β”‚   β”‚   β”œβ”€β”€ authenticity.py       # Authenticity (Alaa et al. 2022)
β”‚   β”‚   β”œβ”€β”€ rarity.py             # Rarity (Han et al. 2023)
β”‚   β”‚   β”œβ”€β”€ motif_usage.py        # motif Gini, hits per sequence
β”‚   β”‚   └── kmer_features.py      # per-seq k-mer space, k-NN radii
β”‚   └── datasets/
β”‚       └── enhancer_gosai_dataset.py
β”œβ”€β”€ scripts/
β”‚   β”œβ”€β”€ masked_separator_trainer.py            # Stage 1: V-REx separator training
β”‚   β”œβ”€β”€ drakes_backbone/
β”‚   β”‚   └── mood_guided_sample_mdlm.py         # Stage 2: LocoGen sampling
β”‚   └── eval/                                  # Stage 3: three-axis benchmark
β”‚       β”œβ”€β”€ manifest.py                        # (method, cell, seed) roster
β”‚       β”œβ”€β”€ aggregate_seeds.py                 # mean Β± std over 3 seeds
β”‚       β”œβ”€β”€ property/
β”‚       β”‚   β”œβ”€β”€ pred_activity.py               # Pred-activity
β”‚       β”‚   └── atac_accessibility.py          # ATAC_acc
β”‚       β”œβ”€β”€ novelty/
β”‚       β”‚   └── authenticity_rarity.py         # Authenticity, Rarity
β”‚       └── plausibility/
β”‚           └── motif_gini_hits.py             # |dGini|, |dhits|
└── configs/
    β”œβ”€β”€ enhancer_gosai_masked_separator_hepg2.yaml     # HepG2 separator
    β”œβ”€β”€ enhancer_gosai_masked_separator_k562.yaml      # K562 separator
    └── enhancer_gosai_masked_separator_sknsh.yaml     # SK-N-SH separator

πŸ“„ License

Distributed under the CC BY-NC-SA 4.0 License.

The DRAKES code, weights and the Gosai et al. MPRA data are not redistributed here and remain under their own licenses.


πŸ“š Citation

@article{jeong2026locality, title={Locality-Controlled Out-of-Distribution Guidance for Robust Regulatory DNA Sequence Design}, author={Jeong, Dabin and Bourdim, Marwane and Lotfollahi, Mo}, journal={arXiv preprint arXiv:2610.41736}, year={2026} }

About

[NeurIPS 2026] Generative model for de novo design of regulatory DNA sequences, using property-guided generation to reach out-of-distribution objectives within a lab-in-the-loop design cycle

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages