Locality-controlled guidance that steers a frozen masked diffusion language model toward high-activity enhancer DNA β no fine-tuning, no reward model training.
- LocoGen
Locality-controlled guidance for masked discrete diffusion of regulatory DNA. LocoGen samples enhancer sequences from a frozen MDLM (Sahoo et al., 2024) backbone β no fine-tuning β using a V-REx-trained masked separator to localize the guidance gradient to invariant regulatory positions.
The separator learns where a sequence must be left alone: positions whose contribution to measured activity is invariant across environments. Guidance and the off-distribution push apply only outside them, so sequences keep their regulatory grammar while the background diversifies β which is what lets property, novelty and plausibility improve together instead of trading off.
This repository trains the separator (Stage 1), samples with LocoGen on a DRAKES MDLM backbone (Stage 2), and reproduces the three-axis benchmark (Stage 3).
- No fine-tuning β the generative backbone stays frozen; nothing in it is retrained.
- V-REx masked separator β finds environment-invariant positions and confines the guidance gradient to them.
- Localized OOD push β
--ood_sil_localizeddamps only separator-background positions. - Cell-type aware β HepG2, K562 and SK-N-SH separators, checkpoints on the Hugging Face Hub.
- Reproducible benchmark β property (Pred-activity, ATAC_acc), novelty (Authenticity, Rarity) and plausibility (|ΞGini|, |Ξhits|) under
scripts/eval/, with a manifest of the runs behind the reported numbers.
- Python β₯ 3.11, < 3.15
- Poetry
- A CUDA GPU β sampling and both reward oracles run on GPU
- ~6 GB free disk β the DRAKES archive is ~5.3 GB zipped
- Git β the DRAKES backbone is cloned, not vendored here
1. Install the package
pip install poetry
poetry install
eval $(poetry env activate)2. Clone the DRAKES code β the backbone definition (diffusion_gosai_update.py), its hydra
config (configs_gosai/config_gosai.yaml) and the oracle loader (oracle.py):
git clone https://github.com/ChenyuWang-Monica/DRAKES.git
export LOCOGEN_DRAKES_REPO=$(pwd)/DRAKES/drakes_dna3. Download the DRAKES weights and data β one ~5.3 GB archive holding the backbone, both reward oracles and the Gosai dataset:
# DRAKES_data.zip β "All data and model weights" in the DRAKES README
wget -O DRAKES_data.zip "https://www.dropbox.com/scl/fi/zi6egfppp0o78gr0tmbb1/DRAKES_data.zip?rlkey=yf7w0pm64tlypwsewqc01wmfq&dl=1"
unzip DRAKES_data.zip # extracts ./data_and_model/
export LOCOGEN_DRAKES_DATA=$(pwd)/data_and_model/Note
A Dropbox share link, so it may be rotated or expire. If it fails, take the current one from the DRAKES README.
LocoGen reads four files from it. Only mdlm/ is needed β pmpnn/ and protein_*/ belong to
the DRAKES protein experiments.
File, relative to $LOCOGEN_DRAKES_DATA |
Size | Used for |
|---|---|---|
mdlm/outputs_gosai/pretrained.ckpt |
54 MB | frozen MDLM backbone |
mdlm/outputs_gosai/lightning_logs/reward_oracle_ft.ckpt |
2.8 GB | guidance oracle P_Ο |
mdlm/outputs_gosai/lightning_logs/reward_oracle_eval.ckpt |
2.8 GB | held-out scoring only |
mdlm/gosai_data/processed_data/gosai_all.csv |
194 MB | Stage 1 training data |
Important
Mind the lightning_logs/ level on the oracle paths. The two oracles are separately trained
models, not copies: guidance uses reward_oracle_ft.ckpt, reported Pred-activity uses
reward_oracle_eval.ckpt. Keeping them apart is what makes that metric non-circular.
4. Link the dataset. The Stage 1 configs default to ./data/gosai_all.csv, which ships in
the same archive:
mkdir -p data
ln -s $LOCOGEN_DRAKES_DATA/mdlm/gosai_data/processed_data/gosai_all.csv data/gosai_all.csvWarning
Both env vars (LOCOGEN_DRAKES_REPO, LOCOGEN_DRAKES_DATA) must be set before any
scripts/drakes_backbone/* entry point.
About gosai_all.csv (data provenance)
DRAKES' repackaging of the Gosai et al. MPRA data: 735,156 rows, each a 200 nt seq plus
measured activity in hepg2, k562 and sknsh. The original release is Supplementary Table 2,
"Processed MPRA data used to train Malinois", of Gosai et al., Machine-guided design of
cell-type-targeting cis-regulatory elements, Nature
2024; raw data and model weights are at
Zenodo 10698014, the RNA-seq reads under NCBI BioProject
PRJNA1075667. LocoGen reads it
only via the DRAKES archive; the upstream links are provenance, not a second download.
Train a separator, sample with it, score the samples. Stage 1 is optional β the released checkpoints go straight to Stage 2.
The separator learns a binary mask over positions such that the masked sequence stays predictive
of activity invariantly across environments (K-means clusters in its x_en embedding). Three
loss terms: invariance (V-REx penalty), L_reg (sparsity), L_smooth (mask smoothness).
python -m scripts.masked_separator_trainer \
--config ./configs/enhancer_gosai_masked_separator_hepg2.yaml \
--out_dir ./runs/masked_separatorSwap the config for cell-aware separators (*_k562.yaml, *_sknsh.yaml). The trainer writes
masked_separator_total_epoch{N}.ckpt each epoch, plus masked_separator_best.ckpt.
The checkpoints behind the reported numbers are released at
DabinJeong/LocoGen β download one to sample
without retraining.
The main entry point. Ξ» controls the OOD push and --alpha_ratio the guidance strength; the
reference setting is Ξ» = 0.7, r = 3.0. Fetch the separator for the cell line you want:
CKPT=$(python -c "from huggingface_hub import hf_hub_download; \
print(hf_hub_download('DabinJeong/LocoGen', 'hepg2/masked_separator.ckpt'))")Full reference settings β --lam and --alpha_ratio alone do not reproduce the reported
numbers:
python -m scripts.drakes_backbone.mood_guided_sample_mdlm \
--name locogen_hepg2 \
--cell_type hepg2 \
--ms_config ./configs/enhancer_gosai_masked_separator_hepg2.yaml \
--ms_ckpt "$CKPT" \
--lam 0.7 \
--alpha_ratio 3.0 \
--alpha_cap 20.0 \
--ood_sil_localized true \
--ood_sil_threshold 0.5 \
--sil_t_threshold 0.3 \
--num_samples 640 \
--batch_size 64 \
--seed 0--num_steps stays at its default of 128; reported results average over seeds 0, 1 and 2.
Outputs are sequences.csv (one row per generated 200 nt sequence) and summary.json with the
run config.
--ood_sil_localized confines the OOD damping to separator-background positions. That is a
different mechanism from --use_masked_separator, which instead rewrites the reward as
P_Ο(xΒ·M + bkgΒ·(1-M)) and stays at its default false in the reported runs.
Caution
Swap cell type, config and checkpoint together. A --cell_type / --ms_config mismatch
aborts at startup β the config's masked_separator.target_idx is checked against the oracle
head. --ms_ckpt is not validated, so a separator trained for another cell line loads
silently and scores against the wrong oracle head.
--cell_type k562 \
--ms_config ./configs/enhancer_gosai_masked_separator_k562.yaml \
--ms_ckpt "$(python -c "from huggingface_hub import hf_hub_download; \
print(hf_hub_download('DabinJeong/LocoGen', 'k562/masked_separator.ckpt'))")"Every measurement script takes an explicit path, so any sequence CSV β a LocoGen run, your own baseline, a sweep point, an ablation β scores directly:
# Property axis
python -m scripts.eval.property.pred_activity --generated <csv> --cell_type hepg2
python -m scripts.eval.property.atac_accessibility --cell hepg2 --label locogen --csv <csv> \
--out_csv runs/atac_hepg2.csv
# Novelty axis
python -m scripts.eval.novelty.authenticity_rarity --generated <csv> --cell hepg2
# Plausibility axis
python -m scripts.eval.plausibility.motif_gini_hits --label locogen --csv <csv> \
--out_dir runs/motif_hepg2Each writes an eval_metrics.json next to the sequences. Aggregate across seeds into the
reported mean Β± std table:
python -m scripts.eval.aggregate_seeds --cell_type hepg2
# β runs/eval_aggregate_hepg2/eval_mean_std_hepg2.csvscripts/eval/manifest.py is the roster aggregate_seeds reads, and the single source of truth
for the runs behind the reported numbers β Ξ»=0.7, r=3.0 over 3 cell lines Γ 3 seeds:
python -m scripts.eval.manifest --count
python -m scripts.eval.manifest --list
python -m scripts.eval.manifest --index 5 # prints KEY=val linesNot in this repository: the third-party baselines LocoGen is compared against (MDLM, SVDD, DRAKES, DNA-Diffusion), whose samplers live upstream and whose sequences therefore cannot be regenerated here; sweeps and ablations; and the LSF dispatcher and per-cell table assembly, which live in the authors' research tree.
- Backbone: frozen MDLM (Sahoo et al., 2024) via the DRAKES release β
pretrained.ckpt - Oracles: DRAKES reward oracles (
reward_oracle_ftfor guidance,reward_oracle_evalfor scoring) and the 7-head ATAC classifier - Deep learning: PyTorch 2.3.1, PyTorch Lightning, hydra-core, ml-collections, timm
- Genomics: gReLU, pyjaspar + Biopython (JASPAR PSSM scanning), selene-sdk
- Data / tracking: pandas, scipy, wandb, huggingface-hub
Pinned ranges live in pyproject.toml; poetry install is the only supported install path.
Expand the directory tree
LocoGen/
βββ pyproject.toml
βββ locogen/ # core package
β βββ model/
β β βββ masked_separator.py # V-REx separator + Y-head (CNN over soft one-hot)
β β βββ property_scorer.py # K-member ensemble: mu_hat / sigma2_hat
β β βββ property_regressor.py # single ensemble member (SeqRegressor)
β βββ utils/
β β βββ train_utils.py
β β βββ grelu_compat.py # loads DRAKES-era grelu checkpoints
β β βββ seq_io.py # sequence loading / encoding
β β βββ seq_stats.py # k-mer counts, Jensen-Shannon
β β βββ jaspar.py # JASPAR PSSM scanning
β βββ metrics/ # one file per reported metric
β β βββ authenticity.py # Authenticity (Alaa et al. 2022)
β β βββ rarity.py # Rarity (Han et al. 2023)
β β βββ motif_usage.py # motif Gini, hits per sequence
β β βββ kmer_features.py # per-seq k-mer space, k-NN radii
β βββ datasets/
β βββ enhancer_gosai_dataset.py
βββ scripts/
β βββ masked_separator_trainer.py # Stage 1: V-REx separator training
β βββ drakes_backbone/
β β βββ mood_guided_sample_mdlm.py # Stage 2: LocoGen sampling
β βββ eval/ # Stage 3: three-axis benchmark
β βββ manifest.py # (method, cell, seed) roster
β βββ aggregate_seeds.py # mean Β± std over 3 seeds
β βββ property/
β β βββ pred_activity.py # Pred-activity
β β βββ atac_accessibility.py # ATAC_acc
β βββ novelty/
β β βββ authenticity_rarity.py # Authenticity, Rarity
β βββ plausibility/
β βββ motif_gini_hits.py # |dGini|, |dhits|
βββ configs/
βββ enhancer_gosai_masked_separator_hepg2.yaml # HepG2 separator
βββ enhancer_gosai_masked_separator_k562.yaml # K562 separator
βββ enhancer_gosai_masked_separator_sknsh.yaml # SK-N-SH separator
Distributed under the CC BY-NC-SA 4.0 License.
The DRAKES code, weights and the Gosai et al. MPRA data are not redistributed here and remain under their own licenses.
@article{jeong2026locality, title={Locality-Controlled Out-of-Distribution Guidance for Robust Regulatory DNA Sequence Design}, author={Jeong, Dabin and Bourdim, Marwane and Lotfollahi, Mo}, journal={arXiv preprint arXiv:2610.41736}, year={2026} }
