Classification Import

Overview

A variant is a change on a locus, and thus is tied to a genome build.

A classification is initially created for a specific build, but it needs to be viewed, exported and checked for discordance in all genome builds used by the system.

This is done using an Allele (a change independent of a genome build, see Variants and Alleles), and ensuring that:

• An Allele exists for the initial variant
• A variant for each genome build is linked to that allele

Assigning a variant to a classification

If a classification is created inside VariantGrid, we already know the variant so it can be linked immediately.

When submitted via API, we require a genome build, and either VCF style coordinates or HGVS, which needs to be resolved to a variant (which may need to be created).

ImportedAlleleInfo

The record that carries an incoming submission's variant identification through resolution is classification.models.ImportedAlleleInfo. It holds what the lab actually sent (imported c.HGVS / g.HGVS / VCF coordinates), and gains a ResolvedVariantInfo per genome build as matching succeeds. Its status (ImportedAlleleInfoStatus) tracks progress, and ImportedAlleleInfoValidation records validation tags/warnings raised along the way.

Because ImportedAlleleInfo is keyed on the imported representation, many classifications sharing the same HGVS share one resolution — so re-matching is done once, not per classification.

Classification import

There can be multiple classifications in an API POST, and BulkInserter creates a new ClassificationImport for each genome build used in the import, and links it to a classification. After processing and saving the classification records, process_classification_import will be called for each build so the records can be processed in batches.

Classifications have been validated, and errors raised if different EKeys resolved to different variant coordinates.

Assigning variants to classifications

process_classification_import (in classification/classification_import.py) walks the import's ImportedAlleleInfo records with status PROCESSING and reads allele_info.variant_coordinate_obj to get a coordinate. Anything that can't produce one is failed with allele_info.set_matching_failed().

Coordinates are batched into a snpdb.variant_pk_lookup.VariantPKLookup, which resolves them against Postgres. Known variants are linked immediately via allele_info.set_variant_and_save().

Unknown coordinates are checked with _is_safe_for_vcf() (all of chrom/position/ref/alt present, and ref/alt within MAX_VCF_FIELD_LENGTH) — anything too large to round-trip through a VCF is failed with a message rather than silently dropped. The rest are written to a VCF and a VCF_INSERT_VARIANTS_ONLY UploadPipeline is created and run with a post-insert step of ClassificationImportProcessVariantsTask.

The pipeline runs even when there is nothing new to insert, because it also has to create Alleles, perform liftover, and set the c.HGVS cache.

The VCF_INSERT_VARIANTS_ONLY UploadPipeline handles normalising and inserting without dupes or race conditions (see VCF import).

ClassificationImportProcessVariantsTask then looks up the newly inserted variant ids and links them back to their ImportedAlleleInfo. A validation tag is raised if the variant was changed by normalisation (bcftools norm — see VCF import).

Alleles and Liftover

ClassificationImportProcessVariantsTask calls populate_clingen_alleles_for_variants with variants from the import, and then liftover_classification_import calls create_liftover_pipelines for all other builds on the system.

See Variants and Alleles and Liftover

Re-matching

Two entry points in classification/classification_import.py re-run resolution over existing records:

• variant_matching_dry_run(queryset) — report what would change for a queryset of ImportedAlleleInfo
• reattempt_variant_matching(user, queryset, clear_existing=False) — actually re-resolve

These are what the re-matching management commands use (classification_re_matching, allele_info_revalidate, and the various fix_legacy_* one-offs), eg after a transcript/annotation update changes how an HGVS resolves.