feat: allow links to declare whether they are required - #71
Merged
Conversation
Links were declared as a parent/multiplicity/child triple and the generator stamped required: true on every one. Across the four dictionaries built with this tool, 41 of 113 links are required: false, and the same triple shape appears with both values - in one repository subject -> site is required while subject -> demographic_measurements, an identical one_to_one triple, is not. No rule derived from the triple could produce both, so those values had been hand-edited into the generated files and were destroyed on every regeneration. A link may now declare `required`. It is read per link, so where a node has two or three parents each one is answered independently rather than the node getting a single blanket setting. It defaults to true, which is what the generator emitted before, so every existing dictionary regenerates byte-identically - verified against bpsych, omix3 and acdc. The subgroup wrapper is unchanged and still means "at least one of these", so a node whose links are all optional is not left orphaned. Also fixes a data_file node with no declared links receiving no core_metadata_collection link while still receiving the matching property, producing a schema that failed this tool's own validation rule.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Links were declared as a
parent/multiplicity/childtriple and the generator stampedrequired: trueon every one, so the input language could not describe an optional parent.That is not a cosmetic gap. Across the four dictionaries built with this tool, 41 of 113 links are
required: false, and the same triple shape appears with both values — in one repositorysubject → siteis required whilesubject → demographic_measurements, an identicalone_to_onetriple, is not. No rule derived from the triple could produce both, so those values had been
hand-edited into the generated files and were destroyed on every regeneration.
It is also a live blocker:
omix3schemadevcannot move to input-driven generation until thisexists, because regenerating would silently make eight optional parent relationships mandatory.
The change
A link may now declare
required:It is read per link, so where a node has two or three parents each is answered independently
rather than the node getting a single blanket setting. The subgroup wrapper is untouched and still
means "at least one of these", so a node whose links are all optional is not left orphaned.
Default is
true— what the generator emitted before — so the change is purely additive.Also fixed
A
data_filenode with no declared links received nocore_metadata_collectionlink while stillreceiving the matching property, producing a schema that failed this tool's own
data_file_link_core_metadatarule. The tool should not be able to generate something its validatorrejects. No dictionary in the fleet has such a node, so this is inert for existing consumers.
Verification
Proof that it is additive: bpsych, omix3 and acdc were regenerated from their real inputs under
3.0.1 and under this build. All three are byte-identical — 19, 31 and 32 files respectively.
tests/test_link_required.pyadds nine tests: the default, an optional link, per-link values acrossa three-link subgroup, the flat single-link path, the injected core-metadata link staying optional,
precedence between a declared value and the fallback parameter, rejection of a meaningless value,
and a regression test for the link-less
data_filebug. 173 tests pass on Python 3.9–3.12.Hands-on: declared a mixed subgroup by hand, confirmed it generates
true/false/falsein theright places and that
validatestill exits 0.Documentation
Detail goes in
docs/gen3_data_modelling/links.md— a newsection on declaring links from the
input_yaml, the multi-parent case, a table distinguishing thetwo levels of
required, and the honest caveat that subgroupexclusive/requiredare still notsettable and that links to
program/project/core_metadata_collectionare discarded.first_dictionary.mdgets a short §4.3 with one example. The quickstart is deliberatelyuntouched — it does not discuss links and stays that way.
🤖 Generated with Claude Code