Skip to content

[FEATURE]: Template-generated specs should fall back to templates/dml and templates/schemas #150

Description

@rederik76

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

functionPath on template-generated specs already searches the instantiating dataflow, then templates/python_functions/. Relative sqlPath and schemaPath do not.

Those values are always resolved under the dataflow group (…/dataflowspec/<subdir>/ then …/<subdir>/). Shared files next to a template (src/templates/dml/…, src/templates/schemas/…) are never considered, so template authors cannot reuse SQL or schemas the way they reuse Python functions.

Regular (non-template) specs should keep resolving DML and schemas only next to the spec. Extra fallbacks are not needed there.

Proposed Solution

For specs tagged _isTemplateGenerated, resolve relative paths in this order:

sqlPath

  1. DML next to the spec (same as today: dataflow group dml/)
  2. DML under the bundle templates directory (templates/dml/<file>)

schemaPath

  1. Schemas next to the spec (same as today: dataflow group schemas/)
  2. Schemas under the bundle templates directory (templates/schemas/<file>)

If a file is found, use that absolute path. Regular specs must not search templates/dml/ or templates/schemas/.

Mirror the existing functionPath pattern in dataflow_spec_builder.py, add unit tests for both spec kinds, and document the search order next to the Python function fallback in the templates docs.

Additional Context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions