Is there an existing issue for this?
Problem statement
functionPath on template-generated specs already searches the instantiating dataflow, then templates/python_functions/. Relative sqlPath and schemaPath do not.
Those values are always resolved under the dataflow group (…/dataflowspec/<subdir>/ then …/<subdir>/). Shared files next to a template (src/templates/dml/…, src/templates/schemas/…) are never considered, so template authors cannot reuse SQL or schemas the way they reuse Python functions.
Regular (non-template) specs should keep resolving DML and schemas only next to the spec. Extra fallbacks are not needed there.
Proposed Solution
For specs tagged _isTemplateGenerated, resolve relative paths in this order:
sqlPath
- DML next to the spec (same as today: dataflow group
dml/)
- DML under the bundle templates directory (
templates/dml/<file>)
schemaPath
- Schemas next to the spec (same as today: dataflow group
schemas/)
- Schemas under the bundle templates directory (
templates/schemas/<file>)
If a file is found, use that absolute path. Regular specs must not search templates/dml/ or templates/schemas/.
Mirror the existing functionPath pattern in dataflow_spec_builder.py, add unit tests for both spec kinds, and document the search order next to the Python function fallback in the templates docs.
Additional Context
Is there an existing issue for this?
Problem statement
functionPathon template-generated specs already searches the instantiating dataflow, thentemplates/python_functions/. RelativesqlPathandschemaPathdo not.Those values are always resolved under the dataflow group (
…/dataflowspec/<subdir>/then…/<subdir>/). Shared files next to a template (src/templates/dml/…,src/templates/schemas/…) are never considered, so template authors cannot reuse SQL or schemas the way they reuse Python functions.Regular (non-template) specs should keep resolving DML and schemas only next to the spec. Extra fallbacks are not needed there.
Proposed Solution
For specs tagged
_isTemplateGenerated, resolve relative paths in this order:sqlPathdml/)templates/dml/<file>)schemaPathschemas/)templates/schemas/<file>)If a file is found, use that absolute path. Regular specs must not search
templates/dml/ortemplates/schemas/.Mirror the existing
functionPathpattern indataflow_spec_builder.py, add unit tests for both spec kinds, and document the search order next to the Python function fallback in the templates docs.Additional Context
LOCALISE_PATHSalready mapssqlPath→./dmlandschemaPath→./schemas._isTemplateGeneratedis set intemplate_processor.py.bundle/templates/python_functions/<file>for generated specs only.sqlPathfallback is implemented in feat: fall back to templates/dml and templates/schemas for generated specs #151.schemaPathis still missing.