Skip to content

Improve Power BI converter round-trip fidelity - #448

Open
eisber wants to merge 2 commits into
apache:mainfrom
eisber:dev/marcozo/improve-pbi-ossie-converter-round-trips-tlt1vs
Open

eisber wants to merge 2 commits into
apache:mainfrom
eisber:dev/marcozo/improve-pbi-ossie-converter-round-trips-tlt1vs

Conversation

@eisber

@eisber eisber commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • preserve explicit Power BI default values such as false key flags and active relationships
  • keep relationship metadata from overriding edited Ossie endpoints or names
  • preserve DAX string-literal calculated columns and exact expression whitespace
  • document the strengthened round-trip guarantee

Private corpus validation

Validated locally against two private PBIX models without committing model-derived artifacts:

  • exact properties: 1,944/2,280 (85.26%) -> 2,280/2,280 (100%)
  • exact semantic elements: 232/428 (54.21%) -> 428/428 (100%)

Tests

  • focused round-trip matrix: 11 passed
  • Microsoft converter suite: 410 passed, 2 skipped
  • coverage gate: 95.39%
  • Ruff, compile, and diff checks passed

Copilot AI lite review requested due to automatic review settings September 23, 2026 13:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 3 Medium severity

Open (3)
What changed in this PR

This PR strengthens round-trip fidelity between Power BI (TMSL) and Apache Ossie by preserving explicitly serialized defaults and DAX expression formatting, and by preventing preserved relationship metadata from overriding user-edited Ossie relationships.

Changes:

  • Preserve explicit false key flags (isKey, isUnique) and isActive relationship state across round trips.
  • Preserve DAX calculated column expressions exactly (including whitespace and string-literal expressions).
  • Prevent stale relationship metadata from replaying when Ossie relationship endpoints/names have been edited; document the stronger guarantee.
File Description
converters/​microsoft/​tests/​test_semantic_model_to_ossie.py Adds focused regression tests for explicit defaults, DAX whitespace/string literals, and relationship metadata precedence.
converters/​microsoft/​src/​ossie_microsoft/​semantic_model_to_ossie.py Preserves explicit false flags and DAX whitespace; expands relationship endpoint metadata tracking.
converters/​microsoft/​src/​ossie_microsoft/​ossie_to_semantic_model.py Refines relationship name/metadata precedence rules; preserves multiline/trailing-newline DAX formatting on export.
converters/​microsoft/​README.md Updates documentation to reflect strengthened round-trip guarantees.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread converters/microsoft/src/ossie_microsoft/semantic_model_to_ossie.py Outdated
Comment thread converters/microsoft/src/ossie_microsoft/semantic_model_to_ossie.py
Comment thread converters/microsoft/src/ossie_microsoft/semantic_model_to_ossie.py
for key, value in stash.items():
if key not in _RELATIONSHIP_CONTROL_KEYS and (
metadata_is_current or key not in _RELATIONSHIP_CARDINALITY_KEYS
metadata_is_current or key not in RELATIONSHIP_ENDPOINT_METADATA

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Widening this gate from the cardinality keys to RELATIONSHIP_ENDPOINT_METADATA means a stash with no normalizedEndpoints and a custom relationship name (e.g. customer_sales) now loses isActive, crossFilteringBehavior and relyOnReferentialIntegrity on export. Those stashes fail the legacy "name equals generated name" check, so metadata_is_current is false.

Before this change only the cardinality keys were dropped, so crossFilteringBehavior: bothDirections was replayed. Now it is dropped silently, so a round trip that used to be lossless loses the cross-filter direction.

Could we keep the old cardinality-only gating for stashes without normalizedEndpoints? Or, if older stashes are intentionally unsupported, could we say so in the README and add a test for a legacy stash with a custom name?


_, expression = _preferred_expression(expressions)
dialect, expression = _preferred_expression(expressions)
if dialect == DIALECT_DAX:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_source_column now returns None for DAX, but _dataset_column_index still adds an alias for any dialect's expression. A DAX-only field whose expression is a bare identifier would be indexed as an alias for a source column. Relationship endpoint validation could then accept a column that is never emitted as a physical column.

Should the index skip DAX expressions so the two stay consistent?

return value.splitlines() if "\n" in value else value
if "\n" not in value and "\r" not in value:
return value
lines = value.splitlines()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: splitlines() splits on more than \n and \r: also \x0b, \x0c, \x1c-\x1e, \x85, \u2028, and \u2029. Expressions containing those characters get re-joined with \n on read, so the "original whitespace preserved" claim doesn't hold for them.

Splitting on \n and \r\n explicitly would preserve them. Otherwise, could we narrow the docstring or the test name?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants