Skip to content

Add datasets to dag graph - #37604

Merged
bbovenzi merged 2 commits into
apache:mainfrom
astronomer:datasets-in-dag-graph
Feb 22, 2024
Merged

Add datasets to dag graph#37604
bbovenzi merged 2 commits into
apache:mainfrom
astronomer:datasets-in-dag-graph

Conversation

@bbovenzi

Copy link
Copy Markdown
Contributor

Add datasets as upstream and downstream nodes in the DAG graph.

Make it easier to visualize what a DAG is connected to. And provide a tooltip to jump over to that dataset to see what else it connects to. This makes it easier to jump back and forth between dag view and a dataset view.

Screenshot 2024-02-21 at 3 10 42 PM

Zoomed out:
Screenshot 2024-02-21 at 3 10 16 PM

After this:

  • we could add the relevant dataset events to a Dataset node too. Perhaps even in both the dataset and dag graphs and we can share the node components between the two graphs.
  • We can add trigger rules to the tooltip for dataset-and-timetable or any/all conditions

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@bbovenzi bbovenzi added this to the Airflow 2.9.0 milestone Feb 21, 2024
@boring-cyborg boring-cyborg Bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Feb 21, 2024
@bbovenzi bbovenzi changed the title Datasets in dag graph Add datasets to dag graph Feb 21, 2024

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about the javascript code but it definitely looks good.

@dirrao dirrao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bbovenzi
Nice work. I believe this is for task view not dag view?

@bbovenzi

Copy link
Copy Markdown
Contributor Author

@bbovenzi Nice work. I believe this is for task view not dag view?

Nope. For the whole dag. The example dag had only a single task.

@bbovenzi
bbovenzi merged commit 1c37891 into apache:main Feb 22, 2024
@bbovenzi
bbovenzi deleted the datasets-in-dag-graph branch February 22, 2024 19:24
sudiptob2 pushed a commit to Satoshi-Sh/airflow that referenced this pull request Feb 22, 2024
* Add dataset nodes to dag graph

* Clean up dataset nodes
@ephraimbuddy ephraimbuddy added the type:new-feature Changelog: New Features label Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:new-feature Changelog: New Features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants