Allow filtering the Dags list by failed and success runs in any run state filter

[yuseok89]

Closes #67094

The Dags list has two run-state filters. "Last run state" matches on a Dag's latest run, and "Any run state" matches a Dag that has any run in the chosen state. The "Any run state" filter only offered queued and running, so a Dag with failed runs but a green latest run could not be found from the Dags list at all.
This adds failed and success to it, so both filters now cover all four states.

What changed

• Backend (_AnyDagRunStateFilter). The filter now supports all four run states. Its query moved from dag_id IN (SELECT DISTINCT dag_id FROM dag_run WHERE state = X) to a correlated EXISTS, which the existing (dag_id, state) index resolves per Dag instead of scanning every run in the state. No new index is needed.
• UI. The "Any run state" selector now lists the same four states as "Last run state".

Performance

Measured on a large dag_run table (a few million rows) across a few thousand Dags(Postgres), filtering by dag_run_state=success (the heaviest state):

• Old DISTINCT scan of all matching runs: around a second or more
• New correlated EXISTS: tens of milliseconds

The EXISTS cost grows only with Dag count (one index seek per Dag, stopping at the first match), not with run volume, so it stays fast well beyond typical Dag counts, and the gap widens further as run history grows.

Demo

Screen.Recording.2026-07-14.at.11.09.50.PM.mov

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  • Opus 4.8

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[bbovenzi]

I think this UX needs to be refined. I don't think any user would realistically search "any dag that has ever had a success", even "any dag that has ever had a failure" could quickly be useless for very old failures. I think we first need a time range filter for use cases like "dags with a failure in the last week", or "dags with any runs in the last 24 hours"

[yuseok89]

@bbovenzi

Thanks for the review.
That makes sense to me. I think a time range is the right direction, so I prototyped two ways of adding it. A standalone time-range select in the filter bar could read as a page-wide filter, so both attach the time axis to the run-state filter itself.

Agreed that searching for "ever had a success" is unrealistic. I had added success for symmetry with "Last run state". With a time window attached though, something like "Dags with a successful run in the last 24 hours" does become meaningful.

Before the backend details, I'd like to settle the UI direction first. Two options:

Option A — attach a time window to the existing "Any run" filter

Screen.Recording.2026-07-19.at.2.27.01.PM.mov

When a state is selected, a within: segment appears joined to the same control(Last 24 hours / 7 days / 30 days / Any time). It defaults to Last 7 days, so the "very old failures" concern is addressed by the default behavior.

Option B — merge the two run-state filters into one

Screen.Recording.2026-07-19.at.2.40.08.PM.mov

Going a step further. Having "Last run state" and "Any run state" side by side is itself a bit confusing, so this merges them into a single Run state control with a scope (Latest run / Last 24 hours / 7 days / 30 days / Any time). Later, adding an "Any state" entry would cover "dags with any runs in the last 24 hours" with the same control.

My preference is B .
It removes the confusion of the two coexisting filters and covers both use cases you mentioned with one model. It does grow the scope of the change, though.
What do you think?

[bbovenzi]

Yes, I prefer B too. Good idea

[pierrejeambrun]

In the current implementation, this is a net gain. (additional options to filter, while still leveraging indexes and keeping performance good)

I would merge this in the current form and follow up with a UX refactoring for those two filters in a second PR.

[pierrejeambrun]

Discussed with Brent, let's merge this iteration which brings value. @yuseok89 please feel free to follow up with a new PR on the UX improvement suggested by Brent.

[yuseok89]

@pierrejeambrun
Sounds good, thanks! I'll open a follow-up PR for the unified filter.