[Feature] Final Built-In Ingress Support with Host, Path, PathType, and TLS Fields

[alimaazamat]

Why are these changes needed?

Make the built in ingress support more configurable as the FINAL ingress support in KubeRay.
Support new fields when present, users can now override the host, path, pathtype, and TLS entries.

Since ingress is deprecated this is the final support we will provide. No more advancements will happen so we don't really need to think about future support for this.

Related issue number

Closes #4598

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef04c2813f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Copilot]

Pull request overview

This PR extends KubeRay’s built-in (Kubernetes) Ingress support by allowing users to configure the generated Ingress rule host/path/pathType and TLS entries via new HeadGroupSpec fields, and ensures existing Ingress resources get updated when the desired spec changes.

Changes:

• Add new API fields on HeadGroupSpec: ingressHost, ingressPath, ingressPathType, and ingressTLS (plus corresponding enum/type definitions).
• Update Ingress construction to honor these fields (host, path, pathType, TLS).
• Reconcile existing Ingresses by updating labels/annotations/spec when they drift from the desired Ingress, with unit tests covering the update helper and the new builder behavior.

Reviewed changes

Copilot reviewed 14 out of 18 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
ray-operator/pkg/client/applyconfiguration/utils.go	Registers apply-configuration type lookup for IngressTLSConfig.
ray-operator/pkg/client/applyconfiguration/ray/v1/ingresstlsconfig.go	Adds generated apply-configuration for the new IngressTLSConfig type.
ray-operator/pkg/client/applyconfiguration/ray/v1/headgroupspec.go	Adds generated apply-configuration fields/builders for ingress host/path/pathType/TLS.
ray-operator/controllers/ray/raycluster_controller.go	Updates ingress reconciliation to update an existing Ingress when desired fields change; adds ingressNeedsUpdate.
ray-operator/controllers/ray/raycluster_controller_test.go	Adds unit tests for ingressNeedsUpdate.
ray-operator/controllers/ray/common/ingress.go	Updates ingress builder to consume new HeadGroupSpec ingress fields (host/path/pathType/TLS).
ray-operator/controllers/ray/common/ingress_test.go	Adds coverage for building Ingress with the new ingress fields set.
ray-operator/config/crd/bases/ray.io_rayservices.yaml	Exposes new ingress fields in RayService CRD schema.
ray-operator/config/crd/bases/ray.io_rayjobs.yaml	Exposes new ingress fields in RayJob CRD schema.
ray-operator/config/crd/bases/ray.io_raycronjobs.yaml	Exposes new ingress fields in RayCronJob CRD schema.
ray-operator/config/crd/bases/ray.io_rayclusters.yaml	Exposes new ingress fields in RayCluster CRD schema.
ray-operator/apis/ray/v1/zz_generated.deepcopy.go	Adds deepcopy support for new ingress-related API fields/types.
ray-operator/apis/ray/v1/raycluster_types.go	Introduces IngressPathType enum and IngressTLSConfig, and wires them into HeadGroupSpec.
helm-chart/kuberay-operator/crds/ray.io_rayservices.yaml	Mirrors CRD schema updates for Helm distribution.
helm-chart/kuberay-operator/crds/ray.io_rayjobs.yaml	Mirrors CRD schema updates for Helm distribution.
helm-chart/kuberay-operator/crds/ray.io_raycronjobs.yaml	Mirrors CRD schema updates for Helm distribution.
helm-chart/kuberay-operator/crds/ray.io_rayclusters.yaml	Mirrors CRD schema updates for Helm distribution.
docs/reference/api.md	Documents the new ingress fields and new IngressPathType/IngressTLSConfig API types.

Files not reviewed (4)

• ray-operator/apis/ray/v1/zz_generated.deepcopy.go: Language not supported
• ray-operator/pkg/client/applyconfiguration/ray/v1/headgroupspec.go: Language not supported
• ray-operator/pkg/client/applyconfiguration/ray/v1/ingresstlsconfig.go: Language not supported
• ray-operator/pkg/client/applyconfiguration/utils.go: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[win5923]

Thanks for the PR! Will review later, have you tested this with different ingress controllers?

[win5923]

Manual Testing with Istio v1.30.1:

1. Create a kind cluster

$ kind create cluster

2. In a separate terminal, run cloud-provider-kind

$ cloud-provider-kind

3. Install Istio (default profile, it ships the istio-ingressgateway)

$ istioctl install -y

4. Build & run the operator locally (from ray-operator/)

$ cd ray-operator
$ make install && make build
$ ./bin/manager -leader-election-namespace default -use-kubernetes-proxy

5. Apply the RayCluster + IngressClass

apiVersion: ray.io/v1
kind: RayCluster
metadata:
  name: raycluster-ingress
  annotations:
    # The operator reads this annotation and sets `spec.ingressClassName` on the
    # generated Ingress. This should match your Ingress controller's class.
    kubernetes.io/ingress.class: istio
spec:
  rayVersion: "2.52.0" # should match the Ray version in the image of the containers
  headGroupSpec:
    rayStartParams: {}
    enableIngress: true
    ingressHost: ray-dashboard.example.com
    ingressPath: /
    ingressPathType: Prefix
    template:
      spec:
        containers:
        - name: ray-head
          image: rayproject/ray:2.52.0
          resources:
            limits:
              cpu: "1"
              memory: "5Gi"
            requests:
              cpu: "1"
              memory: "5Gi"
          ports:
          - containerPort: 6379
            name: gcs-server
          - containerPort: 8265 # Ray dashboard
            name: dashboard
          - containerPort: 10001
            name: client
  workerGroupSpecs:
  - replicas: 1
    minReplicas: 1
    maxReplicas: 5
    groupName: workergroup
    rayStartParams: {}
    template:
      spec:
        containers:
        - name: ray-worker
          image: rayproject/ray:2.52.0
          resources:
            limits:
              cpu: "1"
              memory: "1Gi"
            requests:
              cpu: "1"
              memory: "1Gi"
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller

6. Confirm the operator generated the Ingress from the new fields

$ kubectl get ingress raycluster-ingress-head-ingress -o wide
NAME                              CLASS   HOSTS                       ADDRESS      PORTS   AGE
raycluster-ingress-head-ingress   istio   ray-dashboard.example.com   172.18.0.3   80      28m

7. Send traffic through the Istio gateway

$ GW_IP=$(kubectl -n istio-system get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ curl --resolve ray-dashboard.example.com:80:$GW_IP http://ray-dashboard.example.com

# Should get the following response
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="shortcut icon" href="./favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><title>Ray Dashboard</title><script defer="defer" src="./static/js/main.32399d4c.js"></script><link href="./static/css/main.388a904b.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>

[win5923]

cc @Tom-Newton, since you've raised on ingress-related PRs before, I'd be interested in your thoughts on whether this approach makes sense.

[Tom-Newton]

cc @Tom-Newton, since you've raised on ingress-related PRs before, I'd be interested in your thoughts on whether this approach makes sense.

Thanks for working on this. I think a lot of users will appreciate improved ingress support. I haven't looked particularly throroughly, but I think this PR does not make ingress annotations configurable, so it would not be sufficient for us. We have been using a patched version of kuberay that includes #3871.

[win5923]

but I think this PR does not make ingress annotations configurable, so it would not be sufficient for us.

I think RayCluster's annotations are propagated to the generated Ingress. Would that address this use case, or is there a reason we need ingress-specific annotations instead?

[Tom-Newton]

but I think this PR does not make ingress annotations configurable, so it would not be sufficient for us.

I think RayCluster's annotations are propagated to the generated Ingress. Would that address this use case, or is there a reason we need ingress-specific annotations instead?

Good point. I think that should work for us.

[Tom-Newton]

Actually there is another challenge we have. We would like to configure these settings on the helm chart level, not on the RayCluster, RayJob, etc. On the helm chart we can set it correctly in one place, and it will work automatically for the whole kubernetes cluster.

[rueian]

Actually there is another challenge we have. We would like to configure these settings on the helm chart level, not on the RayCluster, RayJob, etc. On the helm chart we can set it correctly in one place, and it will work automatically for the whole kubernetes cluster.

What stops you from doing that on the helm chart?

[Tom-Newton]

Actually there is another challenge we have. We would like to configure these settings on the helm chart level, not on the RayCluster, RayJob, etc. On the helm chart we can set it correctly in one place, and it will work automatically for the whole kubernetes cluster.

What stops you from doing that on the helm chart?

Maybe I've misunderstood, but it looks like this PR adds the configs to RayCluster https://github.com/ray-project/kuberay/pull/4901/changes#diff-a50470a5d60950286903f3de4362b97a3f309272aa95e82ade0cb3adfb03af6dR138-R149

[alimaazamat]

@Tom-Newton
I believe the community's consensus is to keep this PR scoped to per RayCluster configuration for Ingress for two main reasons:

1. This behavior matches k8s networking behavior of routing config aligned per workload (application specific networking)
2. Operator level defaults require redeploying the operator which affects all managed RayClusters at once (less ideal than incremental rollout per cluster).

[Tom-Newton]

I believe the community's consensus is to keep this PR scoped to per RayCluster configuration for Ingress

Fair enough. We'll probably just keep using our patched version.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit f2987da. Configure here.}

[cursor]

Empty TLS causes update loop

High Severity

When no TLS is configured, BuildIngressForHeadService sets spec.tls to a non-nil empty slice, while the API usually stores that field as nil. The new ingressNeedsUpdate path uses reflect.DeepEqual on the full spec, so those forms never match and the controller can issue an ingress update on every reconcile.

Additional Locations (1)

• ray-operator/controllers/ray/raycluster_controller.go#L543-L546

 

^{Reviewed by Cursor Bugbot for commit f2987da. Configure here.}