NE-2414: Use `trafficDistribution: PreferSameNode` for openshift-dns Service by danwinship · Pull Request #457 · openshift/cluster-dns-operator

danwinship · 2026-01-15T15:55:06Z

ovn-kubernetes and our build of kube-proxy both currently special-case openshift-dns to give it "prefer same node" semantics. As of k8s 1.34 / OCP 4.22, we can do this in a standard way by adding "trafficDistribution: PreferSameNode" to the openshift-dns service.

(trafficDistribution itself is GA as of k8s 1.33, PreferSameNode is Beta in 1.34 and already GA upstream in 1.35.)

(ovn-k probably won't implement trafficDistribution in 4.22, but merging this now will let us get rid of a patch for kube-proxy in openshift/kubernetes.)

openshift-ci-robot · 2026-01-15T15:55:12Z

@danwinship: This pull request references NE-2414 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

ovn-kubernetes and our build of kube-proxy both currently special-case openshift-dns to give it "prefer same node" semantics. As of k8s 1.34 / OCP 4.22, we can do this in a standard way by adding "trafficDistribution: PreferSameNode" to the openshift-dns service.

(trafficDistribution itself is GA as of k8s 1.33, PreferSameNode is Beta in 1.34 and already GA upstream in 1.35.)

(ovn-k probably won't implement trafficDistribution in 4.22, but merging this now will let us get rid of a patch for kube-proxy in openshift/kubernetes.)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

danwinship · 2026-01-15T23:31:08Z

/retest-required

openshift-ci · 2026-01-16T02:10:27Z

@danwinship: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

rikatz · 2026-01-23T16:26:20Z

/approve

Leaving the lgtm for @Miciah

openshift-ci · 2026-01-23T16:31:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rikatz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [rikatz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

danwinship · 2026-01-26T14:28:56Z

following up from slack: this makes the setting of topology aware hints in the existing code redundant, since trafficDistribution: PreferSameNode supercedes that, but also implies falling back to same-zone if same-node is unavailable. But it's fine to leave the topology hint there as well; this would preserve backward compatibility with service proxies that haven't been updated to handle trafficDistribution yet, and there are unit tests upstream that confirm that it's safe to enable both features (topology aware hints and traffic distribution) at the same time.

Miciah · 2026-01-27T19:24:14Z

/assign

candita · 2026-01-29T16:20:54Z

/assign @rikatz

lihongan · 2026-01-30T01:24:16Z

cc @melvinjoseph86

melvinjoseph86 · 2026-01-30T18:17:05Z

Verified using cluster bot

➜  oc get clusterversion
NAME      VERSION                                                AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.22.0-0-2026-01-30-162516-test-ci-ln-y6y27rk-latest   True        False         29m     Cluster version is 4.22.0-0-2026-01-30-162516-test-ci-ln-y6y27rk-latest
➜   oc get svc -n openshift-dns
NAME          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
dns-default   ClusterIP   172.30.0.10   <none>        53/UDP,53/TCP,9154/TCP   55m
➜   oc get svc -n openshift-dns dns-default -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.alpha.openshift.io/serving-cert-signed-by: openshift-service-serving-signer@1769791664
    service.beta.openshift.io/serving-cert-secret-name: dns-default-metrics-tls
    service.beta.openshift.io/serving-cert-signed-by: openshift-service-serving-signer@1769791664
    service.kubernetes.io/topology-aware-hints: auto
  creationTimestamp: "2026-01-30T16:49:23Z"
  labels:
    dns.operator.openshift.io/owning-dns: default
  name: dns-default
  namespace: openshift-dns
  ownerReferences:
  - apiVersion: operator.openshift.io/v1
    controller: true
    kind: DNS
    name: default
    uid: 7173f679-b63a-45a4-9bfb-4dce92c3e5e8
  resourceVersion: "15745"
  uid: e3d0ff61-799e-4f4a-ba9e-c51234aba93b
spec:
  clusterIP: 172.30.0.10
  clusterIPs:
  - 172.30.0.10
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: dns
    port: 53
    protocol: UDP
    targetPort: dns
  - name: dns-tcp
    port: 53
    protocol: TCP
    targetPort: dns-tcp
  - name: metrics
    port: 9154
    protocol: TCP
    targetPort: metrics
  selector:
    dns.operator.openshift.io/daemonset-dns: default
  sessionAffinity: None
  **_trafficDistribution: PreferSameNode_**
  type: ClusterIP
status:
  loadBalancer: {}

Also able to run topology aware hint test case ocp-63553 successfully
passed: (1m10s) 2026-01-30T18:13:44 "[sig-network-edge] Network_Edge Component_DNS Author:mjoseph-ROSA-OSD_CCS-ARO-High-63553-Annotation 'TopologyAwareHints' presents should not cause any pathological events"
/verified by @mjoseph

openshift-ci-robot · 2026-01-30T18:17:16Z

@melvinjoseph86: This PR has been marked as verified by @mjoseph.

Details

In response to this:

Verified using cluster bot

➜  oc get clusterversion
NAME      VERSION                                                AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.22.0-0-2026-01-30-162516-test-ci-ln-y6y27rk-latest   True        False         29m     Cluster version is 4.22.0-0-2026-01-30-162516-test-ci-ln-y6y27rk-latest
➜   oc get svc -n openshift-dns
NAME          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
dns-default   ClusterIP   172.30.0.10   <none>        53/UDP,53/TCP,9154/TCP   55m
➜   oc get svc -n openshift-dns dns-default -oyaml
apiVersion: v1
kind: Service
metadata:
 annotations:
   service.alpha.openshift.io/serving-cert-signed-by: openshift-service-serving-signer@1769791664
   service.beta.openshift.io/serving-cert-secret-name: dns-default-metrics-tls
   service.beta.openshift.io/serving-cert-signed-by: openshift-service-serving-signer@1769791664
   service.kubernetes.io/topology-aware-hints: auto
 creationTimestamp: "2026-01-30T16:49:23Z"
 labels:
   dns.operator.openshift.io/owning-dns: default
 name: dns-default
 namespace: openshift-dns
 ownerReferences:
 - apiVersion: operator.openshift.io/v1
   controller: true
   kind: DNS
   name: default
   uid: 7173f679-b63a-45a4-9bfb-4dce92c3e5e8
 resourceVersion: "15745"
 uid: e3d0ff61-799e-4f4a-ba9e-c51234aba93b
spec:
 clusterIP: 172.30.0.10
 clusterIPs:
 - 172.30.0.10
 internalTrafficPolicy: Cluster
 ipFamilies:
 - IPv4
 ipFamilyPolicy: SingleStack
 ports:
 - name: dns
   port: 53
   protocol: UDP
   targetPort: dns
 - name: dns-tcp
   port: 53
   protocol: TCP
   targetPort: dns-tcp
 - name: metrics
   port: 9154
   protocol: TCP
   targetPort: metrics
 selector:
   dns.operator.openshift.io/daemonset-dns: default
 sessionAffinity: None
 **_trafficDistribution: PreferSameNode_**
 type: ClusterIP
status:
 loadBalancer: {}

Also able to run topology aware hint test case ocp-63553 successfully
passed: (1m10s) 2026-01-30T18:13:44 "[sig-network-edge] Network_Edge Component_DNS Author:mjoseph-ROSA-OSD_CCS-ARO-High-63553-Annotation 'TopologyAwareHints' presents should not cause any pathological events"
/verified by @mjoseph

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

rikatz · 2026-02-11T14:23:47Z

/lgtm
Discussed with @alebedev87 this may not cause impacts
Thanks!

The failure in the `DNS should answer A and AAAA queries for a dual-stack service` test on metal jobs was triggered by the recent merge of PR openshift#457 (`NE-2414: Use trafficDistribution: PreferSameNode for openshift-dns Service`). ### The Root Cause In `pkg/operator/controller/controller_dns_service.go`, the operator compares the `current` DNS Service against the `expected` (static) asset using a function called `serviceChanged()`. To prevent unnecessary updates, the comparison correctly ignores dynamic dual-stack fields. However, when PR openshift#457 added `trafficDistribution: PreferSameNode` to the static asset, it caused `expected` and `current` to differ on existing clusters, legitimately triggering a Service update. When an update is triggered, the operator blindly replaces the spec (`updated.Spec = expected.Spec`) and then tries to manually preserve dynamic fields that the API server or other controllers manage. It only preserved `ClusterIP`, dropping `ClusterIPs`, `IPFamilies`, and `IPFamilyPolicy`. ### The Impact Because `expected.Spec` (from the static YAML) has no IP families defined, the operator pushes a Service update that wipes out the secondary IPv6 Cluster IP. The `openshift-dns` service is abruptly converted back to a single-stack IPv4 service, which permanently breaks the `DNS should answer A and AAAA queries for a dual-stack service` test on metal (dual-stack) jobs. ### The Fix This explicitly preserves the dual-stack fields (`ClusterIPs`, `IPFamilies`, `IPFamilyPolicy`) during an update. It also improves the dual-stack test assertions to verify these fields are preserved field-for-field. Jira: https://issues.redhat.com/browse/OCPBUGS-78053

The failure in the `DNS should answer A and AAAA queries for a dual-stack service` test on metal jobs was triggered by the recent merge of PR openshift#457 (`NE-2414: Use trafficDistribution: PreferSameNode for openshift-dns Service`). ### The Root Cause In `pkg/operator/controller/controller_dns_service.go`, the operator compares the `current` DNS Service against the `expected` (static) asset using a function called `serviceChanged()`. To prevent unnecessary updates, the comparison correctly ignores dynamic dual-stack fields. However, when PR openshift#457 added `trafficDistribution: PreferSameNode` to the static asset, it caused `expected` and `current` to differ on existing clusters, legitimately triggering a Service update. When an update is triggered, the operator blindly replaces the spec (`updated.Spec = expected.Spec`) and then tries to manually preserve dynamic fields that the API server or other controllers manage. It only preserved `ClusterIP`, dropping `ClusterIPs`, `IPFamilies`, and `IPFamilyPolicy`. ### The Impact Because `expected.Spec` (from the static YAML) has no IP families defined, the operator pushes a Service update that wipes out the secondary IPv6 Cluster IP. The `openshift-dns` service is abruptly converted back to a single-stack IPv4 service, which permanently breaks the `DNS should answer A and AAAA queries for a dual-stack service` test on metal (dual-stack) jobs. ### The Fix This explicitly preserves the dual-stack fields (`ClusterIPs`, `IPFamilies`, `IPFamilyPolicy`) during an update. It also improves the dual-stack test assertions to verify these fields are preserved field-for-field. Jira: https://issues.redhat.com/browse/OCPBUGS-78085

DNS queries from pods on primary user-defined networks (UDNs) scatter randomly across dns-default pods on all nodes instead of being handled by the local node's dns-default pod. UDN pods reach the dns-default service (172.30.0.10) through a path that crosses network boundaries: UDN pod -> UDN cluster router -> management port -> default network -> OVN load balancer -> dns-default pod. The OVN load balancer on the default network treats dns-default as a standard ClusterIP service and distributes traffic across all backend pods cluster-wide. Setting internalTrafficPolicy to Local restricts the EndpointSlice to contain only the node-local dns-default backend. Since dns-default runs as a DaemonSet with a pod on every node, this is safe and guarantees that DNS queries are always handled by the local pod. The existing trafficDistribution: PreferSameNode (added in PR openshift#457) provides a soft hint for same-node preference but does not guarantee locality for cross-network UDN traffic. internalTrafficPolicy: Local provides the hard constraint needed. Jira: https://issues.redhat.com/browse/OCPBUGS-55179 Signed-off-by: Parikshit Khedekar <pkhedeka@redhat.com>

Use trafficDistribution: PreferSameNode for openshift-dns Service

e2b909a

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jan 15, 2026

openshift-ci Bot requested review from Miciah and candita January 15, 2026 15:55

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 23, 2026

openshift-ci Bot assigned Miciah Jan 27, 2026

openshift-ci Bot assigned rikatz Jan 29, 2026

danwinship mentioned this pull request Jan 30, 2026

Gather EndpointSlices rather than Endpoints openshift/release#74084

Merged

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jan 30, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Feb 11, 2026

openshift-merge-bot Bot merged commit 5d7c973 into openshift:master Feb 11, 2026
11 checks passed

danwinship deleted the trafficdistribution branch February 11, 2026 14:55

bentito mentioned this pull request Mar 9, 2026

OCPBUGS-78085: Fix dual-stack service update by preserving IP families #467

Merged

Ultimate-etamitlU mentioned this pull request May 5, 2026

OCPBUGS-55179: Set internalTrafficPolicy to Local on dns-default service #474

Closed

Navigation Menu

Uh oh!

NE-2414: Use `trafficDistribution: PreferSameNode` for openshift-dns Service - #457

NE-2414: Use `trafficDistribution: PreferSameNode` for openshift-dns Service#457
openshift-merge-bot[bot] merged 1 commit into
openshift:masterfrom
danwinship:trafficdistribution

danwinship commented Jan 15, 2026

Uh oh!

openshift-ci-robot commented Jan 15, 2026 •

edited by openshift-ci Bot

Loading

Uh oh!

danwinship commented Jan 15, 2026

Uh oh!

openshift-ci Bot commented Jan 16, 2026

Uh oh!

rikatz commented Jan 23, 2026

Uh oh!

openshift-ci Bot commented Jan 23, 2026

Uh oh!

danwinship commented Jan 26, 2026

Uh oh!

Miciah commented Jan 27, 2026

Uh oh!

candita commented Jan 29, 2026

Uh oh!

lihongan commented Jan 30, 2026

Uh oh!

melvinjoseph86 commented Jan 30, 2026

Uh oh!

openshift-ci-robot commented Jan 30, 2026

Uh oh!

rikatz commented Feb 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

danwinship commented Jan 15, 2026

Uh oh!

openshift-ci-robot commented Jan 15, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danwinship commented Jan 15, 2026

Uh oh!

openshift-ci Bot commented Jan 16, 2026

Uh oh!

rikatz commented Jan 23, 2026

Uh oh!

openshift-ci Bot commented Jan 23, 2026

Uh oh!

danwinship commented Jan 26, 2026

Uh oh!

Miciah commented Jan 27, 2026

Uh oh!

candita commented Jan 29, 2026

Uh oh!

lihongan commented Jan 30, 2026

Uh oh!

melvinjoseph86 commented Jan 30, 2026

Uh oh!

openshift-ci-robot commented Jan 30, 2026

Uh oh!

rikatz commented Feb 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

openshift-ci-robot commented Jan 15, 2026 •

edited by openshift-ci Bot

Loading