externalip-manager

A Kubernetes operator to resolve and inject public/external IP addresses for your k8s services running in a private network. This was created to allow apps like external-dns to see the "true" public IP addresses of a service that is behind a NAT/NPT setup. externalip-manager manages this by first resolving, then injecting the public IPs of a service into the services externalIP field. This field is read by apps like external-dns and most Ingress/Gateway API controllers, such as ingress-nginx and istio:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  annotations:
    # 👇 see below, this is how externalip-manager discovers the external IPs
    externalip.spacebird.dev/cluster-external-ip-source: my-source
spec:
  type: LoadBalancer
  externalIPs: 
    - 198.51.100.32 
    - "2001:db8:dead:beef::1"
    # 👆 the true external IP(s), injected by externalIP and read by external-dns + Ingress/GW
status:
  loadBalancer:
    ingress:
      # 👇 IP address assigned by MetalLB or similar
      - ip: "10.47.10.5" 

Note: Although the externalIPs field has been deprecated, this does not affect externalip-manager. The deprecation will cause warnings to be emitted, but the field won't be removed from the API, so as long as the upstream services like external-dns read it, this will continue to work.

Overview

The purpose of this operator is to automate the management of the externalIP field in situations where manual assignment is unfeasible (such as dynamic IP addresses). To do so, it uses a new resource type ClusterExternalIPSource containing one or more solvers for determining the external IP addresses:

apiVersion: externalip.spacebird.dev/v1alpha1
kind: ClusterExternalIPSource
metadta:
    name: public
spec:
  ipv4:
    solvers:
      - dnsHostname:
          host: "cluster-public-ip.example.com"
  ipv6:
    solvers:
      - loadBalancerIngress: {}

Services can then select their externalIP source through an annotation:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  annotations:
    externalip.spacebird.dev/cluster-external-ip-source: public
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: MyApp
  ports:
    - port: 80

externalip-manager will then pick up this service and query the solvers in the ClusterExternalIPSource until valid IP addresses are found. It will then write them into the externalIP field of the service and regularly check the sources for any changes. From there, an Ingress controller can then pick up the externalIP field and use it to advertise Ingress IP addresses for ExternalDNS. In particular, ingress-nginx uses both the externalIP field the loadBalancer.ingress status as provisioned by MetalLB, so your Ingress resources will have both public and internal IPs set. You can then use 'net-filter' parameters for external-dns to further restrict your published IPs, depending on your networking (Hairpin NAT or split-Horizon DNS).

The following solvers are currently available:

• dnsHostname: Perform a DNS query and use the IPs returned in A/AAAA records.
  • Use case: You have a firewall/NAT gateway that sets a DNS record with the public IP.
  • Parameters:
    • host: The host to resolve
• ìpAPI: Uses a "what-is-my-ip" style API to retrieve public addresses
  • Parameters:
    • provider: Which API Provider to use. Current options are myIp and ipify
• loadBalancerIngress: Use the addresses specified in the .status.loadBalancer.ingress field
  • Use case: You have MetalLB or a similar LoadBalancer providing you with some public addresses
  • Parameters: None
• interface: Use the addresses assigned to interfaces on the node. You can optionally specify the interface name, else all public addresses will be used.
  • This requires the pod to run with hostNetworking set to true.
• static: Just return a set of fixed IP addresses. Useful as a fallback or when used in combination with merge
• merge: Create an IP address by merging parts of different IP addresses together. Useful when you have an external network prefix that differs from your node one, such as with NPTv6.
  • This meta-solver queries several sub-solver and then merges their results based on a supplied netmask.
  • It takes a list of partialSolvers, where each partial solver has one regular solver (except merge) and a mask.
  • After all partial solvers have been queried, their results are combined into one address that is then returned.
  • For an example of how to use it, see here

You can optionally define multiple solvers for a single IP source:

piVersion: externalip.spacebird.dev/v1alpha1
kind: ClusterExternalIPSource
metadta:
    name: public
spec:
  ipv4:
    queryMode: firstFound
    solvers:
      - dnsHostname:
          host: "cluster-public-ip.example.com"
      - loadBalancerIngress: {}

By default, the controller will attempt each solver in sequence until one returns a valid address (queryMode == firstFound). You can also have the controller query all solvers and return the combined set of addresses by setting queryMode to all.

For more examples, see the manifests directory in test.

Installation

To install this operator, use the Helm chart at spacebird-dev/charts.

To see the minimum supported k8s version, check the k8s-openapi feature flag in crates/bin/Cargo.toml

Building

This operator is built in Rust, using standard cargo tooling. You may want to install the just command runner to run the recipes in the Justfile. cross is used for cross-compilation.

When making changes to the CRDs, please run just crds before committing any changes. There is also a pre-commit hook that does this for you if you run pre-commit install