-
Notifications
You must be signed in to change notification settings - Fork 67
feat(charts): port nvcf api helm chart #856
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| vault-secrets.json | ||
| nvcr.io.txt | ||
|
|
||
| bin/ | ||
| packaged-charts/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # AGENTS.md - NVCF API Helm chart | ||
|
|
||
| Scope: `deploy/helm/cloud-functions`, the Helm chart for the NVCF API service. | ||
|
|
||
| The chart publishes as `helm-nvcf-api`. Its `Chart.yaml` name is the published | ||
| OCI name and must not be renamed to match the directory or the service. The | ||
| release lane is registered as `cloud-functions-helm` in | ||
| `tools/ci/github-release-subprojects.json`. | ||
|
|
||
| ## Commands | ||
|
|
||
| Run these from this directory. | ||
|
|
||
| ```sh | ||
| make lint # helm lint with the shared CI values | ||
| make template # render to bin/manifest.yaml | ||
| make validate # template, then kubeconform | ||
| make test # tests/sidecar_release_artifacts_test.sh | ||
| ``` | ||
|
|
||
| `lint`, `template`, and `validate` read | ||
| `tools/ci/helm-validate-values/cloud-functions.yaml`. The chart leaves | ||
| `api.image.registry`, `api.image.repository`, and the matching | ||
| `api.accountBootstrap.image` fields empty on purpose, so it does not render | ||
| without those values. | ||
|
|
||
| `make install`, `make uninstall`, and `make status` default to release `api` in | ||
| namespace `nvcf`. | ||
|
|
||
| ## Conventions | ||
|
|
||
| `Chart.yaml` keeps `version: 0.0.0`. The release pipeline sets the real version | ||
| from the `deploy/helm/cloud-functions/v*` tag. `appVersion` tracks the API | ||
| service image and is bumped by hand. | ||
|
|
||
| Match the conventional commit type to the intended version bump, because the tag | ||
| generation pipeline reads it: | ||
|
|
||
| - `fix(<scope>):` for a patch bump | ||
| - `feat(<scope>):` for a minor bump | ||
| - `feat(<scope>)!:` or a `BREAKING CHANGE:` footer for a major bump | ||
|
|
||
| A `chore` commit skips tag creation entirely. | ||
|
|
||
| ## Adjacent subtrees | ||
|
|
||
| - `src/control-plane-services/cloud-functions` builds the image this chart deploys. | ||
| - Sidecar image references in `api.remoteConfig.configData.nvcf.sidecars` use | ||
| Spring placeholders resolved at runtime. `tests/sidecar_release_artifacts_test.sh` | ||
| guards the `release-artifact-*-image` annotations that surface them to stack | ||
| release tooling, so run it after changing any sidecar entry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| @AGENTS.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # Variables | ||
| release ?= api | ||
| namespace ?= nvcf | ||
| helm_dir ?= ./nvcf-api | ||
| values := $(helm_dir)/values.yaml | ||
| # CI-only values for lint/template, shared with the chart validation job. | ||
| validation_values ?= ../../../tools/ci/helm-validate-values/cloud-functions.yaml | ||
|
|
||
| # OPTIONAL for deploy target: Path to an additional Helm values file. | ||
| # Example: make deploy values=my-values.yaml additional_values=override.yaml | ||
| additional_values ?= | ||
|
|
||
| # OCI Registry details | ||
| OCI_REGISTRY_HOST ?= nvcr.io | ||
| OCI_REGISTRY_NAMESPACE ?= 0651155215864979/ncp-dev | ||
|
|
||
| # Automatically determine chart name and version from Chart.yaml | ||
| # IMPORTANT: For this setup, CHART_NAME is expected to include any desired OCI prefix (e.g., "helm-yourchart") | ||
| # as defined in helm/Chart.yaml's 'name' field. | ||
| CHART_NAME := $(shell yq -r .name $(helm_dir)/Chart.yaml) | ||
| CHART_VERSION := $(shell yq -r .version $(helm_dir)/Chart.yaml) | ||
|
|
||
| .PHONY: install uninstall status lint template validate test clean package push-oci | ||
|
|
||
| install: | ||
| ifndef values | ||
| $(error "values" variable is not set. Please specify with 'make deploy values=<path-to-your-values.yaml>') | ||
| endif | ||
| @echo "Deploying $(release) to namespace $(namespace) using values file '$(values)'..." | ||
| @echo "Additional values file: '$(if $(additional_values),$(additional_values),N/A)'" | ||
| helm install $(release) $(helm_dir) \ | ||
| --namespace $(namespace) \ | ||
| --values $(values) \ | ||
| $(if $(additional_values),--values $(additional_values),) \ | ||
| --atomic \ | ||
| --create-namespace \ | ||
| --wait \ | ||
| --wait-for-jobs \ | ||
| --timeout 20m | ||
|
|
||
| uninstall: | ||
| @echo "Deleting $(release) from namespace $(namespace)..." | ||
| helm uninstall $(release) --namespace $(namespace) | ||
|
|
||
| status: | ||
| @echo "Checking status of $(release) in namespace $(namespace)..." | ||
| helm status $(release) --namespace $(namespace) | ||
|
|
||
| lint: | ||
| @echo "Linting chart $(helm_dir)..." | ||
| helm lint $(helm_dir) \ | ||
| $(if $(validation_values),--values $(validation_values),) \ | ||
| $(if $(additional_values),--values $(additional_values),) | ||
|
|
||
| template: | ||
| @echo "Templating chart $(helm_dir)..." | ||
| @mkdir -p bin | ||
| helm template $(release) $(helm_dir) \ | ||
| --namespace $(namespace) \ | ||
| $(if $(validation_values),--values $(validation_values),) \ | ||
| $(if $(additional_values),--values $(additional_values),) \ | ||
| > bin/manifest.yaml | ||
| @echo "Rendered manifest to bin/manifest.yaml" | ||
|
|
||
| validate: template | ||
| @echo "Validating manifest with kubeconform..." | ||
| @kubeconform -strict -summary -output pretty -kubernetes-version 1.31.5 bin/manifest.yaml | ||
|
|
||
| test: | ||
| @echo "Running chart tests..." | ||
| @./tests/sidecar_release_artifacts_test.sh | ||
|
|
||
| # Publish Chart | ||
| # NOTE: this is manual until the CI pipeline is updated to push the chart to the NVCR OCI registry | ||
| clean: | ||
| rm -rf ./bin | ||
| rm -rf ./packaged-charts | ||
|
|
||
| package: clean lint | ||
| @echo "[package] Packaging Helm chart $(CHART_NAME) version $(CHART_VERSION)..." | ||
| @mkdir -p ./packaged-charts | ||
| @helm package $(helm_dir) -d ./packaged-charts/ | ||
| @echo "[package] Packaged chart to ./packaged-charts/$(CHART_NAME)-$(CHART_VERSION).tgz" | ||
|
|
||
| push-oci: | ||
| @echo "[push-oci] Pushing chart $(CHART_NAME) version $(CHART_VERSION) to oci://$(OCI_REGISTRY_HOST)/$(OCI_REGISTRY_NAMESPACE)/$(CHART_NAME):$(CHART_VERSION)" | ||
| @echo "[push-oci] Note: You must be logged into oci://$(OCI_REGISTRY_HOST) for the push to succeed." | ||
| @if [ ! -f ./packaged-charts/$(CHART_NAME)-$(CHART_VERSION).tgz ]; then \ | ||
| echo "[push-oci] Error: Packaged chart ./packaged-charts/$(CHART_NAME)-$(CHART_VERSION).tgz not found. Run 'make package' first."; \ | ||
| exit 1; \ | ||
| fi | ||
| @helm push ./packaged-charts/$(CHART_NAME)-$(CHART_VERSION).tgz oci://$(OCI_REGISTRY_HOST)/$(OCI_REGISTRY_NAMESPACE) | ||
| @echo "[push-oci] Successfully pushed chart to OCI registry." | ||
| @echo "[push-oci] Cleaning up temporary package directory..." | ||
| @rm -rf ./packaged-charts | ||
| @echo "[push-oci] Cleanup complete." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| # NVCF API Helm Chart | ||
|
|
||
| This directory contains the Helm chart for deploying the NVCF API service on Kubernetes. | ||
|
|
||
| ## Overview | ||
|
|
||
| The chart packages the NVCF API deployment together with a post-install account bootstrap hook. | ||
|
|
||
| The default chart values do not set the required image registries and repositories for the API or the account bootstrap job. They must be supplied through an additional values file at install time, and access to those images must be arranged separately. | ||
|
|
||
| Example: | ||
|
|
||
| ```yaml | ||
| api: | ||
| image: | ||
| registry: <your-registry> | ||
| repository: <your-org>/nvcf-api | ||
| tag: <appVersion> | ||
| accountBootstrap: | ||
| image: | ||
| registry: <your-registry> | ||
| repository: <your-org>/nvcf-account-bootstrap | ||
| tag: <version> | ||
| ``` | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Kubernetes cluster | ||
| - Helm 3.x | ||
| - `kubectl` | ||
|
|
||
| ## Getting Started | ||
|
|
||
| Install the chart with the default values plus your own overrides: | ||
|
|
||
| ```bash | ||
| helm install api nvcf-api \ | ||
| --namespace nvcf \ | ||
| --create-namespace \ | ||
| --values nvcf-api/values.yaml \ | ||
| --values path/to/values.yaml \ | ||
| --wait \ | ||
| --wait-for-jobs \ | ||
| --timeout 20m | ||
| ``` | ||
|
|
||
| Upgrade an existing release: | ||
|
|
||
| ```bash | ||
| helm upgrade api nvcf-api \ | ||
| --namespace nvcf \ | ||
| --values nvcf-api/values.yaml \ | ||
| --values path/to/values.yaml \ | ||
| --wait \ | ||
| --wait-for-jobs \ | ||
| --timeout 20m | ||
| ``` | ||
|
|
||
| Uninstall the release: | ||
|
|
||
| ```bash | ||
| helm uninstall api --namespace nvcf | ||
| ``` | ||
|
|
||
| The `Makefile` wraps these with the same defaults (`release=api`, `namespace=nvcf`): | ||
|
|
||
| ```bash | ||
| make install additional_values=path/to/values.yaml | ||
| make status | ||
| make uninstall | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| The default chart configuration lives in `nvcf-api/values.yaml`. | ||
|
|
||
| Important settings to review before deployment: | ||
|
|
||
| - `api.image.*` for the API container image | ||
| - `api.accountBootstrap.image.*` for the post-install bootstrap job image | ||
| - `api.imagePullSecrets` for private registry access | ||
| - `api.replicaCount`, resource requests, and HPA settings for your environment | ||
| - `api.accountBootstrap.accountName`, `api.accountBootstrap.adminClientId`, and `api.accountBootstrap.registryCredentials` for initial account configuration. Use `registryCredentials: []` to create the account without system-provisioned registry credentials. | ||
| - `api.accountBootstrap.limits.*` for per-account quotas | ||
|
|
||
| The default values include development-oriented placeholders. Override them before using the chart in any shared or production environment. | ||
|
|
||
| ## Remote Config RBAC | ||
|
|
||
| `spring-cloud-kubernetes` (v3.3.0, shipped in nvcf-service `1.3.x`) calls `listNamespacedConfigMap` without a `fieldSelector=metadata.name` filter and matches the target name in memory. Because the API request is unfiltered, the Role must grant `list`/`watch` on the configmaps collection itself, with no `resourceNames` scoping for those verbs, so the `nvcf-api` SA can read every ConfigMap in the namespace. Chart assumes none are sensitive; clusters that block broad namespace reads need a policy exception. | ||
|
|
||
| Set `api.remoteConfig.enabled: false` to opt out: the chart no longer renders the broad RBAC, and the service runs on JAR sidecar defaults in `application-ncp.yaml`. The SA keeps default-token automount on (the K8s default) so the vault-k8s injector's fallback service-account discovery continues to find a mount. Hot reload is unavailable. | ||
|
|
||
| ## Sidecar release-artifact annotations | ||
|
|
||
| The `api.remoteConfig.configData.nvcf.sidecars` worker images reference the | ||
| registry via Spring placeholders (`${nvcf.sidecars.hostname}/${nvcf.sidecars.repository}`) | ||
| that resolve at runtime from `NVCF_SIDECARS_HOSTNAME` / `NVCF_SIDECARS_REPOSITORY`. | ||
| Stack release-artifact tooling scans rendered manifests for concrete image | ||
| references and skips any value containing `${...}`, so those worker images would | ||
| otherwise be excluded from stack release artifacts. | ||
|
|
||
| To surface them, the remote-config ConfigMap renders a | ||
| `release-artifact-<sidecar>-image` annotation for every sidecar entry that carries | ||
| the placeholder prefix, with the prefix resolved to | ||
| `<NVCF_SIDECARS_HOSTNAME>/<NVCF_SIDECARS_REPOSITORY>/<image>:<tag>`. This mirrors | ||
| the `release-artifact-*-image` annotations rendered by nvca-operator's | ||
| `self-managed-nvcfbackend-cm.yaml`. Annotations are emitted only when both sidecar | ||
| registry env vars are set, so partially resolved references are never produced. | ||
|
|
||
| ## Account bootstrap | ||
|
|
||
| The chart installs a Helm `post-install` hook Job that calls the API to create an initial NVCF account. The Job waits for the API readiness probe to pass, authenticates to the configured secret store, and issues a single `POST /v2/nvcf/accounts/{ncaId}` request. | ||
|
|
||
| Key points to be noted: | ||
|
|
||
| - The Job is rendered from `nvcf-api/templates/account-bootstrap-hook-job.yaml` and configured by `api.accountBootstrap.*` in `values.yaml`. | ||
| - The bootstrap script lives at `nvcf-api/scripts/account-bootstrap.sh`. | ||
| - `registryCredentials: []` is valid and omits the `registryCredentials` field from the account payload. | ||
| - Non-empty `registryCredentials` may include `CONTAINER` and `HELM` entries. `MODEL` and `RESOURCE` entries are ignored if supplied. | ||
| - Set `DEBUG=true` in the Job environment to enable verbose logging. | ||
|
|
||
| ## Notes | ||
|
|
||
| - If you publish or mirror the required images into another registry, set the image registry, repository, tag, and pull secret values explicitly in your override file. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Patterns to ignore when building packages. | ||
| # This supports shell glob matching, relative path matching, and | ||
| # negation (prefixed with !). Only one pattern per line. | ||
| .DS_Store | ||
| # Common VCS dirs | ||
| .git/ | ||
| .gitignore | ||
| .bzr/ | ||
| .bzrignore | ||
| .hg/ | ||
| .hgignore | ||
| .svn/ | ||
| # Common backup files | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| *.orig | ||
| *~ | ||
| # Various IDEs | ||
| .project | ||
| .idea/ | ||
| *.tmproj | ||
| .vscode/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| apiVersion: v2 | ||
| name: helm-nvcf-api | ||
| description: A Helm chart for NVCF API deployment | ||
|
|
||
| type: application | ||
| version: 0.0.0 # autoversioning enabled via release pipeline | ||
| appVersion: "1.12.6" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.