Skip to content

Add check_cudf, and a few other validation changes - #8038

Merged
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
jcrist:more-validation-functions
May 4, 2026
Merged

Add check_cudf, and a few other validation changes#8038
rapids-bot[bot] merged 2 commits into
NVIDIA:mainfrom
jcrist:more-validation-functions

Conversation

@jcrist

@jcrist jcrist commented May 1, 2026

Copy link
Copy Markdown
Member

This is staging some validation functionality needed to cleanup and improve our preprocessing encoders. I've split out the validation changes into a separate PR (with tests) to ease review.

The main changes here are:

  • Addition of a new check_cudf function. This validates and coerces user input into a cudf object (a DataFrame, Series, or either depending on dimensionality), following the same general validation flow as check_array. This is useful for estimators that make use of cudf functionality, or those that need to work with non-numeric types (like our encoders). The checks and knobs are pared down to just what is needed for these estimators, but all error messages and checks are compatible with what sklearn expects.
  • Addition of a return_index option to check_y. In rare occasions we sometimes want to return an output that aligns with the index of y, and having a way to strip that off in check_y is useful.
  • Addition of ensure_discrete_classes to check_y. Setting this to False disables the continuous value check used when encoding class labels.

These will be used in follow-up PRs to simplify several preprocessing estimators, as well as metrics and other label-handling code.

@jcrist jcrist self-assigned this May 1, 2026
@jcrist
jcrist requested a review from a team as a code owner May 1, 2026 02:55
@jcrist jcrist added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 1, 2026
@jcrist
jcrist requested a review from divyegala May 1, 2026 02:55
@github-actions github-actions Bot added the Cython / Python Cython or Python issue label May 1, 2026
@coderabbitai

coderabbitai Bot commented May 1, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b6016328-2335-4dad-963e-c2cabedb4d4f

📥 Commits

Reviewing files that changed from the base of the PR and between c082073 and 37cd068.

📒 Files selected for processing (2)
  • python/cuml/cuml/internals/validation.py
  • python/cuml/tests/test_validation.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • python/cuml/tests/test_validation.py

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added cuDF-aware validation with optional dimensional coercion and index return
    • check_y gains option to relax discrete-class enforcement and to return aligned index/classes
  • Bug Fixes

    • Improved dtype conversion with robust fallbacks for host/cuDF conversions and class extraction
  • Tests

    • Expanded validation tests covering cuDF behavior, index persistence, dimensional coercion, and new options

Walkthrough

Added index coercion utilities and a new public check_cudf function for cudf-aware coercion and shape validation; refactored check_array to handle cudf dtype fallbacks; extended check_y with ensure_discrete_classes and return_index options and updated class extraction and return signatures to include coerced indices.

Changes

Core validation utilities

Layer / File(s) Summary
Exports
python/cuml/cuml/internals/validation.py
__all__ extended to export check_cudf.
Index coercion helper
python/cuml/cuml/internals/validation.py
Added _index_as_mem_type(index, mem_type=None) to coerce pandas/cudf indices to a requested memory type.
Array conversion fallback
python/cuml/cuml/internals/validation.py
Refactored cudf host dense conversion in check_array: to_numpy(dtype=...) wrapped in try/except NotImplementedError and falls back via to_numpy(dtype="object") then np.asarray(..., dtype=...).
check_array index return
python/cuml/cuml/internals/validation.py
check_array(..., return_index=True) now returns index coerced via _index_as_mem_type.
New public API
python/cuml/cuml/internals/validation.py
Added check_cudf(array, *, ensure_ndim=2, coerce_ndim=False, ensure_min_samples=1, ensure_min_features=1, input_name=None) that coerces inputs to cudf.Series/DataFrame, enforces ndim/coercion policies, and supports min-sample/min-feature checks.
Classification targets & signatures
python/cuml/cuml/internals/validation.py
_check_classification_targets(y) gains ensure_discrete_classes=True. check_y signature updated to accept ensure_discrete_classes and return_index, extracts index from pandas/cudf inputs, derives mem_type, coerces index via _index_as_mem_type, and when returning classes preserves fallback to object dtype on cudf NotImplementedError. Return tuples now include index when return_index=True.

Test coverage

Layer / File(s) Summary
Imports / test entry
python/cuml/tests/test_validation.py
Imported check_cudf into test module.
Label-type tests
python/cuml/tests/test_validation.py
Extended test_check_y_return_classes to include "string" labels with conditional casting for pandas/cuDF; adjusted Hypothesis strategies.
Classifier float behavior
python/cuml/tests/test_validation.py
Updated test_check_y_classifier_floating_input_errors so NaN/Inf/non-integral labels no longer raise when ensure_discrete_classes=False.
Index return tests
python/cuml/tests/test_validation.py
Added test_check_y_return_index to assert check_y(..., return_index=True) returns correct index type per mem_type and consistent class encoding when return_classes=True.
check_cudf tests
python/cuml/tests/test_validation.py
Added test_check_cudf (dimensionality/coercion/warning/error behavior), test_check_cudf_bad_args (invalid arg assertions), and test_check_cudf_persists_index (index preservation across pandas/cuDF inputs).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • csadorf
  • dantegd
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 46.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Add check_cudf, and a few other validation changes' accurately summarizes the main changes: introducing check_cudf function and extending check_y with new parameters.
Description check ✅ Passed The description clearly explains the PR's purpose, detailing the three main changes (check_cudf, return_index option, ensure_discrete_classes option) and their use cases.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Review rate limit: 9/10 reviews remaining, refill in 6 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@python/cuml/cuml/internals/validation.py`:
- Around line 844-848: Add stacklevel=2 to the warnings.warn calls in the
validation helpers so the warning reports the caller's location rather than this
internal function; update the warnings.warn invocations shown (the column-vector
message and the other similar warn at lines ~855-859) to pass stacklevel=2
(i.e., warnings.warn(..., DataConversionWarning, stacklevel=2)) so callers of
check_cudf and related validation functions receive correctly attributed
warnings.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a7bee1e1-8b05-42fc-93a1-35c96c7ec2d7

📥 Commits

Reviewing files that changed from the base of the PR and between 016840c and c082073.

📒 Files selected for processing (2)
  • python/cuml/cuml/internals/validation.py
  • python/cuml/tests/test_validation.py

Comment thread python/cuml/cuml/internals/validation.py
Comment thread python/cuml/cuml/internals/validation.py
Comment thread python/cuml/tests/test_validation.py Outdated
Comment thread python/cuml/cuml/internals/validation.py
Comment thread python/cuml/cuml/internals/validation.py Outdated
jcrist added 2 commits May 4, 2026 11:54
This is staging some validation functionality needed to cleanup and
improve our preprocessing encoders. I've split out the validation
changes into a separate PR (with tests) to ease review.

The main changes here are:

- Addition of a new `check_cudf` function. This validates and coerces
  user input into a `cudf` object (a `DataFrame`, `Series`, or
  either depending on dimensionality), following the same general
  validation flow as `check_array`. This is useful for estimators that
  make use of `cudf` functionality, or those that need to work with
  non-numeric types (like our encoders). The checks and knobs are pared
  down to just what is needed for these estimators, but all error
  messages and checks are compatible with what sklearn expects.
- Addition of a `return_index` option to `check_y`. In rare occasions we
  sometimes want to return an output that aligns with the index of `y`,
  and having a way to strip that off in `check_y` is useful.
- Addition of `ensure_discrete_classes` to `check_y`. Setting this to
  `False` disables the continuous value check used when encoding class
  labels.

These will be used in follow-up PRs to simplify several preprocessing
estimators, as well as metrics and other label-handling code.
@jcrist
jcrist force-pushed the more-validation-functions branch from c082073 to 37cd068 Compare May 4, 2026 16:55
@jcrist

jcrist commented May 4, 2026

Copy link
Copy Markdown
Member Author

Thanks for the review! I believe all comments have been addressed.

@csadorf csadorf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread python/cuml/cuml/internals/validation.py
@jcrist

jcrist commented May 4, 2026

Copy link
Copy Markdown
Member Author

/merge

@rapids-bot
rapids-bot Bot merged commit b932c49 into NVIDIA:main May 4, 2026
172 of 174 checks passed
@jcrist
jcrist deleted the more-validation-functions branch May 4, 2026 21:42
rapids-bot Bot pushed a commit that referenced this pull request May 8, 2026
This cleans up our `LabelEncoder` implementation, preparing for exposing it through `cuml.accel`. This one needed a lot of work, and there are _several breaking changes_.

- The type of `LabelEncoder.classes_` is now a `numpy.ndarray` instead of a `cudf.Series`. Note that `classes_` was undocumented. This mirrors the `classes_` attribute in all our other classifiers (all of which also made this as a breaking change many releases ago). I view this in the same vein.
- Likewise, since `OneHotEncoder` and `OrdinalEncoder` make use of `LabelEncoder` internally, the type of `categories_` has changed there from `list[cudf.Series]` to `list[numpy.ndarray]`. This now better matches with sklearn, and makes sense IMO in the same way that changing `classes_` above did.
- `LabelEncoder` now follows our type-reflection standards, previously it always returned `cudf` objects (even though the docstrings implied type-reflection was followed). I view this as a bugfix.
- `LabelEncoder.inverse_transform` now handles `handle_unknown="ignore"` _as documented_. Previously this would return incorrect values in the presence of missing values. Note that `handle_unknown` is _not_ an argument to `sklearn.preprocessing.LabelEncoder`. I view this as a bugfix.

This required munging in our tests and the implementations of `OrdinalEncoder`/`OneHotEncoder` (and their dask counterparts), but beyond the changes listed above should have no other user-facing ramifications.

Stacked on top of #8038.
Part of #7317.
Precursor for #8015.

Authors:
  - Jim Crist-Harif (https://github.com/jcrist)

Approvers:
  - Simon Adorf (https://github.com/csadorf)

URL: #8039
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cython / Python Cython or Python issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants