Skip to content

Add support for rosidl::Buffer in rosidl Python path for rclpy#250

Open
nvcyc wants to merge 3 commits intorollingfrom
native_buffer/5-rosidl_generator_py
Open

Add support for rosidl::Buffer in rosidl Python path for rclpy#250
nvcyc wants to merge 3 commits intorollingfrom
native_buffer/5-rosidl_generator_py

Conversation

@nvcyc
Copy link
Copy Markdown

@nvcyc nvcyc commented Mar 16, 2026

Description

This pull request adds rosidl::Buffer support for uint8[] fields in the Python code generation path.

This pull request consists of the following key changes:

  • Python message setters (_msg.py.em): For uint8[] fields, the generated setters now allow assigning from rosidl_buffer.Buffer or plain array.array. Existing code that sets msg.data = array.array('B', ...) continues to work unchanged.
  • Python/C conversion (_msg_support.c.em): Messages with uint8[] fields use the sequence's is_rosidl_buffer flag to determine the conversion path:
    • Python to C: For non-CPU rosidl_buffer.Buffer objects, calls _get_buffer_ptr() and sets the C++ Buffer pointer plus is_rosidl_buffer = true so the RMW path uses the existing buffer without copying. For array.array objects, uses the normal byte-copy path.
    • C to Python: When the RMW deserializes into a vendor-backed buffer (is_rosidl_buffer == true), calls _take_buffer_from_ptr() to wrap the pointer in a Python Buffer (taking ownership) and set it on the message. CPU-based data is converted to array.array as before.

Is this user-facing behavior change?

This pull request does not change existing rclpy behavior. Existing code using array.array for uint8[] fields continues to work. The rosidl_buffer.Buffer type is only encountered when a subscriber receives data from a non-CPU buffer backend (and has opted in via acceptable_buffer_backends).

Did you use Generative AI?

Yes. Claude (claude-4.6-opus) via Cursor was used to assist with creating an initial prototype version of the changes contained in this PR.

Additional Information

This PR is part of the broader ROS 2 native buffer feature introduced in this post.

hidmic
hidmic reviewed Mar 18, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@hidmic hidmic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass. Ownership is bit fuzzy.

nvcyc added 2 commits March 22, 2026 06:19
@nvcyc
Add missing rosidl_buffer_py dependency
27de24b 
Signed-off-by: CY Chen <cyc@nvidia.com>
@nvcyc
Add owns_rosidl_buffer assignment
77ffdc5 
Signed-off-by: CY Chen <cyc@nvidia.com>
hidmic
hidmic approved these changes Mar 30, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@hidmic hidmic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

hidmic
hidmic approved these changes Mar 30, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@hidmic hidmic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ahcorde ahcorde mentioned this pull request Apr 1, 2026
Open
@ahcorde
Copy link
Copy Markdown
Contributor

ahcorde commented Apr 1, 2026

Pulls: #250
Gist: https://gist.githubusercontent.com/ahcorde/850e6ecce522c00374d85c84b1381cd8/raw/c7d233d4286d736ba60f776babbe0e0ccba0d270/ros2.repos
BUILD args: --packages-above-and-dependencies rosidl_generator_py
TEST args: --packages-above rosidl_generator_py
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18788

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

This was referenced Apr 1, 2026
Open
Open
@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 1, 2026

Pulls: ros2/rosidl#943, ros2/rosidl_typesupport_fastrtps#144, ros2/rosidl_runtime_py#39, #250
Gist: https://gist.githubusercontent.com/nvcyc/3c858c02920a7febaeeb200342424460/raw/597b9f91085df45556fea2b8a350e6b04438e57f/ros2.repos
BUILD args: --continue-on-error --packages-above-and-dependencies rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp ‎‎rosidl_typesupport_fastrtps_‎c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py
TEST args: --packages-above rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp ‎‎rosidl_typesupport_fastrtps_‎c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18796

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@nvcyc nvcyc mentioned this pull request Apr 1, 2026
Open
@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 1, 2026

Pulls: ros2/rosidl#943, ros2/rosidl_typesupport_fastrtps#144, ros2/rosidl_runtime_py#39, #250
Gist: https://gist.githubusercontent.com/nvcyc/63f913be77e0ac65d906ccf58f94aca7/raw/597b9f91085df45556fea2b8a350e6b04438e57f/ros2.repos
BUILD args: --continue-on-error --packages-above-and-dependencies rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py
TEST args: --packages-above rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18798

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 2, 2026

Pulls: ros2/rosidl#943, ros2/rosidl_typesupport_fastrtps#144, ros2/rosidl_runtime_py#39, #250, ros2/rosidl_core#14, ros2/rmw_cyclonedds#575
Gist: https://gist.githubusercontent.com/nvcyc/dfd06fa52cb41f00f46ec87a19a6a8a6/raw/fa6cc1a80d16030d21aea20bed2065b88f8e3900/ros2.repos
BUILD args: --continue-on-error --packages-above-and-dependencies rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime
TEST args: --packages-above rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18801

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

This was referenced Apr 2, 2026
Open
Merged
Open
wjwwood
wjwwood approved these changes Apr 2, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@wjwwood wjwwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 2, 2026

Pulls: ros2/rosidl#943, ros2/rosidl_typesupport_fastrtps#144, ros2/rosidl_runtime_py#39, #250, ros2/rosidl_core#14, ros2/rmw_cyclonedds#575
Gist: https://gist.githubusercontent.com/nvcyc/d13227291610b614acad252daf9858cb/raw/fa6cc1a80d16030d21aea20bed2065b88f8e3900/ros2.repos
BUILD args: --continue-on-error --packages-above-and-dependencies rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime
TEST args: --packages-above rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18807

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@nvcyc nvcyc mentioned this pull request Apr 3, 2026
Open
@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 4, 2026

Triggered with custom ros2.repos that contain all PRs: https://raw.githubusercontent.com/nvcyc/ros2/refs/heads/rolling-native-buffer-prs/ros2.repos

Started by user CY Chen
Running as SYSTEM
Building on the built-in node in workspace /var/lib/jenkins/workspace/ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18827

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

@nvcyc
Copy link
Copy Markdown
Author

nvcyc commented Apr 4, 2026

Pulls: ros2/rosidl#943, ros2/rosidl_typesupport_fastrtps#144, ros2/rosidl_runtime_py#39, #250, ros2/rosidl_core#14, ros2/rmw_fastrtps#867
Gist: https://gist.githubusercontent.com/nvcyc/2b81ce4a59e71ec57d7d52f520fab590/raw/a28b02c5a17f2e12f08af57ffeb8479e1e399f98/ros2.repos
BUILD args: --continue-on-error --packages-above-and-dependencies rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime rmw_fastrtps_cpp rmw_fastrtps_shared_cpp
TEST args: --packages-above rosidl_buffer rosidl_buffer_py rosidl_runtime_c rosidl_typesupport_introspection_c rosidl_typesupport_introspection_tests rosidl_generator_cpp rosidl_runtime_cpp rosidl_typesupport_introspection_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_runtime_py rosidl_generator_py rosidl_core_runtime rmw_fastrtps_cpp rmw_fastrtps_shared_cpp
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/18835

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wjwwood wjwwood wjwwood approved these changes

@hidmic hidmic hidmic approved these changes

@mjcarroll mjcarroll Awaiting requested review from mjcarroll

@tfoote tfoote Awaiting requested review from tfoote

@karsten-nvidia karsten-nvidia Awaiting requested review from karsten-nvidia

@skyegalaxy skyegalaxy Awaiting requested review from skyegalaxy

@MiguelCompany MiguelCompany Awaiting requested review from MiguelCompany

@ahcorde ahcorde Awaiting requested review from ahcorde

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@nvcyc @ahcorde @wjwwood @mjcarroll @hidmic