Add per-environment geom visibility mask for heterogeneous entities#53
Open
Kashu7100 wants to merge 1 commit into
Open
Add per-environment geom visibility mask for heterogeneous entities#53Kashu7100 wants to merge 1 commit into
Kashu7100 wants to merge 1 commit into
Conversation
Author
|
Local working directory where this branch was developed and built: |
Author
Author
video_old.mp4video_new.mp4 |
The geometry list in gs-madrona is global and shared across all worlds: every world creates the same `numGeoms` RenderEntity instances and the only way to make a geom non-renderable was the global `geomDataIDs == -1` geom-group filter, applied identically to every world. There was no way to express "this geom variant only exists in a subset of environments", so heterogeneous entities (objects whose geometry differs per env) were drawn in every world. This adds an optional per-(world, geom) visibility mask, the renderer-side analog of the `env_active` mask in genesis-world PR #2958 (which fixed the same class of bug in the pyrender rasterizer): - New optional `geom_env_mask` kwarg (int32 [num_worlds, num_geoms], 1=visible / 0=hidden) on MadronaBatchRenderer.__init__; None keeps legacy behavior (all geoms visible everywhere). - Flows through GSModel.geomEnvMask -> H2D copy in Manager::Impl::make -> Sim::Config.geomEnvMask -> Sim::Sim, which disables the per-world instance's renderable when masked off. The per-world instance entity is still created (just non-renderable), so the exported [numWorlds, numGeoms] columns stay aligned and the per-frame pose memcpy is unaffected. The mask is consumed once at construction (heterogeneity is static), so there is no per-frame cost and masked geoms never enter the draw/BVH set. Note: the matching producer side (emitting geom_env_mask and full, non-compacted per-world poses) must be implemented in the Genesis madrona GeomRetriever. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
86f65c6 to
656ca81
Compare
Kashu7100
added a commit
to Kashu7100/Genesis
that referenced
this pull request
Jun 17, 2026
Heterogeneous entities (objects whose geometry variant only exists in a subset of environments) were drawn in every environment by the Madrona batch renderer. Unlike the pyrender path fixed in Genesis-Embodied-AI#2958, the Madrona retriever already feeds dense per-env poses (`vgeoms_state.pos` -> [num_worlds, num_vgeoms]) and never compacts them, so the poses were already aligned -- the missing piece was per-environment visibility. `GenesisGeomRetriever.retrieve_rigid_meshes_static` now emits an optional `geom_env_mask` of shape [num_worlds, num_vgeoms] (1 = visible, 0 = hidden) built from each vgeom's `active_envs_mask`. This is the Madrona analog of the `active_envs` mask added to the pyrender rasterizer in Genesis-Embodied-AI#2958. The mask is omitted for homogeneous scenes, preserving legacy behavior and keeping back-compat with Madrona builds that do not yet accept it. Requires the matching gs-madrona change that consumes `geom_env_mask` (Genesis-Embodied-AI/gs-madrona#53). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Renderer-side support for heterogeneous entities — objects whose geometry variant only exists in a subset of environments. This is the gs-madrona analog of genesis-world#2958, which fixed the same class of bug in the pyrender rasterizer via a per-
(primitive, env)env_activemask.The gap
The geometry list in gs-madrona is global and shared across all worlds: every world creates the same
numGeomsRenderEntityinstances, and the only way to make a geom non-renderable was the globalgeomDataIDs == -1geom-group filter, applied identically to every world (sim.cpp). There was no way to express "this geom variant only exists in envs X, Y", so heterogeneous variant geoms render in every world.The change
Adds an optional per-
(world, geom)visibility mask:geom_env_maskkwarg onMadronaBatchRenderer.__init__—int32 [num_worlds, num_geoms],1= visible /0= hidden. Defaults toNone→ fully backward compatible (all geoms visible everywhere).GSModel.geomEnvMask→ H2D copy inManager::Impl::make→Sim::Config.geomEnvMask→Sim::Sim, which disables the per-world instance's renderable when masked off.[numWorlds, numGeoms]Position/Rotation/Scale columns stay aligned and the per-frame pose memcpy is unaffected.The mask is consumed once at construction (heterogeneity is static), so there is no per-frame cost and masked geoms never enter the draw/BVH set.
Files
src/bridge/mgr.hpp—GSModel::geomEnvMasksrc/bridge/sim.hpp—Sim::Config::geomEnvMasksrc/bridge/sim.cpp— per-world renderable disablesrc/bridge/mgr.cpp— H2D upload + wire-in + freesrc/bridge/bindings.cpp— optionalgeom_env_maskkwargsrc/bridge/gs_madrona/renderer_gs.py— documents the new optionalGeomRetrievercontractBuild / test
Builds cleanly from source (Clang 18.1.8 bundled toolchain, CUDA 12.6):
sim.cpp,mgr.cpp,bindings.cppcompile and link with zero new warnings; the resulting wheel imports and exposes the newgeom_env_maskkwarg.Follow-up (not in this PR)
The matching producer side must be implemented in the Genesis madrona
GeomRetriever: emitgeom_env_maskand full, non-compacted per-world poses for heterogeneous geoms — the direct analog of thevis/rasterizer_context.pychange in genesis-world#2958.Note
Draft — opened for review of the renderer-side approach before wiring up the Genesis producer.
🤖 Generated with Claude Code