Skip to content

Add per-environment geom visibility mask for heterogeneous entities#53

Open
Kashu7100 wants to merge 1 commit into
Genesis-Embodied-AI:mainfrom
Kashu7100:fix/heterogeneous-env-visibility
Open

Add per-environment geom visibility mask for heterogeneous entities#53
Kashu7100 wants to merge 1 commit into
Genesis-Embodied-AI:mainfrom
Kashu7100:fix/heterogeneous-env-visibility

Conversation

@Kashu7100

Copy link
Copy Markdown

Summary

Renderer-side support for heterogeneous entities — objects whose geometry variant only exists in a subset of environments. This is the gs-madrona analog of genesis-world#2958, which fixed the same class of bug in the pyrender rasterizer via a per-(primitive, env) env_active mask.

The gap

The geometry list in gs-madrona is global and shared across all worlds: every world creates the same numGeoms RenderEntity instances, and the only way to make a geom non-renderable was the global geomDataIDs == -1 geom-group filter, applied identically to every world (sim.cpp). There was no way to express "this geom variant only exists in envs X, Y", so heterogeneous variant geoms render in every world.

The change

Adds an optional per-(world, geom) visibility mask:

  • New optional geom_env_mask kwarg on MadronaBatchRenderer.__init__int32 [num_worlds, num_geoms], 1 = visible / 0 = hidden. Defaults to None → fully backward compatible (all geoms visible everywhere).
  • Flows through GSModel.geomEnvMask → H2D copy in Manager::Impl::makeSim::Config.geomEnvMaskSim::Sim, which disables the per-world instance's renderable when masked off.
  • The per-world instance entity is still created (just non-renderable), so the exported [numWorlds, numGeoms] Position/Rotation/Scale columns stay aligned and the per-frame pose memcpy is unaffected.

The mask is consumed once at construction (heterogeneity is static), so there is no per-frame cost and masked geoms never enter the draw/BVH set.

Files

  • src/bridge/mgr.hppGSModel::geomEnvMask
  • src/bridge/sim.hppSim::Config::geomEnvMask
  • src/bridge/sim.cpp — per-world renderable disable
  • src/bridge/mgr.cpp — H2D upload + wire-in + free
  • src/bridge/bindings.cpp — optional geom_env_mask kwarg
  • src/bridge/gs_madrona/renderer_gs.py — documents the new optional GeomRetriever contract

Build / test

Builds cleanly from source (Clang 18.1.8 bundled toolchain, CUDA 12.6): sim.cpp, mgr.cpp, bindings.cpp compile and link with zero new warnings; the resulting wheel imports and exposes the new geom_env_mask kwarg.

Follow-up (not in this PR)

The matching producer side must be implemented in the Genesis madrona GeomRetriever: emit geom_env_mask and full, non-compacted per-world poses for heterogeneous geoms — the direct analog of the vis/rasterizer_context.py change in genesis-world#2958.

Note

Draft — opened for review of the renderer-side approach before wiring up the Genesis producer.

🤖 Generated with Claude Code

@Kashu7100

Copy link
Copy Markdown
Author

Local working directory where this branch was developed and built: /home/kashu/research/gs-madrona

@Kashu7100

Copy link
Copy Markdown
Author
comparison_old_vs_new

@Kashu7100

Copy link
Copy Markdown
Author
video_old.mp4
video_new.mp4

@Kashu7100 Kashu7100 marked this pull request as ready for review June 17, 2026 01:33
The geometry list in gs-madrona is global and shared across all worlds:
every world creates the same `numGeoms` RenderEntity instances and the
only way to make a geom non-renderable was the global `geomDataIDs == -1`
geom-group filter, applied identically to every world. There was no way
to express "this geom variant only exists in a subset of environments",
so heterogeneous entities (objects whose geometry differs per env) were
drawn in every world.

This adds an optional per-(world, geom) visibility mask, the renderer-side
analog of the `env_active` mask in genesis-world PR #2958 (which fixed the
same class of bug in the pyrender rasterizer):

- New optional `geom_env_mask` kwarg (int32 [num_worlds, num_geoms], 1=visible
  / 0=hidden) on MadronaBatchRenderer.__init__; None keeps legacy behavior
  (all geoms visible everywhere).
- Flows through GSModel.geomEnvMask -> H2D copy in Manager::Impl::make ->
  Sim::Config.geomEnvMask -> Sim::Sim, which disables the per-world instance's
  renderable when masked off. The per-world instance entity is still created
  (just non-renderable), so the exported [numWorlds, numGeoms] columns stay
  aligned and the per-frame pose memcpy is unaffected.

The mask is consumed once at construction (heterogeneity is static), so there
is no per-frame cost and masked geoms never enter the draw/BVH set.

Note: the matching producer side (emitting geom_env_mask and full, non-compacted
per-world poses) must be implemented in the Genesis madrona GeomRetriever.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Kashu7100 Kashu7100 force-pushed the fix/heterogeneous-env-visibility branch from 86f65c6 to 656ca81 Compare June 17, 2026 01:39
Kashu7100 added a commit to Kashu7100/Genesis that referenced this pull request Jun 17, 2026
Heterogeneous entities (objects whose geometry variant only exists in a
subset of environments) were drawn in every environment by the Madrona
batch renderer. Unlike the pyrender path fixed in Genesis-Embodied-AI#2958, the Madrona
retriever already feeds dense per-env poses (`vgeoms_state.pos` ->
[num_worlds, num_vgeoms]) and never compacts them, so the poses were
already aligned -- the missing piece was per-environment visibility.

`GenesisGeomRetriever.retrieve_rigid_meshes_static` now emits an optional
`geom_env_mask` of shape [num_worlds, num_vgeoms] (1 = visible, 0 = hidden)
built from each vgeom's `active_envs_mask`. This is the Madrona analog of
the `active_envs` mask added to the pyrender rasterizer in Genesis-Embodied-AI#2958. The mask
is omitted for homogeneous scenes, preserving legacy behavior and keeping
back-compat with Madrona builds that do not yet accept it.

Requires the matching gs-madrona change that consumes `geom_env_mask`
(Genesis-Embodied-AI/gs-madrona#53).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant