This is a followup to #20318. In #20595, we changed how our cameras render slightly to at least change the pink screen into a black screen, which makes the missing frame slightly less noticable.
I documented the cause in other words here, however I will rewrite it here to hopefully better explain the issue.
In order to render with a shader in one frame, these things need to happen in this order:
- Call
asset_server.load("my_shader");
- This starts the task that will load the shader.
- The load task needs to complete.
- Shaders are generally cheap to load (especially with embedded assets), so I will assume this always happens.
- The
handle_internal_asset_events system needs to run (this is scheduled in PreUpdate in the main world).
- This takes the shader asset from the load task and adds it to the ECS.
- The
ExtractSchedule must run.
- This moves the shader asset from the main world to the render world
- The
Render schedule must run.
- This first compiles the render pipelines (using the shaders), then runs your rendering system.
- Note that compiling a render pipeline takes time, but
PipelineCache::block_on_render_pipeline allows your rendering system to wait for the pipeline to be ready. Blocking on the render pipeline only works if the shader assets are already in the render world.
In the current state, the order is:
- The first main world frame runs. Nothing happens.
- So far nothing rendering related has run, so no shader loads have even started.
handle_internal_asset_events runs, but doesn't pick up any shader assets, since we never started any loads.
- As part of "extract"
RenderStartup runs.
- It finally calls
asset_server.load("my_shader");. For argument's sake, let's assume the load finishes instantly.
- The
ExtractSchedule runs.
- There are no shader assets to move from the main world to the render world, since
handle_internal_asset_events didn't add any shaders to the ECS.
- The
Render schedule runs.
- We try to compile the render pipelines, but they are missing shader assets, so they can't compile!
- Rendering systems do nothing when calling
PipelineCache::block_on_render_pipeline since the shader assets are missing.
- We render a black screen!
- The main world starts a new frame. From here, everything goes back to normal: the shader asset is added to the ECS, then gets moved to the render world, then the pipelines are compiled, blocked on, and rendered with.
To fix this, we need to either 1) run RenderStartup earlier -- e.g., #20407, or 2) have shaders live entirely in the render world (which eschews the entire asset system) -- which could be done as part of WESL changes maybe?
This is a followup to #20318. In #20595, we changed how our cameras render slightly to at least change the pink screen into a black screen, which makes the missing frame slightly less noticable.
I documented the cause in other words here, however I will rewrite it here to hopefully better explain the issue.
In order to render with a shader in one frame, these things need to happen in this order:
asset_server.load("my_shader");handle_internal_asset_eventssystem needs to run (this is scheduled inPreUpdatein the main world).ExtractSchedulemust run.Renderschedule must run.PipelineCache::block_on_render_pipelineallows your rendering system to wait for the pipeline to be ready. Blocking on the render pipeline only works if the shader assets are already in the render world.In the current state, the order is:
handle_internal_asset_eventsruns, but doesn't pick up any shader assets, since we never started any loads.RenderStartupruns.asset_server.load("my_shader");. For argument's sake, let's assume the load finishes instantly.ExtractScheduleruns.handle_internal_asset_eventsdidn't add any shaders to the ECS.Renderschedule runs.PipelineCache::block_on_render_pipelinesince the shader assets are missing.To fix this, we need to either 1) run
RenderStartupearlier -- e.g., #20407, or 2) have shaders live entirely in the render world (which eschews the entire asset system) -- which could be done as part of WESL changes maybe?