[BugFix] Fix cudagraph_policy instability and add action std test - #4143

[BugFix] Fix cudagraph_policy instability and add action std test#4143

vmoens merged 2 commits into

pytorch:mainpytorch/rl:mainfrom

coder-jayp:fix/cudagraph-policycoder-jayp/rl:fix/cudagraph-policyCopy head branch name to clipboard