Bug description
On current main 8f92a50cb1e8543d6d7545605e4e5b399224928b, sparse MaskedCategorical.entropy() does not exclude padded indices. With default neg_inf=-inf, a padded logit produces 0 * -inf = NaN. With finite neg_inf, padding incorrectly contributes to entropy instead of being excluded, unlike the dense-mask path and the method's documented valid-outcome behavior.
Reproduction
import torch
from torchrl.modules.distributions import MaskedCategorical
logits = torch.tensor([1., 2., 3., 4.])
indices = torch.tensor([0, 2, -1])
for neg_inf in (float('-inf'), -10.):
dist = MaskedCategorical(logits=logits, indices=indices,
padding_value=-1, neg_inf=neg_inf)
print(dist.entropy()) # NaN or an incorrect finite value
print(torch.distributions.Categorical(logits=logits[[0, 2]]).entropy())
# tensor(0.3653): entropy over the two valid actions
Environment: Windows, Python 3.12.14 / PyTorch 2.10 CPU, TorchRL Python source checkout, TensorDict 0.14.2 official wheel. The optional TorchRL C++ extension is not built; this reproduces in the Python distribution implementation.
I intend to mask padding/non-finite zero-probability contributions before the existing entropy normalization, and test entropy and gradients against independent compact torch.distributions.Categorical references. This is separate from sample/log_prob shape handling in #4511 and mode ties in #4509. Investigated with Codex assistance; reproduction run locally.
Bug description
On current main
8f92a50cb1e8543d6d7545605e4e5b399224928b, sparseMaskedCategorical.entropy()does not exclude padded indices. With defaultneg_inf=-inf, a padded logit produces0 * -inf = NaN. With finiteneg_inf, padding incorrectly contributes to entropy instead of being excluded, unlike the dense-mask path and the method's documented valid-outcome behavior.Reproduction
Environment: Windows, Python 3.12.14 / PyTorch 2.10 CPU, TorchRL Python source checkout, TensorDict 0.14.2 official wheel. The optional TorchRL C++ extension is not built; this reproduces in the Python distribution implementation.
I intend to mask padding/non-finite zero-probability contributions before the existing entropy normalization, and test entropy and gradients against independent compact
torch.distributions.Categoricalreferences. This is separate from sample/log_prob shape handling in #4511 and mode ties in #4509. Investigated with Codex assistance; reproduction run locally.