[Feature Request] Support of MultiDiscrete action space in PettingZoo wrapping #2472

Motivation

I encountered a NotImplementedError while working with the torchrl library, specifically when using the pettingzoo environment. The error occurred during the handling of action spaces of type MultiDiscrete or tuples of Discrete spaces:

) = self._make_group_specs(group_name=group, agent_names=agents)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
group_action_spec["action"].space.n,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 raise NOT_IMPLEMENTED_ERROR
NotImplementedError: method is not currently implemented. If you are interested in this feature please submit an issue at https://github.com/pytorch/rl/issues

The lack of support for MultiDiscrete or tuple-based Discrete action spaces limits the ability to use environments with these specifications, which are common in multi-agent or hierarchical setups. Enhancing the library to support these spaces would make it more flexible and applicable to a broader range of RL environments.

Solution

Implement support for MultiDiscrete and Tuple(Discrete, Discrete, ...) action spaces in torchrl, particularly when used with environments like PettingZoo. This would involve extending the handling of action space specifications in the _make_group_specs method and related functionality to properly recognize and process these action space types

Alternatives

An alternative would be to convert MultiDiscrete and Tuple(Discrete, Discrete, ...) action spaces into a series of individual Discrete actions. However, this approach is less efficient and not as clean, as it requires additional transformations that could introduce complexity and error-prone code into multi-agent scenarios.

Additional context

Adding support for these action space types would enhance the integration of torchrl with multi-agent environments like PettingZoo, allowing for smoother experimentation with a variety of RL tasks. The error currently blocks using these environments directly, which is a significant limitation for developers working with multi-agent setups.

Checklist

• I have checked that there is no similar issue in the repo (required)