Variational Adversarial Training Towards Policies with Improved Robustness

[AISTATS 2025]

Juncheng Dong^* · Hao-Lun Hsu^* · Qitong Gao · Vahid Tarokh · Miroslav Pajic ·

Duke University (^*indicates equal contribution)

This repo contains code for training RL agents with adversarial disturbance agents in our work on Variational Adversarial Training Towards Policies with Improved Robustness. We build heavily on the OpenAI rllab repo and utilize the repo: (RARL) for baseline comparison. We also implement from the repo (SVPG)

Installation instructions

Since we build upon the rllab package for the optimizers, the installation process is similar to rllab's manual installation. Most of the packages are virtually installated in the anaconda rllab3-adv enivronment.

• Dependencies for scipy:

sudo apt-get build-dep python-scipy

• Install python modules:

conda env create -f environment.yml

• Install MuJoCo

• Add rllab-adv to your PYTHONPATH.

export PYTHONPATH=<PATH_TO_RLLAB_ADV>:$PYTHONPATH

Example

# Please change the save path to your own folder before running the example
# Enter the anaconda virtual environment
source activate rllab3-adv
# Train on HopperAdv-v1 using TRPO and single adversary
python adversarial/scripts/train_rarl_adversary.py --env HopperAdv-v1 --folder ~/rllab-adv/results --rl_type trpo --algo_type single

# Train on HopperAdv-v1 using PPO and ROSE-S
python adversarial/scripts/train_population_adversary.py --env HopperAdv-v1 --folder ~/rllab-adv/results --rl_type ppo --algo_type percent_worst