This is a PyTorch implementation of Dueling DQN with action branching architectures for Gym environment with Discrete and Box action space.
Action Branching Architectures
Reference paper: https://arxiv.org/abs/1711.08946
The branching architecture is summarized as following.
For action of Box class, certain number of sub-actions need to be sampled from each continuous action dimension by setting parameter sub_act_dim, which could either be an integer or a list/tuple in the size of action dimension indication number of sub-actions for each action dimension. Given sub_act_dim, actions including lowest and highest possible value are equally sample from each action dimension as following.
Train
Requirements
python=3.8.11
torch=1.9.0
gym=0.19.0
tensorboard=2.8.0
Start Training
Simply run python dqn.py.
Results
CartPole-v1 (discrete action) and MountainCarContinuous-v0 (continuous action) of Gym environment are tested, episode return are show in the following respectively. After around 1.6k and 120 episodes for each case, the agent start to gain steady rewards.
