This is a PyTorch implementation of Dueling DQN with action branching architectures for Gym environment with Discrete and Box action space.

Action Branching Architectures

Reference paper:

The branching architecture is summarized as following.

Action Branching Architectures.

For action of Box class, certain number of sub-actions need to be sampled from each continuous action dimension by setting parameter sub_act_dim, which could either be an integer or a list/tuple in the size of action dimension indication number of sub-actions for each action dimension. Given sub_act_dim, actions including lowest and highest possible value are equally sample from each action dimension as following.

Sub-action sampling.







Start Training

Simply run python


CartPole-v1 (discrete action) and MountainCarContinuous-v0 (continuous action) of Gym environment are tested, episode return are show in the following respectively. After around 1.6k and 120 episodes for each case, the agent start to gain steady rewards.

Episode return of CartPole-v1 in training process.

Episode return of MountainCarContinuous-v0 in training process.


View Github