Repo for the Deep Reinforcement Learning Nanodegree program
Deep Reinforcement Learning Nanodegree
Repo for the Deep Reinforcement Learning Nanodegree program.
Table of Contents
Tutorials
The tutorials lead you through implementing various algorithms in reinforcement learning. All of the code is in PyTorch (v0.4) and Python 3.
 Dynamic Programming: Implement Dynamic Programming algorithms such as Policy Evaluation, Policy Improvement, Policy Iteration, and Value Iteration.
 Monte Carlo: Implement Monte Carlo methods for prediction and control.
 TemporalDifference: Implement TemporalDifference methods such as Sarsa, QLearning, and Expected Sarsa.
 Discretization: Learn how to discretize continuous state spaces, and solve the Mountain Car environment.
 Tile Coding: Implement a method for discretizing continuous state spaces that enables better generalization.
 Deep QNetwork: Explore how to use a Deep QNetwork (DQN) to navigate a space vehicle without crashing.
 Robotics: Use a C++ API to train reinforcement learning agents from virtual robotic simulation in 3D. (External link)
 Hill Climbing: Use hill climbing with adaptive noise scaling to balance a pole on a moving cart.
 CrossEntropy Method: Use the crossentropy method to train a car to navigate a steep hill.
 REINFORCE: Learn how to use Monte Carlo Policy Gradients to solve a classic control task.
 Proximal Policy Optimization: Explore how to use Proximal Policy Optimization (PPO) to solve a classic reinforcement learning task. (Coming soon!)
 Deep Deterministic Policy Gradients: Explore how to use Deep Deterministic Policy Gradients (DDPG) with OpenAI Gym environments.
 Pendulum: Use OpenAI Gym's Pendulum environment.
 BipedalWalker: Use OpenAI Gym's BipedalWalker environment.
 Finance: Train an agent to discover optimal trading strategies. (Coming soon!)
Labs / Projects
The labs and projects can be found below. All of the projects use rich simulation environments from Unity MLAgents. In the Deep Reinforcement Learning Nanodegree program, the projects are reviewed by Udacity experts. These reviews are meant to give you personalized feedback and to tell you what can be improved in your code.
 The Taxi Problem: In this lab, you will train a taxi to pick up and drop off passengers.
 Navigation: In the first project, you will train an agent to collect yellow bananas while avoiding blue bananas.
 Continuous Control: In the second project, you will train an robotic arm to reach target locations. (Coming soon!)
 Collaboration and Competition: In the third project, you will train a pair of agents to play tennis! (Coming soon!)
Resources
 Cheatsheet: You are encouraged to use this PDF file to guide your study of reinforcement learning.
OpenAI Gym Benchmarks
Classic Control
Acrobotv1
with Tile Coding and QLearningCartpolev0
with Hill Climbing  solved in 13 episodesCartpolev0
with REINFORCE  solved in 691 episodesMountainCarContinuousv0
with CrossEntropy Method  solved in 47 iterationsMountainCarv0
with UniformGrid Discretization and QLearning  solved in <50000 episodesPendulumv0
with Deep Deterministic Policy Gradients (DDPG)
Box2d
BipedalWalkerv2
with Deep Deterministic Policy Gradients (DDPG)CarRacingv0
with Deep QNetworks (DQN)  Coming soon!LunarLanderv2
with Deep QNetworks (DQN)  solved in 1504 episodes
Toy Text
FrozenLakev0
with Dynamic ProgrammingBlackjackv0
with Monte Carlo MethodsCliffWalkingv0
with TemporalDifference Methods
Dependencies
To set up your python environment to run the code in this repository, follow the instructions below.

Create (and activate) a new environment with Python 3.6.
 Linux or Mac:
conda create name drlnd python=3.6 source activate drlnd
 Windows:
conda create name drlnd python=3.6 activate drlnd

Follow the instructions in this repository to perform a minimal install of OpenAI gym.

Clone the repository (if you haven't already!), and navigate to the
python/
folder. Then, install several dependencies.
git clone https://github.com/udacity/deepreinforcementlearning.git
cd deepreinforcementlearning/python
pip install .
 Create an IPython kernel for the
drlnd
environment.
python m ipykernel install user name drlnd displayname "drlnd"
 Before running code in a notebook, change the kernel to match the
drlnd
environment by using the dropdownKernel
menu.