SMARTS
SMARTS (Scalable Multi-Agent RL Training School) is a simulaton platform for reinforcement learning and multi-agent research on autonomous driving. Its focus is on realistic and diverse interactions. It is part of the XingTian suite of RL platforms from Huawei Noah's Ark Lab.
Multi-Agent experiment as simple as...
import gym
from smarts.core.agent_interface import AgentInterface, AgentType
from smarts.core.agent import AgentSpec, AgentPolicy
class Policy(AgentPolicy):
def act(self, obs):
return "keep_lane"
agent_spec = AgentSpec(
interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=None),
policy_builder=Policy,
)
agent_specs = {
"Agent-007": agent_spec,
"Agent-008": agent_spec,
}
env = gym.make(
"smarts.env:hiway-v0",
scenarios=["scenarios/loop"],
agent_specs=agent_specs,
)
agents = {
agent_id: agent_spec.build_agent()
for agent_id, agent_spec in agent_specs.items()
}
observations = env.reset()
for _ in range(1000):
agent_actions = {
agent_id: agents[agent_id].act(agent_obs)
for agent_id, agent_obs in observations.items()
}
observations, _, _, _ = env.step(agent_actions)
Setup
# For Mac OS X users, make sure XQuartz is pre-installed as SUMO's dependency
# git clone ...
cd <project>
# Follow the instructions given by prompt for setting up the SUMO_HOME environment variable
./install_deps.sh
# verify sumo is >= 1.5.0
# if you have issues see ./doc/SUMO_TROUBLESHOOTING.md
sumo
# setup virtual environment; presently only Python 3.7.x is officially supported
python3.7 -m venv .venv
# enter virtual environment to install all dependencies
source .venv/bin/activate
# upgrade pip, a recent version of pip is needed for the version of tensorflow we depend on
pip install --upgrade pip
# install [train] version of python package with the rllib dependencies
pip install -e .[train]
# make sure you can run tests (and verify they are passing)
make test
# then you can run a scenario, see following section for more details
Running
We use supervisord to run SMARTS together with it's supporting processes. To run the default example simply build a scenario and start supervisord:
# build scenarios/loop
scl scenario build --clean scenarios/loop
# start supervisord
supervisord
With supervisord
running, visit http://localhost:8081/ in your browser to view your experiment.
See ./envision/README.md for more information on Envision, our front-end visualization tool.
Several example scripts are provided under SMARTS/examples
, as well as a handful of scenarios under SMARTS/scenarios
. You can create your own scenarios using the Scenario Studio. Here's how you can use one of the example scripts with a scenario.
# Update the command=... in ./supervisord.conf
#
# [program:smarts]
# command=python examples/single_agent.py scenarios/loop
# ...
CLI tool
SMARTS provides a command-line tool to interact with scenario studio and Envision.
Usage
scl COMMAND SUBCOMMAND [OPTIONS] [ARGS]...
Commands:
- envision
- scenario
- zoo
Subcommands of scenario:
- build-all: Generate all scenarios under the given directories
- build: Generate a single scenario
- clean: Clean generated artifacts
Subcommands of envision:
- start: start envision server
Subcommands of zoo:
- zoo: Build an agent, used for submitting to the agent-zoo
Examples:
# Start envision, serve scenario assets out of ./scenarios
scl envision start --scenarios ./scenarios
# Build all scenario under given directories
scl scenario build-all ./scenarios ./eval_scenarios
# Rebuild a single scenario, replacing any existing generated assets
scl scenario build --clean scenarios/loop
# Clean generated scenario artifacts
scl scenario clean scenarios/loop
Interfacing with Gym
See the provided ready-to-go scripts under the examples/ directory.
Contributing
Please read Contributing
Building Docs Locally
Assuming you have run pip install .[dev]
.
make docs
python -m http.server -d docs/_build/html
# Open http://localhost:8000 in your browser
Extras
Visualizing Agent Observations
If you want to easily visualize observations you can use our Visdom integration. Start the visdom server before running your scenario,
visdom
# Open the printed URL in your browser
And in your experiment, start your environment with visdom=True
env = gym.make(
"smarts.env:hiway-v0",
scenarios=["scenarios/loop"],
agent_specs=agent_specs,
visdom=True,
)
Using Docker
If you're comfortable using docker or are on a platform without suitable support to easily run SMARTS (e.g. an older version of Ubuntu) you can run the following,
docker build -t smarts .
docker run --rm -it -v $(PWD):/src -p 8081:8081 smarts
# <press enter>
cd /src
pip install -e .[train]
# Run Envision server in the background
$ scl envision start -s ./scenarios -p 8081 &
$ python examples/single_agent.py scenarios/loop
# On your most machine visit http://localhost:8081 to see the running simulation in
# Envision.