/ Machine Learning

RL & SL Methods and Envs For Quantitative Trading

RL & SL Methods and Envs For Quantitative Trading


Personae is a repo that implements papers proposed methods in Deep Reinforcement Learning & Supervised Learning and applies them to Financial Market.

Now Personae includes 4 RL & 3 SL implements and a simulate Financial Market supporting Stock and Future. (Short Sale is still implementing)

More RL & SL methods are updating!


  • The features as inputs are naive.
  • Day frequency is clearly not enough.
  • It's recommended that you could replace the features here to your own.



A basic simulate environment of Financial Market is implemented.

  • Market
    Implement of Market, Trader, Positions as a gym env (gym is not required), which can give a env for regression or sequence data generating for RL or SL model.
    For now, Market support Stock Data and Future Data.

Also, more functions are updating.


Total Profits and Baseline Profits. (Test Set)

Prices Prediction Experiments on 4 Bank Stocks. (Test Set)


Before you start testing, following requirements are needed.

  • Python3.5
  • TensorFlow1.4
  • numpy
  • scipy
  • pandas
  • rqalpha
  • sklearn
  • tushare
  • matplotlib
  • mongoengine
  • CUDA (option)
  • ta-lib (option)
  • Docker (option)
  • PyTorch (option)

It's best that if you are a Docker user, so that you can run the whole project without installing those dependencies manually.

And you can also use Ansible to run CUDA-Playbook and Docker-Playbook to install CUDA and Nvidia-Docker if you want to run tests in a Docker Container.

How to Use

If you use Docker

About base image

My image for this repo is ceruleanwang/personae, and personae is inherited from ceruleanwang/quant-base.

The image ceruleanwang/quant-base is inherited from nvidia/cuda:8.0-cudnn6-runtime. So please make sure your CUDA version and cuDNN version are correct.


First you should make sure you have stocks data in your mongodb.

If you don't have, you can use a spider writen in this repo to crawl stock or future data, but before you start, you should make sure a mongodb service is running.

If you don't have mongodb service running, you can also use a mongodb container (option) by following code:

docker run -p 27017:27017 -v /data/db:/data/db -d --network=your_network mongo

Then, you can use spiders to crawl stock data by following code:

docker run -t -v local_project_dir:docker_project_dir --network=your_network ceruleanwang/personae spider/stock_spider.py

Also, you can crawl future data by following code:

docker run -t -v local_project_dir:docker_project_dir --network=your_network ceruleanwang/personae spider/future_spider.py

But remember to set stock or future codes that you want to crawl, the default stock codes are:

stock_codes = ["600036", "601328", "601998", "601398"]

And the default future codes are:

future_codes = ["AU88", "RB88", "CU88", "AL88"]

And they can be modified in:

Then you can just run a model by:

docker run -t -v local_project_dir:docker_project_dir --network=yuor_network ceruleanwang/personae algorithm/RL or SL/algorithm_name.py

If you use Conda

You can create an env yourself, and install Python3.5 and all dependencies required, then just run algorithm in your way.

One thing should be noticed is that the hostname in mongoengine config should be your own.

About training & testing

For now, all models implemented with TensorFlow support persistence. You can edit many parameters when you are training or testing a model.
For example, following codes show some parameters that could be edited.

env = Market(codes, start_date="2008-01-01", end_date="2018-01-01", **{
    "market": market,
    "mix_index_state": True,
    "training_data_ratio": training_data_ratio,

algorithm = Algorithm(tf.Session(config=config), env, env.trader.action_space, env.data_dim, **{
    "mode": mode,
    "episodes": episode,
    "enable_saver": True,
    "enable_summary_writer": True,
    "save_path": os.path.join(CHECKPOINTS_DIR, "RL", model_name, market, "model"),
    "summary_path": os.path.join(CHECKPOINTS_DIR, "RL", model_name, market, "summary"),


  • More Implementations of Papers.
  • More High-Frequency Stocks Data.