/ Machine Learning

Sequential model-based optimization with a scipy.optimize interface

Sequential model-based optimization with a scipy.optimize interface

Scikit-Optimize

Scikit-Optimize, or skopt, is a simple and efficient library to minimize (very) expensive and noisy black-box functions. It implements several methods for sequential model-based optimization. skopt aims to be accessible and easy to use in many contexts.

The library is built on top of NumPy, SciPy and Scikit-Learn.

Approximated objective function after 50 iterations of gp_minimize. Plot made using skopt.plots.plot_objective.

Install

The latest released version of scikit-optimize is v0.5.2, which you can install with:

pip install scikit-optimize

This installs an essential version of scikit-optimize. To install scikit-optimize with plotting functionality, you can instead do:

pip install 'scikit-optimize[plots]'

This will install matplotlib along with scikit-optimize.

In addition there is a conda-forge package of scikit-optimize:

conda install -c conda-forge scikit-optimize

Using conda-forge is probably the easiest way to install scikit-optimize on Windows.

Getting started

Find the minimum of the noisy function f(x) over the range -2 < x < 2 with skopt:

import numpy as np
from skopt import gp_minimize

def f(x):
    return (np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) +
            np.random.randn() * 0.1)

res = gp_minimize(f, [(-2.0, 2.0)])

For more control over the optimization loop you can use the skopt.Optimizer class:

from skopt import Optimizer

opt = Optimizer([(-2.0, 2.0)])

for i in range(20):
    suggested = opt.ask()
    y = f(suggested)
    opt.tell(suggested, y)
    print('iteration:', i, suggested, y)

Development
The library is still experimental and under heavy development. Checkout the next milestone for the plans for the next release or look at some easy issues to get started contributing.

The development version can be installed through:

git clone https://github.com/scikit-optimize/scikit-optimize.git
cd scikit-optimize
pip install -e.

Run all tests by executing pytest in the top level directory.

To only run the subset of tests with short run time, you can use pytest -m 'fast_test' (pytest -m 'slow_test' is also possible). To exclude all slow running tests try pytest -m 'not slow_test'.

This is implemented using pytest attributes. If a tests runs longer than 1 second, it is marked as slow, else as fast.

All contributors are welcome!

Making a Release

The release procedure is almost completely automated. By tagging a new release travis will build all required packages and push them to PyPI. To make a release create a new issue and work through the following checklist:

  • update the version tag in setup.py
  • update the version tag in init.py
  • update the version tag mentioned in the README
  • check if the dependencies in setup.py are valid or need unpinning
  • check that the CHANGELOG.md is up to date
  • did the last build of master succeed?
  • create a new release
  • ping conda-forge

Before making a release we usually create a release candidate. If the next release is v0.X then the release candidate should be tagged v0.Xrc1 in setup.py and init.py. Mark a release candidate as a "pre-release" on GitHub when you tag it.

GitHub