dKeras is a distributed Keras engine that is built on top of Ray. By wrapping dKeras around your original Keras model, it allows you to use many distributed deep learning techniques to automatically improve your system's performance.
With an easy-to-use API and a backend framework that can be deployed from the laptop to the data center, dKeras simpilifies what used to be a complex and time-consuming process into only a few adjustments.
Why Use dKeras?
Distributed deep learning can be essential for production systems where you need fast inference but don't want expensive hardware accelerators or when researchers need to train large models made up of distributable parts.
This becomes a challenge for developers because they'll need expertise in not only deep learning but also distributed systems. A production team might also need a machine learning optimization engineer to use neural network optimizers in terms of precision changes, layer fusing, or other techniques.
Distributed inference is a simple way to get better inference FPS. The graph below shows how non-optimized, out-of-box models from default frameworks can be quickly sped up through data parallelism:
- Data Parallelism Inference
- Model Parallelism Inference
- Distributed Training
- Easy Multi-model production-ready building
- Data stream input distributed inference
- PlaidML Support
- Automatic optimal hardware configuration
- PBS/Torque support
The first official release of dKeras will be available soon. For
now, install from source.
pip install git+https://github.com/dkeras-project/dkeras
- Python 3.6 or higher
- Linux (or OSX, dKeras works on laptops too!)
Coming Soon: PlaidML Support
dKeras will soon work alongside PlaidML,
a "portable tensor compiler for enabling deep learning on laptops, embedded devices,
or other devices where the available computing hardware is not well
supported or the available software stack contains unpalatable
model = ResNet50() model.predict(data)
from dkeras import dKeras model = dKeras(ResNet50) model.predict(data)
from tensorflow.keras.applications import ResNet50 from dkeras import dKeras import numpy as np import ray ray.init() data = np.random.uniform(-1, 1, (100, 224, 224, 3)) model = dKeras(ResNet50, init_ray=False, wait_for_workers=True, n_workers=4) preds = model.predict(data)
Multiple Model Example
import numpy as np from tensorflow.keras.applications import ResNet50, MobileNet from dkeras import dKeras import ray ray.init() model1 = dKeras(ResNet50, weights='imagenet', wait_for_workers=True, n_workers=3) model2 = dKeras(MobileNet, weights='imagenet', wait_for_workers=True, n_workers=3) test_data = np.random.uniform(-1, 1, (100, 224, 224, 3)) model1.predict(test_data) model2.predict(test_data) model1.close() model2.close()