Pylomin
Pylomin (PYtorch LOw-Memory INference) is a library for low-memory inferencing in PyTorch.
Installation
…
Usage
For example, the following code snippet converts all nn.Linear and nn.Embedding modules in the model into lazy-loading mode.
model = pylomin.lazy_loading(model, target_instances=(nn.Linear, nn.Embedding))
Or, provide a list of modules via target_modules.
model = pylomin.lazy_loading(model, target_modules=your_target_modules)
A detailed documentation is being prepared! ?
Methods
1. Lazy-loading
…
2. Grouped-embedding
…
3. Prefetching
…
