Rising is a high-performance data loading and augmentation library for 2D and 3D data completely written in PyTorch. Our goal is to provide a seamless integration into the PyTorch Ecosystem without sacrificing usability or features. Multiple examples for different use cases can be found in our tutorial docs e.g. 2D Classification on MedNIST, 3D Segmentation of Hippocampus (Medical Decathlon), Example Transformation Output, Integration of External Frameworks
Why another framework?
pip install rising
Editable Installation for development
git clone [email protected]:PhoenixDL/rising.git cd rising pip install -e .
Running tests inside rising directory (top directory not the package directory)
python -m unittest
Check out our contributing guide for more information or additional help.
What can I do with
Rising currently consists out of two main modules:
Dataloader of rising will be your new best friend because it handles all you transformations and applies them efficiently to the data either on CPU or GPU.
On CPU you can easily switch between transformations which can only be performed per sample and transformations which can be applied per batch.
In contrast to the native PyTorch datasets you don't need to integrate your augmentation into your dataset.
Hence, the only purpose of the dataset is to provide an interface to access individual data samples.
DataLoader is a direct subclass of the PyTorch's dataloader and handles the batch assembly and applies the augmentations/transformations to the data.
This module implements many transformations which can be used during training for preprocessing and augmentation.
All of them are implemented directly in PyTorch such that gradients can be propagated through the transformations and (optionally) it can be applied on the GPU.
Finally, all transforms are implemented for 2D (natural images) and 3D (volumetric) data.
In the future, support for keypoints and other geometric primitives which can be assembled by connected points will be added.
rising MNIST Example with CPU and GPU augmentation
rising uses the same
Dataset structure as PyTorch and thus we can just reuse the MNIST dataset from torchvision.
import torchvision from torchvision.transforms import ToTensor # define dataset and use to tensor trafo to convert PIL image to tensor dataset = torchvision.datasets.MNIST('./', train=True, download=True, transform=ToTensor())
In the next step, the transformations/augmentations need to be defined.
The first transforms converts the Sequence from the torchvision dataset into a dict for the following
rising transform which work on dicts.
At the end, the transforms are compose to one callable transform which can be passed to the
import rising.transforms as rtr from rising.loading import DataLoader, default_transform_call from rising.random import DiscreteParameter, UniformParameter # define transformations transforms = [ rtr.SeqToMap("data", "label"), # most rising transforms work on dicts rtr.NormZeroMeanUnitStd(keys=["data"]), rtr.Rot90((0, 1), keys=["data"], p=0.5), rtr.Mirror(dims=DiscreteParameter([0, 1]), keys=["data"]), rtr.Rotate(UniformParameter(0, 180), degree=True), ] # by default rising assumes dicts but torchvision outputs tuples # so we need to modify `transform_call` to support sequences and dicts composed = rtr.Compose(transforms, transform_call=default_transform_call)
rising automatically applies the specified transformations to the batches inside the multiprocessing context of the CPU.
dataloader = DataLoader( dataset, batch_size=8, num_workers=8, batch_transforms=composed)
Alternatively, the augmentations can easily be applied on the GPU as well.
dataloader = DataLoader( dataset, batch_size=8, num_workers=8, gpu_transforms=composed)
If either the GPU or CPU is the bottleneck of the pipeline, the
Dataloader can be used to balance the augmentations load between them.
transforms_cpu = rtr.Compose(transforms[:2]) transforms_gpu = rtr.Compose(transforms[2:]) dataloader = DataLoader( dataset, batch_size=8, num_workers=8, batch_transforms=transforms_cpu, gpu_transforms=transforms_gpu, )
In general you do not need to be familiar with the whole augmentation process which runs in the background but if you are still curious about the
detailed pipeline this section will give a very short introduction into the backend of the
The flow charts below highlight the differences between a conventional augmentation pipeline and the pipeline used in
CPU operations are visualized in blue while GPU operations are green.
The flow chart below visualizes the default augmentation pipeline of many other frameworks.
The transformations are applied to individual samples which are loaded and augmented inside of multiple background workers from the CPU.
This approach is already efficient and might only be slightly slower than batched execution of the transformations (if applied on the CPU).
GPU augmentations can be used to perform many operations in parallel and profit heavily from vectorization.
rising lets the user decide from case to case where augmentations should be applied during this pipeline.
This can heavily dependent on the specific tasks and the underlying hardware.
Running augmentations on the GPU is only efficient if they can be executed in a batched fashion to maximize the parallelization GPUs can provide.
As a consequence,
rising implements all its transformations in a batched fashion and the
Dataloader can execute them efficiently on the CPU and GPU.
Dataloader can still be used to apply transformations on a per sample fashion, e.g. when transforms from other frameworks should be integrated.
rising augmentation pipeline is a superset of the currently used methods, external frameworks can be integrated into
Issues: If you find any bugs, want some additional features or maybe just have a question don't hesitate to open an issue :)
General Project Future: Most of the features and the milestone organisation can be found inside the
Features which are planned for the next release/milestone are listed under
TODO Next Release while features which are not scheduled yet are under