A multi-purpose Video Labeling GUI in Python with integrated SOTA detector and tracker. Developed using PyQt5.
- SSH connection to a remote GPU server (see below to configure the server)
- YOLO and OpenPifPaf integrated object & pose detectors (single frame/video mode)
- Hungarian algorithm for track_id assignment
- SiamMask visual object tracking for missing or mislabeled boxes
- Zoom on video, resizable bounding boxes and skeletons
- Dark mode!
The integrated object detectors and trackers are based on the following codes:
- OpenPifPaf: for human pose estimation
- YOLO darknet: for object detection
- SiamMask: for visual object tracking
- Hungarian algorithm (scipy.optimize): for optimal instance ID assignment
Start by cloning the repository on your computer:
git clone https://github.com/alexandre01/UltimateLabeling.git cd UltimateLabeling
We recommend installing the required packages in a virtual environment to avoid any library versions conflicts. The following will do this for you:
virtualenv --no-site-packages venv source venv/bin/activate pip install -r requirements.txt
Otherwise, just install the requirements on your main Python environment using
pip as follows:
pip install -r requirements
Finally, open the GUI using:
python -m ultimatelabeling.main
Remote server configuration
To configure the remote GPU server (using the code in server files.), follow the steps below:
git clone https://github.com/alexandre01/UltimateLabeling_server.git cd UltimateLabeling_server pip install -r requirements.txt bash siamMask/setup.sh bash detection/setup.sh
The data images and videos should be placed in the folder
data, similarly to the client code.
To extract video files, use the following script:
bash extract.sh data/video_file.mp4
Input / output
To start labeling your videos, put these (folder of images or video file, the frames will be extracted automatically) inside the
Import labels: To import existing .CSV labels, hit
Ctrl+I). UltimateLabeling expects to read one .CSV file per frame, in the format: "class_id", "xc", "yc", "w", "h".
Export labels: The annotations are internally saved in the
outputfolder. To export them in a unique .CSV file, hit
Ctrl+E) and choose the destination location.
If you need other file formats for your projects, please write a GitHub issue or submit a Pull request.
Shortcuts / mouse controls
- A (or Left key): next frame
- D (or Right key): previous frame
- W/S: class up/down
- T: start/stop tracking (last used tracker)
- Numberpad: assign given class_id
- Spacebar: play the video
- Click: select bounding box
- Click & hold: move in the image
- Cmd + click & hold: create new bounding box
- Right click: delete bounding box in current frame (+ in all previous / all following frames if the corresponding option is enabled)
- Scroll wheel (or swipe up/down): zoom in the image