Hotplugger: Real USB Port Passthrough for VFIO/QEMU!
Welcome to Hotplugger! This app, as the name might tell you, is a combination of some scripts (python, yaml, udev rules and some QEMU args) to allow you to pass through an actual USB port into a VM. Instead of passing the USB root hub (which could have the side effect of passing all the ports, including the ones you didn’t want to) or another PCIe hub or something, you can just pass a specific USB port to a VM and have the others free for anything else. Plus, it saves you from using the
vfio-pci driver for the USB root hub, so you can keep using it for evdev or other things on the VM host.
hotplugger.pyrequire Python 3
- Only tested with QEMU 5.0.0. Untested with older or newer versions.
Quick start (Ubuntu 20.10)
git clone https://github.com/darkguy2008/hotplugger.git
python3 monitor.pyand follow the prompts. Basically once you hit Enter you have to plug and unplug an USB device (a thumbdrive or audio device preferred) into the USB ports that you want to know their
DEVPATHroute from. This will help you identify them so you can write them into
portsarray. This array only accepts
config.yaml. It must stay in the same folder as
hotplugger.py. Look at the current example: It’s set for a Windows VM (the name doesn’t matter, as long as it’s unique within the entries of the same file). Make sure the
socketproperty matches the file path of the QEMU
chardevdevice pointing to an Unix domain socket file and in the
portsarray put the list of the
DEVPATHof the USB ports you want to pass through to that VM:
virtual_machines: windows: socket: /home/dragon/vm/test/qmp-sock ports: - /devices/pci0000:00/0000:00:14.0/usb3/3-1 - /devices/pci0000:00/0000:00:14.0/usb3/3-2 - /devices/pci0000:00/0000:00:14.0/usb4/4-1 - /devices/pci0000:00/0000:00:14.0/usb4/4-2
/etc/udev/rules.d/99-zzz-local.rulesfile with the following content:
SUBSYSTEM=="usb", ACTION=="add", RUN+="/bin/bash -c 'python3 /path-to-hotplugger/hotplugger.py >> /tmp/hotplugger.log' 2>&1" SUBSYSTEM=="usb", ACTION=="remove", RUN+="/bin/bash -c 'python3 /path-to-hotplugger/hotplugger.py >> /tmp/hotplugger.log' 2>&1"
Make sure to change
path-to-hotpluggerwith the path where you cloned the repo to, or installed the package. It can be simplified, but this one is useful in case you want to debug and see what’s going on. Otherwise, proceed with a simpler file:
SUBSYSTEM=="usb", ACTION=="add", RUN+="/bin/bash -c 'python3 /path-to-hotplugger/hotplugger.py'" SUBSYSTEM=="usb", ACTION=="remove", RUN+="/bin/bash -c 'python3 /path-to-hotplugger/hotplugger.py'"
Create the QMP monitor Unix domain socket if you haven’t already in your QEMU args. I use this:
-chardev socket,id=mon1,server,nowait,path=./qmp-sock -mon chardev=mon1,mode=control,pretty=on
Have a coffee!
This is a work in progress, but here’s some steps to get you started:
Edit your VM’s XML config like this:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>QEMUGuest1name> <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809uuid> ... <qemu:commandline> <qemu:arg value='-chardev'/> <qemu:arg value='socket,id=mon1,server,nowait,path=/tmp/my-vm-sock'/> <qemu:arg value='-mon'/> <qemu:arg value='chardev=mon1,mode=control,pretty=on'/> qemu:commandline> domain>
xmlnsattribute and the QEMU commandline arguments like that. The
/tmp/my-vm-sockis the name of an unix domain socket. You can use any, just make sure to also put the same path in the
If you get a permissions issue, edit
security_driver = "none"to it to fix apparmor being annoying about it.
How it works
udevrule launches the script on every USB event. For each USB
removeaction there’s around 3 to 5+ events. This allows the app to act at any step in the action lifecycle.
- In the first step it gets the kernel environment variables from
udevand stores them in a temp file. In those variables, the
DEVNUM(host address in QEMU, it seems to change and is sequential…) and the
BUSNUM(bus address in QEMU) are captured. For the subsequent events, the following steps are run:
- It requests QEMU through the Unix socket and the
info usbhostQMP command the USB info from the host. This gives it an extra field: The host port where the device is also connected to. Since I got the
busaddresses in the first event, I can use that to parse through the
info usbhostcommand’s output and find the port connected to the device.
- If the port is found, using the
device_addcommand, a new
usb-hostdevice is added using the USB
portwe got in the previous step, and assigns it a predictable ID that it can use to unplug the device afterwards. To add this of course, the VM should have a
usb-xhcidevice I think. Not sure if it’s required or not, but I prefer to add it as I have USB 3.0 ports and devices.
- The temp file is cleared once the
device_addcommand has run successfully.
- It requests QEMU through the Unix socket and the
Steps 2.1, 2.2 and 2.3 are run on every
udev event. For instance, for an audio device it gets 3 or 4 events: One for the HID device, and two or so for the audio devices. My audio device (Corsair Void Elite Wireless) has both stereo audio and a communications device (mono audio, for mic) so for a single dongle like that I get those many events. Since these steps are ran on all the events, there’s multiple chances to do the hotplug action. When one of them succeeds, the others will silently fail as QEMU will say that the same device ID is being used, so all is good.
If for some reason the app doesn’t seem to work, try these methods:
- Remove the
- Reboot the computer
- Reboot udev:
sudo udevadm control --reload-rules && sudo udevadm trigger
- View udev’s logfile:
sudo service udev restart && sudo udevadm control --log-priority=debug && journalctl -f | grep -i hotplugger
- If you want to see what will be run when you plug a device, try with this command to simulate an udev event:
udevadm test $(udevadm info -a --path=/devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.0) --action=addreplacing
--pathwith the path of the USB port down to the device itself (in this case, I had a device connected to the
usb3/3-1port, identified as
A lot of work and sleepless nights were involved in this procedure, so if this app helps you in any way or another, please consider sending a small donation, it helps a lot in these tough times!
- Initial changelog writing
- App was refactored a bit with improved python mad skillz. It also seems to be a bit more stable and robust, it doesn’t hang much anymore and USB detection seems to work better. This is due to the fact that I added a stupid 1-second delay after all the USB UDEV events have gone through. Since there’s no way to know when UDEV has “finished” sending all the events (and there could be a lot more) the commands being sent to QEMU to add the device will have to wait 1 second now. While it’s not ideal, it should be enough to avoid a VM hanging up and I can live with that.