KVM : GPU Passthrough2025/01/17 |
Configure GPU Passthrough for Virtual Machines. By this configuration, it's possible to use GPU on Virtual Machines and run GPU Computing like Machine learning and so on. Before configuration, Enable VT-d (Intel) or AMD IOMMU (AMD) on BIOS Setting first. |
|
[1] | Enable IOMMU feature and [vfio-pci] kernel module on KVM Host. |
# add kernel parameter # for AMD CPU, specify [amd_iommu=on] [root@dlp ~]# grubby --update-kernel ALL --args intel_iommu=on [root@dlp ~]# grubby --update-kernel ALL --args iommu=pt
# show PCI identification number and [vendor-ID:device-ID] of Graphic card # PCI number ⇒ it matchs [02:00.*] below # vendor-ID:device-ID ⇒ it matchs [10de:***] below [root@dlp ~]# lspci -nn | grep -i nvidia 81:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3060] [10de:2487] (rev a1) 81:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)
[root@dlp ~]#
vi /etc/modprobe.d/vfio.conf # create new : for [ids=***], specify [vendor-ID:device-ID] options vfio-pci ids=10de:2487,10de:228b
[root@dlp ~]#
echo 'vfio-pci' > /etc/modules-load.d/vfio-pci.conf
[root@dlp ~]#
reboot
# confirm IOMMU is enabled [root@dlp ~]# dmesg | grep -E "DMAR|IOMMU" [ 0.009731] ACPI: DMAR 0x000000007A5C3130 000108 (v01 ALASKA A M I 00000001 INTL 20091013) [ 0.009764] ACPI: Reserving DMAR table memory at [mem 0x7a5c3130-0x7a5c3237] [ 0.027651] DMAR: IOMMU enabled [ 0.128332] DMAR: Host address width 46 [ 0.128420] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0 [ 0.128517] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020df [ 0.128634] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1 [ 0.128728] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap d2078c106f0466 ecap f020df [ 0.128843] DMAR: RMRR base: 0x0000007dbcf000 end: 0x0000007dbddfff [ 0.128944] DMAR: ATSR flags: 0x0 [ 0.129031] DMAR: RHSA base: 0x000000c7ffc000 proximity domain: 0x0 [ 0.129124] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1 [ 0.129218] DMAR-IR: IOAPIC id 10 under DRHD base 0xfbffc000 IOMMU 0 [ 0.129329] DMAR-IR: IOAPIC id 8 under DRHD base 0xc7ffc000 IOMMU 1 [ 0.129422] DMAR-IR: IOAPIC id 9 under DRHD base 0xc7ffc000 IOMMU 1 [ 0.129515] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000 [ 0.129607] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 0.129608] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 0.130696] DMAR-IR: Enabled IRQ remapping in xapic mode [ 0.520629] DMAR: No SATC found [ 0.520809] DMAR: dmar0: Using Queued invalidation [ 0.520905] DMAR: dmar1: Using Queued invalidation [ 0.549650] DMAR: Intel(R) Virtualization Technology for Directed I/O # confirm vfio_pci is enabled [root@dlp ~]# dmesg | grep -i vfio [ 4.382874] VFIO - User Level meta-driver version: 0.3 [ 4.394074] vfio-pci 0000:81:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none [ 4.394385] vfio_pci: add [10de:2487[ffffffff:ffffffff]] class 0x000000/00000000 [ 4.418177] vfio_pci: add [10de:228b[ffffffff:ffffffff]] class 0x000000/00000000 |
[2] | That's OK. For example, create a CentOS Stream 10 Virtual machine with GPU. Specify PCI identification number of GPU for [--host-device]. |
[root@dlp ~]# virt-install \
--name centos10 \
--ram 8192 \
--disk path=/var/kvm/images/centos10.img,size=30 \
--vcpus=4 \
--os-variant rhel10.0 \
--network bridge=br0 \
--graphics vnc,listen=0.0.0.0,password=password \
--video vga \
--host-device 81:00.0 \
--features kvm_hidden=on \
--machine q35 \
--cdrom /home/CentOS-Stream-10-latest-x86_64-dvd1.iso
|
[3] | To assign a GPU to an existing Virtual machine that does not have one assigned, edit it as follows. |
[root@dlp ~]#
virsh edit centos10 ..... ..... # add follows in the [features] section <features> ...... ..... <kvm> <hidden state='on'/> </kvm> </features> ..... ..... # add follows in the [devices] section <devices> ...... ..... <hostdev mode='subsystem' type='pci' managed='yes'> <source> # specify host-device for [bus='***'] <address domain='0x0000' bus='0x81' slot='0x00' function='0x0'/> </source> # specify uniq number for [bus='***'] <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </hostdev> </devices> |
[4] | After creating Virtual machine, Confirm GPU is found on it like follows. |
[root@localhost ~]# lspci | grep -i nvidia 05:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3060] (rev a1) |
Sponsored Link |
|