Ubuntu 22.04
Sponsored Link

KVM : GPU Passthrough
2022/04/28
 
Configure GPU Passthrough for Virtual Machines.
By this configuration, it's possible to use GPU on Virtual Machines and run GPU Computing.
Before configuration, Enable VT-d (Intel) or AMD IOMMU (AMD) on BIOS Setting first.
[1] Enable IOMMU feature on KVM Host.
# show PCI identification number and [vendor-ID:device-ID] of Graphic card
# PCI number ⇒ it matchs [01:00.*] below
# vendor-ID:device-ID ⇒ it matchs [10de:***] below

root@dlp:~#
lspci -nn | grep -i nvidia

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 680] [10de:1180] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)

root@dlp:~#
vi /etc/default/grub
# lie 11 : add
# for AMD CPU, set [amd_iommu=on]
# for Intel CPU, set [intel_iommu=on]
# for [vfio-pci.ids==***], set [vendor-ID:device-ID]

GRUB_CMDLINE_LINUX="
intel_iommu=on vfio-pci.ids=10de:1180,10de:0e0a
"
root@dlp:~#
update-grub

root@dlp:~#
reboot
# confirm IOMMU is enabled

root@dlp:~#
dmesg | grep -E "DMAR|IOMMU"

[    0.012821] ACPI: DMAR 0x000000005C6E6B70 0000CC (v01 ALASKA A M I    00000001 INTL 20091013)
[    0.012855] ACPI: Reserving DMAR table memory at [mem 0x5c6e6b70-0x5c6e6c3b]
[    0.195540] DMAR: IOMMU enabled
[    0.431643] DMAR: Host address width 46
[    0.431646] DMAR: DRHD base: 0x000000fbffd000 flags: 0x0
[    0.431653] DMAR: dmar0: reg_base_addr fbffd000 ver 1:0 cap 8d2008c10ef0466 ecap f0205b
[    0.431657] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[    0.431662] DMAR: dmar1: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.431666] DMAR: RMRR base: 0x0000005ce2b000 end: 0x0000005ce3afff
[    0.431669] DMAR: ATSR flags: 0x0
[    0.431672] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[    0.431675] DMAR-IR: IOAPIC id 1 under DRHD base  0xfbffc000 IOMMU 1
[    0.431678] DMAR-IR: IOAPIC id 2 under DRHD base  0xfbffc000 IOMMU 1
[    0.431681] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[    0.431683] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.431684] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.432307] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.740102] DMAR: No SATC found
[    0.740107] DMAR: IOMMU feature sc_support inconsistent
[    0.740110] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.740114] DMAR: dmar0: Using Queued invalidation
[    0.740139] DMAR: dmar1: Using Queued invalidation
[    0.748534] DMAR: Intel(R) Virtualization Technology for Directed I/O

# confirm vfio_pci is enabled

root@dlp:~#
dmesg | grep -i vfio

[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-5.15.0-48-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro intel_iommu=on vfio-pci.ids=10de:1180,10de:0e0a
[    0.195718] Kernel command line: BOOT_IMAGE=/vmlinuz-5.15.0-48-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro intel_iommu=on vfio-pci.ids=10de:1180,10de:0e0a
[    0.797158] VFIO - User Level meta-driver version: 0.3
[    0.797275] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    0.815662] vfio_pci: add [10de:1180[ffffffff:ffffffff]] class 0x000000/00000000
[    0.855690] vfio_pci: add [10de:0e0a[ffffffff:ffffffff]] class 0x000000/00000000
[2] That's OK. For example, create a CentOS Stream 8 Virtual machine with GPU.
Specify PCI identification number of GPU for [--host-device].
root@dlp:~# virt-install \
--name ubuntu2204 \
--ram 8192 \
--disk path=/var/kvm/images/ubuntu2204.img,size=30 \
--vcpus 4 \
--os-variant ubuntu22.04 \
--network bridge=br0 \
--graphics vnc,listen=0.0.0.0,password=password \
--video vga \
--console pty,target_type=serial \
--location /home/ubuntu-22.04-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
--extra-args 'console=ttyS0,115200n8' \
--host-device 01:00.0 \
--features kvm_hidden=on \
--machine q35 
[3] After creating Virtual machine, Confirm GPU is found on it like follows.
ubuntu@ubuntu:~$
lspci | grep -i nvidia

05:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 680] (rev a1)
Matched Content