From 1dca43b3ed6989d10fc0a879a5ed4f62f6320a3a Mon Sep 17 00:00:00 2001 From: Arif Ali Date: Wed, 26 Oct 2022 15:29:58 +0000 Subject: [PATCH 1/2] Add vm_propsed_testing.md Signed-off-by: Arif Ali --- vm_propsed_testing.md | 160 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 vm_propsed_testing.md diff --git a/vm_propsed_testing.md b/vm_propsed_testing.md new file mode 100644 index 0000000..71636cf --- /dev/null +++ b/vm_propsed_testing.md @@ -0,0 +1,160 @@ +# ice driver MTU reproducer + +First install `uvtool` on the hypervisor + +``` +apt install uvtool +``` + +Now download both focal and jammy images for usage to deploy VMs + +``` +mkdir -p /var/uvt + +TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=focal +TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=jammy +``` + +Create a default network for `virsh` to use + +``` +cat > virsh-net-default.xml << EOF + + default + ba71f522-c2c9-4fcd-9792-3c4f294e31da + + + + + + + + + + + + + +EOF +``` + +Now, define it in virsh itself and start it up + +``` +virsh net-define virsh-net-default.xml +virsh net-start default +``` + +Create 4 vms, 2 focal and 2 jammy + +``` +uvt-kvm create focal01 release=focal --mem 1024 +uvt-kvm create focal02 release=focal --mem 1024 +uvt-kvm create jammy01 release=jammy --mem 1024 +uvt-kvm create jammy02 release=jammy --mem 1024 +``` + +Use the document to understand SRIOV basics + +https://www.intel.com/content/www/us/en/developer/articles/technical/configure-sr-iov-network-virtual-functions-in-linux-kvm.html + +The following command show all the bus infos + +``` +lshw -c network -businfo +``` + +We chose the interfaces below + +``` +pci@0000:17:01.4 ens2f0v4 network Ethernet Adaptive Virtual Function +pci@0000:17:01.5 ens2f0v5 network Ethernet Adaptive Virtual Function +pci@0000:17:01.6 ens2f0v6 network Ethernet Adaptive Virtual Function +pci@0000:17:01.7 ens2f0v7 network Ethernet Adaptive Virtual Function +``` + +We can then create a new file `iface01.xml` with the following contents, and similar for all 4 interfaces replacing `function='0x4'` with the number ad the end of the pci address + +```xml + + +
+ + + + + +``` + +``` +virsh attach-device focal01 iface01.xml +``` + +Then we stop and start the VM + +``` +uvt-kvm ssh focal01 +``` + +Add netplan configuration for the interface + +``` +cat > /etc/netplan/90-custom.yaml << EOF +network: + version: 2 + ethernets: + enp7s0: + dhcp4: false + addresses: + - 192.168.1.7/24 + mtu: 9000 +EOF +``` + +``` +netplan apply +``` + +Now check dmesg, and we will find the error as shown below + +``` +[ 61.529605] iavf 0000:07:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6 +``` + +We do the same for all 4 VMs + +* focal01 VF 4, IP: 192.168.1.7/24 +* focal02 VF 5, IP: 192.168.1.8/24 +* jammy01 VF 6, IP: 192.168.1.9/24 +* jammy02 VF 5, IP: 192.168.1.10/24 + +On each of the VMs, we then enable proposed, and install the updated kernel from `-proposed` + + +``` +cat </etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list +# Enable Ubuntu proposed archive +deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe +EOF + + +cat </etc/apt/preferences.d/proposed-updates +# Configure apt to allow selective installs of packages from proposed +Package: * +Pin: release a=$(lsb_release -cs)-proposed +Pin-Priority: 400 +EOF + +apt update +apt -y install linux-image-generic/$(lsb_release -cs)-proposed +``` + +Then test again with the pings, and you should no longer see the issue + +on the focal nodes + +``` +apt -y install linux-image-generic-hwe-20.04/$(lsb_release -cs)-proposed +``` + +then reboot, and re-test \ No newline at end of file From 7adae7befd82326a44f5e50e47ed279acd34ef44 Mon Sep 17 00:00:00 2001 From: Arif Ali Date: Wed, 26 Oct 2022 15:32:55 +0000 Subject: [PATCH 2/2] rename file --- vm_propsed_testing.md => vm_proposed_testing.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename vm_propsed_testing.md => vm_proposed_testing.md (100%) diff --git a/vm_propsed_testing.md b/vm_proposed_testing.md similarity index 100% rename from vm_propsed_testing.md rename to vm_proposed_testing.md