ice driver MTU reproducer

First install uvtool on the hypervisor

apt install uvtool

Now download both focal and jammy images for usage to deploy VMs

mkdir -p /var/uvt

TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=focal
TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=jammy

Create a default network for virsh to use

cat > virsh-net-default.xml << EOF
<network>
  <name>default</name>
  <uuid>ba71f522-c2c9-4fcd-9792-3c4f294e31da</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:f9:0b:40'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>
EOF

Now, define it in virsh itself and start it up

virsh net-define virsh-net-default.xml
virsh net-start default

Create 4 vms, 2 focal and 2 jammy

uvt-kvm create focal01 release=focal --mem 1024
uvt-kvm create focal02 release=focal --mem 1024
uvt-kvm create jammy01 release=jammy --mem 1024
uvt-kvm create jammy02 release=jammy --mem 1024

Use the document to understand SRIOV basics

https://www.intel.com/content/www/us/en/developer/articles/technical/configure-sr-iov-network-virtual-functions-in-linux-kvm.html

The following command show all the bus infos

lshw -c network -businfo

We chose the interfaces below

pci@0000:17:01.4  ens2f0v4     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.5  ens2f0v5     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.6  ens2f0v6     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.7  ens2f0v7     network        Ethernet Adaptive Virtual Function

We can then create a new file iface01.xml with the following contents, and similar for all 4 interfaces replacing function='0x4' with the number ad the end of the pci address

<interface type='hostdev' managed='yes'>
  <source>
    <address type='pci' domain='0x0000' bus='0x17' slot='0x01' function='0x4'/>
  </source>
  <vlan>
    <tag id='999'/>
  </vlan>
</interface>

virsh attach-device focal01 iface01.xml

Then we stop and start the VM

uvt-kvm ssh focal01

Add netplan configuration for the interface

cat > /etc/netplan/90-custom.yaml << EOF
network:
  version: 2
  ethernets:
    enp7s0:
      dhcp4: false
      addresses:
       - 192.168.1.7/24
      mtu: 9000
EOF

netplan apply

Now check dmesg, and we will find the error as shown below

[   61.529605] iavf 0000:07:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6

We do the same for all 4 VMs

focal01 VF 4, IP: 192.168.1.7/24
focal02 VF 5, IP: 192.168.1.8/24
jammy01 VF 6, IP: 192.168.1.9/24
jammy02 VF 5, IP: 192.168.1.10/24

On each of the VMs, we then enable proposed, and install the updated kernel from -proposed

cat <<EOF >/etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF


cat <<EOF >/etc/apt/preferences.d/proposed-updates
# Configure apt to allow selective installs of packages from proposed
Package: *
Pin: release a=$(lsb_release -cs)-proposed
Pin-Priority: 400
EOF

apt update
apt -y install linux-image-generic/$(lsb_release -cs)-proposed

Then test again with the pings, and you should no longer see the issue

on the focal nodes

apt -y install linux-image-generic-hwe-20.04/$(lsb_release -cs)-proposed

then reboot, and re-test

3.6 KiB Raw Blame History

ice driver MTU reproducer

3.6 KiB

Raw Blame History