# ice driver MTU reproducer

First install `uvtool` on the hypervisor

```
apt install uvtool
```

Now download both focal and jammy images for usage to deploy VMs

```
mkdir -p /var/uvt

TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=focal
TMPDIR=/var/uvt uvt-simplestreams-libvirt --verbose sync arch=amd64 release=jammy
```

Create a default network for `virsh` to use

```
cat > virsh-net-default.xml << EOF
<network>
  <name>default</name>
  <uuid>ba71f522-c2c9-4fcd-9792-3c4f294e31da</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:f9:0b:40'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>
EOF
```

Now, define it in virsh itself and start it up

```
virsh net-define virsh-net-default.xml
virsh net-start default
```

Create 4 vms, 2 focal and 2 jammy

```
uvt-kvm create focal01 release=focal --mem 1024
uvt-kvm create focal02 release=focal --mem 1024
uvt-kvm create jammy01 release=jammy --mem 1024
uvt-kvm create jammy02 release=jammy --mem 1024
```

Use the document to understand SRIOV basics

https://www.intel.com/content/www/us/en/developer/articles/technical/configure-sr-iov-network-virtual-functions-in-linux-kvm.html

The following command show all the bus infos

```
lshw -c network -businfo
```

We chose the interfaces below

```
pci@0000:17:01.4  ens2f0v4     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.5  ens2f0v5     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.6  ens2f0v6     network        Ethernet Adaptive Virtual Function
pci@0000:17:01.7  ens2f0v7     network        Ethernet Adaptive Virtual Function
```

We can then create a new file `iface01.xml` with the following contents, and similar for all 4 interfaces replacing `function='0x4'` with the number ad the end of the pci address

```xml
<interface type='hostdev' managed='yes'>
  <source>
    <address type='pci' domain='0x0000' bus='0x17' slot='0x01' function='0x4'/>
  </source>
  <vlan>
    <tag id='999'/>
  </vlan>
</interface>
```

```
virsh attach-device focal01 iface01.xml
```

Then we stop and start the VM

```
uvt-kvm ssh focal01
```

Add netplan configuration for the interface

```
cat > /etc/netplan/90-custom.yaml << EOF
network:
  version: 2
  ethernets:
    enp7s0:
      dhcp4: false
      addresses:
       - 192.168.1.7/24
      mtu: 9000
EOF
```

```
netplan apply
```

Now check dmesg, and we will find the error as shown below

```
[   61.529605] iavf 0000:07:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
```

We do the same for all 4 VMs

* focal01 VF 4, IP: 192.168.1.7/24
* focal02 VF 5, IP: 192.168.1.8/24
* jammy01 VF 6, IP: 192.168.1.9/24
* jammy02 VF 5, IP: 192.168.1.10/24

On each of the VMs, we then enable proposed, and install the updated kernel from `-proposed`


```
cat <<EOF >/etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF


cat <<EOF >/etc/apt/preferences.d/proposed-updates
# Configure apt to allow selective installs of packages from proposed
Package: *
Pin: release a=$(lsb_release -cs)-proposed
Pin-Priority: 400
EOF

apt update
apt -y install linux-image-generic/$(lsb_release -cs)-proposed
```

Then test again with the pings, and you should no longer see the issue

on the focal nodes

```
apt -y install linux-image-generic-hwe-20.04/$(lsb_release -cs)-proposed
```

then reboot, and re-test