How to Configure SR-IOV on Proxmox for Bare-Metal VM Networking

Standard virtual networking relies on software bridges that consume massive CPU resources under heavy load. By enabling SR-IOV (Single Root I/O Virtualization), you can bypass the hypervisor entirely, granting your Proxmox VMs direct hardware access to the network card for near bare-metal speeds.

What You'll Learn

Enabling Hardware IOMMU

To isolate PCIe devices, you must first enable the IOMMU (Input-Output Memory Management Unit) in your bootloader.

Open your GRUB configuration file:

nano /etc/default/grub
                                    

Append intel_iommu=on iommu=pt (or amd_iommu=on for AMD servers) to the GRUB_CMDLINE_LINUX_DEFAULT line.

Apply the changes by running update-grub and reboot your dedicated server:

update-grub
reboot
                                    

Activating Network Virtual Functions

Once IOMMU is active, instruct your physical NIC (like an Intel X710) to split into Virtual Functions (VFs).

Identify your primary interface name by typing:

ip link show
                                    

Check the maximum number of VFs your NIC supports (e.g., using eno1):

cat /sys/class/net/eno1/device/sriov_totalvfs
                                    

Enable 4 Virtual Functions by writing to the sysfs driver:

echo 4 > /sys/class/net/eno1/device/sriov_numvfs
                                    

Passing VFs to Proxmox VMs

Now, map these newly created hardware devices to your virtual machines.

1. Log into your Proxmox web interface and select your target VM.

2. Navigate to Hardware > Add > PCI Device.

3. Select one of the new Virtual Functions associated with your physical NIC. Make sure to select "Raw Device" and uncheck "All Functions", then boot the VM.

Verifying Network Performance

With the VF passed through, your VM communicates directly with the physical network switch.

Inside the guest OS, run the following to confirm the new hardware interface is detected:

ip a
                                    

Assign your public or private EPY Host IP address to this interface to begin routing traffic at bare-metal speeds.

Are you planning to deploy this SR-IOV setup on an Intel or an AMD-based dedicated server?

Depending on your choice of hardware, the configuration nuances for IOMMU and SR-IOV differ slightly at the kernel and driver level. Here is how to handle the specifics for both architectures:

Intel-Based Servers (Intel VT-d)

If you are running an Intel Xeon server, the focus is on the intel_iommu kernel parameter and Intel-specific NIC drivers (like i40e for the popular X710 cards).

• Bootloader Alignment: Ensure your GRUB strictly contains intel_iommu=on iommu=pt. The pt (pass-through) parameter is crucial as it prevents the hypervisor from translating DMA requests for devices not passed to a VM, saving host CPU cycles.

• VFIO Modules: Intel's virtualization technology requires the vfio-pci driver to bind the Virtual Functions. You must explicitly load the modules by adding vfio, vfio_iommu_type1, vfio_pci, and vfio_virqfd to your /etc/modules file.

• Driver State: Some Intel cards require the Physical Function (PF) link to be set to an "up" state on the host before the guest VMs can pass traffic out of the Virtual Functions.

  bash Copy

  
  ip link set eno1 up
                                              

AMD-Based Servers (AMD-Vi)

If you are deploying on an AMD EPYC server, the architecture uses AMD-Vi, and IOMMU groups are handled differently across the massive PCIe lane architecture.

• Bootloader Alignment: Your GRUB must contain amd_iommu=on iommu=pt. While some modern Linux kernels enable AMD IOMMU by default, explicitly setting it ensures stable Virtual Function generation.

• IOMMU Group Isolation: Because AMD EPYC processors have high PCIe lane counts spread across multiple chiplets, it is critical to verify that your generated VFs sit in isolated IOMMU groups. You can check this by listing the directories under /sys/kernel/iommu_groups/. If multiple devices share a group, Proxmox will fail to pass the single VF.

  bash Copy

  
  ls /sys/kernel/iommu_groups/
                                              

• Firmware-Level SR-IOV: AMD servers are frequently paired with 25Gbps or 100Gbps Mellanox cards (using the mlx5_core driver). Unlike Intel, Mellanox requires you to use the mstconfig tool to enable SR-IOV directly inside the network card's hardware firmware before the Linux OS can generate the VFs via sysfs.

Final Thoughts

By implementing SR-IOV on your EPY Host dedicated server, you are fundamentally changing how your virtual machines interact with the network. Bypassing the hypervisor's software bridge not only eliminates CPU bottlenecks during high-traffic surges but also drops network latency to near-zero. Whether you are running a virtualized router, a heavily trafficked load balancer, or a mission-critical database cluster, passing hardware Virtual Functions directly to your VMs ensures you extract every ounce of bare-metal performance from your infrastructure.

Maximize Your Potential with Bare-Metal Servers

To truly unlock the capabilities of SR-IOV, you need a high-performance network stack that doesn't falter. Explore EPY Host's dedicated bare-metal servers designed to give you uncompromising control over hardware acceleration, Intel VT-d, and AMD-Vi environments for ultimate hypervisor efficiency.

Explore EPY Host Dedicated Servers →