mbuf: Make m_unshare() fail on KTLS mbufs
Commit f2202ab5abda did not account for KTLS mbufs.
m_unshare() tries to linearize the original mbuf chain and creates a
writable copy of it, converting unmapped mbufs. Both of them are
unsafe for KTLS mbufs.
It is better to return NULL if the mbuf chain contains a KTLS mbuf.
Reported by: jhb
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D58466
igbv: Retry transient VLAN admission failures
A PF mailbox NACK does not distinguish the SR-IOV VLAN request rate
limit from permanent VLVF exhaustion. Preserve desired VLAN membership
and retry four additions per 500 ms timer tick, matching the PF
sustained allowance.
Bound the whole recovery batch to eight seconds from its first failure
and consolidate restore diagnostics, so a full table cannot create a
permanent mailbox poller or repeated per-VID log bursts.
Sponsored by: BBOX.io
e1000: Sample statistics at timer cadence
Mailbox and link interrupts share iflib admin service with the periodic
timer. Mark timer-driven passes explicitly and run the hardware
statistics sweep only for those samples instead of repeating 66 PF MMIO
reads for every VF mailbox message.
DTrace on the I350 DUT measured the PF sweep at about 79 us on average.
The normal hz/2 timer continues to extend clear-on-read counters
safely; exported counters may trail hardware by up to 500 ms.
Sponsored by: BBOX.io
igb: Rate-limit VF VLAN rebuild requests
Give each VF a burst of 64 VLAN additions and refill it at eight
additions per second. Removals remain unrestricted, idempotent requests
consume nothing, and trusted PF-wide initialization replenishes the
burst while guest resets do not. Checks VLVF capacity before charging
a token.
Do not apply this policy to multicast requests because Linux igbvf and
DPDK ignore their reply status; aggregate MTA write elision bounds
those updates instead.
Sponsored by: BBOX.io
igb: Update only changed IOV VLAN filters
Keep the full VFTA/VLVF software recomputation and clear-map-set
ordering, but compare each phase against the authoritative old value.
Write only VFTA words and VLVF slots whose effective contents change.
I350 uses its software VFTA shadow because erratum 20 makes live reads
unreliable; an invalid shadow forces a complete clear before sparse
restoration. 82576 continues to diff against live VFTA reads.
Add SDT probes for every logical write phase and the final software
images so hardware tests can verify exact elision counts. On my I350
DUT, the old full table path averaged 819 us across 31 VLAN removals
versus about 79 us for the PF statistics sweep.
Sponsored by: BBOX.io
igb: Update only changed IOV multicast hashes
Build the aggregate PF/VF multicast bitmap in software and compare it
with the e1000 MTA shadow. Write only registers whose desired value
changed, while forcing a complete write after PF reset invalidates the
hardware table.
This bounds alternating VF multicast updates without NACKing them.
Linux igbvf and DPDK ignore multicast reply status, so a command-rate
limiter could otherwise acknowledge configuration while leaving
hardware state stale.
Sponsored by: BBOX.io
igb: Stop writing the legacy TADV register
TADV is an em-class interrupt delay register and is absent from the
82575 and later register model. The igb attach path does not expose or
initialize that control, but transmit initialization still wrote its
zero valued storage into a reserved queue-window offset.
Apply the same igb_mac_min boundary already used for TIDV and the
absolute-delay sysctls.
MFC after: 1 week
Sponsored by: BBOX.io
igbv: Support secondary unicast filters
Support the Linux igbvf secondary-MAC mailbox subprotocol, used by
Linux guests running MacVTap.
Replay up to three non-primary unicast addresses after reset and
whenever the address list changes, subject to PF allow-set-mac policy.
Sponsored by: BBOX.io
igbv: Improve VF mailbox and status behavior
Treat VF media as fixed 1000baseT full duplex and report PF not ready
and generated MAC fallback states during attach.
After a successful reset handshake, reconcile a PF rejected MAC back
into the ifnet. If the PF is unavailable, defer MAC, multicast, VLAN,
LPE, and promiscuity replay until CTS is restored. Track a rejected
VLAN removal separately so leaked traffic remains tagged until reset
proves that the stale hardware filter is gone.
Baseline VF counters at attach, collect the four loopback packet and
octet counters with rollover-safe deltas, and account software RX
checksum offload results. Preserve accumulated statistics across PF
resets by rebasing the raw hardware counters, and sample them while
physical link is down because VF loopback can remain active.
Retain the 82576 VFMPRC hardware statistic, but do not read it on I350
VFs because specification update errata 31 says it is unavailable.
[5 lines not shown]
igbv: Isolate VF policy and validate its registers
Give igb virtual functions a separate ifdi method table and move
VF-specific attach, reset, queue, interrupt, and diagnostic policy to
if_igbv.c. Keep shared descriptor-ring mechanisms in if_em.c.
Derive VF identity from IFLIB_IS_VF and assert that hardware
identification agrees. Under INVARIANTS, validate normal VF CSR
accesses against the sparse 82576 and I350 VF register maps.
Stop shared setup from accessing PF-only controls. Require MSI-X and
defer VF sysctls until attach succeeds so failed attachment cannot
leave handlers pointing at freed driver state.
Advertise only VF capabilities, run adaptive moderation without the
PF receive-buffer guard, enable SRRCTL.DROP_EN, and provide a VF-safe
diagnostic register view.
The moved implementation is the existing FreeBSD code. Register model
[3 lines not shown]
igb: Reprogram descriptor queues while disabled
Disable each igb-class transmit and receive queue and flush before
changing its descriptor-ring registers. Restore the head and tail
indices that Intel documents as surviving a VF reset.
Use the igb queue-enable control instead of programming legacy TXDCTL
granularity, low-water, and reserved bits that do not belong to the
82575 and later.
Sponsored by: BBOX.io
MFC after: 1 week
igb: Add SR-IOV PF support for 82576 and I350
Add the PCI IOV schema and PF control plane for up to seven VFs with
one hardware queue per pool. Implement VF mailbox handling, MAC and
VLAN assignment, multicast filtering, promiscuity policy,
anti-spoofing, malicious-driver recovery, reset replay, and queue
lifecycle management.
The basic SR-IOV and VMDq PF implementation follows DPDK Intel e1000
code, including PF pool selection, one queue per pool, mailbox
dispatch, and VF enablement. Intel FreeBSD igb-2.5.31 supplies the
older driver baseline. Linux igb and the Intel SDMs clear up lifecycle,
isolation, reset, and family-specific details absent from DPDK.
Enabling IOV requires the PF to attach with one TX and RX queue.
Systems whose defaults select RSS queues must set the documented iflib
queue override tunables before attach.
Only 82576 and I350 support SR-IOV in silicon. The series has been
[6 lines not shown]
igb: Guard register dump during queue setup
The register-dump sysctl is installed before iflib allocates the queue
arrays and remains visible while they are freed. Return ENXIO outside
the queue lifetime instead of dereferencing a NULL or stale array.
Sponsored by: BBOX.io
igbv: Correct I350 loopback VLAN byte order
I350 loopback receive descriptors report VLAN tags byte-swapped for
both PFs and VFs. The receive path handled the PF device types but
omitted e1000_vfadapt_i350, causing an admitted VF VLAN packet to be
delivered untagged to the VF parent.
Include the I350 VF type in the existing correction. This matches the
dedicated IGB_RXQ_FLAG_LB_BSWAP_VLAN handling in DPDK igbvf.
MFC after: 1 week
Sponsored by: BBOX.io
stand: Cleanup use of cached DHCP response packet
- When cached response is available, actually use xid from one
instead of using its byte-swapped value for BIOS and 1 for UEFI.
- If cached response is not available, generate pseudo-random xid,
since use of a constant may cause conflicts if two systems are
booting same time, and server sends responses as broadcast.
- When cached response is available, skip DHCP DISCOVER/OFFER
and just send REQUEST to the DHCP server from the cached response.
We could skip this phase too and just use the cached response, but
we don't know whether firmware requested all of DHCP options we'd
like to get.
Tested on amd64 Supermicro X11DPI-NT for both BIOS and EFI, with
and without cached response packet.
igbv: Split 82576 and I350 VF registration off igb
Register the 82576 and I350 VF PCI IDs under a separate igbv driver
while continuing to share the igb datapath implementation.
Follow the ixv driver split and give the VF context IFLIB_IS_VF so
iflib does not apply the PF SR-IOV detach guard to a child VF. Program
VTIVAR_MISC in the VF low byte so mailbox and reset notifications reach
the VF admin vector.
The split will become increasingly obvious as bug fixes land, trying to
bias everything with if (sc->vf_ifp) everywhere is error prone in two
directions.
This breaks existing naming/configurations and cannot be MFCed as-is.
I have no plans of adapting it to prior branches at the moment but it
may be possible.
Relnotes: yes
Sponsored by: BBOX.io
mailmap: Fix Nick Price
Somehow a few commits ended up with "null" appended to Nick's name and
email address.
Reviewed by: Nick Price <nick at spun.io>
Differential Revision: https://reviews.freebsd.org/D58517
pci_iov: Clear NumVFs when configuration fails
pci_iov_config() programs NumVFs before validating the final VF RID
layout and allocating all generic resources. A subsequent error ran the
driver uninit callback but left the hardware NumVFs register programmed
while the software VF count returned to zero.
Clear NumVFs in the error path after the driver uninit callback,
matching normal SR-IOV teardown ordering. This prevents stale hardware
state after a failed configuration and permits a clean retry.
MFC after: 1 week
Sponsored by: BBOX.io
pci_iov: Permit non-ARI VFs on a secondary bus
A non-zero VF device number does not always require ARI. The Intel
82576 and I350 [1] explicitly support a non-ARI layout that places VFs
on the next bus.
Check every requested VF RID and reject a non-zero device only when it
is on the PF bus. This retains the ARI guard for invalid same-bus
layouts while permitting the documented second-bus layout.
[1] Intel I350 Datasheet, sections 7.8.2.6.1.2, 9.6.4.6
Sponsored by: BBOX.io
igb: Address queue registers by hardware queue ID
Use each ring's physical queue index for initialization, MSI-X routing,
register dumps, sysctls, and debug output instead of assuming that its
logical array index is also its hardware index.
This is a no-op for the normal queue layout. A later SR-IOV change
moves the PF ring to hardware queue num_vfs, so its hardware ID then
differs from logical queue zero.
Sponsored by: BBOX.io
e1000: Update shared igb SR-IOV code
Update the shared e1000 PF/VF mailbox interfaces for an in-tree igb
SR-IOV implementation.
Intel FreeBSD igb-2.5.31 and DPDK provide the older PF/VF mailbox
baseline.
The retained PF mailbox read and explicit unlock operation follow a
simple Linux igb parameter addition to make PF mailbox acquisition
nonblocking so the driver can retry outside the shared primitive.
Treating a CTS-less E1000_PF_CONTROL_MSG as a reset follows DPDK.
Sponsored by: BBOX.io
