pci: Reserve bus numbers required by SR-IOV VFs
Some firmware assigns only one bus number to each PCI-PCI bridge. This
prevents later SR-IOV VF enumeration when a VF routing ID falls on a bus
number already allocated to a sibling bridge.
Reserve only the additional bus numbers required by SR-IOV PFs.
Enumerate all directly attached functions before child drivers and
bridges attach, inspect their device_t objects for SR-IOV, and grow the
PCI bus resource through the highest possible VF routing ID.
First VF Offset and VF Stride may change when NumVFs changes. Probe
every valid NumVFs value and preserve the original setting. When the
upstream hierarchy uses ARI, temporarily enable the SR-IOV ARI Hierarchy
control in the lowest-numbered PF while sizing, then restore it. Scope
active-VF detection to each conventional PCI slot; an ARI bus remains
one slot-0 hierarchy. If firmware left VFs enabled on a device, do not
modify it and reserve only its active layout.
[21 lines not shown]
pci_iov: Reuse downstream bridge bus window if it already covers VF bus
If the parent bridge's [secondary, subordinate] window already covers
the VF bus (e.g., programmed by BIOS or a prior PF), skip allocating
PCI_RES_BUS. This avoids a duplicate rman allocation in the multi-PF
case while still allocating when growth is actually needed.
Reviewed by: ssaxena
Differential Revision: https://reviews.freebsd.org/D52163
(cherry picked from commit feb0a7e19f3c2e4c7eb90668b1e3dc34b5bb6dd6)
ixl(4): Fix NVM update API
Driver provides an interface for tools operating on adapters'
Non-Volatile Memory using SIOC(G|S)DRVSPEC ioctls. It allows
tools to issue Admin Queue commands to the firmware, which
may return an error code encoded as a negative integer.
The driver passed this negative integer as the return value
of the ioctl handler. This was causing issues because the FreeBSD
kernel reserves negative error codes in system calls for special
purposes such as restarting the handler or detecting incorrect
behavior in VFS operations. Simple remapping of negative values
to positive integers caused currently available tools to misbehave
and could break the NVM update process. Introduce a new API version,
which allows new versions of tools to detect the fixed driver.
Temporarily keep the old API, but issue a warning that tools need
to be updated.
Also refactor the code to use centralized cleanup and move the
debug message from the firmware earlier to make sure it is not masked
[10 lines not shown]
cxgbe(4): Fix netmap rx behavior with nm_split_rss
The driver should look for active queues and one potential default-queue
in both halves of the split instead of stopping at the first valid
default-queue.
Fixes: a9f476580eb0cxgbe(4): fixes for netmap operation with only some queues active
Sponsored by: Chelsio Communications
(cherry picked from commit 77d71f5f36cc01e7fc2ca0828af00a4478bbf53c)
em(4): fix capability bounds needed to access checksum context.
Ensure the offp capability bounds cover entire struct with checksum fields.
This is needed for CHERI systems to avoid bounds violation trap, as
otherwise offp allowed to dereference 4 bytes of csum_flags field only
so bzero failed.
Tested on ARM Morello.
Reviewed by: kbowling
Discussed with: jrtc27
Sponsored by: CHERI Research Centre
Differential Revision: https://reviews.freebsd.org/D53903
(cherry picked from commit 64ee9c166ce5e807e575d205ac2e15cc5cf6581b)
vtnet: remove M_HASHTYPE when there is only one pair of rx/tx queue
Summary:
Doing so will let the upper layer know how to deal with software hash,
with benefits like inp_flowid can be set and m_pkthdr.flowid can be set
in output path.
Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D54929
(cherry picked from commit 20285cad7a55ecd0020f51f3daee74db8b1ea5a0)
tcp: Unifidef use of rss software hash in syncache
Ever since "d9c55b2e8cd6 rss: Enable portions of RSS globally.."
exposed the RSS software hashing functions, it has been possible
to use them without "ifdef RSS". Do so now in the syncache
so as to get flowids recorded.
Note that the use of the rss hash functions is conditional on IP versions,
so we must ifdef INET to ensure rss_proto_software_hash_v4() is available.
Fixes 73fe85e486d2
Sponsored by: Netflix
Reviewed by: glebius, p.mousavizadeh_protonmail.com, nickbanks_netflix.com, tuexen
Differential Revision: https://reviews.freebsd.org/D54534
(cherry picked from commit 21865c970888ad0e2cde06db843ae2a05af550e7)
tcp: Enable symmetric hashing by setting hash on outgoing conns
Now that we can trust NICs to supply an identical hash result
to software, we can setup the inpcb hash on outgoing connections.
This gives us symmetric hashing, meaning packets should enter
and leave on the same NIC queue.
Differential Revision: https://reviews.freebsd.org/D53104
Reviewed by: adrian, cc, kbowling, tuexen, zlei
Sponsored by: Netflix
(cherry picked from commit dd0e6bb996dc46e1d91f3d9aef87979716479465)
em: remove M_HASHTYPE when RSS is not enabled
Summary:
Since "73fe85e486d2 tcp: store flowid info in syncache", inp_flowid can be set
if the incoming packet is not M_HASHTYPE_NONE. But this can introduce dummy
and duplicated flowid when a virtual interface set M_HASHTYPE_OPAQUE. This
change will let the upper layer know how to deal with software hash, with
benefits like inp_flowid can be set correctly and m_pkthdr.flowid can be set
correctly in output path.
This fix is similar to "20285cad7a55"
Reviewed by: kbowling
Differential Revision: https://reviews.freebsd.org/D55137
(cherry picked from commit efcc0423d80e4048e11e997abc46bd0343537e1b)
igc: make the hardware RSS hash agree with the stack's configuration
rss_gethashconfig() is available without options RSS since d9c55b2e8cd6.
Use it to program MRQC, as ixl(4), ice(4) and iavf(4) do, instead of a
fixed field set that included UDP 4-tuple, which the configuration
excludes unless net.inet.rss.udp_4tuple is set. UDP is now hashed on
addresses only by default.
Also report the UDP hash types on receive; they were passed up as
M_HASHTYPE_NONE.
Reviewed by: kbowling
Fixes: 517904de5cca ("igc(4): Introduce new driver for the Intel I225 Ethernet controller.")
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60284
(cherry picked from commit 66ff9289b041720f29205f179337f81b5dd0c365)
ixl: verify flowid is valid before setting rsstype & flowid
According to section 8.3.2.2.1 of the XL710 datasheet,
FLTSTAT indicates if RSS hashing was used (or flow director
or nothing) to arrive at the flowid it is reporting.
We need to check this before reporting the flowid and setting the
rss type. Not checking this allows invalid flowids (0s) to be
reported for some TCP traffic (it seems to mostly impact incoming
connections). This leads to the inp flowid being 0, and egress
traffic being unbalanced (going to tx queue 0, and NIC 0 of an
lacp bundle).
Differential Revision: https://reviews.freebsd.org/D54442
Reviewed by: adrian, kbowling
Sponsored by: Netflix
(cherry picked from commit ac1cd655f647cfd8e8d8c08da00f41abfce212a1)
rss: make toeplitz.c standard part of the kernel
This will fix LINT-NOIP build. This actually adds very little to the
kernel text, e.g. 500 bytes on amd64. A perfect solution would be to
instead declare rss_config.c as 'optional inet | inet6', but that would
fail to build LINT-NOIP in several NIC drivers, that use RSS and
absolutely ignore that both INET and INET6 are optional. It is very
unlikely that vendors who maintain these drivers will will ever chase the
holy grail of a build that doesn't support IPv4 and IPv6.
Fixes: d9c55b2e8cd6b79f6926278e10a79f1bcca27a4b
(cherry picked from commit 407b1e4980189252fade78438aa08191a9883cdc)
sctp: improve handling of SO_SETFIB socket option
Since SCTP keeps fibnum in the SCTP protocol control block,
the code can't just defer the processing to the IPv4 or IPv6
layer. Therefore it needs to be implemented in the the SCTP
layer and it needs to call the socket layer function.
PR: 300091
Reviewed by: tuexen
Fixes: caccbaef8e26 ("socket: Move SO_SETFIB handling to protocol layers")
(cherry picked from commit 7b8151b0cd1b4f077b382b6be2c02c936d1988fa)
bnxt: Initialize RSS with the common key
Use rss_getkey() for the initial VNIC key instead of generating a
per-attachment private key. Preserve the writable rss_key sysctl,
its override lifetime, the hardware DMA representation, and all hash
and indirection settings.
Reviewed by: gallatin
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59795
(cherry picked from commit 62c80d76f7dc960f430cb34f88c2a549612e3e93)
qlnx: Report the programmed RSS key and hash types
Implement SIOCGIFRSSKEY and SIOCGIFRSSHASH for qlnxe and qlnxev using
the cached vport RSS configuration. This lets hn(4) synchronize its
synthetic RSS configuration with the VF without assuming that every
driver programs all of the common RSS hash selections.
Serialize the queries with the device lock and reject them while the
interface is not running. Convert the firmware request's key words back
into network byte order and translate the six representable IPv4/IPv6
TCP/UDP hash selections. Report no hashing when RSS is disabled, including
the driver's single-queue configuration.
For VFs, require a successful RSS extension response from every hardware
function. The overall vport-update reply can succeed while its RSS TLV
is missing or rejected; recording that result prevents reporting an
unconfirmed configuration. Invalidate the confirmation before each RSS
request. Leave existing vport-update return semantics, queue setup, and
the receive datapath unchanged.
[6 lines not shown]
vnic nicvf: Program the common RSS key in network word order
Replace the repeated hardware-reset key with rss_getkey(). Decode each
eight-byte chunk with be64dec() into the register-word array before
programming the five key registers.
This matches the key serialization in DPDK's nicvf_rss_set_key(), added
by commit 7694fab264ac666d62a4cae04f2979991ce64259
("net/thunderx/base: add RSS and RETA configuration"). A raw copy into
the word array on little-endian hosts would reverse bytes within each
register relative to that convention.
Preserve the CPI-algorithm gate, enabled hash fields, and round-robin
indirection table.
Reviewed by: gallatin
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59806
(cherry picked from commit 8c25b70120e8838135cb356c94d350105f45ff7c)
enetc: Use the common RSS key
Replace the random register words with rss_getkey() and decode each
four-byte chunk with le32dec() before writing the port's key registers.
Keep the RSS capability check and existing queue-count and indirection
table programming unchanged.
Reviewed by: gallatin
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59805
(cherry picked from commit 449fc84f370b079619eaa0d70008391ac07718f5)
