ena: Reinit RSS components during queue count change flow
The io_queues_nb sysctl is writable while the interface is down, and
setting it destroys the RSS components. When the interface is not UP
nothing reinitialises them, so the indirection table is left referencing
queues that no longer exist until the next ifconfig up.
Call ena_rss_configure() after the queue count changes when the
interface is down. When it is up, this happens through ena_up().
This can configure RSS twice if the interface is brought up afterwards,
costing three redundant admin commands. That is preferred over changing
ena_up(), which the reset path also uses.
Reviewed by: cperciva
Fixes: 56d41ad5fe9c ("Add sysctl node for ENA IO queues number adjustment")
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Update driver version to v2.8.5
New Features:
* Retrieve RSS indirection table size from device
Bug Fixes:
* Limit IO IRQ setup loop to active queues only
* Reinit RSS table on device restore when queue count shrinks
* Fix RSS not working after driver reload
* Fix NULL deref before NULL check in RSS sysctl
* Reinit RSS components during queue count change flow
Minor Changes:
* Fix ENA_MEM_FREE_COHERENT with an expression size argument
Reviewed by: cperciva
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Reinit RSS table on device restore when queue count shrinks
When ena_restore_device() is called after a reset, pci_alloc_msix()
may return fewer MSI-X vectors than requested, causing num_io_queues
to shrink. When this happens, the RSS indirection table retains
entries referencing queues that no longer exist, routing packets to
non-existent queues and causing connectivity loss.
Extract ena_set_io_queue_count() to pair the queue count update with
ena_com_rss_destroy(), ensuring the RSS table is always invalidated
when the queue count changes. Use it from both ena_update_io_rings()
and ena_restore_device().
Reviewed by: cperciva
Fixes: e2735b095b7e ("Fix assumptions about number of IO queues in the ENA")
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Fix NULL deref before NULL check in RSS sysctl
In ena_sysctl_rss_indir_table(), the code dereferenced
indir->sysctl_buf before checking whether indir itself is NULL.
Move the NULL check above the dereference so the early-return path
fires before any access through indir.
Reviewed by: cperciva
Fixes: 6d1ef2abd330 ("ena: Implement full RSS reconfiguration")
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Retrieve RSS table size from device
The current driver assumes a default RSS table size of 128 entries,
expecting the device's RSS table size to match this assumption.
This patch allows the driver to create RSS indirection table
with the size advertised by the device.
Reviewed by: cperciva
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Fix RSS not working after driver reload
When reloading the driver (kldunload/kldload),
ena_rss_init_default_deferred() is called before ena_attach()
completes. At this point, devclass_get_softc() returns NULL for all
devices because the adapter structs have not been populated yet. As a
result, ena_rss_init_default() is never called and ENA_FLAG_RSS_ACTIVE
is never set. Later, in ena_up_complete(), the code checks the flag,
finds it unset, and skips ena_rss_configure() entirely. RSS is never
initialized, and all traffic is directed to queue 0.
Remove the ENA_FLAG_RSS_ACTIVE gate in ena_up_complete() and always call
ena_rss_configure(), which initialises the table when tbl_log_size is
zero. The deferred SYSINIT is now redundant, so remove it.
Reviewed by: cperciva
Fixes: 9b8d05b8ac78 ("Add support for Amazon Elastic Network Adapter (ENA) NIC")
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Limit IO IRQ setup loop to active queues only
The loop in ena_request_io_irq() iterated over adapter->msix_vecs,
which reflects the maximum number of MSI-X vectors allocated at
init time. When the number of IO queues is reduced at runtime via
the io_queues_nb sysctl, msix_vecs is not updated, causing the
driver to bind interrupts and log CPU affinity for queues that no
longer exist.
Replace the loop bound with ENA_IO_IRQ_IDX(adapter->num_io_queues)
to iterate only over the currently active IO queues.
Also replace the loop in ena_free_io_irq() the same.
Reviewed by: cperciva
Fixes: 56d41ad5fe9c ("Add sysctl node for ENA IO queues number adjustment")
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Retrieve RSS indirection table size from the device in ena-com
This patch allows the driver to create RSS indirection table with the
size advertised by the device.
Reviewed by: cperciva
MFC after: 3 days
Sponsored by: Amazon, Inc.
ena: Fix ENA_MEM_FREE_COHERENT with an expression size argument
ENA_MEM_FREE_COHERENT() discards its size parameter with "(void)size".
A cast binds tighter than '*', so when the argument is an in-place
calculation the macro expands to ((void)(a)) * b, which fails to
compile with "void value not ignored as it ought to be".
Wrap the parameter in parentheses before casting it. No functional
change for existing callers, which all pass a plain variable.
Reviewed by: cperciva
MFC after: 3 days
Sponsored by: Amazon, Inc.
amd64 FRED: normalize the user-provided 'fred' knob value
Assembler accesses 'fred' as byte, ensure that any non-zero
user-provided value works. This is pure cosmetics.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 resumectx(): restore FRED state before loading %cr4
%cr4 has the FRED enable bit. This only matters for NMI and #MC, and the
kernel is really not ready to handle them while in resumectx(), but do
it for formal correctness.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 FRED: no need to store fs/gsbase on user_fred_not_syscall path
The stores are only relevant for PCB_FULL_IRET return to userspace,
and setting the flag stores the bases into the pcb anyway. Also, the
hidden issue is that FRED already did implicit SWAPGS on entry, so
stored gsbase was wrong anyway.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 FRED: handle in-kernel faults same as IDT
Pass #MC and #DF directly to corresponding handlers, avoiding EFI
pcb_onfault case.
If a fault occurs while kdb/ddb is active, handler must call
kdb_reenter() and not do the normal processing, which typically ends up
in panic() for kernel-mode faults. But #MC and #DF should be excluded
there as well.
NMI is not an exception, it is delivered as specific event.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rtwn: fix 8188f channel offset / width programming
* rtwn_chan2centieee() already calculates the proper centre
frequency, so it doesn't need to be further offset.
* Don't program R92C_RF_CHNLBW with further 20MHz wide bits;
the rest of the programming works correctly for 20/40MHz
operation.
Tested:
* Verified against the linux rtl8xxxu/8188f.c code.
* Tested w/ 20 and 40MHz 11n; 40MHz is now operating at higher
speed.
Fixes: 2c38dc04d6f8bc84620bb3b67c8ccd9f910c4d03
Makefile.inc1: Make sure mandoc bootstrap tool depends on libz
It links against both libopenbsd and libz, so reflect this in the
bootstrap tool's dependencies in case of races from highly-parallel
builds.
Reported by: jkh
MFC after: 1 week
contrib/kyua: update to 0.15.1
This new release contains multiple bugfixes. Please see the release
notes mentioned in the merge commit for more details.
MFC after: 2 weeks
Merge commit '3e05143b80d1537cdd51a5dcf742da31f400507e'
pf: Free a table entry that cannot be routed
pfr_insert_kentry() returned the error of pfr_route_kentry() and
leaked the entry it had created; pfr_insert_kentries() stopped at the
first entry it could not route and leaked it and the rest of the work
queue, which no caller frees. Destroy an entry that cannot be routed
and go on with the next.
Reviewed by: kp
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60532
(cherry picked from commit 989d72fd3d35daa6a056afdd0c6b0aba4c5d8ccf)
pf: Do not hash pool address pointers into rule hashes
pf_hash_pool() hashes the first address of a pool as a whole struct
pf_addr_wrap, including the union that holds the kernel's pfi_dynaddr
or pfr_ktable pointer.
An interface pool such as "-> ($ext_if)" gets a new pfi_dynaddr on
every load, so with "set keepcounters" the rule never matches its
predecessor and loses its counters on every reload. The pointers also
differ between hosts: once a filter rule in the main ruleset has an
interface or table pool, pfsync peers disagree on the ruleset checksum
and bind every synced state to the default rule.
Hash the address with that union cleared. The rest of the address,
including the prefix length of an interface address, is hashed as
before, and the hash of a rule with a plain address pool does not
change.
Reviewed by: kp
[6 lines not shown]
pf: Set the reassembly flags under the rules lock
DIOCSETREASS stored V_pf_status.reass twice with no lock, first the
flags it was given and then, if reassembly was not among them, zero.
pf_normalize_ip() and pf_normalize_ip6() read the flags with the rules
lock held for reading, so a packet could see the intermediate value.
Compute the flags first and store them once, with the rules lock held
for writing, as DIOCSETDEBUG and DIOCSETHOSTID do.
Reviewed by: kp
Fixes: 39282ef356db ("pf: backport OpenBSD syntax of "scrub" option for "match" and "pass" rules")
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60531
(cherry picked from commit b951994eddd4d274514df751da06680a32139037)
pf: Drain the overload task before a vnet's rules are freed
pf_src_connlimit() queues an entry for pf_overload_task() when a
source exceeds max-src-conn or max-src-conn-rate and the rule has an
overload table. The entry holds a pointer to the rule, and the task
runs from taskqueue_swi with the vnet of the queue. Nothing drained
the task when the vnet went away: an entry queued just before could
be applied to a rule pf_unload_vnet_purge() had freed, or to a vnet
that was gone.
Drain the task at the start of pf_unload_vnet_purge(), before the
unlinked rules are freed. No packet can queue another by then, as the
hooks are already removed.
Reviewed by: kp
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60529
(cherry picked from commit 92d66a6b474fa6fe42fbe08002803d79a5af6594)
rms: Order a writer's stores before its readers see it leave
Readers check rms->writers without a fence, and rms_wunlock() drops
it with a plain store. rms_wlock() interrupts every CPU before the
first writer blocks readers, but nothing orders them when the last
writer leaves: on a weakly ordered CPU a reader can see the count
reach zero and still read what the writer replaced, such as callbacks
of a module unloaded under the lock.
Fence every CPU before the last writer drops the count: rename
rms_wlock_switch() to rms_block_readers() and add
rms_unblock_readers(), a wrapper for cpus_fence_seq_cst(). Readers
stay fence-free; the cost lands on the rare unlock.
Reviewed by: mjg
Discussed with: kib
Fixes: 1f162fef76a0 ("Add read-mostly sleepable locks")
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
[3 lines not shown]
if_gre: Use OID_AUTO for the net.link.gre sysctl node
if_gre(4) registers net.link.gre with the static OID number IFT_TUNNEL.
That number is an interface type that several tunnel drivers share, not
one that belongs to gre(4). if_me(4) used it too, which made loading
both modules panic until 21f2245de219. Nothing addresses the node by
number, so use OID_AUTO as if_me(4) now does.
Suggested by: markj
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
(cherry picked from commit 80bc1d0e3ef732c6b58d13d0f223453a2410d8c1)
iflib: Do not ring the transmit doorbell when nothing is pending
For a lightly used ring iflib_txd_db_check() may defer zero descriptors,
so its "pending >= limit" test is true even when nothing has been queued
since the last doorbell. iflib_txq_drain() calls it before, inside and
after its loop, so a sender that drains its own packet wrote the tail
register three times per packet, twice with the value the hardware
already had.
The log of 81be655266fa ("iflib: ensure that tx interrupts enabled and
cleanups") calls skipping the doorbell when db_pending is zero "an
obvious missing optimization"; the comparison against a limit of zero
defeated it. vmx(4) and mgb(4) have dropped such repeated requests in
the driver since 2019. Return early when nothing is pending.
Reviewed by: gallatin
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60290
[2 lines not shown]
if_me: Do not reuse gre(4)'s static sysctl OID number
if_me(4) registers its net.link.me node with the static OID number
IFT_TUNNEL, the same number if_gre(4) has used for net.link.gre since
2003, so the two tunnel drivers collide under net.link. Until
d35c4cfad580 that only printed a warning and left two nodes with the
same number; since then sysctl_register_oid() panics, so loading if_me
after if_gre (or the other way round) takes the box down:
kldload if_gre
kldload if_me
panic: sysctl: OID number(131) is already in use for 'me'
Use OID_AUTO like the other tunnel drivers. Nothing addresses the node
by number.
Reviewed by: ae
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC ("Netgate")
[3 lines not shown]
