ena: Batch RX statistics updates
Move per-packet counter_enter/counter_exit pairs out of the RX
processing loop and batch them into a single update after the
loop completes.
Previously, each received packet triggered two separate
counter_enter/counter_exit blocks -- one for bytes and one for
packet count. This commit accumulates totals in local variables
and updates all four counters (ring and hw stats for both packets
and bytes) in a single counter_enter/counter_exit block after the
loop.
Also move the stats update to after the refill and LRO flush
so that the error path (goto update_stats) and the normal path
converge at the same label, avoiding code duplication.
Submitted by: David Arinzon <darinzon at amazon.com>
MFC after: 2 weeks
[5 lines not shown]
ena: Update driver version to v2.8.4
Bug Fixes:
* Fix false 'missing TX completions' warnings due to timestamp race
* Put taskqueues into correct NUMA domain if !RSS
Minor Changes:
* Batch RX statistics updates
* Swap RX/TX completions cleanup order
Submitted by: Arthur Kiyanovski <akiyano at amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58242
(cherry picked from commit 605e699cd6ca4feae6c73c5c5ea8337054897116)
ena: Fix false 'missing TX completions' warnings due to timestamp race
Sporadic 'Found a Tx that wasn't completed on time' warnings appear
under sustained TX load, always reporting '1 msecs since last cleanup'
despite the 5-second timeout threshold.
The per-packet TX timestamp uses struct bintime (128 bits: two 64-bit
fields sec and frac) which is read and written non-atomically. A race
exists between the missing TX completion check
(check_missing_comp_in_tx_queue reading the timestamp) and the TX
submit path or cleanup path writing it on another CPU. Since the two
fields are not updated atomically, the check can observe a partially
written timestamp - one field from the old value and one from the new.
This can produce a timestamp with {sec=0, frac=valid}, causing the
check to compute a time offset equal to system uptime and falsely
exceeding the 5-second timeout.
Confirmed by instrumentation showing all occurrences had sec=0 with
valid frac/mbuf, cleanup_running=0, and ticks==last_cleanup_ticks.
[24 lines not shown]
ena: Swap cleanup order
As RX processing is heavier than TX completions processing, swap the
order and process TX completions first, in order to avoid starving the
completions and causing potential missing TX completions.
Submitted by: Ofir Tabachnik <ofirt at amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58239
(cherry picked from commit f08def9ed97f45700eb0611a3fd9240210c9303e)
nuageinit: Allow the userdata script to run before firstboot* rc services
Allowing nuageinit user scripts to run before these makes it possible to
customize official BASIC-CI and BASIC-CLOUDINIT FreeBSD images.
This was requested by KDE for their CI.
Approved by: cperciva
Pull-Request: https://ron-dev.freebsd.org/FreeBSD/src/pulls/60
tests/sys/capsicum: adjust tests for the new reaping behavior
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D58569
tests/sys/kern: adjust tests for the new reaping behavior
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D58530
processes: add zombie references, each of them prevents reap
Add the p_zombieref bitmask into struct proc, which enumerates all
legitimate waiters on the process exit status. Among them are parent
for PZOMBIEREF_PARENT, and the holder of the process descriptor for
PZOMBIEREF_PROCDESC, if the process was created by pdfork().
Require all zombie refs to be cleared to reap zombie. This prevents
stealing the exit status from the parent by pdwait()ing on a procdesc
obtained by pdopenpid(), or by waitpid() by debugger from the real
parent.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D58264
pdwait(2): change handling of the exited processes
Instead of accessing the struct proc and gathering data from it,
memoize the data needed for pdwait() on exited process in struct
procdesc, at the time of process termination.
This allows unlimited number of calls to pdwait(2) on procdesc for
terminated process.
Change the locking requirements for pd_flags to proctree_lock. This does
not modify the pre-patch locking regime, but the change requires it.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D58407
vfs_mountroot: unmute console in interactive prompt
If boot_mute is set the system appears to hang during the mountroot
prompt. Temporarily unmute the console so the prompt is visible.
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D58549
ixl: enforce the assigned VF MAC address
When allow-set-mac is disabled, the MAC filter validation condition
rejects the assigned VF unicast address while allowing any different
unicast address. The equality test was accidentally inverted when this
code moved to the boolean address helper.
Accept multicast and the assigned unicast address, and reject other
unicast addresses as intended.
Fixes: 7d4dceec1030 ("ixl(4): Fix VLAN HW filtering")
MFC after: 3 days
stdio: *memstream: grow the buffer by 1.5x on write
This improves performance by reducing the number of allocations as we
write into the memstream, both in the fully buffered case with larger
memstreams and also more trivially in the line- and un-buffered case as
they flush back to the underlying buffer more often.
The inspiration for this was taken from Apple's implementation in
https://github.com/apple-oss-distributions/libc, but expanded to include
wmemstream for consistency. I've added a test for the bug that I hit in
libder that caused me to notice this in the first place, and fixed that
bug in this version.
Reviewed by: des, jhb (both slightly previous version)
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D57355
stdio: *memstream: decouple the buffer size from the stream length
It's useful to be able to track both facts with a single variable, but
it also makes it more difficult to change how the buffer size scales.
As an example, Apple's implementation seems to scale the buffer size by
1.5x on growth, presumably in an attempt to reduce trips into realloc().
This might be questionable in the face of stdio buffering, but avoiding
serious churn in the line- or un-buffered case is a net positive if
doing so isn't incredibly invasive.
Reviewed by: des, jhb, obiwac
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D57354
stdio: *memstream: slightly streamline growth function
Inverting the condition after realloc*() is a minor cleanup, but makes
the success path a little cleaner to ease a future change.
Reviewed by: des, jhb
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D57353
e1000: fix rx accounting for multi-descriptor packets
The receive paths accumulate ri->iri_len across the descriptors making
up a packet, then add that running total to rxr->rx_bytes on every
iteration of the loop. A packet spanning descriptors of length l1, l2
and l3 thus contributes 3*l1 + 2*l2 + l3 instead of l1 + l2 + l3.
Single descriptor packets, the common case, are accounted correctly,
so this only shows up on jumbo frames.
Add the per descriptor length instead. iflib memsets the if_rxd_info
before each isc_rxd_pkt_get() call, so summing len gives the same total
as the final iri_len, and the frame error path that returns without
incrementing rx_packets keeps counting bytes exactly as before.
(cherry picked from commit 41a46c2d46aa4078c597ce3a0d19323cab988277)
e1000: Defer link-up notification until after TSO reset
em_automask_tso() changes the enabled TSO capabilities when the link
moves between 10/100 and 1000 Mb/s. A running interface must be
reinitialized to apply the new capability set. Do not publish
LINK_STATE_UP until the requested iflib reset has completed.
Replace link_active with an explicit state machine that distinguishes
the physical link, its publication to iflib, and an outstanding reset
barrier. Preserve that barrier across a link flap with
DOWN_RESET_PENDING, and only publish DOWN if UP was previously
published.
Only request a reset for a running interface or for an initialization
while the interface is administratively up. In other states the next
initialization will apply the capability changes, avoiding a reset
request that iflib's admin task could discard.
Reviewed by: Faraz Vahedi <kfv at kfv.io>
[3 lines not shown]
e1000: fix rx accounting for multi-descriptor packets
The receive paths accumulate ri->iri_len across the descriptors making
up a packet, then add that running total to rxr->rx_bytes on every
iteration of the loop. A packet spanning descriptors of length l1, l2
and l3 thus contributes 3*l1 + 2*l2 + l3 instead of l1 + l2 + l3.
Single descriptor packets, the common case, are accounted correctly,
so this only shows up on jumbo frames.
Add the per descriptor length instead. iflib memsets the if_rxd_info
before each isc_rxd_pkt_get() call, so summing len gives the same total
as the final iri_len, and the frame error path that returns without
incrementing rx_packets keeps counting bytes exactly as before.
(cherry picked from commit 41a46c2d46aa4078c597ce3a0d19323cab988277)
e1000: Defer link-up notification until after TSO reset
em_automask_tso() changes the enabled TSO capabilities when the link
moves between 10/100 and 1000 Mb/s. A running interface must be
reinitialized to apply the new capability set. Do not publish
LINK_STATE_UP until the requested iflib reset has completed.
Replace link_active with an explicit state machine that distinguishes
the physical link, its publication to iflib, and an outstanding reset
barrier. Preserve that barrier across a link flap with
DOWN_RESET_PENDING, and only publish DOWN if UP was previously
published.
Only request a reset for a running interface or for an initialization
while the interface is administratively up. In other states the next
initialization will apply the capability changes, avoiding a reset
request that iflib's admin task could discard.
Reviewed by: Faraz Vahedi <kfv at kfv.io>
[3 lines not shown]
igc: Apply ASPM L1.2 workaround to all I226 devices
Classify I226_LMVP and I226_BLANK_NVM as I226 silicon so they
receive the I226-specific ASPM L1.2 workaround.
PR: 279245
MFC after: 1 week
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2318
nvmecontrol.8: Explain non-operational power modes
`nvmecontrol power -l ...` lists the available power modes.
Non-operational modes are marked with an asterisk. While here,
add <device-id | namespace-id> to the "nvmecontrol power" synopsis.
MFC after: 3 days
Reviewed by: dab, imp, michaelo, ziaee
Differential Revision: https://reviews.freebsd.org/D58480
acpi: parse resources of not-present devices that are kept enabled
acpi_probe_child() keeps PCI link devices, the RTC,
and docking stations enabled even when _STA reports them not present,
but skipped acpi_parse_resources() for them. With an empty resource
list, resource-based hint matching (BUS_HINT_DEVICE_UNIT) cannot wire
such a device to its hinted unit, and the hinted ISA device is then
created as a duplicate.
Modern AMI firmware reports the PNP0B00 RTC as not present while
handing timekeeping to the ACPI Time-and-Alarm device.
Reviewed by: adrian, jhb
Differential Revision: https://reviews.freebsd.org/D58047
ptrace: Propagate errors from set_fpregs()
Otherwise ptrace(PT_SETREGSET) will not return errors to userspace.
Fixes: cef05c5a62ba ("amd64: do not allow to set reserved bits in MXCSR for ptrace(PT_SETFPREGS)")
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D58577
uma: Fix KMSAN integration with malloc zones
In commit 459aa032e872 I dropped kmsan_mark() calls from malloc() on the
basis that UMA and kmem_malloc() would handle updates of the KMSAN
shadow map. However, I missed that UMA explicitly does not handle this.
Modify UMA to only omit origin map updates for malloc zones.
Fixes: 459aa032e872 ("malloc: Refactor redzone and sanitizer handling")
Reviewed by: rlibby
Differential Revision: https://reviews.freebsd.org/D58574
rc.d/bthidd: Correct load_kld invocations
Pass a single module name to load_kld for kbdmux and vkbd, allowing
bthidd_prestart to load both modules successfully.
Fixes: cfe1962a1925 (rc: Fix improper use of load_kld)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
linuxkpi: Add xa_insert_irq wrapper
This just invokes xa_insert similar to other xa_*_irq wrappers.
Reviewed by: bz
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D58576
kld: Reject kernel modules with PT_LOAD segments where filesz > memsz
All sorts of places in the ELF loading code assume that filesz <=
memsz, so check that explicitly up front.
Reported by: Jane Smith <thebugfixers at pm.me> (via D57785)
Reviewed by: jrtc27, kib
Differential Revision: https://reviews.freebsd.org/D58542
