Revert "vm_object: Fix synchronization with teardown in the object list loop"
This was pushed by accident.
This reverts commit 4e6e4fa8be9940e711b791073eee5bcfa77ec5d4.
vm_object: Fix synchronization with teardown in the object list loop
Checking for OBJT_DEAD is racy since pagers update the object type
without the object lock held. Check for the flag as well, as that is
set beforehand. Otherwise pager dealloc can race with, e.g.,
cdev_pager_get_path().
Try to minimize accesses of the object after dropping the object lock in
vm_object_list_handler().
gve: Fix return value in gve_tx_has_desc_room_dqo()
When the cached tx->dqo.desc_head does not show enough available
descriptor ring slots, gve_tx_has_desc_room_dqo() reloads desc_head from
tx->dqo.hw_tx_head and re-checks availability. If the refreshed
desc_head provides sufficient space, the function mistakenly returns 0
(false) instead of true, causing callers to treat a successful
descriptor head refresh as ENOBUFS, stop the TX queue, and retry or
abort packet transmission (including unloading already-mapped DMA
segments) unnecessarily.
Return true when sufficient descriptor space is available after
refreshing desc_head.
In 60s iperf benchmarks between two FreeBSD 14.4 c4-standard-192
instances, this eliminates spurious tx->stopped transitions across all
workloads:
- TCP 1-stream: 54,859 -> 13 (-99.98%)
- TCP 16-stream: 9,623,851 -> 6,324,299 (-34.3%, -3.30M spurious stops)
[9 lines not shown]
gve: Fix DQO TX free-list memory ordering and pending_pkt state
The DQO TX path uses lockless producer/consumer free lists
(free_pending_pkts_prd/csm and free_qpl_bufs_prd/csm) shared between
the completion and transmit taskqueues. On weakly ordered architectures
such as arm64, the current synchronization has three issues:
1. gve_free_pending_packet() updates pending_pkt state and links
pending_pkt->next before pushing the entry onto
free_pending_pkts_prd using a relaxed atomic_cmpset_32(), while
gve_alloc_pending_packet() steals free_pending_pkts_prd using a
relaxed atomic_swap_32(). Without release/acquire ordering, the
transmit thread can observe a stale pending_pkt->next pointer after
stealing the producer list (either orphaning freed entries or
re-allocating entries whose completion tags are still in flight), or
prior completion stores (such as setting state to FREE or clearing
mbuf) can become visible after the slot is re-allocated.
2. Similarly, in QPL mode, gve_tx_alloc_qpl_buf() steals
free_qpl_bufs_prd with a relaxed atomic_swap_32() without an acquire
[53 lines not shown]
route/fib_algo: Fix nexthop reference leak on FLM_BATCH
handle_rtable_change_cb() references the new nexthop before calling
the algorithm. If the algorithm asks to batch the change by
returning FLM_BATCH, queue_rtable_change() references the same
nexthop again for the queued entry, while apply_rtable_changes()
only unreferences the old nexthop of each entry.
Reference new nhop once in handle_rtable_change_cb(), before
deciding whether the change is queued or passed to the algorithm,
and do not reference it again in fill_change_entry(). This also
leaves the queue reallocation as the only way for
queue_rtable_change() to fail.
Currently, dxr is the only algorithm returning FLM_BATCH.
Fixes: 6b8ef0d428c9 ("Add batched update support for the fib algo.")
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D61577
pass(4): Don't drop the periph lock before selrecord() in passpoll()
passpoll() checked the done queue while holding the peripheral lock,
dropped the lock, and only then called selrecord(). A CCB completing
in the window between the unlock and the selrecord() call sent its
selwakeuppri() before the polling thread was on the selinfo list, so
the wakeup was lost and select(2) or poll(2) slept until another
completion arrived or the timeout expired.
Call selrecord() before dropping the lock. This lock order is
already established: passdone() calls selwakeuppri(), which takes
the selinfo lock, while holding the peripheral lock.
kqueue is not affected, since the knlist is backed by the peripheral
mutex.
Reported by: llm-postcr
Fixes: a9934668aa13
MFC after: 1 week
[4 lines not shown]
pass(4): Wake select/kqueue waiters when an immediate CCB completes
In the CAMIOQUEUE ioctl, a queued CCB completes through passdone(),
which moves it to the done queue and calls selwakeuppri() and
KNOTE_LOCKED() so that a consumer blocked in select(2), poll(2) or
kevent(2) wakes up and collects the CCB with CAMIOGET. An immediate
CCB (e.g. XPT_DEV_ADVINFO or XPT_DEV_MATCH) is instead executed
synchronously in the ioctl and moved to the done queue directly,
with no notification, so a consumer sleeping while waiting for its
completion never wakes up until some other queued I/O completes.
Send the same notification that passdone() sends. The peripheral
lock is held at this point and backs the knlist, so KNOTE_LOCKED()
is the correct variant.
A version of this fix has been in production use at Spectra Logic
since 2024.
Co-authored-by: Dave Baukus <daveb at spectralogic.com>
[7 lines not shown]
pass(4): Don't drop the periph lock before selrecord() in passpoll()
passpoll() checked the done queue while holding the peripheral lock,
dropped the lock, and only then called selrecord(). A CCB completing
in the window between the unlock and the selrecord() call sent its
selwakeuppri() before the polling thread was on the selinfo list, so
the wakeup was lost and select(2) or poll(2) slept until another
completion arrived or the timeout expired.
Call selrecord() before dropping the lock. This lock order is
already established: passdone() calls selwakeuppri(), which takes
the selinfo lock, while holding the peripheral lock.
kqueue is not affected, since the knlist is backed by the peripheral
mutex.
Reported by: llm-postcr
Fixes: a9934668aa13
MFC after: 1 week
[4 lines not shown]
pass(4): Wake select/kqueue waiters when an immediate CCB completes
In the CAMIOQUEUE ioctl, a queued CCB completes through passdone(),
which moves it to the done queue and calls selwakeuppri() and
KNOTE_LOCKED() so that a consumer blocked in select(2), poll(2) or
kevent(2) wakes up and collects the CCB with CAMIOGET. An immediate
CCB (e.g. XPT_DEV_ADVINFO or XPT_DEV_MATCH) is instead executed
synchronously in the ioctl and moved to the done queue directly,
with no notification, so a consumer sleeping while waiting for its
completion never wakes up until some other queued I/O completes.
Send the same notification that passdone() sends. The peripheral
lock is held at this point and backs the knlist, so KNOTE_LOCKED()
is the correct variant.
A version of this fix has been in production use at Spectra Logic
since 2024.
Co-authored-by: Dave Baukus <daveb at spectralogic.com>
[7 lines not shown]
sysrc: Add .= operator for STRAPPEND
There are almost 100 *_flags variables in /etc/defaults/rc.conf and
each holds a set of options processed by getopts in one way or
another. The temptation to manage these `*_flags` variables with the
list operator += leads to potentially unexpected behavior.
For example, let's say you have in rc.conf:
cron_flags="$cron_flags -j 15"
Executing `sysrc cron_flags+="-J 15"` to set rootjitter to 15 will
instead result in:
cron_flags="$cron_flags -j 15 -J"
That's because += is not a string append operation but a list add
operation that adds -J but, seeing that 15 is already in the space-
delimited "list," does not add "15".
[30 lines not shown]
sysrc: Add -_*~$ as non-delimiter list operation chars
This is potentially a breaking change (MFC not recommended).
List operators += and -= should not consider some characters as
delimiters when they appear as the first character.
The += and -= operators allow specifying the list delimter as the
first character. For example, `sysrc foo+=,123` will iterate over the
value of foo delimited by commas, and if 123 is not found will append
`,123` to the value of foo.
However, for the sake of convenience, sysrc has always supported
defaulting to using $IFS as delimiter when the first character of the
+= or -= list operator is [a-zA-Z0-9./]. For example, if you execute
`sysrc foo+=abc` or `sysrc bar+=/tmp` or `sysrc baz-=.text` then the
lists foo, bar, and baz are all presumed to be delimited by $IFS.
By adding [-_*~$] we potentially break lists delimited by any of
[14 lines not shown]
sysrc: Disable filename globbing on list assignment
This is potentially a breaking change (MFC not recommended).
Assignments with += or -= should not expand filename globs.
For example, executing `sysrc foo+=' *'` would result in adding to
foo the glob expansion of `*` instead of a literal `*`.
This was unexpected and inconsistent because `sysrc foo='*' would set
foo to literal `*`.
If you need the old behavior, you can do `sysrc foo+="$( echo * )"`.
Reviewed by: kfv
Differential Revision: https://reviews.freebsd.org/D61657
sysrc: Enforce raw (-r) mode for all assignments
This is potentially a breaking change, warranting major version bump
(7.x -> 8.0).
Assignments with =, +=, or -= should not cause expansion of variables.
For example, if rc.conf(5) has one="$1", two="$2", and foo="$one", and
you execute `sysrc foo+=' $two'` you should get foo="$one $two" but
at-present (version 7.x) sysrc stores foo="1 $two". This is because
when querying current value, we read the expanded (read: not raw)
value.
Similary, if rc.conf has foo="$one $two" and you execute the command
`sysrc foo-=' $two'` you should get foo="$one" but at-present sysrc
erroneously stores foo="1 2". Since sysrc was reading the expanded
value prior to modifying, it would read "1 2" prior to attempting to
elide '$two' from the [space-separated] list (and since '$two' does
not appear in the expanded value, no removal is performed and it
subsequently wrote back the expanded value).
[29 lines not shown]
sysrc: Add -r raw and fixup -c assignment check
Add new -r flag to query raw, unexpanded values and display/use raw
values during assignments (including +=/-= assignments).
Constraint -c assignment checks to always use raw values when testing
if an assignment would induce a change or if the existing value is
already set as desired.
PR: 299413
MFC after: 2 weeks
Reviewed by: kfv
Differential Revision: https://reviews.freebsd.org/D61584
dummynet: validate object length before fixed-size copies in do_config()
do_config() checks that each command's o.len fits in the remaining
buffer, but the DN_LINK, DN_PROFILE, DN_SCH and DN_FS cases then copy a
fixed sizeof(struct) out of the command, which can be larger than o.len.
A command with o.len == sizeof(struct dn_id) reads past the end of the
sockopt buffer (DN_PROFILE copies about 4 KB).
Reject a command whose o.len is shorter than the structure it carries.
Add a test.
Reviewed by: kp
MFC after: 1 week
Sponsored by: Advanced Micro Devices, Inc.
Signed-off-by: Osvaldo Janeri Filho <ojanerif at amd.com>
Differential Revision: https://reviews.freebsd.org/D61440
pf tests: Skip unmapped_mbuf:dummynet without dummynet
The dummynet test case in unmapped_mbuf.sh calls pft_init but never
checks whether dummynet(4) is loaded. When it is not, dnctl fails with
"setsockopt(IP_DUMMYNET3): Protocol not available" and the test is
reported as failed instead of skipped.
Call dummynet_init instead. It runs pft_init and skips the test when
dummynet(4) is not loaded, as the other dummynet test cases in
tests/sys/netpfil/pf already do.
Fixes: 8a14fcd23a20 ("pf: fix a crash on sendfile()")
Signed-off-by: Jaswanth Saravanan <jaswanth.s006 at gmail.com>
Reviewed-by: kp
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2556
pf: reject table anchors with empty path components
pfr_fix_anchor() strips leading slashes from table anchors but not
trailing or repeated ones. pf_find_or_create_kruleset() ignores those,
so a table added as "foo/bar/" lands in ruleset "foo/bar" but keeps
"foo/bar/" as its anchor. pfr_table_count() and pfr_skip_table() then
disagree, and with INVARIANTS:
```
panic: pfr_get_tables: corruption detected (1)
```
"foo//bar" does the same once "foo" exists, and pfctl -vsT hits the
same check in pfr_get_tstats().
Loading rules into such an anchor already fails with EINVAL, since
pf_find_kruleset() doesn't find it. Reject it for tables as well.
Reviewed by: kp
[4 lines not shown]
pf: implement RFC 7112 Section 5
RFC 7112 Section 5 and RFC 8200 Section 4.5 require IPv6 fragmented
packets to contain all headers in the first fragment. If this is not the
case we now discard the packet.
We do not send the ICMPv6 Parameter Problem error RFC 8200 requires (but
7112 lists this as "MAY").
This makes bypassing the firewall, in cases where fragment reassembly is
disabled, harder.
RelNotes: yes
Sponsored by: Rubicon Communications, LLC ("Netgate")
pf: improve anchor string validation
Ensure that these are not overly long.
Add a test case.
Reported by: Kevin Day <kevin at your.org>
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC ("Netgate")
opencrypto: Do not requeue a re-dispatched request as ordered
crypto_dispatch_async() marks an ordered request with
CRYPTO_F_ASYNC_ORDERED and gives it a sequence number. The return
thread runs the callbacks in sequence and only takes the head of its
ordered queue when the head carries the next number.
When a driver unregisters, the request completes with EAGAIN and the
ESP and AH callbacks dispatch it again with crypto_dispatch(). The
flag is still set, so crypto_done() puts the request back on the
ordered queue under the number the return thread has already passed.
The head of the queue never matches again, and no ordered request of
any session on that return worker completes after that.
Before crypto_dispatch_async() existed, crypto_dispatch() gave such a
request a new sequence number. Clear the flag when the return thread
takes a request off the ordered queue, so a request that is dispatched
again completes like any other.
[5 lines not shown]
ichsmb: Enable block buffer mode for Elkhart Lake
Elkhart Lake SMBus controllers support the 32-byte block buffer used by
SMBus block process calls. Set ICHSMB_FEATURE_BLOCK_BUFFER for the
Elkhart Lake PCI ID so ichsmb_bpcall() is not rejected with
SMB_ENOTSUPP.
Reviewed by: corvink
MFC after: 1 week
Sponsored by: Beckhoff Automation GmbH & Co. KG
Pull Request: https://github.com/freebsd/freebsd-src/pull/2397
Signed-off-by: YannickV <Y.Vossen at beckhoff.com>
tsearch.3: Clarify node vs. key and improve markup
Pull in clarifications from Apple's libc: describe the comparison
function's arguments as key pointers, refer to the nodes returned by
tfind() and tsearch() rather than to the "datum", explain how to get a
node's key, note that the action function is called only once for leaf
nodes, and note about how VISIT names don't map to the usual traversal
order nomenclature.
The paragraph about the first member of a node being a pointer to its
key was rewritten, as Apple's wording was not strictly accurate.
Apple's Libc also has a paragraph describing what each traversal order
does, but it is mostly redundant, so it is not included here.
While here, mark up NULL and the VISIT values with .Dv, move the VISIT
typedef into a literal block, and fix mandoc lint warnings.
MFC after: 3 days
[4 lines not shown]
tsearch.3: Fix tdestroy(3) entries in NAME and SYNOPSIS
Add tdestroy to the NAME section, remove a stray opening parenthesis
from its node_free argument in the SYNOPSIS, and small grammar fix in
STANDARDS.
Fixes: b0d4b059cf84 ("tdestroy(3) man page")
MFC after: 3 days
Sponsored by: Klara, Inc.
mpr(4)/mps(4): Fix build on 32-bit platforms
bus_addr_t is 32-bit on armv7, so >> 32 triggers -Wshift-count-overflow.
Widen it to uint64_t before shifting, as the other high-address writes
in the same files.
Fixes: 866760ff8338 ("mps(4): add support for hostboot")
Sponsored by: The FreeBSD Foundation
sound: Hold the topology lock across the remaining registration checks
sysctl_dev_pcm_bitperfect(), sysctl_dev_pcm_eq() and
sysctl_dev_pcm_eq_preamp() check PCM_REGISTERED() and then busy the
device with nothing held in between, so pcm_unregister() can run and
destroy the sx before PCM_ACQUIRE() takes it. These sysctls are
registered in the device's newbus context, so they outlive
pcm_unregister().
Hold the topology lock across the check and the busy, as the other
handlers do.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D61224
sound: Fix a LOR between AIOGCAP and SNDCTL_DSP_SYNCGROUP
AIOGCAP locks a dsp_cdevpriv's record channel and then its play channel,
while dsp_oss_syncgroup() locked them the other way round. Verified the
LOR and deadlock by exercising both paths using a duplex FD
concurrently.
Lock the record channel first in dsp_oss_syncgroup() too, and document
the order in sound.h.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D60519
sound: Busy the device in mixer_oss_mixerinfo()
Rather than hold the topology lock across the whole handler, which
blocks detach of every device in the system, busy the one being selected
and used, and drop the topology lock, as the other SNDCTL_*INFO handlers
do.
The mutex now covers only the mixer fields. mixer->cdev is set once in
mixer_init() and never cleared, so reading does not need the lock.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D60518
sound: Retire PCM_LOCK in sound_oss_card_info()
The handler reads the device name, its description and the status
string. All three are set before pcm_register() makes the device
visible, and never change afterwards. The topology lock, which is held
across the whole function, makes sure the device is not freed.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D60515