virtio_random: defer first enqueue until after DRIVER_OK
According to the virtio spec:
> The driver MUST NOT send any buffer available notifications to the
> device before setting DRIVER_OK.
DRIVER_OK isn't set until after attach completes, so defer the first
enqueue. We haven't found any problems as a result of this to the best
of my knowledge: q35 and other devices that we *have* been used with
will apparently check when we go DRIVER_OK.
Tested with q35 virtio-random, which still works.
Reported by: llm-postcr
Reviewed by: kfv, markj
Fixes: f1c5a2e3a62505 ("virtio_random: Pipeline fetching the data")
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
[2 lines not shown]
virtio_random: stop on detach, don't poll
If the device is hung, then polling will block the detach and hang us.
Stop it and drain the queue instead. This fixes a hang on detach in
some circumstances without the upcoming virtio_pci_modern fix.
Reviewed by: bnovkov, kfv
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
(cherry picked from commit cce522853aae3da45b131c2522dd2510edf083e7)
virtio_pci_modern: enable virtqueues before completing attach
Most of our drivers setup interrupt(s) and virtqueues are enabled at
that time, but nothing will currently enable them for devices that we're
only going to poll. This most notably affects virtio-random, and the
problem can be observed with qemu's q35 emulation and a random device
attached. The q35 implementation ignores the ring unless it's enabled,
so vtrnd_harvest() will consistently EAGAIN rather than contribute
anything of value, but with this patch it becomes functional.
This can be reproduced with a qemu invocation like so:
qemu-system-x86_64 -machine q35,accel=tcg -cpu max -smp 2 -m 2G \
-drive file=FreeBSD-16.0-CURRENT-amd64-ufs.qcow2,if=virtio \
-device pcie-root-port,id=rp1,bus=pcie.0,chassis=1 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0,bus=rp1 \
-display curses
[5 lines not shown]
virtio_random: avoid using vtrnd_detach in the failure path
On the off chance that we discover more than one virtio-random device,
the second one will deregister the global instance or trip an invariant
when it fails to attach and calls vtrnd_detach().
We don't have that many resources to free, so just inline those. This
is purely a hygiene thing, we haven't seen an instance of this in the
wild (to my knowledge).
Reviewed by: bnovkov, kfv
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
(cherry picked from commit 26ba71f73e8914ecd9e5b1e7bb864fee41158d04)
virtio_random: defer first enqueue until after DRIVER_OK
According to the virtio spec:
> The driver MUST NOT send any buffer available notifications to the
> device before setting DRIVER_OK.
DRIVER_OK isn't set until after attach completes, so defer the first
enqueue. We haven't found any problems as a result of this to the best
of my knowledge: q35 and other devices that we *have* been used with
will apparently check when we go DRIVER_OK.
Tested with q35 virtio-random, which still works.
Reported by: llm-postcr
Reviewed by: kfv, markj
Fixes: f1c5a2e3a62505 ("virtio_random: Pipeline fetching the data")
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
[2 lines not shown]
virtio_pci_modern: enable virtqueues before completing attach
Most of our drivers setup interrupt(s) and virtqueues are enabled at
that time, but nothing will currently enable them for devices that we're
only going to poll. This most notably affects virtio-random, and the
problem can be observed with qemu's q35 emulation and a random device
attached. The q35 implementation ignores the ring unless it's enabled,
so vtrnd_harvest() will consistently EAGAIN rather than contribute
anything of value, but with this patch it becomes functional.
This can be reproduced with a qemu invocation like so:
qemu-system-x86_64 -machine q35,accel=tcg -cpu max -smp 2 -m 2G \
-drive file=FreeBSD-16.0-CURRENT-amd64-ufs.qcow2,if=virtio \
-device pcie-root-port,id=rp1,bus=pcie.0,chassis=1 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0,bus=rp1 \
-display curses
[5 lines not shown]
virtio_random: stop on detach, don't poll
If the device is hung, then polling will block the detach and hang us.
Stop it and drain the queue instead. This fixes a hang on detach in
some circumstances without the upcoming virtio_pci_modern fix.
Reviewed by: bnovkov, kfv
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
(cherry picked from commit cce522853aae3da45b131c2522dd2510edf083e7)
virtio_random: avoid using vtrnd_detach in the failure path
On the off chance that we discover more than one virtio-random device,
the second one will deregister the global instance or trip an invariant
when it fails to attach and calls vtrnd_detach().
We don't have that many resources to free, so just inline those. This
is purely a hygiene thing, we haven't seen an instance of this in the
wild (to my knowledge).
Reviewed by: bnovkov, kfv
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
(cherry picked from commit 26ba71f73e8914ecd9e5b1e7bb864fee41158d04)
linux: Bound native getdents and getdents64 batches
linux_getdents() and linux_getdents64() round their native read size up
to DEV_BSIZE before converting entries. For an unaligned caller buffer,
the native batch can contain more entries than fit in the Linux buffer.
Conversion then returns EINVAL even after copying complete entries,
and the native read has already advanced the directory position.
For counts of at least DEV_BSIZE, use min(args->count, MAXBSIZE) instead.
A 768-byte request, such as rustix's initial directory read, now uses a
768-byte native buffer rather than 1024. The converted Linux dirent and
dirent64 records are no larger than their native counterparts, so the
whole translated batch fits. Returning a short batch is allowed, and
consuming the whole native batch avoids needing to rewind the directory
position.
Keep the existing sizing for counts below DEV_BSIZE. This fixes the
oversized-batch case in both getdents interfaces, including legacy
getdents for 32-bit and 64-bit callers. The remaining small-buffer,
[8 lines not shown]
cryptodev: zero key material before freeing it
The /dev/crypto cipher and MAC keys are copied from userland and
released with free(), leaving the key bytes in freed kernel memory.
Use zfree() so the key material is cleared on session teardown and on
the setup error path.
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D61324
powerpc/idle: Set 'RL' field in PSSCR on POWER9
The POWER9 does not do automatic state transitions when idling, so the
'MTL' field (Maximum Transition Level) is ignored. Thus, setting '0'
for the 'RL' field (Requested Level) as was done means it
doesn't transition to any power saving level. To fix this, set both the
'MTL' and the 'RL' fields to the same level.
On a Talos II at idle this drops the reported temperatures by ~10-15%.
MFC after: 3 days
libdwarf: standard DWARF 5 form names, DW_LANG_C17 and C23
7bb441c86678 added the fixed-size string and address index forms
0x25 to 0x2c as DW_FORM_ref_strx1..4 and DW_FORM_ref_addrx1..4; the
standard (DWARF 5, table 7.6) names them DW_FORM_strx1..4 and
DW_FORM_addrx1..4, with no "ref". Rename them, keep the old spellings
as aliases since the header is installed, and update dwarf_dump.c's
form names accordingly. Add DW_LANG_C17 (0x2c) and DW_LANG_C23 (0x3e),
which clang emits for -std=c17 and -std=c23 units. ctfconv(1) needs
all of these; it carried them in a private header until now.
Reviewed by: markj, emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D61154
openssl: stop generating asm that nothing builds
Makefile.asm generated eight perlasm outputs that neither libcrypto nor
the kernel builds. Additionally, upstream does not build them for these
architectures with FreeBSD's configuration:
- aarch64: ecp_sm2p256-armv8, sm3-armv8, sm4-armv8, vpsm4-armv8 and
vpsm4_ex-armv8. SM2, SM3 and SM4 are disabled (disable-sm2/sm3/sm4).
- i386: cast-586 (upstream's x86 configuration uses the C CAST code)
and rc5-586 (RC5 is disabled by default).
- powerpc: aes-gcm-ppc. Upstream builds it only for 64-bit
little-endian; libcrypto and ossl(4) use it only on powerpc64 and
powerpc64le, whose copies stay.
Remove them from Makefile.asm and delete the generated files, so the
next vendor import does not regenerate them. If one of these algorithms
is ever enabled, add its script back to Makefile.asm and run "make
buildasm".
[10 lines not shown]
libalias: index fully specified inbound links by remote endpoint
Inbound lookups find the (alias address, alias port, link type) group
with a splay tree and then walk grp->full, a list of every fully
specified link in that group, comparing the remote address and port.
With redirect_addr in front of a busy server, every client connection
to public:443 lands in the same group, so each inbound packet that is
not near the head of the list walks all of it. TCP links live up to
24 hours unless libalias sees a clean close, so the list can grow to
hundreds of thousands of entries and saturate a core at a few hundred
packets per second.
Keep the fully specified links of a group in an RB tree ordered by
(dst_addr, dst_port). Links that share an endpoint are ordered newest
first by a per-instance insertion counter, which keeps the "most recent
link wins" behaviour of the list (tested by 3_natin:2_portoverlap).
Lookups with an unknown remote address and a known port still scan the
group. Each link grows by 16 bytes.
[10 lines not shown]
libalias: Handle GetNewPort() errors properly
AddLink() fails when memory allocation fails or no free port is
available; both are error conditions. However, functions such as
FindUdpTcpIn() were converting such failures to PKT_ALIAS_IGNORED, which
effectively means, "pass the packet without translation," which isn't
what we want.
Fix the problem by making sure that AddLink() errors are converted to
PKT_ALIAS_ERROR where appropriate. The diff is a bit large but is
mostly mechanical: functions like TcpAliasOut() are converted to return
a result code, and an additional out-parameter is added to return the
alias_link pointer.
Reported by: Yuxiang Yang <yangyx22 at mails.tsinghua.edu.cn>
Tested by: Yuxiang Yang <yangyx22 at mails.tsinghua.edu.cn>
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D47778
(cherry picked from commit fb2ea26f3c3681f5ef639af9c798a631d800864b)
libalias: Add support for EIM NAT
Add support for endpoint-independent mapping ("full cone NAT") in
Libalias's UDP NAT.
This conforms to RFC 4787 requirements 1 and 3. All UDP packets sent out from a
particular internal address:port leave via the same NAT address:port,
regardless of their destination.
Add some libalias tests and supporting defines.
Reviewed by: igoro, thj
Differential Revision: https://reviews.freebsd.org/D46689D
(cherry picked from commit 61bf830cbb260c2a046cb44421d319184393e028)
