ufshci: fix the WLUN peripheral lookup timeout
The timeout in ufshci_sim_find_periph() is uint64_t, making the
comparison of timeout - ticks with zero always false. If the requested
peripheral does not become available, the lookup can wait indefinitely.
Use an sbintime_t deadline based on sbinuptime() and mstosbt(), and
compare the current uptime directly with the deadline. Preserve the
successful lookup path and existing timeout cleanup.
On an HP 15-fd0118ca with Intel N150 and Samsung UFS 3.1 storage, a
kernel built from stable/15 revision 84aa3cb3204d with this change
logged the missing-WLUN timeout, attached da0 and mounted root in two
boot trials.
Signed-off-by: Denis Lemire <denis at lemire.name>
Reviewed by: jaeyoon
Fixes: 31407551c3d4 ("ufshci: Add a check for WLUN during driver initialization")
MFC after: 3 days
Pull Request: https://github.com/freebsd/freebsd-src/pull/2520
tcpsso: fix the level error messages
tcpsso incorrectly referred to the optname instead of
the level name/number.
MFC after: 2 weeks
(cherry picked from commit c7cf285bb2069ff5adc3ca63dbdaf3deb0fe3aeb)
Revert "git-arc: add initial non-FreeBSD and `$HOME` install support"
This was in review and should not have been committed.
This reverts commit 010e53f7c6908bee93d1bd32fb4b1658b0f19eef.
GoogleTest: run several tests as plain C++ tests
Some of the tests provided by GoogleTest don't play well with the kyua
GoogleTest engine available in 0.15.0, in part because they test
edgecases that the kyua GoogleTest engine does not expect or handle
gracefully.
Treat the tests as plain C++ tests to keep parity (result-wise) before
switching the testcase type from `plain` (what it is today under the
covers) to `googletest`.
MFC after: 1 month
MFC with: 3fb03fb35b20 ("[tests] switch GoogleTest tests to ...")
[tests] switch GoogleTest tests to `googletest` interface
The new `googletest` kyua test interface--added in 0.15.0 and fixed in
0.15.1--allows tests to be executed and handled in a more fine-grained
manner than the way it was prior to this commit.
This change deletes the `capsicum-test` ATF wrapper script,
`functional.sh`, and all supporting logic in `googletest.test.mk` as
well as this change makes that logic unnecessary.
MFC after: 1 month
MFC with: b68969979d6b ("contrib/kyua: update to 0.15.1")
git-arc: add initial non-FreeBSD and `$HOME` install support
Modify the existing code to permit setting a prefix via the `PREFIX`
make variable, similar to what `--prefix` provides in autotools-based
projects.
Add the required compatibility glue in order to function with vanilla
bmake, which enables installing the commands with Linux and macOS:
- vanilla bmake requires `scripts.mk` to install `SCRIPTS`.
- vanilla bmake does not tack `/man` on to the end of `MANDIR`, like
with FreeBSD.
Adjust the sysctl calls to instead look for `arc` in `$PATH` using
`command -v` instead of hardcoding the path to the `arc` binary. This
allows the command to invoke `phorge` as well as the legacy `arcanist`
tool.
While here, update the origin for `arcanist` as the former is the
non-phorge fork is no longer supported (nor needed).
[9 lines not shown]
style.Makefile(5): update recommendations
- Mention `CXXFLAGS` and `CXXSTD`, in addition to `CFLAGS` and `CSTD`.
- Mention `LIBADD`, in addition to the legacy (in-tree) variables,
`DPADD` and `LDADD`.
- Mention the other option types, e.g., `MK_FOO`, `WITHOUT_FOO`, etc,
added as part of `src.opts.mk` in the past couple releases.
Bump .Dd for the change.
MFC after: 1 week
if_bnxt: fix powerpc build of the MPC ring code
powerpc has no 16-bit acquire/release atomics, so the
atomic_load_acq_16() and atomic_store_rel_16() calls on the MPC TX
ring indices fail to build there. Make the producer and consumer
indices 32 bits wide and use the 32-bit primitives.
Reviewed by: gallatin, chandrakanth.patil_broadcom.com
Sponsored by: Broadcom Inc
Fixes: 088a42c0ff76 ("if_bnxt: add MPC (Mid-Path Channel) ring infrastructure")
Differential Revision: https://reviews.freebsd.org/D60698
(cherry picked from commit b43c4d84c8d172858fc4bbf59e07888bfb8ac472)
if_bnxt: port workqueue, task-queue, and synchronization kPIs to FreeBSD natives
- Replace Linux workqueue (struct workqueue_struct, bnxt_pf_wq) with a
FreeBSD taskqueue (bnxt_taskq) shared by every attached PF. bnxt_taskq
is created once at module load and destroyed once at module unload
(SYSINIT/SYSUNINIT on SI_SUB_KLD), matching the module-owned-taskqueue
idiom used elsewhere in the tree (e.g. sys/dev/cxgbe/t4_main.c's
reset_tq). This replaces an earlier lazy, per-attach/detach
create-or-reuse scheme guarded by a mutex and a "ready" flag, which
had a real race: bnxt_queue_sp_work()/bnxt_queue_fw_reset_work() and
bnxt_attach_pre() re-read the bnxt_taskq/bnxt_taskq_ready globals
outside the mutex right after the lazy-init call returned, so a
concurrent bnxt_detach() on another PF freeing the shared queue once
bnxt_num_pfs hit 0 could race a still-attached PF into enqueuing onto
a freed taskqueue. Tying the taskqueue's lifetime to the module
instead of to any one PF's attach/detach removes that race outright,
since the module can't unload while a PF is still attached.
- Replace struct work_struct sp_task with struct task sp_task;
[43 lines not shown]
if_bnxt: don't reserve RoCE MSI-x on chips that don't support RoCE
RoCE is only supported on Thor and onwards chips.
bnxt_init_sctx_variants() unconditionally set isc_admin_intrcnt to
BNXT_ROCE_IRQ_COUNT for every PF device, stealing 9 MSI-x vectors from
older, non-RoCE-capable chips that will never use them.
Gate the reservation on the device ID: RoCE-capable chips keep
BNXT_ROCE_IRQ_COUNT, other chips fall back to 1, the baseline
isc_admin_intrcnt value used before RoCE support (commit 050d28e13cde)
repurposed it for RoCE MSI-x on every device.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D59091
(cherry picked from commit 15187b27c895e82d153dd1de5324076692023e70)
if_bnxt: support more Tx rings than Rx rings on P5+
Add an unsupported-config guard rejecting nrxqsets > ntxqsets, and
teach the P5+ NQ datapath to handle the opposite asymmetric case:
extra Tx rings beyond the Rx ring count. softc->nq_rings[i].type now
tags each NQ as SHARED_NQ (paired with an Rx CQ) or BNXT_TX_ONLY_NQ
(dedicated to an extra Tx CQ with no backing Rx ring); bnxt_init()
and bnxt_hwrm_resource_free() gate all Rx ring group/ring/stat-ctx
alloc and free on IS_SHARED_NQ() so the extra Tx-only NQs don't touch
nonexistent Rx ring state. Gate the new type-tagging and IS_SHARED_NQ
checks on BNXT_CHIP_P5_PLUS(), since non-P5+ chips never allocate
softc->nq_rings and would otherwise access a nonexistent array.
BNXT_TX_ONLY_NQ is a distinct value from the existing MPC-private
TX_CP_NQ; the two are unrelated and kept apart to avoid confusion.
Suggested by: gallatin
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
[3 lines not shown]
if_bnxt: drop epoch_arr[] to detect EPOCH bit toggle
The EPOCH array size is 4096 but the driver does not enforce any limit
on ring size, so setting ring size >4096 corrupts adjacent memory
and leads to undefined behavior.
Toggle epoch_bit directly at every ring-producer wrap point (TX/RX
encap and refill, MPC crypto commands, kTLS presync/replay) instead of
looking it up per-index in the doorbell path, dropping the now-unused
epoch_arr[] snapshot array.
Reported by: gallatin
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58620
Change-Id: I4673d64670ee0e8ea256b8a8932724bbd73f710d
(cherry picked from commit ed1f24d3e1a1c0f85bf3eda0337cfbf76bc34cf6)
if_bnxt: drop redundant PCI disable in bnxt_fw_reset_close()
bnxt_fw_reset_close() unconditionally called pci_disable_device()
after freeing the Rx IRQs, on every firmware reset, not just the
BNXT_STATE_FW_FATAL_COND path which already disables the device via
bnxt_fw_fatal_close(). BNXT_FW_RESET_STATE_ENABLE_DEV in
bnxt_fw_reset_task() already re-enables the device unconditionally,
so this wasn't leaving the device disabled, just adding an
unnecessary PCI disable/re-enable cycle around every non-fatal reset.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58619
(cherry picked from commit 4fbb63f289ed6a06e715c6882e8fff3201381743)
if_bnxt: fix HWRM failures/timeouts after repeated FW resets
Consecutive firmware-initiated reset cycles produced HWRM
failures/timeouts and traffic didn't come back. Fix the FW-reset
recovery path:
- bnxt_fw_reset_close() called bnxt_stop() and
bnxt_hwrm_func_drv_unrgtr(), sending HWRM ring/VNIC/filter free
commands to a firmware that's already mid-reset and unresponsive,
producing the observed timeouts. Both are unnecessary since
firmware comes back with fresh state anyway; drop them along with
the now-redundant iflib_request_reset().
- bnxt_func_reset() now skips bnxt_hwrm_resource_free() entirely
while BNXT_STATE_IN_FW_RESET is set, for the same reason.
- bnxt_open() (used to reopen after a firmware reset) now issues
bnxt_hwrm_func_reset() up front and drives reinit through iflib's
own reset machinery (iflib_request_reset() plus a new
bnxt_iflib_reset_sync() that waits for
IFF_DRV_RUNNING/OACTIVE to flip 1->0->1) instead of calling
[10 lines not shown]
if_bnxt: ktls: Reject new kTLS sessions once driver is detaching
bnxt_tls_snd_tag_alloc() only gated on BNXT_STATE_OPEN, which is set
once by bnxt_open()/bnxt_attach_pre() but never cleared again, so it
could not reject new sessions once bnxt_detach() started tearing
things down. The snd_tag_alloc_ref-based wait in bnxt_detach(), which
waits for in-flight allocators to exit, was already in place, but
without this gate new sessions could keep being created for the
entire duration of that wait, extending it indefinitely instead of
just draining existing ones.
Reject new sessions once softc->detached is set, and reset it back to
false in bnxt_attach_pre() so it doesn't wrongly persist across a
detach/reattach cycle.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58612
(cherry picked from commit a48a50919da5549b2a5a2141042087a25879435e)
if_bnxt: rework interrupt coalescing onto the AGGINT_QCAPS scheme
Query firmware-advertised coalescing capabilities and program
per-direction rx/tx coalescing settings against them, replacing the
old hardcoded scheme. Add sysctls for the coalescing mode, budget,
and stats-timer knobs the new scheme exposes.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58621
(cherry picked from commit 22f37c46b4841fec22612fd4bc485f0515f27a28)
if_bnxt: add MPC (Mid-Path Channel) ring infrastructure
Add MPC ring support: a pair of extra, driver-private
TX/completion/notification rings the firmware uses as a side-channel
for offload commands. The only current consumer is kTLS TX crypto
command submission; RCE/CFA channel types are not yet implemented by
any consumer.
Adds MPC ring allocation/teardown, HWRM ring alloc/free, a private
interrupt path for the MPC NQ ring, and the TX submit/completion
path. The RoCE-only IRQ table builder is split into a generic,
exported bnxt_populate_irq(softc, irq_count) that both MPC and RoCE
use, plus an incremental bnxt_populate_irq_roce() that grows the
table on top of whatever is already there, so IRQ rids stay
consistent regardless of attach order.
This patch does not link standalone: bnxt_mpc_cmp() calls into
kTLS's completion handler, added by a later patch.
[5 lines not shown]
if_bnxt: Fix HWRM mailbox/DMA teardown race on detach
iflib's generic device-deregister path never drains the admin task
before calling IFDI_DETACH, so bnxt_update_admin_status() (scheduled
once/sec) could still run concurrently with bnxt_detach(), racing on
softc->hwrm_lock and the shared HWRM request/response DMA buffer that
bnxt_detach() destroys. That race could leave the firmware-side HWRM
mailbox inconsistent, surfacing as "Timeout sending HWRM_VER_GET" on
the next module load.
Add a softc->detached guard: set it as the first statement in
bnxt_detach(), and bail out of bnxt_update_admin_status() immediately
when it's set.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58607
(cherry picked from commit 9330b2f73c04f06e39da5cf2279682a607185601)
f_bnxt: add kTLS (kernel TLS) TX offload
Add kTLS TX crypto offload, layered on MPC (crypto add/delete
commands submitted over its TCE channel) and TX completion coalescing
(shared tx_bd_opaque encoding).
Adds kTLS session lifecycle management, a key-context ID allocator,
HWRM key-context alloc/free with partition-mode support, and the TX
submit path: in-order packets go out on the normal iflib path with
CSUM_SND_TAG set, while out-of-order or retransmitted TLS records get
a presync command plus a replayed copy of the record built from the
driver's own replay ring. Also adds kTLS/MPC completion-time counter
sysctls and the max_ktls_entries tunable.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58606
(cherry picked from commit 70d389bede66da585508c1813c10a873d42612e7)
if_bnxt: add bnxt_compat.h version-compat shim
Add bnxt_compat.h, a small __FreeBSD_version-gated compatibility
shim: if_getmtu()/if_name()/if_getsoftc()/if_getflags() fallbacks for
pre-1500000, and a config-task vs. config-gtask abstraction over the
iflib API split at 1403000. On this branch's FreeBSD version it also
unconditionally enables KTLS_IFLIB_SUPPORT/KTLS_REC_SEQ_NUMBER, which
the upcoming kTLS patch depends on.
For stable/15, enable the existing iflib kTLS and TLS record-sequence
interfaces starting at __FreeBSD_version 1501506.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58604
(cherry picked from commit 42988358a9210ccf7ff500895785b49fe5615567)
if_bnxt: consolidate readl_fbsd/writel_fbsd and fix BAR mapping
There are multiple duplicates of readl_fbsd and writel_fbsd spread
accross bnxt_en and bnxt_re drivers. Move a single definition into bnxt.h as static
inline, shared by bnxt_en and bnxt_re, using the correctly sized
bus_space_read_4()/bus_space_write_4() accessors, and drop the other
two copies along with their declarations.
Also, Fix the BAR selection while at it: bar_idx 0 addresses BAR0 (the
HWRM bar) and bar_idx 2 addresses BAR2 (the doorbell bar), matching
every call site (e.g. BNXT_FW_HEALTH_REG_TYPE_BAR0/BAR1 in
if_bnxt.c) and bnxt_pci_mapping()'s own bnxt_map_bar() calls, but
both old copies and bnxt.h's own inline had the two bars swapped.
Add BNXT_HWRM_BAR_IDX/BNXT_DOORBELL_BAR_IDX macros for the two valid
bar_idx values and use them at every readl_fbsd()/writel_fbsd() call
site plus bnxt_pci_mapping(), instead of bare 0/2 literals. Any other
bar_idx is a programming error, so readl_fbsd()/writel_fbsd() panic
on it rather than silently guessing a BAR.
[6 lines not shown]
if_bnxt: add HW-based TX completion coalescing
When firmware advertises BNXT_FLAG_TX_COAL_CMPL, request coalesced
completions via cmpl_coal_cnt on TX ring alloc and decode the
resulting tx_cmpl_coal_t records using a free-running SQ consumer
index per ring, instead of one completion per BD. Add the
tx_hw_coal_cnt sysctl to tune the firmware coalescing count code.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58599
(cherry picked from commit 9e69060da5eaa4a596be96a4a84b6484661466aa)
if_bnxt: add CAGR interrupt coalescing tick-resolution support
Add the P7 CAGR (Completion AGgregation Ring) NQ-aggregation timer
tick-resolution knob, programmed via a direct GRC-window register
write, with a new dev.bnxt.<unit>.cagr_tick_res read-only tunable.
This is independent of the existing per-ring AGGINT_PARAMS coalescing.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58598
(cherry picked from commit 2f3a1ebba81266b5f21640f88d73e03415cf13be)
if_bnxt: add RX timestamping support
Add PHC register mapping, RTC initialization, refclk read with
FW-reset serialization, and sbinuptime-based calibration to convert
the raw hardware timestamp carried in RX completions into an mbuf
timestamp. Advertise IFCAP_HWRXTSTMP, configure the RX timestamp
filter in bnxt_init(), and drain the calibration callout on stop.
Check HWRM_FUNC_QCAPS_OUTPUT_FLAGS_PTP_SUPPORTED in
bnxt_hwrm_func_qcaps() so the capability bit PTP is gated on actually
gets set, without which PTP never armed on real hardware.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58596
(cherry picked from commit f2c300112ea491d69799c5a1085b57c48acef713)
if_bnxt: add host-based TX completion coalescing
Suppress per-BD completions (TX_BD_LONG_FLAGS_NO_CMPL/_COAL_NOW) and
accumulate a running_bds count per ring, forcing a real completion
when iflib requests one or when the count nears overflowing the
opaque bd-count field. Replaces the previous raw (bd_count << 24 |
idx) TX completion opaque encoding, which only had 8 bits for the
count and would overflow well within these coalescing thresholds,
with a tx_bd_opaque bitfield (16-bit idx + 15-bit bds + 1-bit
ktls_replay for later use). Add tx_host_coal_bds/tx_host_coal_enable
sysctls to tune and toggle it.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D58600
(cherry picked from commit 3c995919dc4a7f5de35534b3220f972bf0153fb9)
if_bnxt: dcb: serialize HWRM sends and avoid freeing DMA bufs on timeout
Every _hwrm_send_message() call in bnxt_dcb.c ran without BNXT_HWRM_LOCK,
unlike the rest of the driver, letting concurrent HWRM commands race on
the shared MMIO doorbell and response buffer. Route single-shot sends
through the already-locked hwrm_send_message() wrapper, and wrap the
remaining send+response-read sequences in explicit BNXT_HWRM_LOCK/UNLOCK.
Also stop freeing the transient DMA buffers used for structured-data
get/set on ETIMEDOUT: a host-side timeout doesn't guarantee firmware
actually gave up, so a late completion could still DMA into memory
that's since been freed and reused. Leak the buffer instead in that case.
Reviewed by: gallatin
Sponsored by: Broadcom Inc.
Differential Revision: https://reviews.freebsd.org/D60179
(cherry picked from commit 4f20e02ed0582bd8a3e2b59853690c1d46e08fa8)
if_bnxt: sriov: replace Linux kPIs with FreeBSD native APIs
Convert bnxt_sriov.c and related SRIOV files from Linux to FreeBSD
native equivalents:
- Replace cpu_to_le16/32/64/le16_to_cpu with htole16/32/64/le16toh.
- Replace Linux is_valid_ether_addr, ether_addr_equal, ether_addr_copy
with local bnxt_eth_addr_valid/bnxt_eth_addr_equal/bnxt_eth_addr_copy
helpers built on ETHER_IS_MULTICAST/ETHER_IS_ZERO.
- Replace kcalloc/kzalloc/kfree with malloc/free (M_DEVBUF,
M_WAITOK|M_ZERO).
- Replace dma_alloc_coherent/dma_free_coherent with iflib_dma_alloc/
iflib_dma_free; add struct iflib_dma_info hwrm_cmd_req_mem[4] to
bnxt_pf_info to hold the DMA handles.
- Replace the Linux unsigned long *vf_event_bmap with FreeBSD
bitstr_t *, allocated via bit_alloc(); bit_ffs_at()/bit_clear()/
bit_set() replace find_next_bit()/clear_bit()/set_bit() on it
(if_bnxt.c's vf-event producer side included).
- Replace DIV_ROUND_UP with howmany; rcu_assign_pointer with a plain
[10 lines not shown]
