[LV] Don't tail-fold the epilogue with EVL-based tail-folding (#228452)
Epilogue tail-folding isn't supported yet with the `DataWithEVL`
tail-folding style. The epilogue plan is selected by duplicating an
existing VPlan, and some recipes used only by EVL-based tail-folding
don't implement clone() yet, so cloning a plan that contains them hits
llvm_unreachable. This affects targets that prefer `DataWithEVL`, such
as RISC-V.
Until those recipes implement clone(), fall back to a normal epilogue
when the preferred (or forced) tail-folding style is DataWithEVL. This
bail-out is based on the preferred style rather than the style that
would actually be chosen, so it also rejects fixed-width epilogue VFs,
where EVL isn't used. It will be refined once the epilogue's chosen
tail-folding style is known when epilogue TF gets supported.
[VPlan] Support tailfolded loops in multi-use-reductions (#214455)
Previously `handleMultiUseReductions()` would bail out for
tailfolded-loops, since the backedge value is no longer the reduction
intrinsic but the predicated select based on the header-mask.
This patch adds pattern matching to handle tailfolded loops accordingly.
I compiled the LLVM testsuite (for RISCV with -march=rv64gcv) and it
only triggers in the corresponding unit-tests in
`SingleSource/UnitTests/Vectorizer/`. I guess that's because there are
still some other limitations when handling more generic multi-use
reduction cases (e.g. differing types in CanonicalIV vs. WideIV).
[ConstraintElim] Bound IVs by compares of the phi in header/latch. (#226297)
Extend addInfoForInductions to add bounds for IVs when the phi is
compared in the header or latch:
In that case, every iteration taking the backedge checked
PN ContinuePred B, which guarantees PN != B for NE and LT predicates.
For an increment by one, PN != B together with StartValue <= B (added
precondition) imply PN <= B.
Alive2 Proofs:
* icmp ult in header: https://alive2.llvm.org/ce/z/x37Uvm
* icmp slt in header: https://alive2.llvm.org/ce/z/ibnn5k
* icmp ne in header: https://alive2.llvm.org/ce/z/wLRfZe
Triggers in a number of additional cases on
https://github.com/dtcxzyw/llvm-opt-benchmark-nightly/pull/1431.
As-is, this comes with a slight compile-time increase:
[9 lines not shown]
[Mips] Lower calls to private functions the same way as internal functions (#229975)
As with 85b4fbfecb65cd1f5a06e707e7ec0fc2de562407, this seems to be
another case where the backend was treating private functions
differently for no apparent reason.
This was particularly problematic for building the Zig compiler with
LLVM linked statically because we'd have enough such calls that linking
failed with thousands of these errors:
relocation R_MIPS_CALL16 out of range: 76048 is not in [-32768, 32767]
RegisterPressure: Detect dead physreg defs from LiveIntervals
This reverts the remainder of #222627, which was partially reverted by
on dead flags. This is a prerequisite to deleting LiveVariables.
When constructing the PressureDiff for an instruction during scheduling
DAG construction, dead defs were only recognized from the dead flag on the
operand. This implicitly relied on preprocessing done by LiveVariables to
fixup inconsistent dead flags with overlapping registers in other operands.
Dead flags have no verifier-enforced rules and are thus unreliable.
Before LiveVariables, consider this example:
dead $eax = MOV32r0 implicit-def dead $eflags, implicit-def $rax
; $rax is never used
$rax is never used, but only the $eax def is dead-flagged and the overlapping
implicit-def $rax is not. The shared $eax register units are covered by the
non-dead $rax def and so are counted as live defs. That shared unit is then
[26 lines not shown]
benchmarks/iperf: modernize server mode
Replace the legacy Ruby manager and its persistent socket service with a short-lived Python helper invoked through configd. Store transient jobs and JSON results in /tmp, reject duplicate ports, and support removal of running instances.
Move the server UI to an in-memory Instance model and client-style Bootgrid view under /ui/iperf/index/instance. Drop firewall manipulation, service registration, and Ruby dependencies.
Move 2FA force option to the current API models
## Problem
The HA synchronization change added a required `force` argument to `auth.twofactor.update`, `user.renew_2fa_secret` and `user.unset_2fa_secret`. That change was intended for 28.0, but it was written before the 27 -> 28 API version bump and its `force` field ended up in the v27 pydantic models while the v28 ones were never updated. Current API is v28, so validation never supplies the default and every call fails with `missing 1 required positional argument: 'force'`, which breaks all the 2FA integration tests.
`auth.twofactor.update` had a further problem: `ConfigService.update` only accepts and forwards `data` to `do_update`, so a second positional `force` could never work. Passing it explicitly (the HA retry with `force: true`) failed with `update() takes 4 positional arguments but 5 were given`, and v27 / v25.10 clients failed the same way because the version adapter appends the defaulted `force` to their params.
## Solution
- **User methods**: moved the `force` argument (defaulting to `False`) from the v27 argument models to the v28 ones, since v27 has already shipped. v27 clients keep working as their calls are adapted to the current models.
- **`auth.twofactor.update`**: `force` is part of the `TwoFactorAuthUpdate` payload (`{"force": true, ...}`) rather than a separate argument. `do_update` pops it before merging the payload into the stored configuration, so it is never persisted. This means the user methods take `force` as an argument while `auth.twofactor.update` takes it inside the payload.
- **Audit**: the `audit_extended` lambdas accept `force`, otherwise a caller passing it explicitly (and every adapted older-API call) would get an audit entry without the username.
RegisterCoalescer: Keep remat def dead if it's a copy destination superregister (#230036)
This is a refinement of #226037, which was too strict.
When rematerializing into a physical register that is not exactly the copy's
destination, the def should only stay live if it is a sub-register of the copy
destination, i.e. part of the live value. Checking register unit coverage also
kept the def live when it is a super-register with the same units as the copy
destination, such as $rax for a copy into $eax on x86_64:
dead $rax = MOV64ri32 -11, implicit-def $eax
Only the $eax part is used, so the $rax def is dead. This matches what
LiveVariables produces for a full def with partial uses.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir][tosa] Adding integer data layout operation support to PRO-FP (#229777)
This pull request was made to add integer data layout operation support
to PRO-FP. This is to prevent the need to cast between int and fp to do
these operations, which had the possibility of producing errors or
unwanted behaviour. Partially implements:
https://github.com/arm/tosa-specification/pull/91
Co-authored-by: Luke Hutton <luke.hutton at arm.com>
[OpenMP][DeviceRTL] Report the source location in __kmpc_error diagnostics (#224298)
Follow-up to #220702. Completes #204240.
The device runtime accepted the `ident_t` argument but ignored it, so
`error at(execution)` in a `target` region printed no source location.
This reports it, replicating the host runtime:
```
OMP: error_directive.f90:14:3: Encountered user-directed warning: warning message.
```
When the ident carries no location the result is `unknown:0:0`, same as
the host. flang populates the ident only with `-g`; clang always does.
Assisted-by: Copilot
SelectionDAG: Stop emitting kill flags in InstrEmitter
These is no point to maintaining these before register allocation.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
RuntimeLibcalls: Pass the default calling convention to libraries
Previously a setAvailableLibFuncs_* function computed DefaultCC locally when a
member calling convention referenced it. To do that, the emitter worked
backwards from a library to the SystemRuntimeLibrary records that reference it,
and had to diagnose the cases where that failed, which would be if there is no
referencing system library or several different ones.
The default calling convention belongs to the target, not to a library. The
dispatcher in setTargetRuntimeLibcallSets already computes it, so pass it to
each library function as a parameter. This removes the reverse lookup and both
diagnostics. DefaultCC references are now valid in a library shared by
SystemRuntimeLibrary records with different defaults, and in a library no
SystemRuntimeLibrary references.
This fixes errors when a LibcallLibrary is unused. This will enable defining
the vector math libraries in the future, as well as decoupling the target
specific handling in #229562.
Co-authored-by: Claude Opus 5 <noreply at anthropic.com>
acpi: Tasks: Document why 'acpi_task_count' is accessed unsynchronized
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit e89c3ac5df58a9eae9d01f325ffc9b34258c4b29)
acpi: Tasks: Make OsdSchedule.c whitespace clean
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit f0825f75bab8ae7cb8e0ba717f76110b6f1862f3)
acpi: Tasks: Remove unnecessary includes
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit 2165accd3748c0acb6e82756d71a46509b220fd8)
ifuncs: Include <machine/ifunc.h> instead of <x86/ifunc.h>
All architectures have been providing an 'ifunc.h' header with
DEFINE_IFUNC() et alter working both in kernel and userland since commit
cf41d1113377 ("riscv: implement kernel ifunc resolution"), and separate
<i386/ifunc.h> and <amd64/ifunc.h> headers were introduced in commit
2b1db07bec92 ("x86: add machine/ifunc.h"), so stop including
<x86/ifunc.h> directly and use the common <machine/ifunc.h> idiom.
While here, re-order includes in the blocks featuring <x86/ifunc.h>,
which requires fixing <x86/apicvar.h> so that it can be included before
<machine/intr_machdep.h>.
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60194
(cherry picked from commit a2fbd988638e94b61cc78449e893f2c02d43dec0)
random: Remove unused include of ifuncs
No functional change (intended).
Reviewed by: markj, emaste
Fixes: 3a12982962ce ("random: add RDSEED as a provably unique entropy source")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60193
(cherry picked from commit 18ce3f50f630f64d5e9e898a3c06e9ed6e5a63a5)
NOTES: Multiple schedulers can be compiled in at once
Mention that ULE is the default scheduler when multiple ones are
compiled in and how the tunable 'kern.sched.name' can be used to select
another one.
While here, regroup SCHED_ULE and SCHED_4BSD, as they control if the
respective scheduler instances are compiled in, putting SCHED_STATS
aside.
Reviewed by: mchoo
Fixes: 75a66a92c92f ("- Add an option to compile in SCHED_STATS. ...")
Fixes: 1322760fd127 ("sys: enable both SCHED_ULE and SCHED_4BSD for some configs")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60014
(cherry picked from commit 754d2c69f3589cdd0bc4649d9e85ecd59935d9d3)
ifuncs: Have DEFINE_*IFUNC() macros expand the passed name
While here, make each <machine/ifunc.h> header include <sys/types.h> so
that it can be included standalone.
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60167
(cherry picked from commit ab9fcc5a553a25804b9535190e914e3ba0e746cf)
NOTES: SMP: Move PREEMPTION out of the debugging options section
It has been activated by default for more than 20 years.
Reviewed by: mchoo, srcmgr (imp)
Fixes: 444ba945136b ("Switch the default scheduler to 4BSD...")
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60012
(cherry picked from commit 61ce73643f22f6f8273701eb978faa246bbca27d)
NOTES: IPI_PREEMPTION: Fix documentation, applies to all architectures
Move its description into 'sys/conf/NOTES' and update it to match reality.
Reviewed by: scheduler (mchoo)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60013
(cherry picked from commit d972d832f580c307cdfd87d563a382f40877dd10)
