[VPlan] Match induction before querying truncate lane usage (#233651)
Only query a truncate's lane usage after establishing that its operand
is an optimizable induction. Lane usage is irrelevant when the operand
cannot be narrowed, so checking induction eligibility first avoids
unnecessary use-chain traversal. Truncates without a matching induction
are rejected regardless of lane usage; matching candidates still undergo
the same lane-use and profitability checks before any transformation.
Reordering the checks therefore preserves the narrowing decision in all
cases.
Assisted-By: Opus 5.5
WebAssembly: Use reportFatalUsageError for eh option errors
Avoid using the deprecated report_fatal_error, and add missing
error test coverage.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[CodeGen] Avoid per-function DenseMap growth in LexicalScopes and InstructionOrdering. NFC (#229990)
LexicalScopes::scanFunction keeps a DenseMap from each range's first
instruction to its scope, but only looks it up while walking the ranges
in order. Store the scope next to its range instead.
InstructionOrdering::initialize inserts every instruction of the
function; reserve the map up front.
LLM-aided. O0-g instructions:u: -0.26%
MC: Remove MCAsmInfo::ExceptionsType
The exception model now comes from the module, and the MCAsmInfo
predicates that depend on it take the model as a parameter. The
target's default model is still available from
Triple::getDefaultExceptionHandling.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
PPC: Add MIR examples for missed extsw+word-load fold on subregister input
A gprc LWZ/LWZX feeding EXTSW_32_64 folds into a sign-extending LWA/LWAX
load. The equivalent 64-bit zero-extending word loads (LWZ8/LWZX8) whose
sub_32 feeds EXTSW_32_64 are not folded, leaving a redundant lwz+extsw
(or lwzx+extsw) pair. Add MIR examples documenting the missed fold.
Co-authored-by: Claude (Claude Opus 4.8, claude-opus-4-8) <noreply at anthropic.com>
PPC: Fold 64-bit zero-extending word load feeding extsw subregister
A gprc LWZ/LWZX feeding EXTSW_32_64 is rewritten into a sign-extending
LWA/LWAX load. Extend the same fold to the 64-bit zero-extending word
loads LWZ8/LWZX8 when the EXTSW_32_64 reads their sub_32 subregister,
producing a single LWA/LWAX instead of a redundant lwz+extsw pair.
Co-authored-by: Claude (Claude Opus 4.8, claude-opus-4-8) <noreply at anthropic.com>
CodeGen: Remove TargetOptions::ExceptionModel
The exception model now comes only from the "exception-model" module
flag. Remove the TargetOptions field and its copy into MCAsmInfo, so
MCAsmInfo only knows the target's default model.
The WebAssembly model checks move from the TargetMachine constructor
into the Emscripten EH/SjLj lowering pass. Clang relied on the backend
to reject non-wasm models for wasm targets, so it now rejects them
itself.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
CodeGen: Read the exception model from the module flag
Module::getExceptionModel() now falls back to the triple's default
model instead of falling back to the TargetMachine.
The pass pipeline is built before the module is known, so schedule each
EH preparation pass when the target supports a model, and let the pass
check the module's model. Add TargetMachine::supportsExceptionModel so
the pass config can skip definitely irrelevant passes. SjLjEHPrepare is
now scheduled on every target with exception handling, and WinEHPrepare
only on COFF and Wasm targets. WinEHPrepare and WasmEHPrepare are
scheduled for Wasm by the base pass configs, replacing the WebAssembly
override.
This is almost the last step towards removing TargetOptions::ExceptionModel.
Co-authored-by: Claude Opus <noreply at anthropic.com>
PPC: Fix extsw elimination when the input reads a subregister
The EXTSW_32_64 sign-extend elimination previously assumed its input
was a full register value. It would then try using that value as the
source of the new (unnecessary) INSERT_SUBREG.
The new test would then hit this verifier error:
```
bb.0:
liveins: $x3
%0:g8rc = COPY killed $x3
%1:g8rc = RLDICL killed %0:g8rc, 0, 33
%3:g8rc = IMPLICIT_DEF
%2:g8rc = INSERT_SUBREG %3:g8rc(tied-def 0), %1:g8rc, %subreg.sub_32
$x3 = COPY killed %2:g8rc
BLR8 implicit $lr8, implicit $rm, implicit killed $x3
*** Bad machine code: INSERT_SUBREG expected inserted value to have equal or lesser size than the subreg it was inserted into ***
[8 lines not shown]
[llvm-readobj][ELF] Print the address of each --stack-sizes entry (#232331)
In a linked file, print the function address that each .stack_sizes
entry holds as Address in the LLVM and JSON output. Until now the entry
only gave the function's names, so tools had to look the function up by
name to match the entry with other per-function data.
Assisted-by: Gemini
[llvm-readobj][docs] Update the --stack-sizes description (#232335)
The description said that --stack-sizes is only implemented for GNU
style output, which hasn't been the case for a while. Describe the LLVM
and JSON output instead, including the address that each entry in a
linked file now has.
Assisted-by: Gemini
[llvm-readobj][ELF] Clear the Thumb bit from --stack-sizes addresses (#232338)
On ARM, the address of a Thumb function has bit 0 set, but that bit is
not part of the function's address. --stack-sizes looked up function
symbols at the address with the bit set, so Thumb functions were
reported as unknown, with a warning.
Clear the bit before the lookup, with a helper that is also used by the
two places in the call graph dumping code that already cleared it.
Assisted-by: Gemini
[AMDGPU][NFC] Document buffer_inv without scope bits on GFX942 (#233528)
In the CDNA3/4 ISAs `buffer_inv` without `sc0/sc1` is documented to be a
no-op (see table 51/52 respectively in the ISA pdfs). Implicit to those
documents is that `buffer_inv` increments `VM_CNT` (this is since
`buffer_inv` is a MUBUF instruction and `VM_CNT` is incremented every
time a MUBUF instruction is issued). This PR documents these behaviors
in the AMD GPU docs.
As part of preparing this PR I verified `VM_CNT` is incremented (both
when `EXEC` is and isn't zero), benchmarked walltime of `buffer_inv` on
mi300+mi350, and detected whether the operand-less invocation
invalidates any cache. I put the code in
[danzimm/buffer-inv-bench](https://github.com/danzimm/buffer-inv-bench).
Full transparency: the benchmarks were coauthroed with claude. I audited
and then tweaked the benchmarks myself (e.g. originally claude didn't
use IB_STS, and used timing info to detect cache usage, as opposed to
rocprof as it does now). The full results can be found
[here](https://github.com/danzimm/buffer-inv-bench/blob/main/results.md),
[19 lines not shown]
[RISCV] Return the chain for strict FP_TO_FP16 on RV64 (#230948)
When F or D is available but Zfh is not, RISC-V custom lowers FP_TO_FP16
and STRICT_FP_TO_FP16 to a libcall. On RV64, the lowering returned only
the result, without the chain, even for the strict node. As a result,
strict conversions to half (for example fptrunc from float, or sitofp to
half) hit the assertion `Illegal result number!`.
This happens for any `_Float16` arithmetic with `-ffp-model=strict` on
RV64 without Zfh, because clang computes it in float and rounds the
result back to half.
Merge the chain on RV64 as well, as RV32 already does.
Assisted-by: Claude Opus 5.5
[Clang][C2y] Reject declarations in the condition of a while loop
N3356 only extends the selection statements (`if` and `switch`), but the
shared condition parsing accepted `while (int x = 12) {}` in C as a C2y
extension. `while` passes a null init-statement into
`ParseParenExprOrCondition`, so use that to diagnose a condition
declaration as "expected expression" (matching GCC) instead of emitting
the compatibility/extension warning.
Also only emit the extension warning for a non-simple declaration in the
first clause when the declaration is actually permitted there.
Fixes #233839
Co-Authored-By: Claude Fable 5.1 <noreply at anthropic.com>
Attributor: Don't use callee return attributes for mismatched call types
A call site returned position took the state and attributes of the
callee's returned position, even if the call uses a different return type
than the callee, e.g. calling a void function as returning ptr. This
asserted in AANonNull::isImpliedByIR on the non-pointer callee return
type, and put the callee's nofpclass on calls returning another FP type.
Ignore the callee's returned position, and its returned arguments, if the
types differ.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[AMDGPU][True16] Fix flat/global/scratch 16-bit load patterns (#232855)
Replace the raw (EXTRACT_SUBREG inst, lo16) in the _t16 load patterns
with the vgpr32_lo16 OutPatFrag so the extracted value carries the
VGPR_16 register class instead of relying on type inference.
Also remove the non-t16 pattern class for the sext/zext 16-bit atomic
global load: These are duplicated patterns
NFC.
Co-authored-by: Claude Opus 5 (1M context) <noreply at anthropic.com>
powerpc: powerpc_ipi_handler(): Stop first, preempt last
Preemption should happen last so that other IPIs have a chance to be
executed in due course, in particular hardclockintr(). So, move
handling of IPI_PREEMPT at the end.
An IPI_STOP/IPI_STOP_HARD request should be processed immediately to
avoid risks that further processing changes the system state on panic,
or worse itself causes another panic on corrupted kernel state.
While here, move IPI_AST handling just before IPI_PREEMPT. We currently
do not need to do anything for it, but if we were to, it is likely that
would have to happen after other IPIs but IPI_PREEMPT.
Reviewed by: jhibbits
Fixes: 9d5a22b928c2 ("MFp4: SMP support")
Fixes: 12640815f815 ("MFp4: SMP support")
Fixes: 7ed09a6778d3 ("powerpc: Rework IPI message processing")
MFC after: 1 week
[2 lines not shown]
x86: ipi_bitmap_handler(): Account for IPIs correctly on COUNT_IPIS
A preemption could occur before the updates to the initial CPU's IPI
counters, leading to a potential non-atomic updates of these counters
from another CPU.
Reviewed by: kib
Fixes: ff326a187907 ("x86: Restore the critical section (...)")
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60862
x86: ipi_bitmap_handler(): Preempt last, reduce critical section
Preemption should happen last so that other IPIs have a chance to be
executed in due course, in particular hardclockintr(). So, move
handling of IPI_PREEMPT at the end.
Reduce the critical section to be in force only around hardclockintr(),
which needs it to execute entirely without preemption (because of code
it calls that can enter a critical section; interrupts are already
disabled in this handler, as usual).
The critical_exit() following hardclockintr() may cause preemption if
due. If that happens, there is no need to preempt again on IPI_PREEMPT,
so make sure sched_preempt() is not called in this case.
This fixes inadvertent removal of the optimization that sched_preempt()
should be called outside of a critical section to perform an immediate
context switch in the case of IPI_HARDCLOCK and IPI_PREEMPT being
handled at the same time.
[10 lines not shown]
AVR: Run AtomicExpand (#233207)
Atomics wider than 16 bits were lowered to __sync_* calls. No AVR
runtime provides __sync_* or __atomic_*, but __sync_* requires a
lock-free implementation, which AVR cannot provide at those sizes.
Use the __atomic_* calls instead.
Alternatively it may be appropriate to change getDefaultThreadModel to
singlethread for avr so all atomics are lowered into non-atomic
sequences.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[RISCV][P-ext] Combine (zext_invec (punzipe X, Y)) -> (ppaire X, 0) (#233713)
For RV64, the avgfloor combine creates truncates for both inputs
and an extend fo the result. Those truncates are type legalized to
shufflevectors and bitcasts. The shufflevectors are pulled through
the avgfloor to create a single shuffle after. Then we match that
to PUNZIPE. Now we have a PUNZIPE followed by zext_invec.
For the signed case, the sext_invec is legalized to
aext_invec+sext_inreg. The aext_invec is legalized to zext_invec.
This leaves a redundant pzext.h.b or pzext.w.h.
We could instead combine earlier in shuffle form before the
PUNZIPE is matched, but that would be more complex.
GlobalISel: Add a lowering of G_FCONSTANT to G_CONSTANT
RISCV and ARM both custom legalized G_FCONSTANT into a G_CONSTANT with
the same bits. Add LegalizerHelper::lowerFConstantAsConstant and use it
in both.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
