[NFC][AArch64][FrameLowering] Refactor handling of Callee-Saved spill pairs (#233405)
This PR is preparation for a subsequent PR that will enable quad spills.
The main refactor is adding functions to RegGroupInfo that are dynamic
to the number of registers in a RegGroup.
The subsequent PR will expand these functions for Quad ZPR spills.
A test was also added to show the before/after of this refactor and to
catch any errors in MIR after frame lowering.
[InstCombine] Allow same extract insert shuffle fold for all fixed zero vector types (#233370)
Currently in CollectShuffleMask(), we check for ConstantAggregateZero to
check for a zero vector. However, this does not check for floating point
zero vectors nor null pointer vectors.
I believe this was a side effect from transitioning from CAZ to
ConstantFP and ConstantPointerNull for float and pointer types.
Modify to use type independent zero vector check which can identify both
types allowing for the same transformation to take place independent of
Int/Float/Pointer type.
Since this is an extension of an existing transformation, test coverage
has been kept minimal.
[AArch64][GlobalISel] Ignore debug instructions when reusing carry. (#228128)
Ignore immediately in-between continuous block of debug instructions
when a carry-producing instruction precedes a carry-consuming operation
in AArch64 GlobalISel.
`emitCarryIn()` previously compared the carry producer directly against
`I.getPrevNode()`. As a result, inserting a debug-only instruction
(e.g., `DBG_VALUE`) between otherwise adjacent carry operations
prevented reuse of the `NZCV` flags produced by the previous
instruction. This caused instruction selection to unnecessarily
materialize the carry into a virtual register and reconstruct `NZCV`
before the following carry operation.
Walk backwards over consecutive debug instructions before performing the
adjacency check. Debug instructions do not modify `NZCV`, so they should
not prevent reuse of the existing carry flags.
#### Context
[18 lines not shown]
GlobalISel: Add a lowering of G_FCONSTANT to G_CONSTANT
RISCV and ARM both custom legalized G_FCONSTANT into a G_CONSTANT with
the same bits. Add LegalizerHelper::lowerFConstantAsConstant and use it
in both.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
SelectionDAG: Build FP immediates from integer bits in the legalizer
Teach the generic legalizer to legalize FP immediates as a bitcast of
an opaque integer constant. Migrate LoongArch to use the new helper
instead of doing it in custom lowering.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
CodeGen: Return how to materialize FP immediates from targets
Replace the isFPImmLegal target hook with getFPImmAction, which
returns whether an FP immediate is legal or how it should be
legalized. This will allow adding a way for the legalizer to build an
immediate from its integer bits, instead of each target doing so in
custom lowering or selection.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[InstCombine] Canonicalize atomicrmw sub of a constant to add
Rewrite `atomicrmw sub ptr, C` as `atomicrmw add ptr, -C`. This
matches the canonical form InstCombine already uses for plain `sub`.
In wrapping arithmetic $old - C = old + (-C)$, so both the stored
value and the returned old value stay the same, including when C is
`INT_MIN`. Ordering, syncscope and metadata are kept.
```llvm
; before
%old = atomicrmw sub ptr %p, i32 1 syncscope("agent") monotonic
; after
%old = atomicrmw add ptr %p, i32 -1 syncscope("agent") monotonic
```
This benefits AMDGPU, for example: https://godbolt.org/z/39xsYro3o
[mlir][flang][OpenMP] Privatize LINEAR list items with linear privatizers; support ALLOCATABLE (#228827)
Flang miscompiled `LINEAR` on an `ALLOCATABLE` list item (#182401). The
linear
operand was the descriptor reference, but `linear_var_types` held the
element
type, so translation created an undersized private copy and the program
crashed.
This patch makes linear list items use privatizer recipes, the same way
`private` and `reduction` do.
**MLIR**
- The `linear` clause on `omp.wsloop` and `omp.simd` now carries a
privatizer
symbol and an entry block argument for each list item:
```mlir
omp.private {type = linear} @x.linear : i32
omp.simd linear(@x.linear %x -> %arg0 : !llvm.ptr = %step : i32)
[34 lines not shown]
[AArch64][TTI] Cost extends folded into [US]SHLL as free (#233489)
A doubling NEON zext/sext feeding a shl by a splat immediate
is selected as one [US]SHLL/SHLL, so the extend is free.
[flang] Decouple HLFIR array expansion from optimization level (#233478)
Add optimizeArrayExpression option to the pass pipeline configuration,
set to false by default for now.
It will allow preserving runtime calls for array expressions when set to
false once array temporary
optimizations are default at O0.
Always attempt to inline scalar-to-array assignments, including host
code at O0.
Update pipeline expectations and add coverage for broadcasts, array
expressions, and reductions.
The goal is to make this option always true by default once we start
emitting some fir.assert operation.
Part of
https://discourse.llvm.org/t/rfc-flang-expanding-array-expressions-and-assignments-at-o0/91971
[mlir][vector][NFC] Rename getShapeForUnroll to getIterationShapeToUnroll (#232648)
The old name was ambiguous about whether it returned the original shape
or the target shape for unrolling.
The new name makes clear that it returns the shape of the iteration
space to be unrolled. This can differ from the result shape: for
example, vector.contract includes reduction dimensions in its iteration space
that are absent from its result shape.
Implements: https://github.com/llvm/llvm-project/issues/233408
[AArch64][TTI] Cost NEON Nots absorbed by And/Or users as free (#232788)
Cost a NEON Not as free when every user is an And/Or in the same block,
since the Not folds into its BIC, ORN or BSL. If a user negates both of
its operands, only one Not can fold, so it still counts.
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
[AArch64][TTI] Cost the Ands of a NEON bit select as free (#232787)
Or(And(M, A), And(Not(M), B)) selects to a single BSL/BIT/BIF. Charge
the
Or and treat the single-use Ands in its block as free, using the matcher
shared with operand sinking.
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
[AArch64][TTI] Drop single-use requirement on the mask from BSL sinking (#232786)
Drop the one-use requirement on the mask's Not. Sinking puts a copy of
the
Not next to the And, and the original stays for its other users.
ISel can then form bsl/bif even when the Not has other uses.
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
[AMDGPU][NFC] Explicitly narrow conversions in libcalls, TTI and InstCombine (#215196)
Fixes 34 instances of MSVC warning C4244 and 3 of C4267 ("possible loss of data") across 4 files in llvm/lib/Target/AMDGPU.
Assisted-by: Claude <noreply at anthropic.com>
[Clang][AST][NFCI] Make Decl reference attributes directly (#219138)
This commit changes `Decl` attributes from being stored in a map to
being managed by an allocator and letting the individual `Decl` objects
reference the attributes directly. Doing so reduces the additional
complexity of maintaining a map and improves performance by reducing the
levels of directions used when accessing `Decl` attributes.
As an additional benefit, this removes the need for the `hasAttr` field
as the pointer can be used to indicate whether a `Decl` has attributes
or not. This coincidentally preserves the size of `Decl` on most
systems, as the single-bit `hasAttr` field caused additional padding for
the class.
Assisted-by: Claude Opus 5
---------
Signed-off-by: Steffen Holst Larsen <sholstla at amd.com>
