[RISCV][P-ext] Support Packed Slide (#234629)
This patch supports packed slide intrinsics using funnel shifts.
On RV32, the 64-bit funnel shift expands into select instructions, so it
involves branches.
[CodeGen] Print x86 memory operands with their operand count in tests
The MIR printer now writes an inline asm flag's operand count when it isn't
1, so the five operands of an x86 memory reference print as "mem:m ops:5".
Assisted-by: Claude Opus 5.5
[RegAllocFast] Print x86 memory operands with their operand count in tests
The MIR printer now writes an inline asm flag's operand count when it isn't
1, so the five operands of an x86 memory reference print as "mem:m ops:5".
Assisted-by: Claude Opus 5.5
[CodeGen] Address review: report a memory constraint on a label as such
A direct label operand with a memory constraint (e.g. "!m") isn't an
input, so it was reported as a direct memory output. Give it its own
error, and update the comment above the check, which still described only
memory inputs.
Assisted-by: Claude Opus 5.5
[SPARC] Treat %g0 as a constant in the delay slot pass (#234152)
This allows instructions such as clr, that reads %g0, to be moved past a
compare, that writes to %g0, and into a delay slot. %g0 is always 0, so
there is no dependency.
Assisted-by: Claude
[mlir] Fix signed ceiling division of INT_MIN
Negating INT_MIN causes signed overflow even when its quotient is
representable. For INT_MIN / 2, the Index expansion included:
%zero = llvm.mlir.constant(0 : i64) : i64
%n = llvm.mlir.constant(-9223372036854775808 : i64) : i64
%m = llvm.mlir.constant(2 : i64) : i64
%neg_n = llvm.sub %zero, %n : i64 // Overflow: -INT_MIN is unrepresentable.
%div = llvm.sdiv %neg_n, %m : i64
%result = llvm.sub %zero, %div : i64
The first subtraction wraps back to INT_MIN, producing +2^62 instead of
-2^62. A range inference workaround copied this incorrect sign.
Compute a truncating quotient and increment it only for inexact divisions
with same-sign operands. Use this for Index folding and LLVM/SPIR-V
lowering, and remove the inference workaround to match the Arith expansion.
[mlir] Fix signed ceiling division of INT_MIN
Negating INT_MIN causes signed overflow even when its quotient is
representable. For INT_MIN / 2, the Index expansion included:
%zero = llvm.mlir.constant(0 : i64) : i64
%n = llvm.mlir.constant(-9223372036854775808 : i64) : i64
%m = llvm.mlir.constant(2 : i64) : i64
%neg_n = llvm.sub %zero, %n : i64 // Overflow: -INT_MIN is unrepresentable.
%div = llvm.sdiv %neg_n, %m : i64
%result = llvm.sub %zero, %div : i64
The first subtraction wraps back to INT_MIN, producing +2^62 instead of
-2^62. A range inference workaround copied this incorrect sign.
Compute a truncating quotient and increment it only for inexact divisions
with same-sign operands. Use this for Index folding and LLVM/SPIR-V
lowering, and remove the inference workaround to match the Arith expansion.
[clang][Sema] Expand `#embed` in braced lists that initialize non-aggregates (#231938)
Fixes #222796
A multi-byte `#embed` inside braces is one `EmbedExpr`, and only
aggregate initialization slices it. Every other consumer of the braced
list counted it as a single element, so `X x = { #embed ... }` called
the constructor with the first byte only, and a reference or fixed enum
silently took the first byte instead of being rejected.
`Sema::getInitsWithEmbedExpanded()` hands those consumers the list with
each multi-element `EmbedExpr` split into one single-element `EmbedExpr`
per byte, the same slices aggregate initialization already builds.
Constructor selection, braced-argument overload resolution and CTAD use
it; the single-element and excess-element checks go through the new
`InitListExpr::hasZeroInits()` / `hasOneInit()`. Aggregates still keep
the whole `EmbedExpr`.
For more reference, see https://github.com/llvm/llvm-project/pull/218262
[2 lines not shown]
[MLIR][XeGPU] Rematerialize layout conversion sources after conflict resolution (#230669)
This PR separates layout conflict resolution from rematerialization. The
resolver now inserts xegpu.convert_layout for conflicts ....a following
step clones eligible, cheap vector producer chains in the target layout
and replaces those conversions. This avoids SLM round trips.
[RISCV][P-ext] Add isel patterns for mul.h01/mul.h11 and macc.h01/macc.h11 (#234976)
`mul.h01` multiplies the lower half word of rs1 by the upper half word
of rs2, and `mul.h11` multiplies the upper half words of both.
`macc.h01/h11` add the product to rd.
Previously, (sra X, 16) was selected as a separate srai feeding
`mul/macc.h00`. Match it directly so the shift is folded. As with
`macc.h00`, the macc patterns require the multiply to have a single use.
[VPlan] Move extract-lane(0, canWideIV) combine before unrolling
Some extract-lane combines can't be performed before unrolling, e.g. the one that combines it to Instruction::ExtractElement can end up extracting from a different part. But the extract-lane(0, canWideIV) can because at index 0 you're always extracting from the first part. Do this before unrolling, which improves codegen of unrolled code.
[VPlan] Support interleaving early exit loops with side effects
We couldn't do this on main previously because the exit mask `active-lane-mask(0, FirstActiveLane(Cond))` doesn't get unrolled properly across parts. Now that we use mask.beforefirst we can unroll it properly in a chain. The unrolled form is:
unroll(mask.beforefirst, P) = mask.beforefirst(cond[P] | broadcast(anyof(not(unroll(mask.beforefirst, P-1)))))
On AArch64 this can be combined to brkpb in another patch.
[VPlan] Use llvm.mask.beforefirst for computing early exit mask
This makes the cost more accurate and improves codegen for RISC-V. The codegen for an early exit loop with side effects then goes from:
.LBB0_3: # %vector.body
# =>This Inner Loop Header: Depth=1
vl1r.v v16, (a3)
vsetvli zero, zero, e8, m1, ta, ma
vmseq.vi v16, v16, 0
vfirst.m a7, v16
srli t0, a7, 63
czero.nez a7, a7, t0
czero.eqz t0, a1, t0
or a7, t0, a7
vsetvli zero, zero, e64, m8, ta, ma
vmsltu.vx v0, v8, a7
vle8.v v17, (a5), v0.t
vcpop.m t1, v16
vsetvli zero, zero, e8, m1, ta, ma
[22 lines not shown]
[CIR][CMake] Fix MSVC PCH ambiguity after 0173f46b7d9d (#235058)
0173f46b7d9d added a PCH for clangCIR that includes
CIRGenFunction.h, which forward-declares ScalarExprEmitter in an
anonymous namespace. MSVC treats the anonymous namespace from the PCH
as distinct from the one in CIRGenExprScalar.cpp, causing error C2872
('ScalarExprEmitter': ambiguous symbol). Skip the PCH for that file on
MSVC.
Assisted-By: Claude Opus 5.5
[CodeGen] Drop dead SlotIndexes before allocation
SlotIndexes keeps the index list entry of an erased instruction and only
clears its instruction pointer. Live range sizes are measured in slot
indexes and greedy ranks ranges by size, so the leftovers inflate some
ranges more than others and reorder allocation, spilling heavily on
register-starved functions.
Add SlotIndexes::compactIndexes() to erase them. Erased entries are
unlinked, so holding an index to one is now a bug; linkedEntry() asserts
on it. The one analysis that did was LiveIntervals, so add
removeUnreferencedIntervals() to retire the intervals responsible.
Off by default behind -greedy-compact-slot-indexes, since it changes
allocation across much of the test suite.
The test file greedy-compact-slot-indexes-priority.mir by Q. Colombet.
[TargetInstrInfo] Address review: write the two-register flag symbolically
Now that the operand count of an inline asm flag can be written in MIR, use
"reguse:GR64 foldable ops:2" instead of an immediate whose register class ID
changes whenever a register class is added. Regenerate the checks, which
now print the five operands of each x86 memory reference as "ops:5".
Assisted-by: Claude Opus 5.5
