[mlir] Fix signed ceiling division of INT_MIN
Negating INT_MIN causes signed overflow even when its quotient is
representable. For INT_MIN / 2, the Index expansion included:
%zero = llvm.mlir.constant(0 : i64) : i64
%n = llvm.mlir.constant(-9223372036854775808 : i64) : i64
%m = llvm.mlir.constant(2 : i64) : i64
%neg_n = llvm.sub %zero, %n : i64 // Overflow: -INT_MIN is unrepresentable.
%div = llvm.sdiv %neg_n, %m : i64
%result = llvm.sub %zero, %div : i64
The first subtraction wraps back to INT_MIN, producing +2^62 instead of
-2^62. A range inference workaround copied this incorrect sign.
Compute a truncating quotient and increment it only for inexact divisions
with same-sign operands. Use this for Index folding and LLVM/SPIR-V
lowering, and remove the inference workaround to match the Arith expansion.
[clang][Sema] Expand `#embed` in braced lists that initialize non-aggregates (#231938)
Fixes #222796
A multi-byte `#embed` inside braces is one `EmbedExpr`, and only
aggregate initialization slices it. Every other consumer of the braced
list counted it as a single element, so `X x = { #embed ... }` called
the constructor with the first byte only, and a reference or fixed enum
silently took the first byte instead of being rejected.
`Sema::getInitsWithEmbedExpanded()` hands those consumers the list with
each multi-element `EmbedExpr` split into one single-element `EmbedExpr`
per byte, the same slices aggregate initialization already builds.
Constructor selection, braced-argument overload resolution and CTAD use
it; the single-element and excess-element checks go through the new
`InitListExpr::hasZeroInits()` / `hasOneInit()`. Aggregates still keep
the whole `EmbedExpr`.
For more reference, see https://github.com/llvm/llvm-project/pull/218262
[2 lines not shown]
[MLIR][XeGPU] Rematerialize layout conversion sources after conflict resolution (#230669)
This PR separates layout conflict resolution from rematerialization. The
resolver now inserts xegpu.convert_layout for conflicts ....a following
step clones eligible, cheap vector producer chains in the target layout
and replaces those conversions. This avoids SLM round trips.
[RISCV][P-ext] Add isel patterns for mul.h01/mul.h11 and macc.h01/macc.h11 (#234976)
`mul.h01` multiplies the lower half word of rs1 by the upper half word
of rs2, and `mul.h11` multiplies the upper half words of both.
`macc.h01/h11` add the product to rd.
Previously, (sra X, 16) was selected as a separate srai feeding
`mul/macc.h00`. Match it directly so the shift is folded. As with
`macc.h00`, the macc patterns require the multiply to have a single use.
[VPlan] Move extract-lane(0, canWideIV) combine before unrolling
Some extract-lane combines can't be performed before unrolling, e.g. the one that combines it to Instruction::ExtractElement can end up extracting from a different part. But the extract-lane(0, canWideIV) can because at index 0 you're always extracting from the first part. Do this before unrolling, which improves codegen of unrolled code.
[VPlan] Support interleaving early exit loops with side effects
We couldn't do this on main previously because the exit mask `active-lane-mask(0, FirstActiveLane(Cond))` doesn't get unrolled properly across parts. Now that we use mask.beforefirst we can unroll it properly in a chain. The unrolled form is:
unroll(mask.beforefirst, P) = mask.beforefirst(cond[P] | broadcast(anyof(not(unroll(mask.beforefirst, P-1)))))
On AArch64 this can be combined to brkpb in another patch.
[VPlan] Use llvm.mask.beforefirst for computing early exit mask
This makes the cost more accurate and improves codegen for RISC-V. The codegen for an early exit loop with side effects then goes from:
.LBB0_3: # %vector.body
# =>This Inner Loop Header: Depth=1
vl1r.v v16, (a3)
vsetvli zero, zero, e8, m1, ta, ma
vmseq.vi v16, v16, 0
vfirst.m a7, v16
srli t0, a7, 63
czero.nez a7, a7, t0
czero.eqz t0, a1, t0
or a7, t0, a7
vsetvli zero, zero, e64, m8, ta, ma
vmsltu.vx v0, v8, a7
vle8.v v17, (a5), v0.t
vcpop.m t1, v16
vsetvli zero, zero, e8, m1, ta, ma
[22 lines not shown]
[CIR][CMake] Fix MSVC PCH ambiguity after 0173f46b7d9d (#235058)
0173f46b7d9d added a PCH for clangCIR that includes
CIRGenFunction.h, which forward-declares ScalarExprEmitter in an
anonymous namespace. MSVC treats the anonymous namespace from the PCH
as distinct from the one in CIRGenExprScalar.cpp, causing error C2872
('ScalarExprEmitter': ambiguous symbol). Skip the PCH for that file on
MSVC.
Assisted-By: Claude Opus 5.5
[CodeGen] Drop dead SlotIndexes before allocation
SlotIndexes keeps the index list entry of an erased instruction and only
clears its instruction pointer. Live range sizes are measured in slot
indexes and greedy ranks ranges by size, so the leftovers inflate some
ranges more than others and reorder allocation, spilling heavily on
register-starved functions.
Add SlotIndexes::compactIndexes() to erase them. Erased entries are
unlinked, so holding an index to one is now a bug; linkedEntry() asserts
on it. The one analysis that did was LiveIntervals, so add
removeUnreferencedIntervals() to retire the intervals responsible.
Off by default behind -greedy-compact-slot-indexes, since it changes
allocation across much of the test suite.
The test file greedy-compact-slot-indexes-priority.mir by Q. Colombet.
[TargetInstrInfo] Address review: write the two-register flag symbolically
Now that the operand count of an inline asm flag can be written in MIR, use
"reguse:GR64 foldable ops:2" instead of an immediate whose register class ID
changes whenever a register class is added. Regenerate the checks, which
now print the five operands of each x86 memory reference as "ops:5".
Assisted-by: Claude Opus 5.5
arm64 linux: Allow FUTEX_REQUEUE on arm64 by setting the brand flag
Better change is to provide the dedicated musl brandinfo, as it is done
on amd64/i386, which should be done later.
PR: 299836
MFC after: 1 week
[MIR] Serialize the operand count of inline asm operand flags
An inline asm operand flag is followed by the number of operands that it
says: e.g. two registers for an i128 or an i386 "A" output, or the five
operands of an x86 memory reference. The symbolic flag syntax had no way to
write that count, so the printer dropped it and the parser assumed one.
Printed MIR with such a flag didn't parse back correctly: the operands
after the first were read as further flags or as stray operands, which the
machine verifier rejects. Tests had to write such a flag as an immediate,
whose register class ID changes whenever a register class is added.
Print the count as "ops:N" at the end of the flag when it isn't 1, and
parse it back, e.g.:
INLINEASM &"", attdialect, mem:m ops:5, %stack.0, 1, $noreg, 0, $noreg
INLINEASM &"", attdialect, reguse:GR64 foldable ops:2, %0, %1
A count of 0 is accepted, since the printer emits whatever count the flag
holds; peephole-copy.mir writes "1 /* clobber */", which is a reguse flag
[3 lines not shown]
