[CIR] Reject void record members, array elements and union padding (#233898)
!cir.void has no data layout, so a record or array that holds one has
none either. The verifiers accepted such types, and the first size query
crashed. CallConvLowering hit a DataLayout fatal error on a union
argument, and LowerToLLVM asserted in LLVMStructType::setBody.
The record and array verifiers now reject void, so no consumer has to
guard against it. CallConvLowering still accepts void as a parameter
type, which the x86_64 classifier ignores.
Assisted-by: Claude Code (Claude Opus 5.5)
[mlir] Fix false positive loops in RegionBranchOpInterface::hasLoop (#233784)
`RegionBranchOpInterface::hasLoop` reported a loop whenever its
traversal popped a region it had already visited. The worklist gets one
entry per successor of each terminator, so a region reachable along two
paths was popped twice even without a cycle, for example when two
terminators of a region branch to the same successor region.
`hasLoop` now runs one iterative depth-first search from the entry
regions and reports a loop only when it reaches a region that is still
on the DFS stack. Each region is queried at most once, through
`RegionBranchOpInterface::getSuccessorRegions(Region &)` instead of
asking each terminator with null operand attributes. The stop condition
of the remaining traversal loses its `visited` argument.
Assisted-by: Claude Code (Claude Opus 5.5).
[lldb-dap] Don't drop active sessions when another client disconnects. (#234292)
When the connection-timeout is specified the server stop only after all
clients have disconnected and the connection-timeout has passed,
Disconnecting a client should not trigger server shutdown
Fixes #234254
[analyzer] Fix crashing PaddingChecker when using dependent context lambdas in noexcept specifiers (#232986)
A lambda in the `noexcept` specifier of an abbreviated function template
is not marked as dependent, even when it captures a dependent entity.
This Sema bug is tracked in #233102.
```cpp
void crasher(auto&& fn) noexcept(noexcept([_ = fn] {})) {}
```
Because the closure type is not marked dependent, `PaddingChecker` tries
to compute its record layout and crashes in
`ASTContext::getTypeInfoImpl` on the dependent field type.
This PR adds a unit test documenting the Sema bug, a FIXME in
`Sema::ActOnLambdaExpressionAfterIntroducer`, and a temporary workaround
in `PaddingChecker::shouldSkipDecl` that skips fields of dependent type
so the analyzer no longer crashes. The workaround should be removed
together with the FIXME once the Sema bug is fixed.
Fixes #232874
[LV] Implement legality checks for compressing patterns (#230546)
This implements the legality checking for "compressing" patterns for
example:
E.g., Loops like:
```
int dst_idx = 0;
for (int i = 0; i < n; i++) {
// Can be vectorized as a llvm.masked.compressstore.
if (cond[i])
dst[dst_idx++] = src[i];
}
```
This uses the new `ConditionalInductionDescriptor` to recognize
compressing patterns.
[16 lines not shown]
[lldb] Python3 fixes for "lui" script (#234548)
Which I was trying out after reading
https://discourse.llvm.org/t/rfc-removing-gui-mode/92570.
This is fixes by 2to3, some that I found testing
it, and a few I saw looking through the sources.
Just enough to get it to run, I bet there are
all sorts of issues with it.
[LoongArch] Lower FP_TO_SINT_SAT to [X]VFTINTRZ/FTINTRZ (#233760)
The ftintrz.{w,l}.{s,d} and [x]vftintrz.{w.s,l.d} instructions saturate
out-of-range inputs to the minimum/maximum integer and convert NaN to 0
(when the invalid operation exception is not trapped), which matches the
semantics of llvm.fptosi.sat when the saturation width is the native
32/64 bits. This was verified on LA664 for all scalar, LSX and LASX
variants and matches QEMU's LoongArch model. Use them instead of the
generic expansion, which emits three compares and selects per element
and fully scalarizes vectors.
Also override shouldConvertFpToSat so DAGCombiner only folds min/max
clamps and NaN-guarded selects into FP_TO_SINT_SAT when that lowers to a
single ftintrz, avoiding the generic expansion for mismatched vector
element widths and for i64 results without the D extension.
This is what Rust emits for `f32 as i32`. rustc-perf svg-render-1 on
LA664: instructions -6.4%, cycles -4.0%.
[CIR] Use `llvm::ElementCount` to hold `cir::VectorType` size (#231906)
A separate scalability flag is a design issue inherited from MLIR's
VectorType. It makes dropping/ignoring scalability an easy mistake to
make in common transforms and lowerings.
This patch aligns ClangIR with LLVM and uses `ElementCount` to hold the
vector size. This makes it simple to propagate the entire size (element
count and scalability) and forces users to query the size they expect:
- `.getFixedValue()`
- Asserts the size is fixed and converts it to an integer
- `.getKnownMinValue()`
- Allowed for fixed/scalable vectors
- Returns the element count at vscale=1
A few small bugs/missed verifier checks have been handled with this
patch and are now tested:
[4 lines not shown]
[AMDGPU] Only fold 32-bit shift sources in isExtractHiElt (#215245)
D16 image sample returns 4-time 16-bit samples packed into 64 bits. The
backend legalizes extractelement <4 x half>, 1 (channel 1 — the upper
half of the first dword) into trunc(srl(i64, 16)). At normal
optimization levels the DAG combiner splits the i64 into two i32s before
instruction selection, hiding the problem. At -O0 the i64 survives
intact. This creates an invalid COPY from a 64-bit virtual register to a
32-bit physical register, which asserts in SIInstrInfo::copyPhysReg.
To fix it, in isExtractHiElt (both the SelectionDAG version in
AMDGPUISelDAGToDAG.cpp and the GlobalISel version in
AMDGPUInstructionSelector.cpp), check that the shift source is exactly
32 bits before folding.
[lld][LoongArch] Fix crash with TLS IE relocations in extreme code model (#234508)
Restore TLS IE and DESC relocation expressions needed by the extreme
code model fallback in relocateAlloc().
Fixes: #234507
[TargetLowering][X86] Address review: keep "rm" in memory for half and bfloat
The check that an "rm" value fits in one 'r' register accepted any value no
wider than the register. X86 gives a half or bfloat value a GR64 (GR32 on
i686), so "rm"(half) printed as %rax and failed to assemble, where it used
to pick memory. Require the value to fill the register, and test half and
bfloat.
Also check the target, opt level, and isIndirect before scanning the
constraint codes, don't offer 'm' to a MayFoldRegister output with a
matching input (as for any other operand), assert a single register in
GlobalISel as SelectionDAG does, and name the instruction selector in the
new tests' RUN lines.
Assisted-by: Claude Opus 5.5
[ADT] Add SmallVectorWithFlags with packed user flags
Structures that hold a SmallVector followed by a bool or a couple of
flag bits pay up to eight bytes of padding for those bits, and there is
no padding inside the vector to reuse: the header is exactly a pointer
and two 32-bit fields, and the inline storage of pointer-sized elements
is a multiple of eight bytes. SmallVectorWithFlags stores one to four
user flag bits in the high bits of the capacity field, so it has the
same size and inline capacity as SmallVector.
The capacity field is the one header field not read on the hot path:
the buffer pointer's free low bits depend on the element alignment and
are read on every element access, and the size is read by size() and
end(). Only capacity() masks the flags out, which costs one extra `and`
in the growth check of push_back; begin(), end(), and size() are
untouched. The cost is a maximum capacity divided by 2^FlagBits, which
with the 32-bit size type and four flag bits is 268 million elements.
The SmallVector implementation layers take a FlagBits template
[35 lines not shown]
[TargetLowering][X86] Prefer 'r' over 'm' for foldable "rm" inline asm operands
An "rm" (register-or-memory) inline asm operand has always resolved to
'm', because getConstraintPreferences() picks the most general
constraint present, and 'm' is more general than 'r'. That's safe, since
memory can't run out, but it forces a value that could stay in a
register through a stack slot even when there's no register pressure
(https://github.com/llvm/llvm-project/issues/20571).
Prefer 'r' instead where the register allocator can fold the register
back to a stack slot when it runs out of registers, and mark the
register operand foldable (InlineAsm::Flag::setRegMayBeFolded()) so it
does. Both allocators can: the greedy allocator folds an operand when it
spills its value, and the fast allocator folds operands up front when
the asm's register operands wouldn't fit.
ParseConstraints() sets AsmOperandInfo::MayFoldRegister for an operand
whose constraint codes include 'r' and 'm' and are otherwise only
immediate codes, above -O0, on a target that opts in through the new
[26 lines not shown]
[X86] Address review: test a tied input's untied read under -regalloc=fast
Add the case from the #229630 review, one value passed to an untied "r"
input and to the "0" input tied to an "rm" output, to show that the fast
register allocator no longer folds the output when everything fits.
Drop the -O0 RUN line from inline-asm-direct-mem-output-error.ll: it only
showed that "=rm" still picks memory at -O0, which asm-constraints-rm.ll
already tests, and there make the -O0 line name the instruction selector
and say that it tests the optimization level.
Assisted-by: Claude Opus 5.5
[RegAllocFast] Address review: don't fold registers whose slot can't hold the value
Folding a register stores the value the asm reads into the register's slot,
which can't work in two cases, so leave the register unfolded:
- A def that is also the tied input of another def. Its slot would need both
its old value, for that def, and its own tied input's value. The store put
the old value there, so the asm read the wrong input.
- An operand, or a def's tied input, that reads a subregister. The slot holds
the whole register; storing a GR64 tied input for a GR32 def failed the
machine verifier.
ISel never emits either form. Also skip the analysis for inline asm without
a foldable operand, which is most of it.
Assisted-by: Claude Opus 5.5
[CycleInfo] Move invalidate() to the result types (#234518)
The new pass manager looks for a custom invalidate() method on the
analysis result type, not on the analysis pass. #188443 added these
methods to CycleAnalysis and MachineCycleAnalysis, so they were never
called and the results were still invalidated by passes that preserve
the CFG. Move them to CycleInfo and MachineCycleInfo.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[RegAllocFast] Fold foldable inline asm operands under register pressure
An inline asm register operand marked foldable (from an "rm" constraint)
may be replaced with a stack slot when the register allocator runs out
of registers. The greedy allocator does that when it spills the value.
The fast allocator can't: it assigns the operands of an instruction one
at a time, and folding replaces the instruction. So it reported
"inline assembly requires more registers than available" instead.
Before allocating a block, estimate from each inline asm's own operands
whether they fit in registers, and fold as many foldable registers as
needed to make them fit, so the common case without pressure still gets
a register. Values that only live across the asm don't count, since the
allocator spills them when it needs their registers. The estimate
follows allocateInstruction():
- First all defs get distinct registers, avoiding physreg defs; then all
uses, along with the defs still occupied while the uses are read
(early-clobber and tied defs, see isLiveThroughDef()), get distinct
[26 lines not shown]
[RegAllocFast] Address review: don't recount a tied input's untied reads
When one value is passed to both a tied operand and an untied one, e.g.
"=rm"(r) : "r"(v), "0"(v), with an explicit -regalloc=fast above -O0 (so
TwoAddress doesn't run and the ties reach RegAllocFast), the fold estimate
counted three registers where lowerTiedUse() uses one. Tied uses are
allocated first, so the value moves into the def's register and the untied
read is rewritten to it. tiedUseNeedsOwnReg() still assumed another read of
the value could get it a register first, and the untied read was counted on
its own as well, so the asm's output was folded to the stack needlessly.
Model the tied input itself: while one of its tied defs is in a register,
it needs no register of its own unless tiedUseNeedsOwnReg() says so (only
an early-clobber def now makes another read matter), and its untied reads
share whichever register it has. Once its tied defs are folded, the untied
reads need a register again, so choose what to fold by counting again with
each candidate folded, and fold both halves when neither alone helps.
Assisted-by: Claude Opus 5.5
[CodeGen] Address review: error for "=m|r" direct outputs too
A direct output whose chosen alternative is memory, e.g. "=m|r", still
crashed SelectionDAG, which exempted multiple-alternative operands from the
check and then asked for the address of an output with no value. Drop the
exemption and test "=m|r" with both instruction selectors. Name the
selector explicitly in the RUN lines rather than relying on -O levels,
since X86 GlobalISel can't lower inline asm and both X86 lines ran
SelectionDAG anyway. Drop "yet" from the error: Clang never emits a
direct memory output, so there are no plans to support one.
Assisted-by: Claude Opus 5.5
[CodeGen] Report an error for a direct inline asm output in memory
An inline asm output returned by value has no memory to write to, yet a
constraint such as "=rm" picks memory, the most general constraint, as
does "=m". SelectionDAG asserted on that ("Can only indirectify direct
input operands!"), and GlobalISel dereferenced a null pointer. Clang
never emits such an output, since it passes the address of a memory
output, but other IR can. Report "cannot handle direct memory outputs
yet for constraint 'm'" instead, like the other inline asm errors there.
Assisted-by: Claude Opus 5.5
[TargetInstrInfo] Address review: don't fold one register of a multi-register operand
A memory operand replaces exactly one register, so return nullptr for a
folded operand whose flag holds more than one, rather than relying on an
assert that release builds drop. Also assert that every folded operand
holds the same register, and say that a caller folding a def whose tied use
is another register must store that register's value into the slot.
Test both the new bail-out and folding a rematerialized load into two "rm"
operands, which takes the fold-a-load path.
Assisted-by: Claude Opus 5.5
