[GlobalISel][AArch64] Fold vector multiplication by 0/1 masks into AND (#240134)
Multiplying a vector by a constant whose elements are all 0 or 1 only
keeps or clears lanes, so it can be done with an AND mask, as
SelectionDAG does. This adds a generic combine,
`mul_by_zero_one_vector`: `(mul x, <0/1...>) -> (and x, <0/-1...>)`, for
either operand order. All-zero and all-one vectors are left to the
existing combines.
```
define <2 x i32> @test(<2 x i32> %a) {
%r = mul <2 x i32> <i32 1, i32 0>, %a
ret <2 x i32> %r
}
```
Before:
```
adrp x8, .LCPI1_0
[7 lines not shown]
[mlir][SPIR-V] Collect availability of optional enum attributes (#240532)
The availability generator skipped OptionalAttr enums, so update-vce
gave ImageRead with Nontemporal v1.0 instead of v1.6
[BasicAA] Drop nsw when folding an overflowing constant offset (#242541)
Folding the constants in a chain of `nsw` adds or subtracts can overflow
even when the operations themselves are defined. Keeping `nsw` on the
resulting linear expression lets BasicAA return `NoAlias` for overlapping
accesses. Check the folded offset for signed overflow and drop `nsw` if
it wraps.
Split from #241380.
Proofs: https://alive2.llvm.org/ce/z/nq85ze
Assisted-by: Claude Code
Assisted-by: OpenAI Codex
[CIR] Keep volatile and atomic return loads before cleanups
When FlattenCFG routes a cir.return through a cleanup, it sinks a load
of an entry-block alloca that feeds the return into the block after the
cleanup. For a volatile or atomic load this moved the access past the
cleanup code. Such a load is now spilled to a temporary before the
cleanup, like any other return operand.
Assisted-by: Claude Code (claude-opus-5-5).
[CIR][NFC] Add isVolatileOrAtomic() to cir.load and cir.store
Several places spelled out getIsVolatile() || getMemOrder() to decide
whether a load or store must stay as written. Name that check once on
both ops and use it in the load effects, mem2reg, CIRSimplify and the
ABI rewriter.
Assisted-by: Claude Code (claude-opus-5-5).
[CIR] Report side effects of volatile loads and memory asm operands (#242691)
Dead code elimination erased some unused CIR ops that have side effects,
because their declared memory effects were too weak:
- `cir.load`, `cir.get_bitfield` and `cir.matrix.column_major_load`
declared only a read of their address, so an unused volatile or atomic
load was erased. A volatile or atomic load now also reads and writes
unknown memory, as `llvm.load` does.
- `cir.asm` reported nothing for a memory operand such as `"=*m"`, so an
asm whose only output is in memory was erased. It now reads and writes
each memory operand. The new `InlineAsmOp::isMemoryOperand` identifies
those operands, and the LLVM lowering uses it too.
- `cir.asm` now has a verifier: `operand_attrs` needs one entry per
operand, either a unit attribute on a pointer operand (a memory operand)
or an empty dictionary.
Assisted-by: Claude Code (claude-opus-5-5).
[CIR] Keep volatile and atomic return loads before cleanups
When FlattenCFG routes a cir.return through a cleanup, it sinks a load
of an entry-block alloca that feeds the return into the block after the
cleanup. For a volatile or atomic load this moved the access past the
cleanup code. Such a load is now spilled to a temporary before the
cleanup, like any other return operand.
Assisted-by: Claude Code (claude-opus-5-5).
[CIR][NFC] Add isVolatileOrAtomic() to cir.load and cir.store
Several places spelled out getIsVolatile() || getMemOrder() to decide
whether a load or store must stay as written. Name that check once on
both ops and use it in the load effects, mem2reg, CIRSimplify and the
ABI rewriter.
Assisted-by: Claude Code (claude-opus-5-5).
[BasicAA] Restrict cross-iteration separate_storage hints (#241381)
A `separate_storage` assumption applies to its operand values when it
executes. If the operands change across loop iterations, it does not
imply that an operand from one iteration is separate from the other
operand in another iteration. BasicAA can nevertheless use it to return
`NoAlias` in cross-iteration queries, causing GVN and loop-vectorization
miscompiles.
After finding a matching assumption that is valid in the query context,
require both underlying objects to be invariant according to
`isValueEqualInPotentialCycles()` before using it. Queries without a
usable assumption skip these checks. Same-iteration queries and
hints about loop-invariant objects remain supported.
Fixes #241378.
Assisted-by: Claude Code
Assisted-by: OpenAI Codex
[LoongArch] Cast to unsigned before LSHing in LoongArchMergeBaseOffsetOpt::detectAndFoldOffset() (#242656)
Make one step ahead of #242761 to fix a latent, similar problem in
`LoongArchMergeBaseOffsetOpt::detectAndFoldOffset`, as suggested by
@heiher.
[RISCV][P-ext] Restrict P widening shift legalization to left shifts (#242773)
The RV32 P-extension custom legalization for i64 shifts shares a path
for `SHL`, `SRL`, and `SRA`. Its known-bits checks select `WSLL` or
`WSLA` when the input fits in an unsigned or signed 32-bit value, but
both instructions perform a left shift.
This commit restricts this selection to `SHL` so right shifts use the
existing legalization paths.
Closes https://github.com/llvm/llvm-project/issues/242771
[LLD] Disable bitcode verification in release builds (#242663)
Similar to the plugin for other linkers, disable running the IR verifier
on bitcode input files. This improves ThinLTO link times by 1.5-1.8% and
stage2-clang build times by 0.57%.
Hexagon: Use LastSubtargetCache in getSubtargetImpl
Migrate to the new common target pattern
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir][SPIR-V] Deserialize null array and struct constants (#231626)
Deserialize OpConstantNull for arrays and literal structs by recursively
building the ArrayAttr representation used by spirv.Constant. Preserve
rejection of unsupported constituent types and report the nested type
when a null value cannot be represented.
Allow OpConstantComposite with no constituents for empty structs so that
imported null empty structs can round-trip. Diagnose OpUndef
constituents explicitly instead of reporting them as missing normal
constants.
Document the supported representations and add binary deserialization,
round-trip, and negative tests for aggregate constants.
---------
Co-authored-by: Claude Fable 5.2 <noreply at anthropic.com>
AMDGPU: Fix double interval removal in SIOptimizeExecMaskingPreRA
An exec copy folded after its use was erased from an s_endpgm block
had its interval removed twice. Fixes an assert after 6a483810f0da.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[RISCV][P-ext] Use EVT in `combinePExtWidening*Acc` (#242687)
Fixes https://github.com/llvm/llvm-project/issues/242679.
The previous implementation assumes a simple value type, which is wrong
for weird-sized integers.
Assisted-by: Codex
DAG: Allow narrowing sint_to_fp source to the sext source type
Same as 241c94e3919e for uint_to_fp. The narrowing fold skipped
sint_to_fp (sext x) when the narrow type matched x, so sitofp (sext i32
to i64) was expanded as a 64-bit conversion on AMDGPU.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Add more sitofp of sext from i32 tests
These currently get the full i64 conversion expansion.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
CSKY: Use LastSubtargetCache in getSubtargetImpl
Migrate to the new common pattern for targets to cache the last
result.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
SystemZ: Use LastSubtargetCache in getSubtargetImpl
Migrate to the new common pattern and infrastructure.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
WebAssembly: Use LastSubtargetCache in getSubtargetImpl
Cache the last subtarget and use the new common pattern for targets.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Mips: Use LastSubtargetCache in getSubtargetImpl
Migrate to use the new common infrastructure and pattern which caches
based on the last used subtarget.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Bitstream: Move the slow path of Read out of line
SimpleBitstreamCursor::Read was too large to inline into its
callers. Separate out the slow error path.
CTMark instructions:u: ThinLTO -0.23% (kimwitu++ -0.72%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AliasAnalysis: Use SmallVectors in AAResults
AAResults is rebuilt for each function whenever AAManager is invalidated,
and the default pipeline adds 3 or 4 AA results to it. With std::vector
each one regrows its storage on the second and third addition.
callgrind instructions, opt -O3: kimwitu++ k.bc -0.45%, SPASS cnf.bc
-0.12%, sqlite3 -0.03%. CTMark instructions:u: O3 -0.02%, ThinLTO -0.01%.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[ARM] Prevent debug values from causing extra Thumb1 spills (#237017)
Under Thumb1 execute-only code with an AAPCS frame chain, a debug value
for a stack-passed argument is represented with a fixed frame index.
canSpillOnFrameIndexAccess treats that debug-only operand as a real
access, so enabling debug info spills two additional callee-saved
registers.
Skip debug instructions in the scan. Keep a real stack-argument use
covered to ensure that frame-index elimination still reserves enough
registers.
Assisted-by: OpenAI Codex, co-reviewed by Claude and Yongqiang Tian.
