TwoAddressInstructions: Keep undef INSERT_SUBREG lanes defined (#230931)
%reg = INSERT_SUBREG undef %reg, %subreg, subidx defines all of %reg, so
reading the lanes outside subidx afterwards is valid. Rewriting it to
undef %reg.subidx = COPY %subreg narrows the definition to subidx and
leaves those reads without a live subrange, failing the "No live subrange at use"
machine verifier check.
When a subrange outside subidx is still live past the def, insert an
IMPLICIT_DEF of the full register and drop the undef flag. In the common
case, where the lanes the INSERT_SUBREG left undefined are dead, keep
the undef flag and avoid an IMPLICIT_DEF that survives to the end of codegen
when the COPY is not coalesced.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply at anthropic.com>
[mlir][LLVM] Add ignore_denormal_mode UnitAttr to LLVM::AtomicRMWOp (#217588)
Add an `ignore_denormal_mode` unit attribute to `llvm.atomicrmw`. It
maps 1:1 to the target-independent `!atomic.ignore.denormal.mode`
instruction metadata (#217585): export attaches the metadata when the
attribute is set, and import sets the attribute when the metadata is
present.
The metadata tells the backend that the atomic may ignore the function's
denormal mode, so a native atomic instruction with fixed denormal
behavior can be used instead of a CAS loop. AMDGPU already honors it,
and NVPTX does as of #217586. Until now MLIR could only produce it on
AMDGPU, through the discardable `rocdl.ignore_denormal_mode` attribute.
That leaves frontends like XLA with no way to request it for NVPTX, and
forces target-specific code for something that is target-independent in
LLVM IR.
The attribute is modeled like `volatile_`: an inherent unit attribute
with an optional `ignore_denormal_mode` keyword in the assembly format
[12 lines not shown]
RegAllocGreedy: Skip redundant hint recoloring walks
Each broken hint starts a walk over its copy-connected registers. When
many broken hints share one large component, as happens after PHI
elimination splits many critical edges into a block, the same component
is walked over and over, which is quadratic.
A walk that recolors nothing changes no assignments, so a later walk
from any register it reached would also recolor nothing. Skip those
seeds until the next recoloring.
On a stress test with 4000 split critical edges, Greedy drops from 40.8G
to 3.8G instructions.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[InstCombine] Fold fpto[su]i.sat of NaN guarded select
fpto[su]i.sat returns 0 for both NaN and $\pm 0.0$, so a select that
replaces a NaN input with zero does not change the result, and the call
can take X directly.
```llvm
%not.nan = fcmp ord float %x, 0.0
%sel = select i1 %not.nan, float %x, float 0.0
%r = call i32 @llvm.fptosi.sat.i32.f32(float %sel)
-->
%r = call i32 @llvm.fptosi.sat.i32.f32(float %x)
```
The inverted form `select (fcmp uno X, 0.0), 0.0, X` is folded too.
Only the canonical `fcmp ord/uno X, 0.0` is matched, which also covers
`oeq/une X, X` after fcmp canonicalization.
[InstCombine] Add tests for fpto[su]i.sat of NaN guarded select. NFC
fpto[su]i.sat returns 0 for NaN and +/-0.0, so the select is redundant.
Cover ord/oeq/uno/une checks, scalar and vector, and negative cases.
[CIR][HIP][SPIR-V] Emit constant globals in addrspace(1) (#230813)
CIR treated HIP `__constant__` like OpenCL constant, which is
addrspace(2) on SPIR-V
Align behavior with classic codegen
[Mips] Fix R6 floating-point condition register handling (#226838)
Sharing a predicate between R6 floating-point selects can produce copies
between incompatible register classes and crash in copyPhysReg. The
condition classes model both CMP.S and CMP.D results as i32, despite the
instructions writing 32-bit and 64-bit FPRs respectively.
Use ordinary FPR classes for compare results and tied select operands.
Represent the legalized i32 condition with a same-width copy for CMP.S
and a low-word subregister for CMP.D. Insert the condition into the low
word for SEL.D, leaving the unused upper word undefined. This lets
register coalescing handle shared predicates and mixed-precision users,
removing the special condition classes, copy heuristics and
register-flags pass.
Select BC1EQZ/BC1NEZ directly for FP comparison conditions, including
their microMIPS counterparts, and support them in branch analysis. These
branches read bit zero, avoiding a GPR transfer and boolean
normalization. Fold normalization of known 0/-1 masks into zero/nonzero
[14 lines not shown]
[clang][bytecode] Check for invalid record decls when returning a pointer from an evaluation (#230907)
We do allow creating such records for diagnostic purposes, but we
shouldn't allow returning them from an evaluation.
TableGen: Cache resolved class instantiations
The same class instantiation is often referenced many times in one
record, and each reference was resolved from scratch along with
everything nested in its arguments. Cache the results in MapResolver and
RecordResolver, and share them with any TrackUnresolvedResolver wrapping
those.
Gigainstructions executed by llvm-tblgen --null-backend
before after
AMDGPU.td 71.75 18.22 -74.6%
X86.td 6.10 6.15 +0.8%
AArch64.td 3.28 3.32 +1.3%
RISCV.td 8.18 8.26 +1.0%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Skip resolving values that cannot change (#231083)
Lists, dags, bits and arguments made only of leaf values always resolve
to themselves, but resolving them still walked every element. Record
this when they are created and return early.
Gigainstructions executed by llvm-tblgen --null-backend
| |before |after | |
|-------------|-------|------|------|
| AMDGPU.td |79.12 |71.73 | -9.3%|
| X86.td | 6.25 | 6.12 | -2.1%|
| AArch64.td | 3.40 | 3.28 | -3.7%|
| RISCV.td | 8.34 | 8.19 | -1.8%|
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
RuntimeLibcalls: Fix wrong types of some vector math functions
The f32 x86 libmvec __expf_finite, __logf_finite and __powf_finite
variants were assigned to the f64 libcalls, and amd_vrd4_erf and
amd_vrd8_erf to the f32 ones.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[VectorCombine] Ignore frees when checking dereferenceability for load widening (#218657)
For load widening, we can ignore frees in dereferenceability checks: The
presence of the load proves that the object cannot have been freed.
The only concern here is in-place shrinkage of allocations. In that
case, it could be that the object is no longer dereferenceable for the
wide load type, but still dereferenceable for the narrow load type.
LangRef currently does not allow shrinking allocations. We discussed
whether we plan to allow this in the future in a recent formal spec WG
meeting, and I believe there was a quite strong consensus that we do not
want to allow shrinking allocations in the future. I believe that
follow-up discussions on the Rust side also failed to produce any
compelling use-cases.
So this PR makes use of the fact that allocations can't shrink in
VectorCombine.
[SDAG] Don't expand forceExpandWideMUL to libcall with unsuitable types. (#230522)
When the wide type is legal or the narrow type isn't, emit a wide MUL
and let it legalise normally to avoid setting up the call incorrectly.
This came up while doing more testing with [su]mulh intrinsics. It seems
it could also be triggered with umulfix, for instance.
[IR] Remove unused debug intrinsic methods (NFC) (#231075)
DbgVariableIntrinsic::replaceVariableLocationOp(unsigned, Value *),
DbgVariableIntrinsic::setKillLocation, and
DbgVariableIntrinsic::setRawLocation: The last callers were removed on
July 18, 2025 in commit c9d8b68676dbf51996a76475313088f750697343.
DbgVariableIntrinsic::addVariableLocationOps,
DbgAssignIntrinsic::setAddressExpression, and
DbgAssignIntrinsic::setKillAddress: The last callers were removed on
July 18, 2025 in commit 2a1869b9815c1f59af9eae91a3ef7e7d78e8f4f2,
leaving DbgAssignIntrinsic::isKillAddress unused as well.
DbgVariableIntrinsic::setVariable and DbgLabelInst::setLabel: The last
callers were removed on June 17, 2025 in commit
9eb0020555fc643582b2802abb8c1bc92059c248.
DbgVariableIntrinsic::isAddressOfVariable: The last caller was removed
on July 21, 2025 in commit c9ceb9b75fd547c7d2e79837075370f4c8db8faa.
Assisted-by: Antigravity
[clang] Replace PointerUnion::dyn_cast with llvm::dyn_cast (NFC) (#231074)
PointerUnion::dyn_cast has been soft-deprecated in favor of
llvm::dyn_cast and llvm::dyn_cast_if_present. This patch replaces the
former with llvm::dyn_cast where the operand is guaranteed to be
nonnull.
Note that VDTemplSpec->getInstantiatedFrom() here returns
getSpecializedTemplateOrPartial() because the enclosing if statement
checks isTemplateInstantiation(...). In turn,
getSpecializedTemplateOrPartial() always returns a nonnull pointer by
construction.
Assisted-by: Antigravity
[CodeGen] Return an IR type from getCmpLibcallReturnType (NFC) (#227192)
Refactor `getCmpLibcallReturnType()` to return an IR `IntegerType`, as
suggested by @arsenm in #226813.
This hook is only overriden by AVR and MSP430, so only those two targets
are changed.
TableGen: Cache resolved class instantiations
The same class instantiation is often referenced many times in one
record, and each reference was resolved from scratch along with
everything nested in its arguments. Cache the results in MapResolver and
RecordResolver, and share them with any TrackUnresolvedResolver wrapping
those.
Gigainstructions executed by llvm-tblgen --null-backend
before after
AMDGPU.td 71.75 18.22 -74.6%
X86.td 6.10 6.15 +0.8%
AArch64.td 3.28 3.32 +1.3%
RISCV.td 8.18 8.26 +1.0%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Skip resolving values that cannot change
Lists, dags, bits and arguments made only of leaf values always resolve
to themselves, but resolving them still walked every element. Record
this when they are created and return early.
Gigainstructions executed by llvm-tblgen --null-backend
before after
AMDGPU.td 79.12 71.73 -9.3%
X86.td 6.25 6.12 -2.1%
AArch64.td 3.40 3.28 -3.7%
RISCV.td 8.34 8.19 -1.8%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
TableGen: Use DenseMap for the IntInit pool (#231082)
Gigainstructions executed by llvm-tblgen --null-backend
before after
AMDGPU.td 79.34 79.13 -0.3%
X86.td 6.30 6.25 -0.7%
AArch64.td 3.43 3.41 -0.6%
RISCV.td 8.40 8.34 -0.7%
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[DAGCombiner] Narrow the integer source of sint_to_fp (#224242)
Truncate the source of a `sint_to_fp` when it is known to fit in a
narrower
type the target can convert from directly.
For example:
```
sitofp (sext i32 %x to i64) to float
```
On AMDGPU this becomes a single `v_cvt_f32_i32` instead of the generic
i64 to f32 expansion.
Note: X86 marks i16 sint_to_fp as Custom, but SSE only supports
conversion from
i32, so the custom lowering sign-extends i16 back to i32. Mark i16 as
undesirable to avoid a redundant movswl.
[clang][CIR] Add tests for SVE MINV intrinsics (#230081)
adds CIR tests for svminv (plain SVE) intrinsics.
Moves and adapts sve-intrinsics/acle_sve_minv.c to sve/minv.c.
Part of #223963
[AArch64] Mark masked load/store data types that get legalized to scalars illegal (#228122)
Masked load/stores cannot handle data types that get scalarized during
type legalization. This PR marks them illegal in AArch64 TTI so they get
expanded before reaching SDAG.
Fixes e.g. the crash:
```llvm
; llc -mtriple=aarch64 -mattr=+sve crash.ll
define <1 x half> @masked_load_v1f16(ptr %p, <1 x i1> %mask) vscale_range(2,2) {
%load = call <1 x half> @llvm.masked.load.v1f16.p0(ptr %p, <1 x i1> %mask, <1 x half> poison)
ret <1 x half> %load
}
```
