CodeGen: Drop inert -fp-contract flags from tests
-fp-contract is nearly dead, but referenced in tests. These have
no output change with or without the flag now.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[AArch64] Combine scalar_to_vector(C) -> buildvector (#220948)
This helps keep a single canonical form of a constant vector, helping a
number of the existing buildvector combines trigger.
[CoroSplit] Don't go sub-word for suspend index (#221374)
Currently, `CoroFrame` selects the coroutine suspend index type
(`SwitchIndexType`) using `Type::getIntNTy(Context, IndexBits)`. For
coroutines with fewer than 256 suspend points, this produces
sub-byte/sub-word integer types (such as `i1`, `i2`, or `i8`) in the
coroutine frame struct.
On targets whose `DataLayout` specifies `-n32` (no legal sub-32-bit
integer types), storing and loading sub-word integer fields in memory
forces the backend to emit read-modify-write sequences on every
coroutine suspend and zero-extension masks on every resume.
Use `DataLayout::getSmallestLegalIntType(Context, IndexBits)` when
selecting `SwitchIndexType` so that targets respect their legal native
integer width.
[X86] X86CompressEVEX - Fold AVX-512 sign-mask compares to MOVMSK (#216884)
## Summary
Fold AVX-512 complementary signed sign-test masks to the corresponding
MOVMSK instructions in `X86CompressEVEX`.
AVX-512 lowers comparisons such as:
- `X >= 0`
- `X > -1`
to `VPCMP*` producing a k-mask followed by `KMOV`.
For 128-bit and 256-bit B/D/Q forms, these masks can instead be obtained
from the source sign bits using:
- `VPCMPB` -> `VPMOVMSKB`
- `VPCMPD` -> `VMOVMSKPS`
[43 lines not shown]
[ARM] Handle debug instructions in LowOverheadLoop (#221699)
The last instruction in the block might be a debug instruction, in which
case we should get the last non-debug instruction.
Fixes #221633.
[AMDGPU] Use AMDGPUMnemonicAlias in DS_Real_gfx13 (NFC) (#217912)
Emit an AMDGPUMnemonicAlias in DS_Real_gfx13 so it carries the
PredicateControl fields.
NFC: 46 gfx13 aliases gain the PredicateControl fields at their
defaults; Predicates is unchanged on all of them.
[AArch64] Correct the throughput of reductions on Neoverse V3ae (#221151)
This tightens up the throughput of sve saddv, sminv, smaxv, along with
the unsigned variants. They were a bit high due to the use of too many
micro ops.
[mlir][llvm] Gate unhandled function metadata warning
Emit the warning for unhandled function metadata only when expensive
warnings are requested, matching how the flag is already used for
instruction metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[ORC] Add SymbolNameSpec, hoist name mangling into Shared.
Introduce two Shared, ExecutionSession-independent pieces:
- SymbolNameSpec (Shared/SymbolNameSpec.h): a StringRef paired with a
SymbolNameKind (Verbatim, Linker, IR, C) describing the naming level
a symbol name is expressed in. A lightweight, dependency-free value
type, so it can describe interface-name constants in Shared headers.
- Mangler (Shared/Mangler.h, Shared/Mangler.cpp): the linker
name-mangling logic factored out of MangleAndInterner. It holds only
a ManglingMode (no ExecutionSession or SymbolStringPool), so names
can be mangled on the executor side as well as the controller
side. It exposes withMangledNameDo (which avoids allocating when
mangling is a no-op) and a mangledCopy convenience, both dispatching
on the spec's SymbolNameKind.
MangleAndInterner becomes a thin wrapper: it derives a ManglingMode
(from an ABIName / ManglingMode / DataLayout), holds a Mangler, and
[6 lines not shown]
[APInt][NFC] Add branch hints to APInt single-word (#219437)
The fast-path for APInt is generally done with for "single word"
integers, i.e. ones that can be represented in a single 64-bit word.
This commit adds branch hints to the single-word branching to help
indicate to the compilers prioritize branch-prediction towards the
fast-path. The trade-off of this is that the >64-bit APInt case will be
slower.
---------
Signed-off-by: Steffen Holst Larsen <sholstla at amd.com>
[Support] [CMake] Only pass -Wno-c99-extensions to Clang, not GCC (#221822)
GCC doesn't support this option, only Clang.
GCC doesn't warn if an unrecognized `-Wno-<foo>` option is passed, but
if GCC produces other warnings, it also adds a note about `-Wno-<foo>`
not being recognized, and that it may have been intended to silence the
warning that was produced.
[CMake] Move -Wno-unused-but-set-parameter from MLIR up to LLVM (#221824)
This warning is known to have false positives with GCC versions before
14.
Since 3aec6a40bb4e49f9ea181ac5c949b6c7c20a5465, this warning appears
when building LLVMSupport as well - thus move the disabling of the
warning up from MLIR to all of LLVM.
In HandleLLVMOptions, the common procedure is to not check for whether
the options are supported or not, but to just hardcode the version
ranges where the options are supported, per compiler.
This option (and -Wunused-but-set-parameter) is available in GCC since
long before our minimum required version.
[X86] Fold atomic loads into compares (#221290)
A compare of an atomic load kept the load in a register (mov + cmp/test)
where a compare of a regular load folds the load into the compare's
memory operand. The folded form performs a single full-width load, and
on x86 an ordinary load already provides every ordering up to seq_cst,
so folding is legal for any non-extending atomic load.
Add isel patterns folding an atomic load into CMP (mi/mr/rm forms across
i8/i16/i32/i64). Compare-to-zero already reaches these via EmitTest;
nonzero and register compares are emitted as a flag-only X86ISD::SUB (to
enable CSE with a real subtract) whose load the peephole then folds. The
peephole cannot move an ordered access, so those never folded for atomic
loads. Emit X86ISD::CMP instead when an operand is a foldable atomic
load: it has no non-atomic sibling to CSE with anyway.
x86lint reports 5854 "load foldable into compare" sites in a release
build of Firefox's libxul.so; sampling shows the majority are
atomic-load compares of the kind this folds -- mozilla::Atomic and
[5 lines not shown]
[UpdateTestChecks][X86] Allow --no_x86_scrub_sp to disable folded stack spill/restore stack math scrubing (#221788)
We were handling cases to show general load/store stack math, but had
missed the r/w/rmw folded cases
[LLVM][NVPTX] Add Rubin extensions for G2S Tensor intrinsics (#220029)
This change adds support for the Rubin-extended Global-to-Shared tensor
copy intrinsics for both the Cluster and CTA variants:
1. 32-bit multicast mask support: the G2S Cluster intrinsics are now
overloaded on the multicast mask type (i16/i32); an i32 mask lowers to
`.multicast::cluster::32b`.
2. Data-validity reporting through the mbarrier: all G2S Cluster and CTA
intrinsics, including the property-override forms, take a trailing `i32
%validate_pattern` immediate (range [0, 6)) that selects a
`.mbarrier::report::validity` qualifier. It defaults to 0 (disabled).
3. Override tensor map properties: the G2S Cluster and CTA intrinsics
now support `.override::global_address`, optionally followed by
`.override::global_dim` / `.override::global_dim_stride`, across all
load modes. The dim/stride overrides apply to tile mode only; im2col and
gather4 support the global-address override only.
The new `validate_pattern` argument changes the IR signatures of all G2S
[3 lines not shown]
[GlobalISel] Lower bytecast in IRTranslator
Add translateByteCast and move byte-specific cast lowering out of
translateBitCast. Byte-pointer crossings use G_INTTOPTR/G_PTRTOINT; other
byte casts reuse the existing G_BITCAST/COPY paths.
Co-authored-by: Cursor <cursoragent at cursor.com>
[Bitcode] Add ByteCast encoding and decoding
Add CAST_BYTECAST to the bitcode cast opcode table and wire it through the
reader and writer so bytecast instructions round-trip in bitcode.
Co-authored-by: Cursor <cursoragent at cursor.com>
[LLVM-C] Add LLVMBuildByteCast
Expose a C API builder for the bytecast instruction, mirroring the existing
LLVMBuildBitCast entry point, and teach llvm-c-test --echo to clone bytecast
so the new entry point is exercised by a round-trip test.
Co-authored-by: Cursor <cursoragent at cursor.com>
[IR] Add ByteCast opcode and ByteCastInst
Introduce a dedicated bytecast instruction for casts involving byte types.
Split byte cast validation out of bitcast, add ByteCastInst, constant folding,
and verifier support. Renumber subsequent opcodes to keep Instruction.def unique.
Add LLVMByteCast to the C API opcode enum for Core.cpp mapping tables, minimal
codegen stubs, assembler/llubi support, and bitcast-to-bytecast fallback in the
parser so existing IR text continues to assemble.
Co-authored-by: Cursor <cursoragent at cursor.com>
[GlobalISel] Lower bytecast in IRTranslator
Add translateByteCast and move byte-specific cast lowering out of
translateBitCast. Byte-pointer crossings use G_INTTOPTR/G_PTRTOINT; other
byte casts reuse the existing G_BITCAST/COPY paths.
Co-authored-by: Cursor <cursoragent at cursor.com>
