[LoongArch] Load scalars into vector lane 0 with FLD/VLDREPL
Inserting a loaded integer into lane 0 of an undefined vector, which is
also how sub-128-bit vector loads are legalized, was lowered to a scalar
load plus [X]VINSGR2VR. VINSGR2VR merges into its destination, so it
carries a false dependency on the previous value of that register, and
it needs a GPR-to-vector move.
Use FLD.{S/D} for 32/64-bit elements and [X]VLDREPL.{B/H} for 8/16-bit
elements instead. They write the whole register in one instruction.
When the register allocator reuses one vector register for several such
loads, the false dependency serializes otherwise independent chains. In
the GSM autocorrelation loop (9 x i64 multiply-accumulate over i16 taps)
on LA664:
VF2: 0.953s -> 0.724s
VF4: 1.446s -> 0.856s
[LTO] Prevent crashes if a diagnostic is emitted while compiling with `--lto-partitions`
Claude found this bug for me while working on the earlier path.
I checked it and I think this fix makes sense. Without the fix in Config.h, the test crashes with `bad_function_call`.
There were other ways to fix it but I think this makes the most sense as it fixes all users.
I only check the full LTO path as all thinLTO paths set a diagnostic handler, and I don't think adding a unit test just to check
the thinLTO path without a diag handler was worth it.
Assisted-By: Claude Sonnet 5.5
[SPIR-V] Lower llvm.expect.with.probability to OpExpectKHR (#237712)
The hint was silently dropped with SPV_KHR_expect_assume
OpExpectKHR has no probability operand, so lower to spv_expect with the
first two operands
[mlir][ODS] Add EnumInfo accessors for the bit enum separator (NFC) (#237962)
The generators read the bit enum `separator` field through
`getValueAsString` in nine places, and EnumsGen and OpFormatGen each had
a copy of the `StringSwitch` that maps it to the `AsmParser` method that
parses it. This adds `EnumInfo::getBitEnumSeparator`,
`getBitEnumSeparatorKind` and `getBitEnumSeparatorParser` and uses them
at every site; the comma checks now compare the kind. Generated code is
unchanged for the in-tree enums. A separator other than `|` or `,` used
to produce C++ that did not compile and is now a TableGen error. Split
out of llvm/llvm-project#237149, which adds a space-only separator on
top of this.
Assisted-by: Claude Code (Claude Opus 5.5).
Add explicit map for lastprivate value
Add explicit map for lastprivate scalar value to allow copy back due to
missing implicit map (known issue #49496).
[InstCombine] Canonicalise Vector Shuffle that clears upper bits
Currently, shuffle vectors that are equivalent to preserving lane 0 and clearing the uppers lanes of a vector result in different shuffle vectors and masks
Recognise and canonicalise the shuffleVectors to the same form allowing for easier recognition in the backend
[AArch64][NFC] Fix fixed-width gather/scatter expand tests (#238013)
The tests were ensuring that the IR didn't contain a call to
`@llvm.masked.gather.v4i32(`, etc. However, in reality the CHECK-NOT
lines always pass even if we do generate a masked gather call because
the intrinsic name is actually llvm.masked.gather.v4i32.v4p0, etc.
[AsmPrinter] Emit null pointer vector splats as zero bytes (#235980)
Emitting a global such as `@x = global <2 x ptr> zeroinitializer` with
`llc -data-sections -filetype=obj` asserts on 64-bit targets.
`ConstantPointerNull` can represent a vector splat since #195486, but
AsmPrinter passes the entire allocation size to
`MCStreamer::emitIntValue`, which accepts at most eight bytes.
Route pointer-vector splats through `emitGlobalConstantVector`, matching
the existing handling of integer and floating-point splats. Its
zero-value path emits the full allocation with `emitZeros`. Keep scalar
pointers on the existing `emitIntValue` path.
Fixes #235977
Assisted-by: OpenAI Codex
[MachineLICM] Fix incorrect hoisting from a speculation cache shared between loops (#234138)
Since 5ec9699, MachineLICM may check a block against both an outer loop
and its subloops, but whether the block is guaranteed to execute was
cached once per block. A result computed for an inner loop could then be
reused for the outer loop, so a load was hoisted above the guard that
made it safe. This causes wrong code in #233340, and in the reverse case
some loads were no longer hoisted to the inner preheader (the existing
test changes).
Use `SmallDenseMap` to cache the result per loop.
Fixes #233340
Assisted-by: Claude Opus 5.5
[CMake][NFC] Add llvm_tablegen and use it instead of tablegen(LLVM ...) (#235796)
Other subprojects invoke their TableGen through a project-specific
wrapper (clang_tablegen, mlir_tablegen, lldb_tablegen). Add
llvm_tablegen for llvm-tblgen and replace every tablegen(LLVM ...) call
with it.
The llvm-tblgen-specific logic in tablegen() moves into llvm_tablegen:
the DAGISel/GlobalISel coverage and -omit-comments flags, and the extra
outputs of -gen-register-info, which tablegen() now accepts generically
through a new EXTRA_OUTPUTS argument.
My motivation is general cleanliness but also as a use-case to add the
`EXTRA_OUTPUTS` cmake option to the `tablegen` command
[mlir][transform] Fix crash on empty type converter region (#237717)
The default type converter region of transform.apply_conversion_patterns is
declared MaxSizedRegion<1>, so it may have no blocks. The verifier called
front() on it unconditionally, which hits an assertion on an empty region.
Check for an empty region first and report the existing "expected exactly one op
in default type converter region" error. Add tests for both an empty block
(custom syntax) and a region without blocks, which is only expressible in the
generic form.
Assisted-by: Claude
Fixes #234636
[Flang] KIND De-templatization (#216960)
Avoid template instantiation based on a type's kind, but pass the kind
as runtime value. The cross-product of template instantiations
(including nested `visit` calls of methods that are already per-kind)
leads to super-linear compile-time bloat.
For more details, see the RFC:
https://discourse.llvm.org/t/rfc-the-cost-of-templates-when-building-flang/91197.
### Implementation notes
* The conversion is meant to be mostly mechanical, wherever possible.
That is, the following rules were applied:
1. If a function implementation is independent of KIND, just remove the
template argument
2. If another object is passed as argument (or exists as a field of the
class) with the same KIND, get KIND from its `.kind()` method, and store
it in a variable `const int kind`.
[74 lines not shown]
[OpenMP][MLIR] Add virtual collapse for constant static worksharing (#235822)
This patch adds virtual collapse for OpenMP loop nests in
`OpenMPIRBuilder` and enables it in Flang through MLIR-to-LLVM IR
translation. It retains the physical nest and decodes the start of each
assigned logical range once.
This is the first patch in a proposed series, introducing the basic
functionality. Virtual collapse is enabled by default when these
conditions hold:
- Host standalone worksharing with `collapse(2)` and directly nested
loops.
- Constant normalized trip counts.
- Static or default scheduling without a chunk size.
- No `ordered` or `linear` clauses.
Other cases retain the existing lowering. Follow-up patches will extend
the supported cases. Flang can opt out with `-mmlir
[9 lines not shown]
[compiler-rt][Darwin] Add SANITIZER_ macros for individual Darwin OSes (#237765)
`SANITIZER_IPHONEOS` is now exactly `TARGET_OS_IOS` (iOS only), while
`SANITIZER_IOS` (derived from `TARGET_OS_IPHONE`) remains "iOS-like".
Also add `SANITIZER_VISIONOS` for `TARGET_OS_VISION`.
Update the shared macOS-aligned mapping computation to use the new
macro.
Assisted-By: Claude
rdar://191534752
[Offload] Add multi-device testing (#233387)
This PR changes the way our fixtures group devices - instead of grouping
each device separately, we group compatible devices into a vector.
`OFFLOAD_TESTS_INSTANTIATE_DEVICE_FIXTURE` still has the same behavior -
a test is instantiated for each device separately, but
`OFFLOAD_TESTS_INSTANTIATE_MULTI_DEVICE_FIXTURE` can be used so a test
is instantiated with many `std::vector<ol_device_handle_t> Devices`. I
kept `ol_device_handle_t Device`, so single device tests stay simple.
For now I added simple memcpy tests, later we can add more complex tests
with kernels that communicate across devices. Such tests might require
additional APIs to express p2p connection between devices though.
Assisted with: claude
[CodeGen] Lower differences from __ImageBase to image relative references on MinGW (#237118)
`TargetLoweringObjectFileCOFF::lowerRelativeReference` returned early
for Cygwin and MinGW, so a constant of the form `ptrtoint(X) -
ptrtoint(__ImageBase)` reached the object writer as a plain symbol
difference and failed with `symbol '__ImageBase' can not be undefined in
a subtraction expression`. MSVC targets lower the same expression to
`X at IMGREL`, an `IMAGE_REL_AMD64_ADDR32NB` relocation
The early return looks inherited and not decided. The transform was
added in dee105772c5b (D2523) behind `isTargetWindows()`, which then
meant MSVC only. 98117a47e649 (D47783) moved it into the shared COFF
code and wrote the MinGW and Cygwin exclusion explicitly, and
2f07c2e9d9cc turned it into `isOSCygMing()`. GNU ld has defined
`__ImageBase` since 2008 and lld defines it too, and the object file
only needs the relocation, so there is no reason to treat the
environment differently
The MinGW cases are added to `llvm/test/MC/COFF/ir-to-imgrel.ll` and
[7 lines not shown]
