[mlir][ArmSVE] Implement `ConvertToLLVMPatternInterface`
ArmSVE could only be lowered to LLVM through
`convert-vector-to-llvm{enable-arm-sve}`. This PR implements the
`ConvertToLLVMPatternInterface` for the ArmSVE dialect so that `convert-to-llvm` lowers ArmSVE operations as well.
Since there are `arm_sve` dialect lowering patterns (`vector.create_mask`) that lower from vector dialect, an `add-lowering` pass option was also added to `convert-to-llvm` that allows loading a dialect and its conversion patterns, even if the dialect is not part of the input.
Part of a greater work in trying to make the VectorToLLVM pass not require linking every target dialect it can lower to.
[Offload][L0] Swap Level Zero NAME/PRODUCT_NAME (#235817)
The Level Zero plugin reports `OL_DEVICE_INFO_NAME` and
`OL_DEVICE_INFO_PRODUCT_NAME` backwards from how other plugins report
these values, e.g.:
|Backend|NAME|PRODUCT_NAME|
|-|-|-|
|Level Zero|Intel(R) Arc(TM) B580 Graphics|Intel GPU Xe HP|
|AMDGPU|gfx1201|AMD Radeon RX 9070 XT|
|CUDA|NVIDIA GeForce RTX 3060|NVIDIA GeForce RTX 3060|
Change the Level Zero plugin to report the product name through
PRODUCT_NAME and the architecture string through NAME (mirroring
AMDGPU), clarify the `NAME` description in Device.td, and change libsycl
to use `PRODUCT_NAME` for `sycl::info::device::name` so it still reports
the product name.
[LLVM][CodeGen][SVE] Add lowering for bfloat strict-fp arithmetic operations. (#230470)
This extends coverage to match the half, float and double support.
[flang] Cache in flang-licm which locations a loop may modify
Hoisting a load asks the alias analysis whether the loop may modify each
location the load reads, and the answer depends only on the loop and the
location. The loads of a loop read the same few locations over and over, so
flang-licm asked the same question many times: on a large OpenACC region 97%
of the queries of a run repeated an earlier one, and with alias analysis
implementations that are expensive to query the pass took minutes.
Cache the answers of a run, keyed by the loop and the location, in a small
LoopModifiesLocationCache that the pass owns. It is valid for the whole run
because the pass only hoists operations that have no effects or only read: it
never moves a write, so a location that is not modified in a loop does not
become modified in it. Only whether the location may be modified is used, as
before.
-debug-only=flang-licm-cache prints, for each run, how many queries were
answered from the cache and how many asked the alias analysis.
[Clang] Don't skip lambda constraints when substituting type constraints (#236425)
This follows up to https://github.com/llvm/llvm-project/pull/207966 in order to transforms type
constraints with a ForConstraintSubstitution flag.
Assisted-by: Codex
[mlir][x86] Make AMX nanokernel independent of result write (#236699)
The pattern relied on the presence of a
`vector.store`/`vector.transfer_write` of the accumulation result back
to memory. This patch reworks the handling in the following way:
- Enable direct tile-stores to the output memref for the VNNI-packed,
_zero-initialized_ case as well.
- All remaining paths require that the tile register contents are moved
through a temporary buffer. However, the shape of that buffer only
matters if we still need to accumulate the results onto the initial
value in memory – we can derive it from the source buffer's shape, as
before. Otherwise, we can just pick a shape to hold all output tiles. On
the online-packing path, we ensure that we have two neighboring tiles in
the N-dimension to allow shuffling.
Assisted-by: Copilot
[BasicAA] Account for wrapping scaled indices in `computeConstantOffsetHeuristic` (#236719)
Similarly to what already being done in `computeMinAbsVarOffset` since
1b7ef6a, the constant offset heuristic previously assumed `MinDiff *
Scale` to be a byte distance that cannot wrap. That does not hold if
`Scale*ext(V0)` or `Scale*ext(V1)` may wrap in the index width, thus
ensure this is checked explicitly.
Fixes: https://github.com/llvm/llvm-project/issues/236396.
[OpenMP] Switch the underlying type of VersionRange to unsigned (#236089)
It was int before. The goal is to unify Version and VersionRange,
and Version stores the value as unsigned.
[CIR] Append the fenv interface to complex FP op traits
Use let append traits instead of extending the base trait list.
Assisted-by: Claude Code (Opus 5.5).
[TableGen] Keep dumps when instantiating defs from a multiclass (#236438)
The Record copy constructor copied assertions but not dumps, so a dump
in the body of a def inside a multiclass, or inherited by it from a
superclass, was silently dropped from the defs instantiated by defm.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[CIR] Store nsw/nuw as an OverflowFlags bit enum and split out saturated add/sub
cir.add, cir.sub and cir.mul carried nsw and nuw as two separate unit
properties, and lowering mapped them to LLVM with an early return that
picked nuw over nsw, so an op with both flags lowered to `overflow<nuw>`
and lost nsw. cir.add and cir.sub also carried a `sat` flag that changed
the operation's result rather than adding a poison condition.
Store the flags as one OverflowFlags bit-enum property whose layout
matches mlir::LLVM::IntegerOverflowFlags, so lowering casts it directly
and keeps both flags. The property uses the default EnumProp syntax,
printed as `nuw`, `nsw` or `nuw|nsw`, so no custom parser or printer is
needed. cir.mul now shares the add/sub base class and so becomes Pure.
Saturating arithmetic moves to new cir.add.sat and cir.sub.sat ops, which
lower to the signed or unsigned saturation intrinsics by element type.
That leaves cir::OverflowBehavior only describing nsw/nuw, so the builder
takes cir::OverflowFlags instead.
[6 lines not shown]
[AMDGPU] Align FPTRUNC_ROUND_F32_F64_PSEUDO VGPR source operand (#236195)
The 64-bit VGPR source of FPTRUNC_ROUND_F32_F64_PSEUDO used the plain
VReg_64 class, while SIModeRegister lowers the pseudo to V_CVT_F32_F64_e32,
whose source must be even-aligned on subtargets that require aligned VGPRs.
Use VReg_64_AlignTarget so the pseudo already carries the alignment of the
instruction it becomes.
Co-Authored-By: Claude <noreply at anthropic.com>
---------
Co-authored-by: Claude <noreply at anthropic.com>
Co-authored-by: Jay Foad <jay.foad at gmail.com>
[AMDGPU] Enable SlotIndexes compaction by default
Add TargetRegisterInfo::shouldCompactSlotIndexes() so targets can opt
in to dropping dead SlotIndexes entries before greedy allocation, one
target at a time. An explicit -greedy-compact-slot-indexes still
overrides the target. Enable it for AMDGPU, where the leftover entries
skew live range priorities enough to cause heavy spilling.
[Clang][CodeGen] Don't crash on a struct inline asm output in a mask register
A struct register output with "=k" crashes when x86's
'adjustInlineAsmType' returned '<N x i1>' for "k". The conversion chain
then bitcast that vector to the struct type, which is invalid IR.
Fix this by casting the vector to an integer that's the same size as the
vector, then truncate it to the destination's size.
Fixes: 8d6e82d501cd ("[X86] Use vXi1 for `k` constraint in inline asm (#77733)")
Assisted-by: Claude Opus 5.5
[lldb][test] Force lldb-server use in fail_watchpoint_command.test (#236711)
Turns out that our Windows on Arm bot hasn't been testing lldb-server
this entire time. We set LLDB_USE_LLDB_SERVER, but to test it you have
to set LLDB_TEST_USE_LLDB_SERVER.
And I don't fancy setting that and waiting a week to find out what
breaks.
So we will update in the future bot but for now, make this test always
use lldb-server.
With in-process debug, you cannot set any watchpoints on Windows on Arm.
With lldb-server, you can set 1, at least you can on the bot's hardware.
Fixes #235985. Relates to #232235.
[libc] Use a syscall wrapper for semop
This PR move semop behind the centralized syscall wrappers, following libc/docs/dev/syscall_wrapper_refactor.md.
[libc] Use a syscall wrapper for semget
This PR move semget behind the centralized syscall wrappers, following libc/docs/dev/syscall_wrapper_refactor.md.
[libc] Use a syscall wrapper for semctl
This PR move semctl behind the centralized syscall wrappers, following libc/docs/dev/syscall_wrapper_refactor.md.
[OpenMP][offload] Apply the low-trip rule to no-loop kernels
A no-loop kernel gets one thread per iteration. Like an SPMD kernel, it
now gets fewer threads per block, down to
LIBOMPTARGET_MIN_THREADS_FOR_LOW_TRIP_COUNT, and more blocks if its loop
is too short to fill the default number of blocks. A thread_limit is
never changed. For example, a no-loop kernel of 512 iterations gets
16x32 instead of 2x256 on MI210. For BLAS level-3 kernels of that shape,
each iteration a long serial loop, that is 3-6x faster.
This is an upstream port of the corresponding downstream.
Claude assisted with this patch.
[libc] Use a syscall wrapper for semop
This PR move semop behind the centralized syscall wrappers, following libc/docs/dev/syscall_wrapper_refactor.md.
[libc] Use a syscall wrapper for semget
This PR move semget behind the centralized syscall wrappers, following libc/docs/dev/syscall_wrapper_refactor.md.
