Reland "[AMDGPU] Fix llvm.amdgcn.ballot with return width != wavefront size" (#213635)
Reverts https://github.com/llvm/llvm-project/pull/212628
This relands #211493, which was reverted because
ockl_dm_alloc/ockl_dm_dealloc in device-libs emit an i32 ballot on
wave64, which GlobalISel cannot select (one bit per lane doesn't fit).
[#212813](https://github.com/llvm/llvm-project/pull/212813) widens the
clang ballot builtins to the wavefront size so a narrower-than-wave
ballot is no longer emitted, fixing the root cause.
[LAA] Properly report strided access preventing store-to-load forwarding (#208791)
Original test by @fhahn in https://github.com/llvm/llvm-project/pull/191867, further reduced here.
Before this change LAA results in
> maximum safe store-load forward width of 32|0 bits
for `i32` accesses, effectively meaning that only `VF == 1` is safe, yet
not explicitly returning `false` from `couldPreventStoreLoadForward`.
This PR fixes that.
[libc++] Fold deque iterator benchmarks into algorithm benchmarks (#212279)
The deque::iterator benchmarks were not truly about deque::iterator, but
about specialized algorithm implementations we have for segmented
iterators. This patch handles them as such, like we do for other
specialized algorithms like vector<bool>.
[flang] - Call _FortranAAssignSimple instead of _FortranAAssign for intrinsic-type array assignments.
This patch adds support for calling _FortranAAssignSimple, a faster-path for array assignments.
`_FortranAAssignSimple` is called when ALL the following conditions are true:
1. Intrinsic element type (not derived type)
2. Matching ranks (no scalar-to-array broadcasting)
3. Non-volatile
4. Not polymorphic
5. Not explicit-length character
6. Not temporary LHS
Otherwise, uses `_FortranAAssign` (or specialized variants like `_FortranAAssignPolymorphic`, `_FortranAAssignExplicitLengthCharacter`).
This is a (perhaps final) part of the fix for https://github.com/llvm/llvm-project/issues/203915
[flang-rt] - Lightweight runtime assignment function (AssignSimple) for intrinsic-type assignments.
This PR introduces a lightweight assignment runtime path (`_FortranAAssignSimple`) for intrinsic-type arrays
with the goal of reducing compile-time overhead seen primarily in the form of severly increased time taken by LTO.
This PR includes only the changes to the runtime (flang-rt) and as such just with this PR compile-time improvements
will not be visible.
**Problem**
When compiling Fortran code with OpenMP GPU offload and `firstprivate(allocatable_array)`, LLVM's Attributor creates excessive abstract attributes analyzing complex runtime assignment machinery:
**Symptom:**
- **Test case:** 8-element allocatable integer array with `firstprivate` clause
- **Compile time:** 24.97s (vs 0.78s for `private` - **32x slower**)
- **Root cause:** LLVM Attributor analyzing complex Fortran runtime functions
**Why this happens:**
1. `firstprivate` requires copying arrays from host to device
[43 lines not shown]
[libcxx][test] Fix a few test conditions (#213512)
libcxx/vendor/apple/{availability-with-pedantic-errors.compile.pass.cpp,disable-availability.sh.cpp}
both require the platform to support availability markup but weren't
labeled as such. std/time/time.hash/time.hash_enabled.pass.cpp has some
leap second tests that are missing the guards <chrono> uses to include
<__chrono/leap_second.h> and will fail if experimental tzdb is set but
the other ones aren't.
GlobalISel: Add type size guards in tryCombineMergeLike
Bug in LegalizationArtifactCombiner when:
DstSize < UnmergeSrcSize case can create unmerge with one element.
DstSize > UnmergeSrcSize case can end up attempting to create merge
with one source element and hits assert(TmpVec.size() > 1).
AMDGPU/GlobalISel: Precommit tests for upcoming bug fix
Here we have:
artifact combiner creating one element unmerge and
unmerge lowering of FP source using FP type for bit twiddling.
[dyndbg][llvm][ELF] Add ELF section type for dynamic debugging (#208803)
Add ELF section type `SHT_LLVM_DYNDBG_ELF` for embedding the "inner"
unoptimized dynamic debugging ELF object within the "outer" optimized
ELF object.
RFC: https://discourse.llvm.org/t/90113
[benchmark] Fix -Wunused-but-set-variable warning in basic_test (#213637)
Fix a build error when building benchmark unit tests with modern
GCC/Clang compilers under strict warning options (-Werror).
In `test/basic_test.cc`, the variable `sum` in `BM_OneTemplateFunc` was
assigned but never read, triggering `-Wunused-but-set-variable`.
Silenced the warning by marking sum with `[[maybe_unused]]`, preserving
the benchmark function's logic while ensuring clean build output.
### Description
Fixes a build error when compiling `third-party/benchmark` unit tests
with modern GCC/Clang compilers under `-Werror`.
In `test/basic_text.cc`, the variable `sum` in `BM_OneTemplatcFunc` was
assigned but never read, triggering `-Wunused-but-set-variable`.
### Solution
[13 lines not shown]
[lldb][test] Give a directly created lldb-dap session the init commands (#213553)
create_debug_adapter passes the commands that carry the configuration
the test suite was invoked with, and a test that builds a
DebugAdapterServer itself got none of them, so that session ran
unconfigured. It matters wherever the suite configures the debugger
through settings, such as pointing a platform at the runtime it
launches.
[libc++][CI] run-buildbot and libcxx-lit need a way to pass the paths to cmake and ninja (#213511)
macOS/Xcode don't have cmake or ninja anywhere in a default PATH, so
run-buildbot and libcxx-lit fail unless you do some PATH surgery before
running them. Allow passing them as environment variables instead, so
run-buildbot can be invoked as `CMAKE=$(xcrun --find cmake)
NINJA=$(xcrun --find ninja) CC=$(xcrun --find clang) CXX=$(xcrun --find
clang++) run-buildbot` on macOS. Allow cmake to be passed to libcxx-lit
in a similar fashion.
[offload][test] Instruct clang-format to not reflow comments (#213696)
Without this, clang-format attempts to reflow check lines in tests,
resulting in broken tests.
[OpenMPIRBuilder] Handle empty blocks in restoreIPandDebugLoc (#212535)
`restoreIPandDebugLoc` previously only recovered a debug location when
the insertion block was non-empty, using its last instruction. For an
empty block it left the current debug location unchanged so instructions
emitted afterwards could have wrong debug location.
This PR enhance `restoreIPandDebugLoc` to also handle the empty-block
case: when the insertion point is at the end of an empty block,
synthesize a location scoped to the parent function's subprogram
provided the function has debug metadata.
This helps us get a valid debug location when we switch to `CodeGenIP`
in `emitOffloadingArrays` even when `CodeGenIP` is pointing to an empty
`BB`.
Fixes https://github.com/llvm/llvm-project/issues/212488
Co-authored-by: Cursor <cursoragent at cursor.com>
[libsycl][Unit] Tighten mock liboffload API argument checks (#211035)
Prior to this patch, unit test default actions for mock liboffload
returned errors when receiving invalid arguments. There are only a few
valid scenarios where the runtime should expect and handle error codes
returned by liboffload (for example, checking whether a pointer is USM
or not). In most cases, the calls should not be made with invalid
arguments by libsycl at all, and especially not with the type of invalid
arguments that we can check for in default actions, like nullptrs or
invalid size values.
This patch changes such default action checks to trigger test failures
directly instead of mocking error codes.
[X86] vector-compare-all_of/any_of - updates tests to use reduction intrinsics (#213683)
These more closely match middle-end IR and currently expand to the
existing shuffle/bitcast patterns.
[offload] Add `dlwrap::loaded` function to check for optional symbols (#210737)
This PR adds a new template into `dlwrap` namespace that can be used to
check if a symbol was correctly loaded. It adds and easy way to see if
version of shared object in a system has required capability. We could
use it to improve prefetch in CUDA backend as noted
[here](https://github.com/llvm/llvm-project/blob/main/offload/plugins-nextgen/cuda/src/rtl.cpp#L912)
without breaking compatibility with older platforms using CUDA older
than 13.
In the case of prefetch, the new `dlwrap` API could be used like:
```cpp
bool BatchedPrefetchAvailable = dlwrap::loaded<cuMemPrefetchBatchAsync>();
if (BatchedPrefetchAvailable)
cuMemPrefetchAsync(....)
else
// Current implementation
```
[5 lines not shown]
[offload] Remove `omptarget` references from tests (#208205)
Make check lines more generic so that we can move and rename components
without breaking the tests
This is in preparation for splitting off parts of libomptarget into
libompaccsupport, which will be used by both OpenACC and OpenMP. Some
debug prints will be printed from `ompaccsupport` and not `omptarget`,
thus the need for this change.
[AMDGPU] Support partial and empty WWM pools for SGPR spills
SGPR lane spilling currently treats the WWM VGPR pool as all-or-nothing. This
can fail compilation when the requested pool cannot be formed, even though
scratch spilling or a smaller spillable pool could make progress.
This PR lets ordinary SGPR spills fall back to scratch when the pool is empty
and lets WWM register allocation use a nonempty partial pool. It keeps the
full-pool requirement for strict WWM/WQM and explicit spill-carrier
preallocation.
The no-pool fallback is recorded in SIMachineFunctionInfo so frame lowering can
provide enough emergency scavenging slots. The state is also serialized to
preserve the behavior across MIR round trips.
[offload] add nodiscard support to offload-tblgen and mark ol_errc_t (#209727)
Offload API functions may fail with error codes that shouldn't be
ignored. Most notably, if `olInit` fails and its error return value is
ignored, it is easy to use the library in an invalid uninitialized
state, which can and has caused confusion. In those cases, it may be
useful to have the ability to mark some API function with
`[[nodiscard]]`
This PR adds an optional `nodiscard` property to offload-tblgen's
`Function`, `Enum`, and `Struct`. If set, an `OL_NODISCARD` macro is
emitted, which expands to `[[nodiscard]]` on >=C++17 and >=C23, and to
nothing otherwise.
`nodiscard` is set for `ol_errc_t`, meaning every call to a function
that returns it will emit a compiler warning if the return value is
ignored and the TU is compiled on a supported language mode.
`libsycl` and `llvm-gpu-loader` still build cleanly and are unaffected
[5 lines not shown]
[OpenMP] Analyze the loop-body callback of the static-loop runtime entries (#211287)
Fixes #211132. Also removes the trigger for #198621, see below.
`AAKernelInfo` treats the loop body passed to the
`__kmpc_*_static_loop_*` entries as opaque and records an unknown
parallel region for it, per the TODO at the site. Consequently
`NestedParallelism` is true for any kernel whose parallel region
contains a device workshare loop, and `MayUseNestedParallelism` is
written to the kernel environment as 1 where it should be 0.
The callback is a direct function operand at the callsite, so resolve it
and consult its `AAKernelInfo`, exactly as the `__kmpc_parallel_60`
handling already does for its parallel-region operand a few lines away.
Only record an unknown region when it does not resolve, or does reach
parallel regions. The SPMD-izability half of the TODO is left alone.
Only flang lowers device workshare loops through these entries; clang
emits `__kmpc_for_static_init_4` plus an explicit loop, which the
[54 lines not shown]
