[Offload] Avoid shadowing of test variables in the unittest ASSERT macros (#243921)
ASSERT_SUCCESS, ASSERT_SUCCESS_OR_UNSUPPORTED, ASSERT_ERROR and
ASSERT_ANY_ERROR store the checked result in a local variable named Res.
This is a ticking bomb: Res is also the name the test code itself uses
for results, e.g. 'ol_result_t Res = olInit(nullptr)' in olInit.cpp and
Environment.cpp, and several helpers in Fixtures.hpp and Properties.hpp.
Passing such a variable to one of the macros, e.g. ASSERT_SUCCESS(Res),
expands to 'ol_result_t Res = Res'. The macro's local shadows the test's
variable, is initialized from itself and is then dereferenced, so the
test crashes instead of checking the result. It only works today because
nobody happened to pass a variable named Res to these macros yet.
Rename the macro local to Res_ so it unlikly to collide.
Assisted-by: Claude
[CGP] Vector LSR postcommit comments addressed (#240839)
Fixing up postcommit problems found in #226197
* Use CreatePtrAdd instead of CreateGEP
* Use TypeAllocSize of SourceElementType
* Add test for i24 indexing
[flang][OpenMP] Exclude user-defined reductions from assumed bounded recipe effects (#240814)
Currently, user-defined reduction recipes may have their memory effects
assumed to be bounded, for example through the presence of
a`task_reduction_assume_bounded_recipe_effects` attribute. However
user-defined reduction recipes may access storage beyond their operands,
violating the operation contracts of the
`*_reduction_assume_bounded_recipe_effects` attributes.
Add a new flag `hasUserDefinedReduction` tracking whether a construct
with a reduction has any user-defined reduction recipes.
I'm not aware of any other way of doing this, for example in the
existing `canAssumeBoundedRecipeEffects` helper functions a variable
symbol may be involved in multiple reductions.
AI-Assisted-by: Codex
[JITLink] Skip allocation for graphs with only empty sections (#239712)
An object whose only definitions are absolute symbols still has an
empty __text section on MachO. JITLinkerBase sent its graph to the
memory manager, and FinalizedAlloc asserted on the empty allocation.
Skip sections with no blocks, as BasicLayout does.
Assisted-by: Claude Code
Co-authored-by: Lang Hames <lhames at gmail.com>
[AMDGPU] Preserve noclobber after acquire operations from a different addrspace (#231855)
Follow-up to #219991 to address reviewer comments:
AMDGPUAnnotateUniformValues should annotate uniform global loads as
`!amdgpu.noclobber` if the memory they are reading is never clobbered (since
kernel launch) before the load is executed.
PR #219991 strengthened the logic to consider acquire operations before the
load as clobbering (because they cause stores from other threads to be observed
by the load without a data race).
This PR weakens this criterion to ignore acquire operations that don't affect
the global address space (because they don't affect what the load in question
may or may not see). This addresses the performance-relevant case of s_barriers
with LDS-only fences.
The logic is still overly pessimistic for one-as load acquires, they could also
be ignored when looking for clobbering operations.
Disclaimer: Claude gave review comments and suggestions that I addressed during
[4 lines not shown]
[LI] Test shrinkSubRangeToUses handling of early-clobber uses
An instruction could potentially read a subrange that also gets
redefined at the early-clobber slot. The test explicitely uses an
instruction with no register ties.
This makes the test a bit weird, because LLVM is supposed to guarantee
that the early-clobber register won't be used by any other input (unless
it's tied to the def).
See https://llvm.org/docs/LangRef.html#output-constraints
[PowerPC] Make `isa-v207-instructions` imply `isa-v206-instructions` (#233704)
Most `FeatureISA*` features imply former versions, with the exception of
`FeatureISA2_07` to `FeatureISA2_06`. There isn't a reason for this
because 2.07 should be backward compatible with 2.06 ([hwcap] sets 2.06
for all future ISA versions), and it wouldn't make sense to disable 2.06
instructions if 2.07 is enabled. Thus, make the impliciation explicit.
Note that this is effectively non-functional since 2.06 is already
implied as part of `P7InheritableFeatures`, which are picked up by all
future ISA versions.
[hwcap]: https://docs.kernel.org/arch/powerpc/elf_hwcaps.html
[VPlan] Mask all in-loop reduction links with the blend's edge mask (#243899)
When an in-loop reduction chain reaches the exit value through a phi
(converted to a VPBlendRecipe), the blend was folded to its non-phi
operand and each link was masked only by its own block mask. Links that
execute on paths where the phi selects the reduction phi instead, e.g.
a link in the loop header with the phi in the latch, were reduced
unmasked.
The chain only contributes on the blend's edge from the chain, so mask
all links with the blend's mask for that operand and place the
reductions in the blend's block, where the mask is defined.
PR: https://github.com/llvm/llvm-project/pull/243899
[IndVarSimplify][SCEV][NFC] Add tests for LFTR losing the constant max trip count
When LFTR of IndVars rewrites a loop exit like i >s C
to i + 1 == smax(start, C + 1),
SCEV's constant max backedge-taken count of the loop gets worse.
Add tests showing it.
Assisted-By: Claude
[IndVarSimplify] Use context to prove a narrow IV def non-negative
When IndVars widens an IV, it checks for each narrow def
(the IV and values computed from it) whether it is
non-negative on every iteration.
If it is, zext and sext of it are equal. Then when IndVars
idens the def's users, they can choose to use either
the sext or zext of the def.
Previously, we didn't use context to prove the non-negativity of def.
Now ask SCEV about the def non-negativity at its context -
the def instruction itself.
This allows SCEV to infer facts from the dominating guards and assumes.
This in turn allows eliminating unnecessary truncs/zexts. E.g. in
if (m > 0)
[9 lines not shown]
[IndVarSimplify][NFC] Add a negative test for zext elimination
Add a test for a zext of an nsw add of a sign-extended IV, where the
add may be negative.
Also rewrite a stale comment of an existing test.
Assisted-By: Claude
[LoopVectorize][NFC] Add tests for runtime checks with a widened IV index
In
if (start <u len && start - 32 <u len) // len >=s 0
for (int32_t i = start; i < n; i++) a[i] = a[i - 32];
i - 32 >=s 0, so the distance between a[i] and a[i - 32] is always 32.
If the loop has an i64 IV and i - 32 is computed in i64, it is
vectorized without runtime checks.
If i - 32 is computed in i32 and zero-extended, the vectorizer
emits an overflow check and an alias check.
Add tests for both forms.
Assisted-By: Claude
[SPIR-V] Type the capture_event_profiling_info value pointer as i8 (#230548)
The SPIR-V spec requires Value of OpCaptureEventProfilingInfo to point
to an 8-bit integer, but its pointee was inferred from other uses
Discussed in https://github.com/llvm/llvm-project/pull/229023
[AMDGPU] Describe MFMAs with a variant in MAIInstInfoTable (NFC)
- Add an MFMAVariant enum. Each MFMA pseudo gets its variant from
MAIInstInfoTable, a TableGen GenericTable.
- Look up an MFMA's co-execution info by its variant, as gfx1250 does
for WMMA, instead of matching every opcode.
- Give MFMAs without a co-execution model an empty window instead of
one that allows everything for 9 cycles, and have the hazard
recognizer start no window for them. The schedule does not change.
- Replace the is_dgemm and is_gfx940_xdl fields with the variant.
- Real MFMA opcodes get no variant, since nothing needs one.
- Strictly speaking this is not NFC, but no code path asks for the
co-execution info of a real opcode.
Assisted-by: Claude Opus 5.5
[SCEVExpander] Also apply 0/poison-free expansion to URem. (#243898)
Also apply SAfeUDivMode to expanding URem in SCEVExpander, which
triggers UB on zero or poison, similar to UDiv.
The code for expanding a safe divisor is moved to a helper.
PR: https://github.com/llvm/llvm-project/pull/243898
[LV] Don't fall back on legacy cost model for unknown replicates (#243844)
In VPReplicateRecipe::computeCost we currently call getLegacyCost if we
encounter an unknown opcode, however I think the cost model here is
sufficiently complete for us to mimic the behaviour of
getInstructionCost in LoopVectorize.cpp and simply return the cost of a
mul. I tried putting a llvm_unreachable in the default case and no test
failed and I also didn't hit the error when compiling the LLVM test
suite.
[X86] Fix f64 mask encoding in demanded-elts VPERMV to VPERMILPV narrowing (#238829)
SimplifyDemandedVectorEltsForTargetNode narrows a 256-bit X86ISD::VPERMV
whose upper elements are not demanded to a 128-bit X86ISD::VPERMILPV. It
passes the VPERMV mask through unchanged. VPERMV masks hold plain
element indices, but VPERMILPD selects with bit 1 of each mask element.
So for f64, index 1 decodes as element 0, and the narrowed permute then
folds to its input.
According to Claude (I left it here as it might be helpful):
The bad narrowing is older. It became reachable after 961ab7a36967
("[SDAG][X86] Uplift pseudo fmin/fmax from X86"), which routes the x86
fmin/fmax intrinsics through ISD::PSEUDO_FMAX. I found this by reading
the code. I did not confirm it with a build bisect.
Assisted-by: Claude (Anthropic)
[X86] Lower i16 partial reductions to PMADDWD on targets without VNNI (#240904)
Part of #223374. The first commit adds the tests (NFC, #240903); the
second commit is the change.
Without VNNI, an i16 × i16 → i32 partial reduction was expanded and only
partly folded back into PMADDWD. That came to 11 instructions for a
v4i32 accumulator on SSE2, and pmulhw/pmullw sequences for v8i32. This
marks signed i16 × i16 → i32 partial reductions as Custom (512-bit only
with BWI registers, and not when PMADDWD is slow) and lowers them to
`add(acc, pmaddwd(a, b))`. The partial reduction leaves the lane
grouping unspecified. A wrap case (all four inputs 0x8000) matches
VPDPWSSD.
Instruction counts for v4i32 / v8i32 / v16i32 accumulators:
- SSE2: 11 / 16 / 36 → 2 / 4 / 8.
- AVX2: 6 / 10 / 20 → 2 / 2 / 4.
- AVX512BW v16i32: 10 → 2.
[18 lines not shown]
[mlir][ComplexToStandard] Fix intermediate result fmf in exp lowering (#243397)
When computing `complex.exp(x + iy)`, the intermediate result `exp(x)`
may overflow even though the final result doesn't. Passing the `ninf`
flag to it (and to the inf comparison) is thus incorrect, and leads to
the branch that handles the overflow case being optimized out.
[libc++abi] Remove has_path_to_static_ptr from dynamic_cast
`path_dst_ptr_to_static_ptr` already contains whether we found a path to
static_ptr, so there is no need to track it explicitly.
[AMDGPU] Preserve noclobber after acquire operations from a different addrspace
Follow-up to #219991 to address reviewer comments:
AMDGPUAnnotateUniformValues should annotate uniform global loads as
`!amdgpu.noclobber` if the memory they are reading is never clobbered (since
kernel launch) before the load is executed.
PR #219991 strengthened the logic to consider acquire operations before the
load as clobbering (because they cause stores from other threads to be observed
by the load without a data race).
This PR weakens this criterion to ignore acquire operations that don't affect
the global address space (because they don't affect what the load in question
may or may not see). This addresses the performance-relevant case of s_barriers
with LDS-only fences.
The logic is still overly pessimistic for one-as load acquires, they could also
be ignored when looking for clobbering operations.
Disclaimer: Claude gave review comments and suggestions that I addressed during
[5 lines not shown]