[AArch64][SVE] Prefer SVE over scalarization for masked load/store of bf16 vectors (#210986)
During lowering of masked load/store for bf16 fixed-length vectors,
prefer emitting SVE instructions instead of scalarized branching code
whenever "+sve" is enabled in "target-features", regardless of whether
"+bf16" is present. The resulting changes can be seen in the following
test cases.
- llvm/test/CodeGen/AArch64/sve-fixed-length-masked-64-128bit-loads.ll
- llvm/test/CodeGen/AArch64/sve-fixed-length-masked-64-128bit-stores.ll
The direct cause of generating scalarized branching code is that the
"scalarize-masked-mem-intrin" optimization uses isLegalMaskedLoadStore
to determine whether the target natively supports masked load/store
operations. In AArch64TTIImpl::isLegalMaskedLoadStore,
isElementTypeLegalForScalableVector only treats bf16 vector operations
as legalizable when "+bf16" is enabled in "target-features". This
introduces an unnecessary restriction.
Therefore, this patch removes this restriction from
[3 lines not shown]
[LV] Simplify Blend/Select chain for tail folding outer loop reductions (#209659)
Predicated tail-folding outer loop reductions will generate a `Blend`
recipe for the predicate in-loop op and a `select` for tail-folding.
E.g.
```llvm
%blend = select %pred.mask, %inloop.op, %red.phi
%sel = select %header.mask, %blend, %red.phi
```
This patch simplifies this Blend/Select chain to
```llvm
%logical.and = select %header.mask, %pred.mask, false
%sel = select %logical.and, %inloop.op, %red.phi
```
This enables further optimization opportunities from CSE and further
simplification since the `%header.mask` will become an all-true-mask in
[3 lines not shown]
[flang][acc] fix array-section reductions on boxed arrays (#211648)
Reduction lowering (genReductions in flang/lib/Lower/OpenACC.cpp) only
remapped the reduced symbol to the acc.reduction result when the object
was a whole array (`isWholeSymbol` guard). For sections, the remap was
skipped, so hlfir.designate for element accesses inside the compute
region kept resolving through the original host declare's box instead of
the private reduction copy — the reduction ran on a private buffer that
never got read back.
Checked and confirmed dropping the boolean is safe for: non-1-starting
sections (recipe already compensates via a deliberate base-pointer
shift), SIZE/UBOUND/LBOUND inside privatized sections (intended
semantics), OPTIONAL dummies (pre-existing TODO, now fails loudly
instead of silently), combined constructs.
[clang][sema] Call expressions are constructed for undeduced deleted functions (#208488)
The deleted function path of overload resolution always constructed a CallExpr node
for the called function even if the function could not be deduced.
This case is handled in other paths by DiagnoseUseOfDecl which does perform that
test. The delete path cannot use that path though, as DiagnoseUseOfDecl rejects
deleted functions, and the entire point of this code is to permit the continued
evaluation of code even if the resolved function was deleted.
To fix this we now manually check for a complete type before continuing to construct
a potentially bogus CallExpr.
Fix error reported by -Wrange-loop-construct (#211657)
One system I'm building on adds -Wrange-loop-construct to the list of
-W<group> options. This results in the compile error:
```
/home/perry/llvm/Woz/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp:37:19: error: loop variable '[ID, N]' creates a copy from type 'std::pair<unsigned int, llvm::MDNode *> const' [-Werror,-Wrange-loop-construct]
for (const auto [ID, N] : MD) {
^
/home/perry/llvm/Woz/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp:37:8: note: use reference type 'std::pair<unsigned int, llvm::MDNode *> const &' to prevent copying
for (const auto [ID, N] : MD) {
^~~~~~~~~~~~~~~~~~~~
&
1 error generated.
```
[LLVM] Auto-upgrade legacy coro.end results (#211081)
When `llvm.coro.end` and `llvm.coro.end.async` changed from returning
`i1`
to returning `void` in #159278, the bitcode auto-upgrader continued to
handle
only the older two-argument `llvm.coro.end` form.
As a result, valid bitcode produced before that transition fails
verification
when a current LLVM ThinLTO backend materializes a module containing
either:
* the three-argument, `i1`-returning `llvm.coro.end`; or
* the `i1`-returning `llvm.coro.end.async`.
Teach `AutoUpgrade` to recognize both legacy declarations and rebuild
their
calls with the current `void`-returning intrinsics. If the legacy result
[15 lines not shown]
[lldb] Fix qSpeedTest radix mistake, make number parsings explicit (#211495)
The documentation for qSpeedTest says
```
send packet: qSpeedTest:response_size:response-size;
read packet: data:<response data>
response-size is a hex encoded unsigned number up to 64 bits in size.
```
debugserver implements qSpeedTest as per this documentation, but lldb
sends the number in decimal (base 10), and lldb-server parses it as base
10. I changed lldb and lldb-server to base 16. This is a maintenance
command used by lldb developers exclusivley, so IMO I'm not handling a
migration for old/new servers or defining a new packet or key. I was
running some packet transmission tests with debugserver and noticed
debugserver was sending much larger packets than requested; that's the
kind of failure you see when there is a mismatch.
[21 lines not shown]
[OpenMP] Propagate PRESENT to pointee entries in mapper codegen
Extend mapper map-type-modifier propagation to include PRESENT, gated to entries
that have an attach pointer (HasAttachPtr): the pointee data, whose storage
differs from the struct being mapped. A present modifier on the outer map/motion
clause must require that pointee to be present on the device; the present-check
on the struct's own storage does not cover it.
The propagation is gated on a new PropagatePresentToPointee argument to
emitUserDefinedMapper, set by callers only for OpenMP >= 6.0. Before 6.0 the
present modifier is treated as not applying to the pointee: the spec committee
confirmed the divergence between the present 'motion' modifier (to/from) and the
present map-type modifier (map) was unintentional, to be fixed as an OpenMP 6.0
erratum, so for 5.2 present is ignored for the pointee for both map and to/from.
Regular struct members receive only ALWAYS/DELETE/CLOSE; attach-ptr/pointee
entries additionally receive PRESENT at OpenMP >= 6.0. ATTACH entries receive no
modifier bits.
[8 lines not shown]
[OpenMP] Track attach-ptr entries in mapper codegen (HasAttachPtr)
Add a per-entry HasAttachPtr flag to MapInfosTy. It is set for entries that
have an attach pointer (and thus an accompanying ATTACH entry linking that ptr
to its pointee): pointee/combined entries whose storage differs from the struct
being mapped. It is NOT set for the ATTACH entries themselves.
In emitUserDefinedMapper, entries with HasAttachPtr (or the ATTACH bit, or the
Flang/MLIR PreserveMemberOfFlags) do not receive a new outer MEMBER_OF: pointee
data occupies a different storage block than the struct, and ATTACH entries
just link a ptr to its ptee. Existing inner MEMBER_OF bits are still shifted.
Clang (CGOpenMPRuntime) and the MLIR translator populate HasAttachPtr in
parallel with the other per-entry arrays; the MLIR/Flang side currently pushes
false with a TODO to set it for pointee-storage entries (e.g. s%p(0:10)) and
eventually drop PreserveMemberOfFlags in favor of it.
Co-Authored-By: Claude Opus 4.8 <noreply at anthropic.com>
[OpenMP] Propagate ALWAYS/DELETE/CLOSE map-type modifiers to mapper entries
Per OpenMP 6.0:281:34, when a map/motion clause uses a mapper modifier, any
map-type-modifying modifier on that clause applies to each map the declared
mapper specifies. Propagate the ALWAYS, DELETE, and CLOSE bits from the outer
clause's map type into every entry emitted by emitUserDefinedMapper, except
ATTACH entries (ATTACH|ALWAYS is reserved for attach(always), and the other
bits have no meaning for an ATTACH entry).
PRESENT is intentionally NOT propagated here: it requires distinguishing
pointee entries from the struct's own storage and is handled in a follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply at anthropic.com>
braces
The 1:N conversion callback returned MlirLogicalResult, which the binding
could only map to success or nullopt (try-another), leaving the C++
failure() state (fail without trying another conversion) unreachable.
Return a MlirTypeConverterConversionStatus enum instead, with Success,
Failure, and Declined states mapped to success(), failure(), and
std::nullopt respectively. Add a test covering the hard-failure path.
[mlir-c] Strengthen testGetParentWithName coverage
Address review feedback: the previous IR chain
(arith.constant -> func.func -> builtin.module) had each queried name
appear exactly once, so an implementation returning the outermost match
instead of the nearest would still pass. Use nested modules so
builtin.module appears twice in the ancestor chain, assert the walk
returns the inner (nearest) module, and add a boundary case for a
top-level op whose parent is null.
[clang-tidy] Add performance-expensive-value-or check (#200166)
Finds calls to `value_or` (and alternative spellings `valueOr`,
`ValueOr`) on optional types where the return type is expensive to copy.
Objects not trivially copyable, or larger than a configurable size threshold
are treated as expensive.
Rationale: While `value` and `operator*` return references, `value_or` for
these types always returns by value.
[mlir][EmitC] Add `class-name-format` option to `wrap-emitc-func-in-class` pass (#211147)
Added the `class-name-format` option that takes a format string used to
generate the wrapper class name.
This provides greater control over the generated `ClassOp` name. When
multiple identically structured IR files need to be programmatically
lowered with this pass and linked, there will be naming conflicts which
this patch provides a method of resolution for.
[OpenACC] set CollapseCountAttr for scf.parallel (#211676)
During compute lowering, also set the collapse count attribute when
creating scf.parallel ops from acc.loop ops. While this collapse count
is implied by the number of IVs, later canonicalizations may use this
information to decide whether or not to collapse the loop nest further
