[libc][cpp::string] Allocate fewer temp strings in operator= and +=
This PR generally updates `cpp::string` to avoid incidental allocations. Specifically, it:
- Updates `opreator=(string_view)` to avoid allocating a temporary string: https://github.com/llvm/llvm-project/blob/67ebc4b221c3e94028b33004cd5cd08deee95048/libc/src/__support/CPP/string.h#L106-L108
- Changes `operator+=(const string&)` to accept a `string_view` so that strings may be appended without allocation.
- Makes the `string(string_view)` constructor explicit. Before, there were non-obvious allocations because of the implicit conversion.
This PR assumes that self-assignment and self-appends are not supported with `cpp::string`, and adds debug asserts preventing them. Currently, assignment doesn't work. `cpp::string s = "abc"; s = s;` will just zero out the string. This PR is a slight regression in that `operator+=(const cpp::string&)` previously worked, but now it errors in debug mode, and would result in a use-after-free without the assert statement. I don't think self-append/self-assignment is something `cpp::string` needs to support, so I omitted it, but if a reviewer thinks this behavior is important happy to amend this PR.
[profcheck][coro] Adding Branch weights PGO in Coroutines Passes (#184466)
The CoroSplit and CoroFrame generate several control flow instructions
(`br`, `select`, `switch`) that previously lacked `!prof` branch weight
metadata.
When a coroutine is split, most basic blocks are simply cloned into the
generated functions (`.resume`, `.destroy`, and `.cleanup`) and don't
actually change their internal structure or layout. This PR changes:
1. Dispatch Branch Weights:
Since the original coroutine already ran through PGO, we already have
the `SuspendFreqs` records telling us exactly how many times each
suspension point was executed in the real world.
In the new clone `.resume` functions, the compiler generates new routing
`switch` instruction to direct the traffic. Instead of guessing the
probabilities for these new routes, we simply look back at the
`SuspendFreqs` records, calculate the exact execution frequency and
[55 lines not shown]
[clang][sema] Call expressions are constructed for undeduced deleted functions (#208488)
The deleted function path of overload resolution always constructed a CallExpr node
for the called function even if the function could not be deduced.
This case is handled in other paths by DiagnoseUseOfDecl which does perform that
test. The delete path cannot use that path though, as DiagnoseUseOfDecl rejects
deleted functions, and the entire point of this code is to permit the continued
evaluation of code even if the resolved function was deleted.
To fix this we now manually check for a complete type before continuing to construct
a potentially bogus CallExpr.
[libc][cpp::string] Allocate fewer temp strings in operator= and +=
This PR generally updates `cpp::string` to avoid incidental allocations. Specifically, it:
- Updates `opreator=(string_view)` to avoid allocating a temporary string: https://github.com/llvm/llvm-project/blob/67ebc4b221c3e94028b33004cd5cd08deee95048/libc/src/__support/CPP/string.h#L106-L108
- Changes `operator+=(const string&)` to accept a `string_view` so that strings may be appended without allocation.
- Makes the `string(string_view)` constructor explicit. Before, there were non-obvious allocations because of the implicit conversion.
This PR assumes that self-assignment and self-appends are not supported with `cpp::string`, and adds debug asserts preventing them. Currently, assignment doesn't work. `cpp::string s = "abc"; s = s;` will just zero out the string. This PR is a slight regression in that `operator+=(const cpp::string&)` previously worked, but now it errors in debug mode, and would result in a use-after-free without the assert statement. I don't think self-append/self-assignment is something `cpp::string` needs to support, so I omitted it, but if a reviewer thinks this behavior is important happy to amend this PR.
[mlir][VectorToLLVM] Fix comment still referring to undef instead of poison (NFC) (#211196)
When updating operators from undef (deprecated) to poison, the
associated comment wasn't updated accordingly. This PR fixes it.
[X86] Don't emit x87 register scrub when x87 is unavailable (#211146)
### Summary
`-fzero-call-used-regs=all` always emits the x87 `fldz`/`fstp` scrub
sequence, without checking whether x87 is enabled. In a function built
with `-mno-x87` or `-msoft-float`, this injects 16 `x87` instructions
into a translation unit that must not touch the FPU:
```asm
// test.c
int f(int x) { return x + 1; }
$ clang -O2 -S -mno-x87 -msoft-float -mno-sse \
-fzero-call-used-regs=all --target=x86_64-linux-gnu test.c -o -
f:
leal 1(%rdi), %eax
fldz
... # 8x fldz
[30 lines not shown]
[LoongArch] Support i128 operands for LSX inline assembly (#211464)
Allow `i128` values to be used with the `f` inline assembly constraint
when targeting LSX.
Although `i128` is not a legal LSX value type, it naturally maps to a
single 128-bit LSX vector register for inline assembly. This enables
instructions such as `vadd.q` to operate directly on `__int128` operands
without requiring explicit vector types.
Reland "[mlir][tosa] Combine unranked/ranked tensor types into single type" (#211708)
Reverts llvm/llvm-project#211701
Relands llvm/llvm-project#209737
The buildbot failure was misattributed to this change.
Real cause is https://github.com/llvm/llvm-project/pull/207274
[ARM] Use .reloc for dso_local weak symbols in PIC mode instead of GOT indirection (#209660)
In ARM ELF PIC mode, weak symbols referenced via the constant pool use a
PC-relative expression like `.long sym-(.LPC+8)`. The assembler eagerly
resolves this when the symbol and reference are in the same section,
which prevents the linker from overriding a weak definition with a
non-weak one from another object file.
The previous approach forced *all* weak symbols through GOT indirection.
That works, but `dso_local` weak symbols with hidden/protected
visibility are non-preemptible — they don't need GOT indirection, and
the extra load is wasteful. This patch emits a `.reloc` directive for
such symbols instead, forcing the assembler to produce an `R_ARM_REL32`
relocation while keeping a direct PC-relative reference from the
constant pool.
Default-visibility weak symbols (even `dso_local`) remain routed through
the GOT because they stay exported and preemptible in shared objects,
and `R_ARM_REL32` against an external symbol cannot be used when making
[2 lines not shown]
[AMDGPU] Fix gfx1250 WMMA latencies
- 16x16x64 FP8/BF8 WMMA run in 4 cycles, not 8. Add a 4-cycle
WriteXDL1PassWMMA write and split the FP8/BF8 InstRW by shape so that
16x16x128 FP8/BF8 (and F16/BF16) keep the 8-cycle latency.
- f8f6f4 WMMA run in 4 cycles when both matrix inputs are f4 and 8 cycles
when any input is f6 or f8, instead of the previous 8/16. Update the
scheduling predicate accordingly.
The co-execution hazard category is derived from the WMMA latency. The
4-cycle WMMAs have a single co-execution slot, so give them their own
category (2 wait states before a dependent WMMA, 1 before a dependent VALU)
rather than bucketing them with the 8-cycle WMMAs.
[AArch64][SVE] Prefer SVE over scalarization for masked load/store of bf16 vectors (#210986)
During lowering of masked load/store for bf16 fixed-length vectors,
prefer emitting SVE instructions instead of scalarized branching code
whenever "+sve" is enabled in "target-features", regardless of whether
"+bf16" is present. The resulting changes can be seen in the following
test cases.
- llvm/test/CodeGen/AArch64/sve-fixed-length-masked-64-128bit-loads.ll
- llvm/test/CodeGen/AArch64/sve-fixed-length-masked-64-128bit-stores.ll
The direct cause of generating scalarized branching code is that the
"scalarize-masked-mem-intrin" optimization uses isLegalMaskedLoadStore
to determine whether the target natively supports masked load/store
operations. In AArch64TTIImpl::isLegalMaskedLoadStore,
isElementTypeLegalForScalableVector only treats bf16 vector operations
as legalizable when "+bf16" is enabled in "target-features". This
introduces an unnecessary restriction.
Therefore, this patch removes this restriction from
[3 lines not shown]
[LV] Simplify Blend/Select chain for tail folding outer loop reductions (#209659)
Predicated tail-folding outer loop reductions will generate a `Blend`
recipe for the predicate in-loop op and a `select` for tail-folding.
E.g.
```llvm
%blend = select %pred.mask, %inloop.op, %red.phi
%sel = select %header.mask, %blend, %red.phi
```
This patch simplifies this Blend/Select chain to
```llvm
%logical.and = select %header.mask, %pred.mask, false
%sel = select %logical.and, %inloop.op, %red.phi
```
This enables further optimization opportunities from CSE and further
simplification since the `%header.mask` will become an all-true-mask in
[3 lines not shown]
[flang][acc] fix array-section reductions on boxed arrays (#211648)
Reduction lowering (genReductions in flang/lib/Lower/OpenACC.cpp) only
remapped the reduced symbol to the acc.reduction result when the object
was a whole array (`isWholeSymbol` guard). For sections, the remap was
skipped, so hlfir.designate for element accesses inside the compute
region kept resolving through the original host declare's box instead of
the private reduction copy — the reduction ran on a private buffer that
never got read back.
Checked and confirmed dropping the boolean is safe for: non-1-starting
sections (recipe already compensates via a deliberate base-pointer
shift), SIZE/UBOUND/LBOUND inside privatized sections (intended
semantics), OPTIONAL dummies (pre-existing TODO, now fails loudly
instead of silently), combined constructs.
[clang][sema] Call expressions are constructed for undeduced deleted functions (#208488)
The deleted function path of overload resolution always constructed a CallExpr node
for the called function even if the function could not be deduced.
This case is handled in other paths by DiagnoseUseOfDecl which does perform that
test. The delete path cannot use that path though, as DiagnoseUseOfDecl rejects
deleted functions, and the entire point of this code is to permit the continued
evaluation of code even if the resolved function was deleted.
To fix this we now manually check for a complete type before continuing to construct
a potentially bogus CallExpr.
Fix error reported by -Wrange-loop-construct (#211657)
One system I'm building on adds -Wrange-loop-construct to the list of
-W<group> options. This results in the compile error:
```
/home/perry/llvm/Woz/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp:37:19: error: loop variable '[ID, N]' creates a copy from type 'std::pair<unsigned int, llvm::MDNode *> const' [-Werror,-Wrange-loop-construct]
for (const auto [ID, N] : MD) {
^
/home/perry/llvm/Woz/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp:37:8: note: use reference type 'std::pair<unsigned int, llvm::MDNode *> const &' to prevent copying
for (const auto [ID, N] : MD) {
^~~~~~~~~~~~~~~~~~~~
&
1 error generated.
```
[LLVM] Auto-upgrade legacy coro.end results (#211081)
When `llvm.coro.end` and `llvm.coro.end.async` changed from returning
`i1`
to returning `void` in #159278, the bitcode auto-upgrader continued to
handle
only the older two-argument `llvm.coro.end` form.
As a result, valid bitcode produced before that transition fails
verification
when a current LLVM ThinLTO backend materializes a module containing
either:
* the three-argument, `i1`-returning `llvm.coro.end`; or
* the `i1`-returning `llvm.coro.end.async`.
Teach `AutoUpgrade` to recognize both legacy declarations and rebuild
their
calls with the current `void`-returning intrinsics. If the legacy result
[15 lines not shown]
