[AArch64] Cap upper-bound unrolling of loops with uncomputable trip counts (#205102)
SCEV can compute a small maximum trip count for a loop even when it
cannot compute the exact backedge-taken count. This is typical of
data-dependent exits, such as the varint-length loops in sqlite3:
for (i = 1; (v >>= 7) != 0; i++)
;
Unrolling to the maximum trip count turns the loop into a chain of
copies containing an exit test for every possible iteration. This
increases code size and the number of static branches without knowing
whether enough iterations usually execute to make unrolling profitable.
These loops were not unrolled before #197292. That change improved
SCEV's maximum backedge-taken count for shift recurrences, reducing the
maximum trip count of these loops to 6. This made them eligible for
AArch64 upper-bound unrolling and caused an approximately 3% regression
in sqlite3 from SPEC CPU 2026.
[6 lines not shown]
[mlir][tosa] Add more informative error messages to block scaled types (#211569)
This commit improves block scaled tensor type verification to provide a
specific reason about why type verification failed. Previously the error
message was a very generic "must be tosa-conformant tensor of number
values".
This commit restores previous functionality that was reverted by
https://github.com/llvm/llvm-project/pull/207995, but uses the ODS
string interpolation mechanism instead to prevent incompatibility with
PDLL.
[AArch64] Enable FEAT_LSFE CodeGen only when exception can be ignored (#212799)
This patch restricts the generation of new atomic instructions added
under FEAT_LSFE to functions where no-trapping-math flag is set and
strictfp attribute is not set. This is done to preserve the exception
behaviour of the original codegen.
[NVVM][NVPTX] Add im2col_w support for S2G and reduction intrinsics (#214436)
PTX ISA 9.4 adds the im2col_no_offs::w mode to shared-to-global tensor
copy and reduction instructions for Rubin family targets.
This change adds the corresponding NVVM intrinsics and NVPTX lowering.
[libc] Add getpwent, setpwent, and endpwent entrypoints (#213076)
Added getpwent, setpwent, and endpwent functions using the internal
pwd_utils line parser.
* Implemented getpwent.cpp, setpwent.cpp, and endpwent.cpp entrypoints
* Added database iteration state and helpers (open, close, read_next)
under namespace passwd in pwd_utils
* Registered entrypoints in config/linux/*/entrypoints.txt
* Added CMake target guards for File and platform_file to support
overlay builds and cross-platform targets
* Added unit tests in libc/test/src/pwd/getpwent_test.cpp
Assisted-by: Automated tooling, human reviewed.
[libc][NFC] Format all CMake files under libc/ (#213132)
Use cmake_format.py from #213102 on all CMakeLists.txt files in libc/
Assisted-by: Automated tooling, human reviewed.
[lldb] Make RegisterFlagsDetector into RegisterTypesDetector
In future it may be generating things other than flags. Functionality
is the same, but the interface changes to use RegisterType.
[lldb] Refactor RegisterTypeBuilder
This prepares it for emitting union types. Major changes:
* Entry function is now a dispatcher to builder functions for each type.
* Name mangling is standardised.
* The register name parameter is no longer needed and so was removed.
Revert "[SCEV] Speed up forgetLoop by avoiding def-use walk for loop-header PHIs" (#212485)
Reverts https://github.com/llvm/llvm-project/pull/201572
Multiple miscompilations are reported, see
https://github.com/llvm/llvm-project/issues/207744,https://github.com/llvm/llvm-project/issues/212027
That commit made forgetLoop() rely on LoopUsers[L] and stop walking the
def-use chain of the loop-header PHIs. This is insufficient, because
some cached data is derived from the underlying IR of SCEVUnknown, it is
not reachable from LoopUsers[L].
After that commit, forgetLoop() no longer invalidated them, so stale
UnsignedRanges / SignedRanges, ConstantMultipleCache, ValuesAtScopes
cause miscompilations.
(cherry picked from commit 1c0eda0d2371a6a755f90299892a7b89dc917442)
[InstCombine] Remove binop(concat(x,y),concat(a,b)) -> concat(binop(x,a),binop(y,b)) fold (#214285)
A similar fold is cost-driven in VectorCombine, but this InstCombine
fold was then undoing the decision
Seems to an old fold added by @rotateright (#32373) before we created
VectorCombine to do this properly
Fixes #205707
[lldb] Store all XML register types in a single string map
We are assuming that their ID's are unique, so there's no need to keep
separate maps. We can do basic type checking by checking the kind of
the type pointed to.
A few more methods were added to the base RegisterType. GetSize()
returns 0 for enums because enums don't have a size until they are
used by a register. This is not ideal but it works for now.
[libc++][libc++abi] Merge the operator new implementations into a shared file (#213427)
Instead of copying the code around we can simply share a file that
actually implements operator new/delete and include that file in the two
places that need the definitions.
[mlir][affine] Implement ValueBoundsOpInterface for affine.for (#214614)
`scf.for` has provided induction variable bounds through
`ValueBoundsOpInterface` for a long time, but `affine.for` has no model
at all. `ValueBoundsConstraintSet` therefore cannot derive any bound for
an affine induction variable, not even `iv >= lowerBound`. Queries just
return "unknown", which is easy to miss because it looks the same as a
bound that genuinely cannot be proven.
Add a model for the induction variable. The lower bound of an
`affine.for` is the maximum over the results of its lower bound map and
the upper bound is the minimum over the results of its upper bound map,
so the induction variable is constrained by every individual result.
This follows how `AffineMinOpInterface` and `AffineMaxOpInterface` in
the same file align their maps with the constraint set; the shared logic
is factored into `alignBoundExpr`.
When both maps have exactly one result the step is taken into account as
well, following `scf::ForOpInterface`: the induction variable is always
[29 lines not shown]
[SPIR-V] Lower nested aggregate insertvalue operands (#204239)
Fix a crash in the SPIR-V backend when an aggregate `extractvalue`
result is used as the base of a later `insertvalue`.
The failing pattern is:
```llvm
%e = extractvalue [1 x [1 x i64]] %a, 0
%i = insertvalue [1 x i64] %e, i64 %x, 0
%r = insertvalue [1 x [1 x i64]] %a, [1 x i64] %i, 0
```
`SPIRVPrepareFunctions` rewrites aggregate function arguments and
returns to `i32` SPIR-V value IDs. The `llvm.spv.insertv` intrinsic also
models its composite operand as an `i32` value ID. However, an
intermediate aggregate `extractvalue` could still be rewritten to
`llvm.spv.extractv` with an LLVM aggregate result type. Passing that
aggregate-typed value to `llvm.spv.insertv` made
[14 lines not shown]
[RISCV] Remove isBarrier from PseudoCALLReg/PseudoTLSDESCCall (#214155)
`PseudoCALLReg` models a returning call with an explicit link register.
The machine outliner uses it for normal outlined calls that return
through `x5`.
Marking this pseudo as a barrier is inconsistent with call sites at the
end of a block that falls through to a successor after the outlined
function returns. MachineVerifier correctly diagnoses those blocks as
malformed.
Remove the barrier flag and add a MIR regression test that outlines a
sequence at the end of a fall-through block. The test runs for RV32 and
RV64 with `-verify-machineinstrs`.
The attribute isBarrier from PseudoTLSDESCCall is removed as well for
the similar reason and a MachineVerifier regression test is added for
the TLSDESC call at the end of a fall-through block.
[flang][MIF] Fix team_type usage in all MIF operations #205953 (#214207)
This PR resolves the issue #205953. Its purpose is to correct the use of
`team_type` by lowering it to Addr rather than to Box.
(cherry picked from commit 0563f0d88d8314edcde7deab465759a84696e894)
[LLD][COFF] Emit ARM64X relocations for TLS directory (#213529)
The current behavior of merging EC and native chunks to have a single
TLS directory for both views matches the MSVC linker, but it has its
shortcomings. In addition to merging, that solution requires all TLS
callbacks to use -arm64xsameaddress, leaving it to the CRT to handle. If
the appropriate EC object files are not pulled in by linked EC code and
thus never have a chance to mark the callback with -arm64xsameaddress,
this may lead to an invalid image that crashes at runtime.
This patch avoids the entire problem by using entirely separate TLS
directories for EC and native views along with the standard ARM64X
dynamic relocation mechanism. Since callback lists are now separate, a
missing -arm64xsameaddress is no longer a problem.
Also, mingw-w64-crt currently doesn't mark its TLS callbacks with
-arm64xsameaddress. That could be changed if needed, but with this
change, it is no longer necessary.
(cherry picked from commit 65e92e073539c3fe3523256c4fba35ac1dedcb92)
[compiler-rt] Disable executable stack on `aeabi_u{read,write}*.S` (#214465)
These were missing `NO_EXEC_STACK_DIRECTIVE` to add `.note.GNU-stack`;
without it, a binary including any of these files will have the stack
marked executable. Add the directive here, matching other similar files.
Symtab diff before:
$ clang compiler-rt/lib/builtins/arm/aeabi_uread4.S
--target=arm-unknown-linux-gnueabi -c
$ llvm-readelf aeabi_uread4.o -S
There are 5 section headers, starting at offset 0xe4:
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .strtab STRTAB 00000000 0000a8 000039 00 0 0 1
[ 2] .text PROGBITS 00000000 000034 000020 00 AX 0 0 4
[ 3] .ARM.attributes ARM_ATTRIBUTES 00000000 000054 000022 00 0 0 1
[21 lines not shown]
[flang][MIF] Fix undef reference to a coarray_handle in mif.dealloc_coarray #193157 (#213890)
This PR fixes the behavior reported in issue #193157. The coarray_handle
was only defined if a call to mif.alloc_coarray was present.
If a call to mif.dealloc_coarray was encountered without a prior call to
mif.alloc_coarray, then the coarray_handle was missing, and therefore
llvm.address_of pointed to a non-existent address, which is not allowed.
We now define a coarray_handle that has not been allocated by PRIF for
each coarray variables.
(cherry picked from commit a9426776e78a89dec776c34efdb7725de591d878)
workflows/llvm-abi-tests: Cache the baseline abi (#211968)
This way we don't need to recompute it for every workflow.
(cherry picked from commit 09734937d37ca9c8e8c9cdaa55d00339068922c3)
[SLP][modularisation][NFC] Move full-vector width helpers to SLPTypeUtils (2/2)
Move the following BoUpSLP-independent helpers out of SLPVectorizer.cpp
into SLPVectorizer/SLPTypeUtils.{h,cpp}:
getFullVectorNumberOfElements
getFloorFullVectorNumberOfElements
getMaskedDivRemType
hasFullVectorsOrPowerOf2
They build on the type helpers moved in (1/2). Behavior is unchanged.
Part of the SLPVectorizer.cpp modularization effort:
https://discourse.llvm.org/t/modularizing-slpvectorizer-cpp/90922
[SLP][modularisation][NFC] Move type/width helpers to SLPTypeUtils (1/2)
Move the following BoUpSLP-independent type/width helpers out of
SLPVectorizer.cpp into a new SLPVectorizer/SLPTypeUtils.{h,cpp}:
isValidElementType
getValueType
getWidenedType
SLPReVec is made non-static and extern-declared in SLPTypeUtils.cpp so
the moved helpers can reference it; behavior is unchanged.
Part of the SLPVectorizer.cpp modularization effort:
https://discourse.llvm.org/t/modularizing-slpvectorizer-cpp/90922
[ValueLattice][SCCP] Consider provenance for predicate-derived pointer constants (#160083)
Similarly to what it is being already done in GVN
(fb632ed2377d280b581b8d4653b855e60d611f77), make sure pointers
equalities derived via PredicatedInfo may be propagated so long as
pointer constants' provenance is taken into account, which may be
non-existent. This is achieved by tracking provenance in
ValueLatticeElement and restrain replacement to uses allowed by
`canReplacePointersInUseIfEqual`.
Fixes: https://github.com/llvm/llvm-project/issues/159565.
[BOLT][AArch64] Expand cmpbr when reversing would overflow (#202998)
AArch64 compare-and-branch instructions can usually be reversed by
changing the condition and adjusting the immediate. At boundary values,
that adjustment can underflow or overflow, leaving the branch
non-reversible.
When condition flags are dead at such a branch, split the compare branch
into an explicit compare followed by a conditional branch. The new
sequence lets branch fixup reverse the condition without relying on an
out-of-range adjusted immediate.
Teach the branch-fixing paths to use cached branch liveness information
when deciding whether this expansion is legal. The liveness snapshot is
built before branch relaxation/fixup and is safe for the current users:
they only insert trampolines/stubs between existing CFG edges or invert
branches without changing program semantics.
Since expansion can grow the source basic block, update the local
[2 lines not shown]
