LLVM/project 25e2c68llvm/lib/Transforms/Vectorize VPlanAnalysis.cpp, llvm/test/Transforms/LoopVectorize revec-reg-usage.ll

[LV][REVEC] Correctly compute register usage

For REVEC, the initial types might already be vectors, so make sure the
right register class is picked.
DeltaFile
+39-0llvm/test/Transforms/LoopVectorize/revec-reg-usage.ll
+17-12llvm/lib/Transforms/Vectorize/VPlanAnalysis.cpp
+56-122 files

LLVM/project 31ba2c4llvm/test/Transforms/LoopVectorize revec-unroll.ll

Update test after rebase
DeltaFile
+1-1llvm/test/Transforms/LoopVectorize/revec-unroll.ll
+1-11 files

LLVM/project c7eb31ellvm/lib/Transforms/Vectorize VPlanRecipes.cpp

Restore type checks in computeScalarTypeForInstruction()
DeltaFile
+7-0llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+7-01 files

LLVM/project 61f1220llvm/lib/Transforms/Vectorize LoopVectorizationPlanner.cpp, llvm/test/Transforms/LoopVectorize revec-maximize-bandwidth.ll

Revert REVEC-specific useMaxBandwidth() change and add tests
DeltaFile
+50-0llvm/test/Transforms/LoopVectorize/AArch64/revec-maximize-bandwidth.ll
+45-0llvm/test/Transforms/LoopVectorize/revec-maximize-bandwidth.ll
+1-3llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
+96-33 files

LLVM/project 6184586llvm/lib/Target/AArch64 AArch64TargetTransformInfo.h, llvm/lib/Transforms/Vectorize LoopVectorizationPlanner.cpp LoopVectorizationLegality.cpp

Stop using isElementTypeLegalForScalableVector as type check for REVEC
DeltaFile
+5-10llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
+0-9llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
+2-7llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
+7-263 files

LLVM/project 629ececllvm/lib/Transforms/Vectorize VPlan.h VPlanRecipes.cpp, llvm/test/Transforms/LoopVectorize revec-liveout.ll

Test get(VPV, LaneIdx) for "vector lanes"

This requires a slight adjustment to type asserts in
VPInstruction::execute to be safe for REVEC.
DeltaFile
+77-0llvm/test/Transforms/LoopVectorize/revec-liveout.ll
+8-4llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+2-1llvm/lib/Transforms/Vectorize/VPlan.h
+87-53 files

LLVM/project 384387bllvm/test/Transforms/LoopVectorize/AArch64 revec-memory-interleaved.ll revec-memory-contiguous.ll

[LV][REVEC][AArch64] Proof of concept for re-vectorisation

This shows the changes required to enable basic re-vectorisation support in LoopVectorizer. Most of the diff comes from the added tests, the changes to LoopVectorizer files are rather minimal. This proof-of-concept has obvious limitations and only represents the first building block.

My hope is that this helps discussions and complements the RFC at https://discourse.llvm.org/t/rfc-re-vectorisation-to-wider-vectors-in-loopvectorizer/91071.

Support for re-vectorisation is hidden behind a -vectorize-vector-loops flag and LV will bail out if it encounters constructs that are not yet supported. For example:
 - shufflevectors
 - gather/scatter and interleaved accesses
 - target intrinsics
 - reductions
 - if-conversion or tail folding
DeltaFile
+293-0llvm/test/Transforms/LoopVectorize/AArch64/revec-select.ll
+118-0llvm/test/Transforms/LoopVectorize/AArch64/revec-memory-gather-scatter.ll
+102-0llvm/test/Transforms/LoopVectorize/AArch64/revec-predication.ll
+100-0llvm/test/Transforms/LoopVectorize/AArch64/revec-unroll.ll
+90-0llvm/test/Transforms/LoopVectorize/AArch64/revec-memory-contiguous.ll
+89-0llvm/test/Transforms/LoopVectorize/AArch64/revec-memory-interleaved.ll
+792-013 files not shown
+1,170-4119 files

LLVM/project 44640dallvm/test/Transforms/LoopVectorize revec-predication.ll revec-memory-gather-scatter.ll, llvm/test/Transforms/LoopVectorize/AArch64 revec-predication.ll revec-memory-gather-scatter.ll

Remove aarch64 triple and move tests out of AArch64/
DeltaFile
+0-293llvm/test/Transforms/LoopVectorize/AArch64/revec-select.ll
+293-0llvm/test/Transforms/LoopVectorize/revec-select.ll
+119-0llvm/test/Transforms/LoopVectorize/revec-memory-gather-scatter.ll
+0-118llvm/test/Transforms/LoopVectorize/AArch64/revec-memory-gather-scatter.ll
+103-0llvm/test/Transforms/LoopVectorize/revec-predication.ll
+0-102llvm/test/Transforms/LoopVectorize/AArch64/revec-predication.ll
+515-51312 files not shown
+962-95918 files

LLVM/project 59040f8llvm/lib/CodeGen/SelectionDAG LegalizeDAG.cpp, llvm/lib/Target/AArch64 AArch64ISelLowering.cpp

Support expansion to shufflevector
DeltaFile
+68-0llvm/test/CodeGen/AArch64/vector-broadcast.ll
+19-0llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+3-0llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+90-03 files

LLVM/project d717b71llvm/docs LangRef.md

Update LangRef
DeltaFile
+26-0llvm/docs/LangRef.md
+26-01 files

LLVM/project f399c1cllvm/lib/CodeGen/SelectionDAG LegalizeTypes.h LegalizeVectorTypes.cpp, llvm/test/CodeGen/AArch64 sve-vector-broadcast.ll

Support splitting results
DeltaFile
+51-0llvm/test/CodeGen/AArch64/sve-vector-broadcast.ll
+46-0llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
+1-0llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h
+98-03 files

LLVM/project eaa9029llvm/include/llvm/IR Intrinsics.td, llvm/lib/CodeGen/SelectionDAG SelectionDAG.cpp LegalizeIntegerTypes.cpp

[IR] Define llvm.vector.broadcast intrinsic

This is used broadcast a smaller vector into a wider one. The patch adds
basic legalisation support and ISel for AArch64.
DeltaFile
+318-0llvm/test/CodeGen/AArch64/sve-vector-broadcast.ll
+36-6llvm/lib/Target/AArch64/SVEInstrFormats.td
+30-0llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp
+12-0llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+9-0llvm/test/CodeGen/AArch64/sve-vector-broadcast-unsupported.ll
+7-0llvm/include/llvm/IR/Intrinsics.td
+412-65 files not shown
+430-611 files

LLVM/project 62537bellvm/test/Transforms/LoopVectorize compress-store-vec-epilogue.ll compress-idioms.ll

Rebase fixups
DeltaFile
+8-8llvm/test/Transforms/LoopVectorize/compress-idioms.ll
+3-3llvm/test/Transforms/LoopVectorize/compress-store-vec-epilogue.ll
+11-112 files

FreeBSD/ports 495cf99devel/py-sqlfluff Makefile distinfo

devel/py-sqlfluff: Update to 4.3.0
DeltaFile
+3-3devel/py-sqlfluff/distinfo
+1-1devel/py-sqlfluff/Makefile
+4-42 files

LLVM/project ddd7b23llvm/lib/Target/AArch64 AArch64SVEInstrInfo.td AArch64SelectionDAGInfo.cpp, llvm/test/CodeGen/AArch64 neon-partial-reduce-dot-product.ll sve-partial-reduce-dot-product.ll

[AArch64] Use [SU]ADALP for partial (nx)v4i32 -> (nx)v2i64 add reduce. (#213915)

The previously used AArch64ISD::[SU]ADDW[BT] became unused after this
change, so I've removed them.
DeltaFile
+111-154llvm/test/CodeGen/AArch64/sve-fixed-length-partial-reduce.ll
+32-32llvm/test/CodeGen/AArch64/sve-partial-reduce-dot-product.ll
+14-28llvm/test/CodeGen/AArch64/neon-partial-reduce-dot-product.ll
+19-18llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+0-19llvm/lib/Target/AArch64/AArch64SelectionDAGInfo.cpp
+4-12llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
+180-2636 files

LLVM/project 09375e3offload/test/ompt register_both.h veccopy_disallow_both.c, openmp/runtime/src ompt-general.cpp

[OpenMP][OMPT] Error when registering EMI and non-EMI callbacks (#213697)

The OpenMP specification includes a restriction for registering both EMI
and non-EMI callbacks (e.g. `ompt_callback_target`, OpenMP v5.2, p. 503,
l. 18-20):

  These callbacks must not be registered at the same time.

However, this was not respected in the runtime. A tool was able to
register both, receiving `ompt_set_always` for each `ompt_set_callback`
call. When an event associated with the EMI or non-EMI callbacks was
dispatched and both callbacks were registered, only the EMI version was
dispatched.

To align the runtime behavior with the OpenMP specification, return
`ompt_set_error` when a tool tries to register both the EMI and non-EMI
variant for a callback. With this, only the tool's first registration is
used.


    [5 lines not shown]
DeltaFile
+26-1openmp/runtime/src/ompt-general.cpp
+11-8offload/test/ompt/veccopy_disallow_both.c
+13-0openmp/runtime/src/include/omp-tools.h.var
+3-1offload/test/ompt/register_both.h
+53-104 files

LLVM/project 662221bllvm/test/Transforms/LoopVectorize compress-idioms.ll

Test uncond increment
DeltaFile
+96-2llvm/test/Transforms/LoopVectorize/compress-idioms.ll
+96-21 files

LLVM/project 0c9ec14llvm/lib/Transforms/Vectorize VPlanRecipes.cpp, llvm/test/Transforms/LoopVectorize compress-idioms-negative-tests.ll compress-idioms.ll

Move negative tests to new file
DeltaFile
+0-105llvm/test/Transforms/LoopVectorize/compress-idioms.ll
+94-0llvm/test/Transforms/LoopVectorize/compress-idioms-negative-tests.ll
+0-1llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+94-1063 files

LLVM/project 7456fd8llvm/lib/Transforms/Vectorize VPlanConstruction.cpp VPlan.h

Don't allow null phi
DeltaFile
+4-6llvm/lib/Transforms/Vectorize/VPlan.h
+1-1llvm/lib/Transforms/Vectorize/VPlanConstruction.cpp
+5-72 files

LLVM/project 3dc3541llvm/lib/Transforms/Vectorize LoopVectorize.cpp VPlan.h, llvm/test/Transforms/LoopVectorize compress-store-vec-epilogue.ll

Fix epilogue resume handling
DeltaFile
+98-0llvm/test/Transforms/LoopVectorize/compress-store-vec-epilogue.ll
+5-5llvm/lib/Transforms/Vectorize/VPlan.h
+2-1llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+105-63 files

LLVM/project d1bd26bllvm/lib/Transforms/Vectorize VPlan.h VPlanRecipes.cpp, llvm/test/Transforms/LoopVectorize compress-idioms.ll

[LoopVectorize] Support vectorization of compressing patterns in VPlan

RFC link: https://discourse.llvm.org/t/rfc-loop-vectorization-of-compress-store-expand-load-patterns/86442

This adds loop vectorizer support for "compressing" patterns,
for example:

```
int dst_idx = 0;
for (int i = 0; i < n; i++) {
  if (cond[i])
    dst[dst_idx++] = src[i];
}
```

Can be vectorized with a `llvm.masked.compressstore` as:

```
int dst_idx = 0;

    [52 lines not shown]
DeltaFile
+424-0llvm/test/Transforms/LoopVectorize/compress-idioms.ll
+157-0llvm/test/Transforms/LoopVectorize/VPlan/compress-idioms.ll
+132-0llvm/test/Transforms/LoopVectorize/AArch64/compress-idioms.ll
+111-14llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+70-4llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+60-4llvm/lib/Transforms/Vectorize/VPlan.h
+954-2213 files not shown
+1,096-2719 files

LLVM/project a2a1c45llvm/lib/Analysis IVDescriptors.cpp

remove ;
DeltaFile
+1-1llvm/lib/Analysis/IVDescriptors.cpp
+1-11 files

LLVM/project 2b4e8afllvm/include/llvm/Analysis IVDescriptors.h, llvm/lib/Analysis IVDescriptors.cpp

[IVDescriptors] Implement MonotonicDescriptor

RFC link: https://discourse.llvm.org/t/rfc-loop-vectorization-of-compress-store-expand-load-patterns/86442

"Monotonic" variable is similar to induction variable, but its value is updated under some condition, e.g.:
```
int idx = 0;
for(int i = 0; i < n; ++i) {
  // some uses of idx
  if (cond)
    ++idx;
}
```
In this example, `i` is induction variable and `idx` is monotonic variable: it's updated only when cond == true. In LLVM IR, this looks like:
```
loop_header:
  %monotonic_phi = [%start, %prehader], [ %chain_phi0, %latch]

step_bb:

    [26 lines not shown]
DeltaFile
+153-0llvm/unittests/Analysis/IVDescriptorsTest.cpp
+121-0llvm/lib/Analysis/IVDescriptors.cpp
+39-0llvm/include/llvm/Analysis/IVDescriptors.h
+313-03 files

LLVM/project 0257996llvm/include/llvm/Analysis IVDescriptors.h

Add docs
DeltaFile
+27-4llvm/include/llvm/Analysis/IVDescriptors.h
+27-41 files

LLVM/project 78ee0e6lldb/source/Plugins/Process/Utility RegisterTypeDetector_arm64.h RegisterTypeDetector_arm64.cpp, lldb/test/API/linux/aarch64/mte_core_file TestAArch64LinuxMTEMemoryTagCoreFile.py

[lldb][AArch64] Use unique_ptr instead of statics in RegisterTypeDetector

Fixes #214264.

I used static variables for the created types, on the assumption
that only one detector would be used and that the host's
features would not change.

That is true for an lldb-server on a real Linux/FreeBSD system.
It is not true when we use the detector with core files. In the
same LLDB session you might load several files that came from
systems with different features.

The result was that the first detection sets up the static variables
and future detections do not update them. So subsequent core
files can have incorrect types.

(and in future if types vary per-process, we could have the same
issue in lldb-server)

    [31 lines not shown]
DeltaFile
+64-74lldb/source/Plugins/Process/Utility/RegisterTypeDetector_arm64.cpp
+57-27lldb/source/Plugins/Process/Utility/RegisterTypeDetector_arm64.h
+19-0lldb/test/API/linux/aarch64/mte_core_file/TestAArch64LinuxMTEMemoryTagCoreFile.py
+140-1013 files

LLVM/project ac04848lldb/source/Plugins/Process/Utility RegisterTypeDetector_arm64.h

put back initialiser
DeltaFile
+1-1lldb/source/Plugins/Process/Utility/RegisterTypeDetector_arm64.h
+1-11 files

LLVM/project 46feebbclang/test/OpenMP taskloop_simd_reduction_messages.cpp taskloop_reduction_messages.cpp, flang/test/Semantics/OpenMP allocate-clause-version.f90

[flang][OpenMP] Gate the allocate clause at OpenMP 5.0 (#213980)

`allocate` is an OpenMP 5.0 clause, but 58 of the 62 directives that
allow it declare it as
bare `VersionedClause<OMPC_Allocate>`. The default in `DirectiveBase.td`
is `min = 1`:

```
class VersionedClause<Clause c, int min = 1, int max = 0x7FFFFFFF> : Versioned<min, max>
```

so those 58 accept the clause at every version. Only four are gated
today: `do`, `taskgroup`
and `parallel do` at 50, and `scope` at 52.

This is reachable in practice because flang defaults to OpenMP 3.1
(`newestFullySupported = 31`, `CompilerInvocation.cpp`). An invocation
with no
`-fopenmp-version=` lands in the ungated range, semantics accepts the

    [81 lines not shown]
DeltaFile
+168-166clang/test/OpenMP/for_lastprivate_codegen.cpp
+58-58llvm/include/llvm/Frontend/OpenMP/OMP.td
+47-0flang/test/Semantics/OpenMP/allocate-clause-version.f90
+20-20clang/test/OpenMP/taskloop_simd_reduction_messages.cpp
+20-20clang/test/OpenMP/taskloop_reduction_messages.cpp
+20-20clang/test/OpenMP/target_teams_reduction_messages.cpp
+333-28488 files not shown
+903-74294 files

FreeBSD/src 2947a48sys/amd64/amd64 exception.S

amd64: try to fix the build with old clang that does not know about FRED

(cherry picked from commit 72952bf6a307391e127d3ab4a6f073664ce62d89)
DeltaFile
+17-0sys/amd64/amd64/exception.S
+17-01 files

FreeBSD/src 408031fsys/amd64/include md_var.h

amd64: Remove a prototype for an unimplemented function

(cherry picked from commit f42c68291d6803abc404fa8d6915a9f5de3a3448)
DeltaFile
+0-1sys/amd64/include/md_var.h
+0-11 files

FreeBSD/src 5e1fdbdsys/amd64/amd64 initcpu.c cpu_switch.S

amd64: FRED support

(cherry picked from commit 6e93f5e4d6932c423b89dff8fc08d86f8bdeb7b9)
DeltaFile
+554-0sys/amd64/amd64/trap.c
+282-0sys/amd64/amd64/exception.S
+73-31sys/amd64/amd64/machdep.c
+55-21sys/amd64/amd64/mp_machdep.c
+68-5sys/amd64/amd64/cpu_switch.S
+43-0sys/amd64/amd64/initcpu.c
+1,075-5716 files not shown
+1,160-8522 files