[LV][AArch64] Provide option to use partial reductions by default (#216001)
Partial reduction intrinsics are explicitly unordered, which may allow
us to perform extra optimizations that may result in a different
ordering of lanes within a vector. This would only be done if the target
indicated the intrinsic was at least as cheap as the normal accumulation
operation.
Off by default to start with.
As an example of an optimization which would be easier/less fragile with
this change, see #206047 which currently tries to match the entire
reduction. If the in-loop update was a partial reduction intrinsic (even
at the same VF), then we could just implement the whole thing as a
DAGCombine.
Some follow-up work will be needed before turning on by default; there's
25 AArch64 check-all failures (just need to update test checks) and some
build failures + regressions in some spec benchmarks to look at.
bsd.crunchgen.mk: Add CRUNCH_HOSTPROGS_${P} and fix awk in rescue
The usr.bin/awk consists of two subdirs: (1) 'awk' the actual program;
(2) 'maketab' a host program to help generate additional source.
Previously, the initrd/rescue/Makefile pointed the awk source to
'usr.bin/awk', and that caused the following warning:
```
crunchgen: rescue.conf: awk: warning: could not find any .o files
```
And pointing the source to 'usr.bin/awk/awk' couldn't fix it because the
required 'maketab' program would be unavailable.
To fix the problem as well as to extend the crunchgen framework, add the
CRUNCH_HOSTPROGS_${P} variable to specify the host programs required by
the build. Implement the rules and specify the dependencies to build
the host programs.
[2 lines not shown]
[LLVM][SelectionDAG] Implement unrolling of scalable vectors. (#216985)
We cannot typically unroll scalable vectors because they have an
unknown-at-compile-time vector length. However, when a function has the
vscale_range attribute, we can unroll based on their maximum possible
length.
[AArch64][PAC] Reset `killed` operand flags in outlined functions
Presently, MachineOutliner does not take `killed` operand flags into
account when merging instruction sequences. While it sounds perfectly
reasonable not to inhibit merging of the instruction sequences that
only differ in `killed` flags (for N flags there is technically 2^N
valid ways to drop some subset of them), copying these flags from
an arbitrarily chosen representative instruction may result in
incorrect codegen of PAuth-related pseudo instructions on AArch64.
To keep `killed` flags conservatively correct as if `OUTLINED_FUNCTION`s
are virtually re-inserted at every call site, this patch takes the
simplest approach of resetting every `killed` flag inside the
outlined functions.
py-strich: updated to 0.20
0.20
All formats: numeric arguments (cell size, render options, encoder settings)
are validated for type and range up front
GS1-128: Code128Data.gs1 rejects payloads over the GS1 maximum of 48 data
characters
[CIR][NFC] Share getSuccessorRegions across region-branch ops
Six of the ten CIR ops implementing RegionBranchOpInterface reported the same
successors: any of their regions may be entered from the parent operation, and
every region exit goes back to it. Add a CIR_EnterAnyRegionBranchOpBase class
that appends that definition to the one inherited from CIR_RegionBranchOpBase,
retarget the six ops onto it and delete their hand-written definitions.
The shared definition walks getRegions() rather than naming region accessors.
For all six ops the entry regions were exactly the declared regions in
declaration order, so it reports the same successors in the same order. The doc
comments of the deleted ScopeOp and TernaryOp definitions go with them, instead
of being left behind on the neighbouring builders.
IfOp, GlobalOp, TryOp and AwaitOp stay on the base class, since their successors
depend on the operation: IfOp falls back to the parent when the else region is
empty, GlobalOp skips its optional ctor and dtor regions, TryOp iterates
variadic handler regions, and AwaitOp routes ready to resume and suspend.
[CIR] Add RegionBranchOpInterface unit tests and fix cir.await successors
Five of the ten ops implementing RegionBranchOpInterface have no unit test
coverage: cir.case, cir.cleanup.scope, cir.global, cir.await and
cir.coro.body. Add tests for all five.
Covering cir.await exposes a disagreement with its own terminator.
cir.condition terminates the ready region and reports {resume, suspend} as
its successors when the parent is an await, but AwaitOp::getSuccessorRegions
listed all three regions as entry successors and reported the parent op as
the successor of every region exit. Fix it to match cir.condition: ready is
the only entry successor, exiting ready branches to resume or suspend, and
exiting suspend or resume returns to the parent operation.
cir.await declares no results and carries NoRegionArguments, so successor
operand and input counts stay at zero along every edge and the MLIR verifier
is unaffected.
[CIR][NFC] Share getSuccessorInputs across region-branch ops
The ten CIR ops implementing RegionBranchOpInterface each hand-wrote
getSuccessorInputs, and all ten bodies were equivalent: regions take no
inputs, and returning to the parent yields the parent's results. Three did
not look equivalent but are: CleanupScopeOp and CoroBodyOp returned an empty
ValueRange unconditionally and declare no results, and AwaitOp returned
region block arguments but carries NoRegionArguments, so those ranges are
always empty.
Add a CIR_RegionBranchOpBase ODS class that declares the method and generates
the single shared body through extraClassDefinition, mirroring the existing
CIR_LoopOpBase, and retarget all ten ops onto it.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
ten definitions removed from CIRDialect.cpp.
[CIR] Give the cleanup kind a proper standalone attribute spelling (#220878)
CleanupKindAttr overrode its assembly format to a bare `$value` so
`cir.cleanup.scope` would print `cleanup all`. The cost was that the
attribute had no readable standalone form, falling back to
`#cir<cleanup_kind all>`.
The `enum($attr)` operation directive removes the tradeoff. The
attribute
keeps CIR_EnumAttr's bracketed default and now spells
`#cir.cleanup<all>`,
while the operations ask for the bare keyword. The mnemonic drops the
`_kind`
suffix the C++ class name carries.
[CodeGen] Add Action types for vector_(de)interleave ops (#220859)
Targets could not set lowering actions with setOperationAction for
Interleave operations due to dependence on type and factor. Instead of
requiring two API to find legality of operation one for type and one for
factor. It is cleaner to add a separate action type for these operation.
umtx: do not sleep on an unowned mutex after a spurious CAS failure
On ll/sc architectures casueword32() may report a spurious
store-conditional failure (reservation lost to an interrupt, preemption,
or another CPU touching the same reservation granule), and this is
indistinguishable from a genuine comparison mismatch: both return 1.
That is intentional since D20772 and documented in casueword(9) ("The
store can fail on load-linked/store-conditional architectures."), so
callers must cope.
do_lock_normal() does not fully cope. When the initial
UMUTEX_UNOWNED -> id acquire CAS fails spuriously, the observed owner is
still UMUTEX_UNOWNED, so neither the UMUTEX_CONTESTED branch nor the
real-owner case applies, and execution falls through past the "rv == 1
but not contested, likely store failure" comment into the sleep path.
There, the contested-bit CAS (expecting the observed owner, i.e.
UMUTEX_UNOWNED) succeeds because the mutex really is unowned, stamping
m_owner = UMUTEX_CONTESTED with no owner tid, and the thread sleeps on
"umtxn" forever: nobody owns the mutex, so no unlock and no wakeup ever
[28 lines not shown]
[mlir][linalg] Scaled contraction op (#203958)
A new contraction op with extended semantics to represent input scaling
using additional scale values as described by their corresponding
indexing maps.
RFC: https://discourse.llvm.org/t/rfc-linalg-scaled-contraction/91075
Assisted-by: Claude
Co-authored-by: Md Asghar Ahmad Shahid <md.asghar.ahmad.shahid at intel.com>
[VPlan] Use VPlan-based costs for all integer IVs. (#217766)
Continue the migration of induction costs to be completely VPlan-based.
VPScalarIVStepsRecipe now supports compute its cost for integer
inductions. Skip the legacy precomputeCost code for them.
The main change is that previously we never explicitly accounted for the
canonical IV, which we always generate. It was instead folded into
computing cost of the existing integer IVs with only scalar uses, which
in
most cases are actually served by the canonical IV.
The new code accounts for the cost explicitly, as
VPInstruction::computeCost returns 0 for all recipes w/o underlying
instruction.
Floating point and pointer induction will be handled as follow-ups to
keep the diffs managable and make it easier to track down regressions.
[28 lines not shown]
